15:02:13 <ihrachys> #startmeeting neutron_upgrades
15:02:13 <openstack> Meeting started Mon Jun 20 15:02:13 2016 UTC and is due to finish in 60 minutes.  The chair is ihrachys. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:17 <openstack> The meeting name has been set to 'neutron_upgrades'
15:02:19 <ihrachys> hello everyone!
15:02:19 <korzen> hi
15:02:30 <dasm> g'morning yall
15:02:43 * ihrachys waves at rossella_s
15:02:50 <rossella_s> hi all!
15:03:01 <ihrachys> #link https://wiki.openstack.org/wiki/Meetings/Neutron-Upgrades-Subteam Agenda
15:03:08 <ihrachys> #topic Actions from the last meeting
15:03:15 <ihrachys> "ihrachys to clean up agenda page"
15:03:37 <ihrachys> I think I did it. Left just a stub with no references to patches since gerrit apparently is a better tool to track patches :)
15:03:48 <rossella_s> :D
15:03:49 <ihrachys> it's now quite bare, but that's ok
15:04:13 <ihrachys> "rossella_s to come up with specific list of TODOs for port object"
15:04:31 <rossella_s> ihrachys, I put that in the commit message
15:04:36 <ihrachys> rossella_s: link?
15:04:52 <rossella_s> https://review.openstack.org/#/c/253641/
15:05:37 <ihrachys> mm, great. I will take a look.
15:05:41 <rossella_s> ihrachys, tests are currently failing...I don't think it's hard to fix them though...they are failing because something has changed after rebading
15:05:45 <rossella_s> *rebasing
15:06:19 <ihrachys> rossella_s: unless you get to that till next week, I plan to take it over from you next Mon. does it sound ok?
15:06:51 <rossella_s> ihrachys, perfect
15:06:54 * ihrachys will be mostly unavailable this week due to visas and PTOs
15:07:15 <rossella_s> ihrachys, this week I will be mostly unavailable too...due to the opnfv summit
15:07:26 <ihrachys> rossella_s: have fun :)
15:07:28 <ihrachys> "ihrachys to come up with a plan for getting dvr grenade multinode voting"
15:07:41 <ihrachys> that did NOT happen, even though I actually started looking at it today.
15:07:57 <ihrachys> I hope to come up with some plan, at least in my mind, tomorrow
15:08:13 <ihrachys> "ihrachys to send a bi-weekly to openstack-dev@"
15:08:20 <ihrachys> that DID happen, though very late
15:08:28 <ihrachys> I spotted that I haven't done it today only
15:08:36 <ihrachys> #link http://lists.openstack.org/pipermail/openstack-dev/2016-June/097750.html the report
15:08:51 <korzen> ihrachys, nice :)
15:09:20 <ihrachys> thanks
15:09:26 <ihrachys> and that's the end of action items
15:09:49 <ihrachys> #topic Announcements
15:10:13 <ihrachys> I don't have much, except that there will be a mid cycle meetup and some of us may meet there
15:10:22 <ihrachys> I actually go there if I get visa in time
15:10:27 <ihrachys> anyone else going?
15:10:30 <johndperkins> o/
15:10:44 <korzen> I will know in a week or two
15:11:12 <dasm> the same for me. i need to clarify this
15:11:52 <ihrachys> cool
15:11:56 <ihrachys> #topic Partial Multinode Grenade
15:12:14 <ihrachys> as I mentioned, not much happened there because I am a slacker
15:12:32 <ihrachys> one thing to note is a governance change that affected *aas assert:supports-rolling-upgrade tagging
15:12:49 <ihrachys> #link https://review.openstack.org/#/c/323522/
15:13:13 <ihrachys> the change effectively removed the tag, as long as supports-upgrade tag, from all *aas repos
15:13:22 <ihrachys> that's because there are no grenade jobs for those repos whatsoever
15:13:41 <ihrachys> but there is no technical change, just a reflection of reality
15:14:02 <ihrachys> #topic Object implementation
15:14:38 <ihrachys> most of the patches currently in review are mentioned in the email I sent today, so I won't repeat myself
15:14:40 <korzen> rossella_s, can you take a look at: https://review.openstack.org/#/c/326477/10 DNSNameServer patch
15:14:43 <ihrachys> #link http://lists.openstack.org/pipermail/openstack-dev/2016-June/097750.html
15:15:19 <ihrachys> I think the most critical bits right now is subnet adoption, that should prove objects are applicable for core resources
15:15:24 <rossella_s> korzen, will do but probably tomorrow
15:15:48 <ihrachys> apart from that, we obviously want to land smaller objects like DNSNameServer
15:16:13 <ihrachys> and another critical direction is port and all related objects (like security groups that slunkad_ is working on)
15:16:27 <ihrachys> I think sec groups are ready except a missing unit test for is_default field.
15:16:48 <slunkad_> ihrachys: yes I should be able to push that by tomorrow
15:17:07 <ihrachys> slunkad_: great. I hope we will land it then and then port object will be mostly unblocked
15:17:15 <slunkad_> though there are other tests which are failing too
15:17:17 <ihrachys> slunkad_: thanks for pushing it
15:17:29 <rossella_s> slunkad_, thanks!
15:17:38 <korzen> for subnet patch, I have 25 unit tests to be fixed and need to look at functional and API tests
15:18:04 <ihrachys> korzen: do you have estimates considering your current load with other tasks?
15:18:47 <korzen> currently subnet patch is my P1 so I would like to finish this by EOW
15:19:44 <ihrachys> oh great. I will get to it with reviews on Monday.
15:20:05 <korzen> I would need to spin a new patch for create_subnet patch, to see if Subnet object is OK
15:20:23 <korzen> but I'm working hard to get this done ;)
15:20:55 <ihrachys> one general thing to note is: kevinbenton was asking whether we have any devref page that describes all the 'boilerplate' that we have in the base class (fields_no_update, fields_need_translation, db_model, synthetic_fields, ...) for other neutron devs.
15:21:18 <rossella_s> ihrachys, it's a good point we should do that
15:21:27 <ihrachys> I think we promised to the community before that while we won't follow thru with a spec, we will need to produce some devref docs as part of the effort.
15:21:46 <ihrachys> so, we have that action item on the plate. do we have volunteers? :)
15:21:51 <korzen> I guess that base class has some comments
15:22:13 <johndperkins> there is versionedobject dev documentation already, do you mean neutron-specific?
15:22:27 <ihrachys> korzen: right. but apparently it's not enough, it would be probably wise to cover base pieces that we consider part of public API for objects to be described in laymen's terms.
15:22:49 <ihrachys> johndperkins: yes. we have a lot of things implemented in base neutron class that are not part of the library.
15:23:23 <ihrachys> korzen: once we have it in devref, we can actually expand the documentation right away while landing more base class features.
15:23:30 <johndperkins> ihrachys: I noticed, like how we don't import all the objects in objects/__init__.py as instructed
15:23:41 <johndperkins> but perhaps that was an oversight
15:23:46 <ihrachys> johndperkins: yeah, testing interface is also of interest.
15:24:18 <ihrachys> so, that's a huge piece to swallow. anyone brave? :)
15:24:26 <korzen> I can spin a patch
15:24:36 <ihrachys> korzen: you win the prize!
15:24:58 <korzen> if it is not a money or beer I do not want it ;)
15:25:07 <ihrachys> #action korzen to spin up a devref patch describing base neutron object class features, testing interface, tips&tricks, etc.
15:25:22 <ihrachys> korzen: no, the action item!
15:25:28 <korzen> :(
15:25:31 <ihrachys> :D
15:26:18 <korzen> action items are adorable also
15:26:23 <ihrachys> apart from those things already discussed, I don't have anything for objects. anything of particular interest for the team that I am not aware?
15:27:01 <korzen> any news from new features adopting OVOs?
15:27:20 <johndperkins> I'm working on routers but hitting duplicate errors
15:27:28 <johndperkins> I could use some expert eyes
15:27:28 <ihrachys> korzen: nope. I've heard from carl_baldwin in the email thread for the prev weekly update, but nothing actionable in gerrit.
15:27:41 <ihrachys> johndperkins: link to the patch CI run?
15:27:58 <johndperkins> https://review.openstack.org/#/c/307964/
15:28:22 <johndperkins> and low-priority PS on OVO itself: https://review.openstack.org/#/c/329742/
15:28:24 <slunkad_> well like I mentioned earlier even for the sec groups we have a lot of unit tests failing with foreign key contraints...anyone familiar with those?
15:29:13 <ihrachys> johndperkins: "ObjectFieldInvalid: Field destination of RouterRoute is not an instance of Field" is that the one?
15:29:20 <johndperkins> no, I'm past that
15:29:27 <slunkad_> oh sorry I think I am wrong. they just seem to have passed!
15:29:45 <johndperkins> It's "Failed to create a duplicate RouterPort..." now
15:30:08 <rossella_s> johndperkins, without the logs it's hard to detect the problem
15:30:25 <johndperkins> true, I'll push
15:30:25 <korzen> slunkad_, if foreign keys are missing, you need to create the missing entity that object is referring to, or add mock somewhere
15:30:46 <rossella_s> johndperkins, anyway maybe you are using some key that it's the same value for 2 different object?
15:31:00 <ihrachys> johndperkins: right. we need CI run with logs to be able to have a meaningful discussion on your latest patch set.
15:31:04 <johndperkins> rossella_s: that's what the error suggests but I can't find it
15:31:05 <slunkad_> korzen: well it seems to be passing as of now. bull will keep that in mind
15:31:58 <ihrachys> johndperkins: ok, keep me posted once we have CI logs and the latest patch on gerrit. ping me in irc, no need to wait for the next meeting or smth. :)
15:32:15 <johndperkins> ihrachys: understood
15:32:41 <korzen> anyone is familiar with TestMl2DbOperationBoundsTenant.test_subnet_list_queries_constant unit tests? in Subnet ovo patch, the test is failing when you call list subnet 2 times, the ;constant; number of queries is raising
15:33:20 <ihrachys> korzen: I know the intent of the test, yes
15:33:35 <ihrachys> korzen: it validates that the number of sql queries does not raise with the number of resources in the database
15:33:55 <korzen> and if it does ;) ?
15:33:55 <ihrachys> meaning, no code triggers additional fetches apart from the very base model query from common_db_mixin
15:34:15 <ihrachys> this is to avoid scaling issues
15:34:23 <ihrachys> well if it does, we should think of how to avoid that :)
15:34:29 <korzen> that what I thought so..
15:34:29 <ihrachys> korzen: link to the code that triggers that?
15:34:53 <korzen> in ovo?
15:34:57 <korzen> or in UT?
15:35:26 <ihrachys> korzen: in db code I guess. what's the operation that triggers more queries? list_subnets?
15:35:35 <korzen> yes
15:36:29 <korzen> http://logs.openstack.org/01/321001/3/check/gate-neutron-python27/b9a1a1c/console.html.gz#_2016-06-17_11_01_26_004840
15:36:32 <korzen> smth like this
15:39:14 <korzen> if I understood correctly, the UT is checking if queries are the same if we have 10 vs 20 subnets to list?
15:39:20 <ihrachys> oh I think that's because of rbac load you have
15:39:28 <ihrachys> korzen: yeah, along those lines
15:39:46 <ihrachys> in https://review.openstack.org/#/c/331009/1/neutron/objects/subnet.py, you seem to load rbac .shared field after initial fetch
15:40:47 <ihrachys> we need to look at how we currently achieve the constant scaling for sql queries, and model it into the subnet object
15:40:54 <ihrachys> korzen: does it ring a bell?
15:41:08 <korzen> yes, I have changed it to use RBAC https://review.openstack.org/#/c/331009/1/neutron/objects/subnet.py@189
15:41:38 <korzen> ok, thx ihrachys I will look at it tomorrow
15:41:46 <korzen> lets continue ;)
15:42:00 <ihrachys> korzen: ok, I guess you will need to load under transaction.
15:42:13 <ihrachys> hm, or maybe that's not enough. anyway...
15:42:14 <ihrachys> let's move on :)
15:42:20 <ihrachys> #topic Other patches on review
15:42:43 <ihrachys> anything of interest apart from objects?
15:43:19 <ihrachys> I bet nope, moving on :
15:43:26 * ihrachys #topic Open discussion
15:43:39 <ihrachys> ok, I have one small thing that may be of relevance
15:44:03 <jlibosva> ihrachys: note that you haven't changed the topic :)
15:44:13 <ihrachys> #topic Open discussion
15:44:18 <ihrachys> jlibosva++
15:44:32 <korzen> ihrachys, changed a topic in his mind
15:44:38 <korzen> ;)
15:44:58 <ihrachys> in mitaka, neutron and nova try to set mtu on data path devices to network mtu. due to some reasons that I am not going to cover here but that you can read on http://openvswitch.org/pipermail/dev/2016-June/073190.html it may not work in some cases.
15:45:13 <ihrachys> but basically, we use ovs in a way that is not supported by ovs folks
15:45:29 <ihrachys> and maybe that discussion will trigger bridge remodeling for existing workloads
15:45:57 <ihrachys> which... now we get to upgrades part... may trigger questions on how to migrate from existing setup to a better one for existing workload
15:46:29 <ihrachys> so... since everyone here should be aware of no-downtime requirement on the data plane, it may be of interest to some of you
15:46:45 <ihrachys> there is a link to openstack-dev@ thread in the email too
15:47:00 <ihrachys> apart from that, I don't have anything.
15:47:20 <ihrachys> anyone has other topics in mind?
15:47:26 <korzen> will it happen in newton?
15:47:35 <ihrachys> korzen: it probably won't. if at all.
15:47:54 <ihrachys> korzen: atm it's in early discussion mode, but I wanted to have you aware of that.
15:48:14 <ihrachys> I would be more happy to find a solution that would avoid bridge remodeling
15:48:18 <ihrachys> for obvious reasons
15:48:32 <korzen> upgrading the OVS is causing the dataplane downtime as well so...
15:49:46 <jlibosva> that's a good point. afaik node are evacuated before upgrading - so you can migrate workloads away, change how bridges are wired and migrate workload back
15:50:18 <ihrachys> korzen: that's fair. I guess it means once we require a new ovs version we may use the opportunity to remodel bridges/drop support for old model too.
15:50:43 <ihrachys> jlibosva: yeah, but we generally allowed for an in-place upgrade where you just restart the agent and it gracefully keep up maintaining connectivity
15:51:35 <korzen> we need to see how containers can help with taht
15:51:52 <korzen> but that affect/relay on deployment model
15:51:53 <jlibosva> also this topic is probably relevant for vlan aware vms blueprint
15:52:37 <ihrachys> jlibosva: loosely for my taste, at least from upgrades perspective, since they model bridge-per-port for new trunk ports only. so there is no existing workload that would be affected.
15:53:42 <ihrachys> or at least it's my limited understanding of what they try to achieve
15:54:03 <jlibosva> I imagine it has pretty similar - you basically inject a bridge between integration bridge and an instance's tap
15:55:02 <ihrachys> jlibosva: how would per-network bridges interact with that? having another layer in between? so you would have br-int - br-<netid> - br-<portid>?
15:55:58 <jlibosva> ihrachys: no, I meant that migration from current wiring to mtu is similar as migration to vlan-aware vms with respect to not braking data plane
15:56:41 <jlibosva> br-int <-> br-<netid> is like br-int <-> br-<portid>
15:56:43 <ihrachys> jlibosva: but for vlan-aware-vms, you don't change the bridge setup for 'usual' ports right?
15:56:55 <ihrachys> it applies to new trunk ports only
15:57:00 <jlibosva> yep, only for trunk ports
15:57:18 <ihrachys> while for the mtu discussion, we talk about disrupting for all existing ports
15:57:50 <ihrachys> because those will need to be rewired into per-network bridges without users opting into using new features.
15:59:07 <ihrachys> ok, I guess we are running out of time
15:59:13 <ihrachys> thanks everyone for joining
15:59:15 <ihrachys> keep up the good work
15:59:18 <ihrachys> #endmeeting