15:00:54 <ihrachys> #startmeeting neutron_upgrades
15:00:55 <openstack> Meeting started Mon Aug 29 15:00:54 2016 UTC and is due to finish in 60 minutes.  The chair is ihrachys. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:57 <ankur-gupta-f1> yo yo yo
15:01:00 <openstack> The meeting name has been set to 'neutron_upgrades'
15:01:07 <ihrachys> hi folks!
15:01:20 <dasanind_> hi
15:01:31 <ihrachys> #link https://wiki.openstack.org/wiki/Meetings/Neutron-Upgrades-Subteam Agenda
15:01:38 <electrocucaracha> o/
15:01:42 <manjeets_> o/
15:01:55 <ihrachys> #topic Announcements
15:02:10 <tonytan4ever> o/
15:02:13 <ihrachys> I have nothing specific, but note we have newton-3 milestone cut-off this week
15:02:25 <ihrachys> some time after that, stable/newton branch will be created
15:02:33 <ihrachys> which should open master for Ocata development
15:02:36 <sindhu> hi
15:02:52 <ihrachys> at the moment the team works on delivering pending features targeted for Newton
15:03:09 <ihrachys> so attention may be not ideal for things that are not targeted as such
15:03:17 <electrocucaracha> ihrachys: sometime this week means monday or friday?
15:03:26 <ihrachys> electrocucaracha: deadline is Sep 1
15:03:34 <ihrachys> Thu
15:03:46 <ihrachys> but stable branch will take a bit more to emerge
15:03:50 <ihrachys> as history shows
15:04:09 <ihrachys> any questions on the milestone impact?
15:04:34 <ihrachys> nope, moving on :)
15:04:35 <tonytan4ever> One question
15:04:40 <manjeets_> one from me as well
15:04:42 <ihrachys> tonytan4ever: shoot
15:04:57 <ihrachys> manjeets_: same
15:04:57 <tonytan4ever> is there a feature freeze during the tim ? If yes, how long is the freeze going to be ?
15:05:28 <ihrachys> #link https://releases.openstack.org/newton/schedule.html Newton release schedule
15:05:47 <ihrachys> feature freeze is this week, happening with the milestone cut-off
15:05:47 <manjeets_> I have some patches which were dependent on patch relocating l3 models, how long it would take to clear l3, if it is going to take long then i can remove the dependency
15:05:52 <manjeets_> for other patches
15:06:01 <tonytan4ever> I see thanks.
15:06:16 <ihrachys> after that moment and until master is open for Ocata, new features are generally hold off except when you get an exception from PTL
15:06:40 <ihrachys> manjeets_: ideally we remove dependency from l3 one because it won't land until master is open
15:06:42 <tonytan4ever> Seems like all the way to Sep 2nd.
15:06:54 <ihrachys> manjeets_: due to large number of conflicts that would ensue with it landing
15:07:02 <manjeets_> okay need to do that
15:07:18 <ihrachys> usually stable branches are cut off around RC1
15:07:33 <ihrachys> meaning we'll have 1-2 weeks of tight control of the gate
15:07:48 <ihrachys> during that time it will be hard to justify landing anything non-release critical
15:08:25 <ihrachys> ok, I hope it's clear. let's move on.
15:08:26 <ihrachys> #topic Partial Multinode Grenade
15:09:03 <ihrachys> for linuxbridge, I asked jschwarz and abregman lately to take a look at what's missing there
15:09:24 <ihrachys> at least we need https://review.openstack.org/#/c/346282/ to land in devstack
15:09:26 <manjeets_> for multinode grenade do we just need to add gate jobs for testing upgrades ?
15:09:34 <ihrachys> but for some reason it does not want to, even with W+1
15:09:58 <ihrachys> sc68cal: https://review.openstack.org/#/c/346282/ does not want to land, can you try to W+1 it once more?
15:10:14 <ihrachys> manjeets_: we already have jobs, both dvr and legacy
15:10:39 <ihrachys> manjeets_: we need a linuxbridge one though, the job is present in experimental gate but it currently fails due to some pieces missing in devstack and devstack-gate.
15:10:43 <manjeets_> what are the missing bits for multinode grenade then ?
15:10:43 <electrocucaracha> Revert "Revert"?
15:11:00 <ihrachys> manjeets_: there are also concerns around whether we run all the needed tests, but that's a separate story
15:11:12 <ihrachys> electrocucaracha: yeah, the first attempt broke the gate so was reverted :)
15:11:41 <ihrachys> multiple Reverts are usually a monument of shame for those who pushed the first time  :)
15:11:55 <tonytan4ever> double negatives...
15:12:02 <manjeets_> lol
15:12:25 <ihrachys> there is more to linuxbridge job than this patch. some work will probably belong to devstack-gate and may project-config. we'll see.
15:12:59 <ihrachys> also, armax wanted the whole neutron team to move from testing against legacy l3 mode towards dvr+ha gating only.
15:13:28 <ihrachys> but that's probably in scope of some broader discussion on how we gate, not just grenade.
15:14:05 <ihrachys> otherwise, I'll wait until jschwarz gets to the job and has progress to report. :)
15:14:12 <ihrachys> moving on to...
15:14:14 <ihrachys> #topic Object implementation
15:14:28 <ihrachys> ok, P1 (ports, networks, secgroups) are either in tree or on review
15:14:55 <ihrachys> I haven't seen much traction from push-notifications author (kevinbenton) which sucks but... what can I do? :)
15:15:12 <ihrachys> networks: https://review.openstack.org/#/c/269658/ and ports: https://review.openstack.org/#/c/351368/24
15:15:37 <ihrachys> I don't believe there is any known issue there, we just wait for kevinbenton to get to those
15:15:39 <electrocucaracha> regarding P1 objects, do we have plans to integrate them in newton 3?
15:15:58 <manjeets_> i think port and network (intro ovo ) are very close to clear
15:16:10 <electrocucaracha> I'm referring to subnet which hasn't been integrated
15:16:19 <ihrachys> electrocucaracha: I had such plans, but in the end it all depends on whether kevinbenton will have time to make use of them in push-notifications.
15:16:33 <ihrachys> the latter may as well slip into Ocata, in which case there is no rush to land those in Newton
15:16:52 <ihrachys> electrocucaracha: integrations are out of scope for Newton
15:17:05 <ihrachys> electrocucaracha: this work should happen start of Ocata though
15:17:28 <manjeets_> ihrachys, would integration be high priority in ocata ?
15:17:44 <ihrachys> manjeets_: yes, as well as convertion of all the code base to objects.
15:18:07 <electrocucaracha> it would be nice to have an idea how long take us to merge a single object since the creation
15:18:14 <ihrachys> manjeets_: this, and making sure that all is ready in tree for contract-less upgrades.
15:18:16 <dasm> ihrachys: which bp covers it? that one? https://blueprints.launchpad.net/neutron/+spec/online-upgrades
15:18:50 <ihrachys> electrocucaracha: it really depends on where you stand in terms of framework. This cycle we spent significant time to prepare base class for actual usage.
15:19:00 <ihrachys> electrocucaracha: some of us believe that now it should be an easier path
15:19:15 <ihrachys> dasm: this bp covers contract-less migrations
15:19:28 <electrocucaracha> ihrachys: I'm one of them :)
15:19:32 <ihrachys> dasm: another one is for object adoption: https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/adopt-oslo-versioned-objects-for-db
15:19:43 <ihrachys> the latter is a dependency for the former
15:19:43 <dasm> ihrachys: thanks
15:20:02 <dasm> electrocucaracha: always look for the bright side of life :)
15:20:14 <ihrachys> electrocucaracha: we have plenty of folks on board now, we should be at good position to make progress.
15:20:37 <ihrachys> that said, we have some N things still...
15:21:12 <ihrachys> one thing I spotted lately in gate logs is we issue a warning when sorting against keys that do not contain 'id'
15:21:17 <ihrachys> it's a bug in oslo.db.
15:21:22 <ihrachys> I sent a patch: https://review.openstack.org/#/c/362061/
15:21:35 <ihrachys> will take some time to bake, and also we are past library freeze.
15:21:44 <ihrachys> so no requirements bump for oslo.db for us
15:21:58 <ihrachys> though I don't think it's needed if we just backport and release a new version.
15:22:06 <ihrachys> in the end, the warning does not break anything...
15:23:01 <ihrachys> another N thing is, we currently pass subnetpool objects instead of models to registered plugin extension functions
15:23:10 <ihrachys> there are some patches in flight but they are not ready to merge
15:23:17 <ihrachys> https://review.openstack.org/#/c/348279/ and https://review.openstack.org/#/c/348987/
15:23:32 <ihrachys> jschwarz once started looking at those but I don't know the latest state
15:23:51 <ihrachys> need to sync with John to make sure it's on radar for N
15:23:58 <ihrachys> otherwise we potentially break some plugins
15:24:10 <ihrachys> (though we did not have any actual reports)
15:24:52 <ihrachys> anyone knows of more patches that are Newton-critical?
15:24:54 <electrocucaracha> more likely he's waiting for passing jekins jobs
15:25:44 <ihrachys> electrocucaracha: considering pep8 fails there, it may take some time :P
15:25:53 <ihrachys> I will ask John to take a look
15:26:09 <ihrachys> oh I remember one more N thing
15:26:11 <electrocucaracha> well, the UT for ForeignKeyNotFoundError is not critical but it's nice to have it
15:26:12 <electrocucaracha> https://review.openstack.org/#/c/360699/
15:26:34 <ihrachys> the thing when passing an unknown query filter to API makes objects raise an exception.
15:26:57 <ihrachys> before objects, those queries were silently ignored, so we need to continue handling those gracefully
15:27:11 <ihrachys> jschwarz was looking at filtering those out at API layer: https://review.openstack.org/#/c/352809/
15:27:36 <ihrachys> but we realized that due to the hackish way we evolve and implement our API, we currently don't have enough info to do it there.
15:27:50 <ihrachys> (there are plans to move into this direction long term though)
15:28:12 <ihrachys> so we will probably need to ignore validating filters in objects layer when filters are passed from API
15:28:20 <ihrachys> afaik there is no patch for that right now
15:28:27 <ihrachys> and I would appreciate someone to take the piece
15:29:04 <electrocucaracha> I think that I can
15:29:07 <ihrachys> basically, it would involve adding a hacking argument to get_objects like validate_filters=True and pass False where we pass filters from API layer.
15:29:15 <ihrachys> like in get_subnetpools in base db class
15:29:33 <ihrachys> electrocucaracha: ok cool, please take that on your plate :)
15:30:07 <ihrachys> also, lots of folks currently chew model relocation: https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bug/1597913
15:30:19 <manjeets_> ihrachys, I had question regarding online schema migrations ?
15:30:52 <ihrachys> I tried to land some of them where I saw they are ready AND don't result in conflicts. Most remaining may need to wait until Ocata is open though. Thanks everyone on getting this covered.
15:30:54 <ihrachys> manjeets_: shoot
15:31:03 <manjeets_> we support those with alembic migration scripts ?
15:31:40 <ihrachys> manjeets_: depending on what you mean
15:31:50 <manjeets_> we can update shcema without putting the service down ? just wanted to check how well they work ?
15:31:51 <ihrachys> manjeets_: we split the scripts into expansion and contraction
15:32:04 <ihrachys> manjeets_: only expansion scripts are safe to execute online
15:32:18 <ihrachys> manjeets_: so you still need to shutdown all services before contraction
15:32:46 <manjeets_> there is no downtime while expansion ?
15:32:52 <ihrachys> manjeets_: the plan is to forbid contract data migration in Ocata, and postpone schema contraction to when old data is not accessed
15:33:13 <ihrachys> manjeets_: expansion, no, but consider that it's mostly adding columns and tables, nothing time consuming
15:33:22 <ihrachys> manjeets_: most of downtime come from contraction
15:34:10 <ihrachys> manjeets_: but the last statement from PTL and other team members is, for Ocata, we start forbidding contraction, and will revert the resolution only if we can't manage the complexity of the new world.
15:34:29 <manjeets_> cool, yea I tried running expansion and contract script at the same time expansion succeeded but contraction failed
15:34:45 <ihrachys> manjeets_: so it will depend on the team that we deliver alternative solution to existing problems and help the other team members to evolve their features in contract-less manner.
15:35:18 <ihrachys> otherwise we will just revert to existing world, and with bad publicity to the rolling scenario :)
15:35:30 <ihrachys> everyone should feel motivated to kick ass :)
15:35:30 <manjeets_> ihrachys, if we forbid contract completely are we gonna keep dead fields in database ?
15:36:00 <ihrachys> manjeets_: schema contraction will happen, but just later (2 cycles since when you introduced the feature that triggered the change)
15:36:16 <ihrachys> manjeets_: so that at that time there are no neutron-servers that access that old columns/tables
15:36:31 <ihrachys> it will be tricky to maintain, we'll see how well we thought it through on paper
15:36:47 <manjeets_> ok something like clean up after certain time , thanks
15:36:49 <ihrachys> does that answer the question?
15:37:00 <manjeets_> ihrachys, yes thank you very much
15:37:16 <dasm> ihrachys: afaik cinder is currently doing something like this (or at least preparing to do this)
15:37:40 <ihrachys> ok. any other stuff of whole team interest that you have on your plates?
15:37:52 <manjeets_> dasm you mean forbiding contract ?
15:38:12 <dasm> manjeets_: i mean: rolling upgrade without contract phase. everything is phased-out in 3 releases.
15:38:28 <manjeets_> ok
15:38:38 <electrocucaracha> ihrachys: I was wondering if we have plans to merge the devref patch https://review.openstack.org/#/c/336518/
15:38:51 <dasm> manjeets_: fyi: http://docs.openstack.org/developer/cinder/devref/rolling.upgrades.html
15:39:04 <manjeets_> thanks dasm
15:39:14 <ihrachys> electrocucaracha: we could take a look at that once we cover release critical bits
15:39:33 <ihrachys> electrocucaracha: at this point, most of the core team is deep into release preparation
15:39:44 <ihrachys> so it will probably wait a bit for more feedback
15:39:57 <electrocucaracha> ihrachys: yeah, documentation who cares
15:40:02 <ihrachys> :)
15:40:24 <ihrachys> everyone does. but I don't see your vote there?
15:40:48 <ihrachys> please don't think it's on core reviewers to do all reviews.
15:40:52 <electrocucaracha> I didn at some point
15:41:41 <ihrachys> ok. eventually it will get its attention. probably some time closer to N release.
15:42:06 <ihrachys> #topic Other patches on review
15:42:20 <ihrachys> anything non-object related to upgrades?
15:43:08 <ihrachys> great, nothing
15:43:14 <ihrachys> #topic Open discussion
15:43:20 <manjeets_> https://review.openstack.org/#/c/354447/
15:43:23 <ihrachys> if you have questions/concerns/appraisals, shoot
15:43:40 <manjeets_> for this i saw dns-integration is already in list of api extensions
15:43:57 <manjeets_> but for some reason test crashes because it is not enabled
15:44:03 <ihrachys> manjeets_: you mean configured in gate_hook?
15:44:04 <electrocucaracha> I just wondering if someone has tried to generate documentation -> https://review.openstack.org/#/c/353158/
15:44:10 <manjeets_> ihrachys, yes
15:44:43 <ihrachys> manjeets_: yeah, and also the test would be skipped if it would not find the extension configured for tempest.
15:44:48 <ihrachys> manjeets_: sec, checking server logs
15:45:24 <ihrachys> manjeets_: http://logs.openstack.org/47/354447/1/check/gate-neutron-dsvm-api/9c278da/logs/screen-q-svc.txt.gz#_2016-08-12_00_35_43_143
15:45:30 <ihrachys> "Extension dns-integration not supported by any of loaded plugins"
15:46:53 <ihrachys> manjeets_: I think the reason is that ml2 extension driver is not enabled
15:47:25 <manjeets_> should i enable that for adding tests for port and network dns ?
15:47:28 <ihrachys> manjeets_: in http://logs.openstack.org/47/354447/1/check/gate-neutron-dsvm-api/9c278da/logs/etc/neutron/plugins/ml2/ml2_conf.ini.txt.gz
15:47:32 <ihrachys> extension_drivers = port_security,qos
15:47:35 <ihrachys> note no dns
15:47:50 <manjeets_> i see
15:48:03 <manjeets_> if i enable dns extension driver it would be fine then
15:48:16 <ihrachys> manjeets_: yeah, otherwise your tempest is configured to run dns tests but extension is off
15:48:30 <manjeets_> ok, thanks will fix that
15:50:07 <ihrachys> electrocucaracha: not following the intent of the question
15:50:13 <ihrachys> electrocucaracha: if you mean this should land, sure
15:50:25 <ihrachys> electrocucaracha: or was that a legit question? no, I have not tried it
15:52:05 <ihrachys> ok, probably we lost Victor :)
15:52:17 <ihrachys> anyhow, I think it's all we needed to cover today. keep up the good work!
15:52:23 <ihrachys> cheers and see you next Mon
15:52:28 <ihrachys> or in the #neutron channel
15:52:31 <ihrachys> #endmeeting