14:00:18 <ihrachys> #startmeeting networking
14:00:19 <openstack> Meeting started Tue Aug 30 14:00:18 2016 UTC and is due to finish in 60 minutes.  The chair is ihrachys. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:22 <openstack> The meeting name has been set to 'networking'
14:00:25 <ihrachys> hi all
14:00:27 <njohnston> o/
14:00:27 <mlavalle> o/
14:00:30 <amotoki> o/
14:00:30 <jlibosva> \o
14:00:30 <jschwarz> \o/
14:00:31 * ihrachys letting the dust to settle
14:00:35 <hoangcx> hi
14:00:35 <haleyb> hi
14:00:37 <hichihara> hi
14:00:41 <HenryG> o/
14:00:46 * john-davidge stirs the dust
14:01:13 <ihrachys> #link https://wiki.openstack.org/wiki/Network/Meetings Agenda
14:01:15 <electrocucaracha> 0/
14:01:34 <akamyshnikova> hi
14:01:43 <ihrachys> #topic Announcements
14:01:49 <andreas_s> o/
14:01:57 <ihrachys> I have a lot of announcements for everyone
14:02:03 * ihrachys looks where to start
14:02:18 <ihrachys> first order of business, we are approaching milestone3 cut-off
14:02:25 <ihrachys> it should happen till Sep 1
14:02:30 <ihrachys> = this week
14:02:49 <ihrachys> release gods are to prepare release request in due course
14:03:13 <ihrachys> same goes for the client, same deadline
14:03:17 <ihrachys> amotoki already pushed a release request: https://review.openstack.org/#/c/362880/
14:03:18 <jschwarz> :Qa!
14:03:21 <jschwarz> oops :P
14:04:01 <ihrachys> after the milestone3 cut-off, the branch will be strictly monitored not to allow any breakage in
14:04:36 <ihrachys> the only things to be allowed after it happens are bug fixes, doc cleanups, plus things that happen to get an FFE (feature freeze exception) approved
14:04:59 <ihrachys> #link http://lists.openstack.org/pipermail/openstack-dev/2016-August/102321.html Armando email on the matter and the process
14:05:24 <ihrachys> the process to apply for FFE is by updating the post mortem document Armando spun up: https://review.openstack.org/#/c/360207/
14:06:13 <ihrachys> even if you don't have an exception to request (huge thanks for that!), you should still follow the link and update it with comments on features you led or helped with or just happen to know details about
14:06:36 <ihrachys> it will be of enormous help for PTL and other folks involved in release process if the review is updated with latest state for all tracked bits
14:07:05 <ihrachys> the next events ahead are, RC1 and stable/newton branches' creation
14:07:22 <ihrachys> that will happen some time 2 weeks+ from now
14:07:28 <ihrachys> more details at: https://releases.openstack.org/newton/schedule.html
14:07:52 <ihrachys> once stable/newton is cut-off, master is open for Ocata development and everyone can proceed merging featureful patches.
14:08:15 <ihrachys> so, we have 2 weeks of master being kinda locked for anything but release critical
14:08:31 <HenryG> Are any outstanding DB migrations still hoping to get into Newton?
14:08:31 <ihrachys> cores, make sure you ain't land crap
14:08:54 <ihrachys> HenryG: are you intending to tag the latest migrations with Newton?
14:09:11 <jschwarz> HenryG, yes
14:09:21 <ihrachys> HenryG: I think there are patches in flight that should land in Newton that happen to touch schema, I suggest we wait a week or so
14:09:21 <jschwarz> HenryG, https://review.openstack.org/#/c/357965/10
14:09:22 <HenryG> I usually do the tagging quite late.
14:09:51 <amotoki> HenryG: what repos do you plan to cover? neutron, lbaas, fwaas, vpnaas and dynamic-routing?
14:10:06 <ihrachys> good news we have a pre-release check list now: http://docs.openstack.org/developer/neutron/policies/release-checklist.html
14:10:30 <ihrachys> we will follow the list during the upcoming weeks to make sure nothing is left behind
14:10:59 <HenryG> amotoki: Good question. It may also be a good point to stop this tagging that no one uses as far as I know.
14:11:49 <amotoki> I am not sure no one uses it. do we hear it from operators?
14:11:52 <ihrachys> HenryG: The question is more about the contract we happened to define and whether we can break it.
14:12:15 <ihrachys> HenryG: a good start to stop doing it would be reaching out ops and deployment tools teams
14:12:25 <ihrachys> for N, it will need to happen anyhow
14:12:44 <HenryG> ihrachys: ack
14:13:07 <ihrachys> the last thing to note, we had a midcycle, and Armando with the help of his minions prepared a report
14:13:13 <ihrachys> #link http://lists.openstack.org/pipermail/openstack-dev/2016-August/102366.html
14:13:22 <ihrachys> everyone is encouraged to comment on the thread
14:13:49 <ihrachys> I think it captures most prominent discussions that happened during the event rather neatly, so please make use of it
14:14:18 <ihrachys> ok, announcements are done!
14:14:35 * ihrachys wipes off sweat from his head
14:14:39 <ihrachys> #topic Blueprints
14:15:13 <ihrachys> not much to cover here, just a note that feature owners and approvers should update the dashboards for targeted features
14:15:13 <ihrachys> #link https://launchpad.net/neutron/+milestone/newton-3
14:15:43 <ihrachys> also it's probably a good time to make more use of the dashboard to target and track bugs that we believe are release critical.
14:16:53 <ihrachys> reviewers, now that we cut off milestonel, please focus on bugs and FFEs in your reviews; other stuff should anyway be delayed to when the branch is open
14:17:02 <ihrachys> #topic Bugs and Gate failures
14:17:17 <ihrachys> we had armax covering the bug deputy role for the last 2 weeks
14:17:43 <ihrachys> well, actually 1.5 weeks. he told me he did not pay attention the last half of the last week :-|
14:17:43 <njohnston> Yay armax!
14:17:51 <ihrachys> so we could have some bits falling thru cracks
14:18:07 <ihrachys> we really need a volunteer to cover for this week, atm we have no one
14:18:16 <ihrachys> actually, we also need a volunteer for the next one too
14:18:26 <ihrachys> don't be shy, raise your hands!
14:18:29 <jschwarz> ihrachys, I'd like to volunteer for the next week
14:18:48 <ihrachys> jschwarz: Sep 5+ is yours!
14:18:55 <jschwarz> ihrachys, yay!
14:18:55 <ihrachys> who has capacity for this week?
14:20:13 * ihrachys taps the desktop with his fingers
14:20:43 * HenryG watches the tumbleweeds rolling by
14:20:44 <haleyb> i can do the rest of this week, but monday is a holiday around here
14:20:55 <ihrachys> it's very important to have someone covering during pre-release weeks
14:20:59 <ihrachys> haleyb: next Monday?
14:21:11 <haleyb> yes
14:21:14 <jschwarz> haleyb, Monday is mine! ^_^
14:21:23 <ihrachys> haleyb: that's ok, jschwarz starts on Sun :)
14:21:43 <haleyb> good, i figured i had until the meeting
14:21:44 <ihrachys> jschwarz: haleyb: thanks folks, you rock. please one of you, update the agenda page with the set dates.
14:21:54 <jschwarz> on it
14:22:26 <ihrachys> haleyb: if possible, you may look at what happened the last Thu-Fri since those may be without coverage.
14:22:35 <ihrachys> thanks again for all the volunteers.
14:23:13 <ihrachys> now that we approach release, we should also keep an eye on deprecation bugs:
14:23:13 <ihrachys> #link https://bugs.launchpad.net/neutron/+bugs?field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.tag=deprecation
14:23:52 <ihrachys> if something should be deprecated in N, it's the time to do it, otherwise a thing prolonges till O
14:24:24 <ihrachys> also, we should really start cleaning up all the automatically created doc bugs we have on our plate:
14:24:26 <ihrachys> #link https://bugs.launchpad.net/neutron/+bugs?field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.tag=doc
14:25:01 <ihrachys> please folks, walk thru the list and assign yourself to relevant bits, and make them happen before N final. We should have the best docs. :)
14:25:31 <ihrachys> I believe best candidates for those doc bugs are authors of patches that triggered their creation in the first place
14:25:59 <ihrachys> you may need to update openstack-manuals or neutron-lib (api-ref) or devref, depending on the kind of the bug.
14:26:18 <john-davidge> ihrachys: Is there a way to modify the script to auto-assign them it that way? Could a a useful future improvement
14:26:41 <ihrachys> john-davidge: I guess it's a matter of some patch in infra repos
14:26:53 <ihrachys> john-davidge: you are obviously welcome to start it, or discussion on openstack-dev@
14:26:59 * john-davidge will do some digging
14:27:05 <ihrachys> I suspect that it may be a lot easier to do for someone already involved in the repos
14:27:13 <ihrachys> john-davidge: thanks!
14:27:43 <ihrachys> the last bugs list to keep an eye on is gate failures:
14:27:43 <ihrachys> #link https://bugs.launchpad.net/neutron/+bugs?field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.tag=gate-failure
14:28:28 <ihrachys> looks like we don't have anyone working on first two
14:29:03 <ihrachys> would be great to have someone digging those so that we proactively keep our gate in order during next weeks.
14:29:34 <ihrachys> speaking of gate... we may have some work to do to clean up our logs before release
14:29:57 <ihrachys> we looked at current service logs from full job yesterday, and those look quite bad.
14:30:13 <ihrachys> tracebacks, deadlocks, irrelevant warnings
14:30:20 * ihrachys digs for a link
14:30:33 <ihrachys> example: http://logs.openstack.org/30/271830/10/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/52ca52d/logs/screen-q-svc.txt.gz?level=TRACE
14:30:58 <ihrachys> I'd like to remind that openstack logging guidelines require that no tracebacks are shown in logs
14:31:30 <ihrachys> for one, that deadlock/InternalError red thing, it seems relevant to provision blocks and standardattributes.
14:31:43 <ihrachys> kevinbenton: did you happen to look at those?
14:32:13 <ihrachys> I also see ipam mentioned in deadlock failures.
14:32:34 <ihrachys> so carl_baldwin may also be interested
14:32:46 <ihrachys> another case to look at is this traceback: http://logs.openstack.org/30/271830/10/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/52ca52d/logs/screen-q-svc.txt.gz?level=TRACE#_2016-08-29_11_27_28_095
14:32:56 <haleyb> ihrachys: there were a couple patches last night to deal with dhcp traces
14:33:15 <ihrachys> is it a negative test triggering the traceback? we may have some work to do to sanitize the exception
14:33:22 <ihrachys> haleyb: cool. links?
14:33:34 <haleyb> two were merged, let me find links
14:34:00 <amotoki> i think https://bugs.launchpad.net/bugs/1618216 and https://bugs.launchpad.net/bugs/1618187
14:34:00 <openstack> Launchpad bug 1618216 in neutron "dhcp agent RPC handler doesn't retry DBError" [High,Fix released] - Assigned to Kevin Benton (kevinbenton)
14:34:01 <openstack> Launchpad bug 1618187 in neutron "InvalidInput exceptions from DHCP RPC handler" [High,Fix released] - Assigned to Kevin Benton (kevinbenton)
14:34:10 <haleyb> https://review.openstack.org/362458
14:34:32 <haleyb> and https://review.openstack.org/362343
14:34:59 <ihrachys> ok, cool. I have a more recent one here: http://logs.openstack.org/74/362774/1/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/c4a8799/logs/screen-q-svc.txt.gz?level=TRACE
14:35:03 <haleyb> yes, those two, but i think the underlyng bug is still there, can't tell yet
14:35:22 * ihrachys notes that he should look at 'Failed to determine MTU for segment None:None; network 694e0047-c236-4fb4-9070-4e33d12c6c2b MTU calculation may be not accurate' thingy. haven't seen it before.
14:35:47 <ihrachys> also still seeing pymysql.err.IntegrityError in ipam code at: http://logs.openstack.org/74/362774/1/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/c4a8799/logs/screen-q-svc.txt.gz?level=TRACE#_2016-08-30_11_03_10_703
14:36:57 <ihrachys> ok, I just wanted everyone to be aware that's part of release process that logs should be clean
14:37:30 <ihrachys> there are some gate news that may be worth mentioning too
14:38:08 <ihrachys> jlibosva has a controversial patch adding scenarios job to the gate (or reusing existing one to run those tests): https://review.openstack.org/355344
14:38:23 <jlibosva> controversial? :)
14:38:41 <ihrachys> looks like people have OPINIONS! :)
14:39:00 <ihrachys> also of note, we now gate neutronclient on stable branches of neutron: https://review.openstack.org/#/c/361603/
14:39:20 <ihrachys> there are also Xenial based jobs baking for functional/fullstack: https://review.openstack.org/#/c/359843/
14:39:43 <ihrachys> the latter should give us ovs 2.5 from apt repos, not through compilation
14:40:04 <ihrachys> ok, let's move thru specific topics
14:40:07 <ihrachys> #topic Transition to OSC
14:40:22 <ihrachys> rtheis: your stage?
14:40:52 <rtheis> status update from midcycle is available via armax's email
14:41:07 <ihrachys> rtheis: where do we stand Newton release wise?
14:41:29 <ihrachys> rtheis: f.e. we have some qos patches in OSC. would they land if ready?
14:41:29 <rtheis> OSC is working on 3.x release stability so new function merges are on hold until that is complete
14:42:00 <ihrachys> I see. so we proceed with neutronclient and maybe in Ocata we land OSC bits.
14:42:12 <rtheis> the qos patches can proceed and land in the next osc release
14:42:33 <ihrachys> rtheis: thanks for the update
14:42:34 <ihrachys> #topic Keystone v3
14:42:40 <ihrachys> dasm: you're up
14:43:44 <ihrachys> anyone except dasm aware of the state of affairs? how close are we? any remaining bits to land?
14:44:07 <ihrachys> I see this WIP https://review.openstack.org/#/c/357977/
14:44:26 <ihrachys> it does not seem really ready. I wonder whether it means it slips into O
14:44:51 <ihrachys> ok, moving on
14:44:53 <ihrachys> #topic Neutron-lib
14:45:03 <ihrachys> boden: care to update? :)
14:45:32 <boden> neutron-lib 0.4.0 was recently released to pypi
14:46:01 <amotoki> and stable/newton branch has been created.
14:46:14 <HenryG> Yes, 0.4.0 will most likely be the "Newton version" of the lib
14:46:36 <HenryG> We haven't spotted any issues yet.
14:46:51 <boden> I would encourage folks to take a spin through the neutron-lib reviews if they get time
14:47:19 <boden> HenryG: anything else noteworthy?
14:47:41 <amotoki> I am looking thru api-ref cleanup patches but it is not so fast.. let me give some more time.
14:48:02 <ihrachys> amotoki: that work is not blocked by release times, right?
14:48:13 <ihrachys> it's docs only, so should be always ok to land
14:48:15 <amotoki> ihrachys: yes. it is not a release blocker
14:48:29 <ihrachys> great.
14:48:34 <ihrachys> #topic Open Floor
14:48:38 <amotoki> it is what we want to have before release though
14:48:59 <ihrachys> I am looking thru on demand agenda, nothing new pops out
14:49:16 <ihrachys> anyone has a topic interesting to the team to raise?
14:49:44 <boden> next monday is holidy in US
14:50:09 <boden> s/holidy/holiday/g
14:50:15 <njohnston> so the next of these meetings will be lightly attended
14:50:20 <dasanind_> ihrachys: there are some patches related to centralize config options with +2s do you think they will make it in N release
14:50:48 <ihrachys> dasanind_: that's a question to ask the approver. I don't personally think it's release critical, probably best effort.
14:51:01 <ihrachys> does it mean we should cancel the next meeting?
14:51:13 <ihrachys> I will leave it up to armax to decide though.
14:51:34 <ihrachys> ok, I guess we'll call it a day. stay tuned, be aware of release implications, and behave.
14:51:37 <ihrachys> #endmeeting