21:01:09 <armax> #startmeeting networking
21:01:09 <openstack> Meeting started Mon Jan  9 21:01:09 2017 UTC and is due to finish in 60 minutes.  The chair is armax. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:12 <openstack> The meeting name has been set to 'networking'
21:01:15 <njohnston> o/
21:01:17 <mlavalle> o/
21:01:23 <armax> hello folks, welcome back after the looong break
21:01:30 <bzhao_> o/
21:01:36 <armax> I hope you’re well recharged
21:01:44 <armax> and ready to roll
21:01:55 <armax> also, happy new year!
21:02:00 <manjeets> o/
21:02:01 <blogan> likewise!
21:02:03 <mlavalle> Likewise!
21:02:52 <armax> the agenda for the meeting
21:02:58 <armax> #link https://wiki.openstack.org/wiki/Network/Meetings
21:03:18 <armax> even though there’s no announcement as stated
21:03:21 <armax> #topic Announcements
21:03:50 <armax> I am gonna remind folks that this cycle FF comes closer
21:03:52 <armax> than usual
21:04:12 <armax> we only have a few weeks left to the end of ocata
21:04:20 <kevinbenton> o/
21:04:37 <lindycoder> o/
21:04:50 <armax> any announcement/reminder from any of you?
21:05:02 <dasm> just dates. next week is neutron-lib release
21:05:09 <dasm> and week after, ocata-3
21:05:29 <armax> dasm: yes, in fact we gotta nail the week for requirement freeze
21:05:30 <dasm> so, currently we should wrap up all outstanding changes for these
21:05:41 <mlavalle> Also remind folks about the PTG
21:05:57 <mlavalle> I checked this morning and there are about 150 places left open
21:06:11 <boden> is there an outline of topics for the PTG, or we just going to play it by ear?
21:06:35 <armax> mlavalle: good point
21:06:41 <dasm> boden: first comes first served? who has topic, can discuss it? :)
21:06:48 <mlavalle> #link https://www.openstack.org/ptg/
21:06:58 <armax> boden: I am still trying to figure out what the format is going to be like
21:07:13 <dasm> and, i believe. other announcement.
21:07:20 <dasm> armax: thanks for your service as PTL
21:07:26 <armax> I’ll communicate something between this week and the next
21:07:38 * armax blushes
21:08:42 <armax> dasm: let’s talk about neutron-lib in its section
21:08:53 <armax> any other reminder/announcement?
21:09:36 <armax> looks like we’re good
21:09:39 <armax> #topic blueprints
21:09:51 <armax> as ocata-3
21:09:58 <armax> we got the following outstanding plan:
21:09:59 <armax> #link https://launchpad.net/neutron/+milestone/ocata-3
21:10:12 <armax> as usual we’re cronically late
21:10:48 <armax> we should probably do some pruning before FF
21:11:19 <armax> of the high priority ones
21:11:47 <armax> neutron-lib is ongoing, so it’ll roll over
21:12:24 <armax> though boden and I optmistically and probably irrealistically wanted to have one subproject entirely off neutron as dependency I feel we did churn quite a bit of code so far
21:12:55 <armax> we made some progress, but we have one week to cut another ocata release, so we probably don’t have much time left
21:13:03 <boden> armax: there’s a good bit in review right now as well
21:13:28 <armax> boden: we should find some 1-1 time this week to go over the list and prioritize
21:13:48 <boden> armax: I’ll be around all week
21:14:02 <armax> boden: could you please hit me via email with some times where we can catch up?
21:14:25 <armax> I don’t see ihrachys
21:14:47 <armax> kevinbenton: how’s push notification going?
21:15:10 <kevinbenton> armax: need some reviews from ihrachys
21:15:23 <armax> kevinbenton: anything the team at large can help you with?
21:15:40 <kevinbenton> armax: if there is an OVO person
21:16:01 <kevinbenton> armax: there is a warning that someone should look into on every get_object call
21:16:31 <kevinbenton> http://logs.openstack.org/39/388939/16/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/26aa4ca/logs/screen-q-svc.txt.gz?level=TRACE
21:16:40 <kevinbenton> Unique keys not in sort_keys. The sorting order may be unstable.
21:16:59 <armax> kevinbenton: is that generated by patch 388939
21:17:00 <armax> ?
21:17:27 <kevinbenton> armax: this patch results in it appearing a whole bunch
21:18:00 <armax> kevinbenton: so you’re saying it shows in the gate but not as frequently as 388939?
21:18:21 <kevinbenton> do we have qos in the gate?
21:18:33 <armax> kevinbenton: not on the full job
21:18:41 <armax> but the API one
21:18:43 <armax> we do
21:19:37 <kevinbenton> i don't see the errors there
21:19:44 <armax> ok
21:19:53 <armax> mlavalle: what about routed networks?
21:19:58 <kevinbenton> but this is the first code with neutron server running loading ports, networks, security groups, and sg rules
21:20:11 <armax> kevinbenton: aye
21:20:23 <mlavalle> armax: over the past few weeks I was able to make great progress
21:20:34 <mlavalle> completed the funtionality and the unit tests
21:20:45 <mlavalle> tested the code extensively in my deployment
21:21:03 <mlavalle> and founf a bug on the Nova side: https://bugs.launchpad.net/nova/+bug/1652642
21:21:03 <openstack> Launchpad bug 1652642 in OpenStack Compute (nova) "Nova API doesn't return aggregate's uuid, which is needed when using the placement API" [High,Fix released] - Assigned to Matt Riedemann (mriedem)
21:21:26 <mlavalle> mriedem and jaypipes were super responsive
21:21:26 <armax> nice
21:21:37 <mlavalle> and the fix merged Friday night
21:21:59 <mlavalle> so I tested it over the weekend and it works fine
21:22:19 <armax> that looks like quite a nice catch
21:22:34 <mlavalle> yeah, they were surprised about it
21:23:03 <mlavalle> later today I will push the updated complete code
21:23:27 <mlavalle> It's two patchsets. the first one is the placement API client:
21:23:49 <mlavalle> #link https://review.openstack.org/#/c/414726/
21:24:13 <mlavalle> and the second one is the piece that uses that client to talk to the placement and nova APIs:
21:24:35 <mlavalle> #link https://review.openstack.org/#/c/358658
21:24:40 <armax> mlavalle: ack
21:24:53 <mlavalle> as I said, code is complete and tested. All I need after tonight is reviews
21:25:11 <mlavalle> that's my update :-)
21:25:20 <armax> mlavalle: thanks
21:25:36 <mlavalle> armax: i'll update the blueprint tomorrow
21:26:25 <armax> anything else on blueprints from someone in the team?
21:27:35 <armax> mlavalle: is there any integration testing done with nova?
21:28:00 <armax> mlavalle: we can take this offline though
21:28:09 <mlavalle> armax: I've been doing that manually in my deployment. I intend to follow up with tempest tests after this code merges
21:28:31 <armax> mlavalle: ok
21:28:41 <armax> moving on
21:29:08 <armax> as I don’t see ihar in the channel let’s skip an update on ovo/no api downtime effort, unless someone else can?
21:29:47 <armax> ok
21:29:48 <armax> moving on
21:29:53 <armax> #topic bugs
21:30:16 <armax> I was the last known bug deputy
21:30:38 <armax> I think we’re been unstaffed the last week or so, but things have been relatively quiet
21:30:47 <armax> no major fire to put out
21:31:25 <armax> grafana dashboards don’t like they are screaming
21:31:36 <armax> I did notice something in channel earlier today
21:31:45 <armax> about nova being broken
21:32:13 <armax> https://bugs.launchpad.net/python-novaclient/+bug/1654806
21:32:14 <openstack> Launchpad bug 1654806 in devstack "Functional tests (with neutron) failing with "ERROR (Conflict): Multiple possible networks found, use a Network ID to be more specific. (HTTP 409)" since 1/6" [High,In progress] - Assigned to Matt Riedemann (mriedem)
21:32:18 <kevinbenton> nova client functional job i think
21:32:48 <armax> we should look into it
21:32:53 <kevinbenton> based on what i've seen it's not a neutron problem
21:33:12 <kevinbenton> well at least not neutron server
21:33:38 <kevinbenton> they are using the admin credentials for regular tenant operations
21:33:53 <kevinbenton> so when it lists networks, it also sees the external network that belongs to the admin and gets confused
21:33:58 <armax> anyone interested, except kevinbenton?
21:34:18 <kevinbenton> armax: sc68cal was also looking into it
21:34:27 <armax> sc68cal: any update?
21:35:13 <armax> I’ll see if I can spot anything
21:35:38 <armax> we need a deputy for the week, any volunteer?
21:35:58 <armax> someone new, come on!
21:36:06 <armax> new year’s resolution
21:36:08 <armax> you know you want it!
21:36:29 <armax> burn some calories with the extra typing on the keyboard
21:37:15 <kevinbenton> armax: you quit being PTL, you should have extra time :P
21:37:29 <boden> I’m more familiar with neutron-lib, but I can give it a shot if there’s no one else
21:37:32 <armax> kevinbenton: I am still the PTL until there’s a new one
21:37:50 <armax> kevinbenton: one we have a new one, I can be the deputy as much as you like
21:37:58 <armax> boden it is!
21:38:16 <armax> #action boden deputy for the week of Jan 9
21:38:21 <mlavalle> boden: Thanks!
21:38:25 <armax> boden: all you have to do is dispatch appropriately
21:38:40 <boden> armax: ack.. I’ll read up on the process
21:38:47 <armax> http://docs.openstack.org/developer/neutron/policies/bugs.html#neutron-bug-deputy
21:39:17 * haleyb wanders in late
21:39:33 <armax> haleyb: hello
21:39:46 <armax> ok, moving on
21:40:02 <armax> I don’t see john-davidge around
21:40:11 <armax> let’s go straight to OSC
21:40:20 <armax> #topic Transition to OSC
21:40:26 <armax> abhiraut: hi
21:40:31 <abhiraut> cleaned up wiki here
21:40:32 <abhiraut> https://etherpad.openstack.org/p/osc-transition-priorities
21:40:39 <abhiraut> thanks to ankur on that one
21:41:07 <armax> looking
21:41:59 <abhiraut> barring vpnaas and lbaasv2 i see patches for almost all commands
21:42:00 <armax> ok, once we squashed this list, I guess we’re at virtually parity
21:42:06 <abhiraut> yeah
21:42:13 <armax> nice
21:42:33 <abhiraut> for sdk only a couple of patches remain
21:42:52 <armax> abhiraut: added the etherpad link to the team meeting wiki page
21:42:59 <abhiraut> ill work on them this week.. so hoepfully can get people to look at them
21:43:07 <abhiraut> thanks
21:43:23 <armax> thank you
21:43:30 <johnsom> The security groups breakage has been causing problems for a bunch of teams....
21:43:40 <dasm> fyi. in osc+networking there are multiple problems: https://bugs.launchpad.net/manila/+bug/1652317/
21:43:40 <openstack> Launchpad bug 1652317 in Manila "OpenStackSDK refactoring caused various OSC networking commands to fail" [Critical,New]
21:43:45 <ankur-gupta-f> indeed, reviews may be slow coming in.
21:43:46 <armax> johnsom: which one?
21:43:48 <johnsom> ^^^ Yeah, that
21:44:02 <johnsom> It has all of our gates blocked at the moment
21:44:03 <armax> ouchy
21:44:23 <abhiraut> yeah security group is one of the 2 patches which remain to be completed
21:44:30 <abhiraut> as you can see here
21:44:31 <abhiraut> https://etherpad.openstack.org/p/osc-network-command-sdk-support#L71
21:44:43 <abhiraut> ill get on to that
21:45:15 <ankur-gupta-f> *aaS service patchsets I have seen require some more attention to be meet OSC standards. Could use more eyes on them
21:45:18 <johnsom> The tests got "skipped" so it made it in with broken securitygroup create
21:45:25 <johnsom> This is going in: https://review.openstack.org/#/c/417879/
21:46:04 <armax> johnsom: is anyone working on the fixes that make the networking commands compliant with the latest sDK?
21:46:19 <johnsom> One of our folks did this: https://review.openstack.org/#/c/417813/
21:46:44 <johnsom> It fixes our issue, but of course requires a python-openstackclient release before the gates pick it up.
21:46:47 <armax> though that needs a release
21:47:07 <johnsom> Right.  I think that is why the revert is going into global
21:47:30 <armax> the revert is not solving all the issues by the looks of it though
21:47:31 <armax> ok
21:47:45 <abhiraut> thanks for bringing this into attention.
21:48:07 <armax> how did this not get caught in the gate?
21:48:30 <johnsom> They put in skips on all of the tests
21:48:34 <armax> we did switch to using the opentsack client in the gate
21:48:38 <armax> johnsom: where?
21:49:02 <johnsom> armax https://review.openstack.org/#/c/417813/6/openstackclient/tests/functional/network/v2/test_security_group.py
21:49:05 <johnsom> As an example
21:49:45 <armax> oh my
21:50:20 <armax> I am afraid to ask why
21:50:21 <dtroyer> armax: FWIW the root cause of these recent OSC/SDK network command issues is the OSC functional tests were broken for a time and we did not catch either a) real failures or b) SDK incompatibilities
21:50:47 <armax> dtroyer: I see
21:51:02 <dtroyer> the functioanl tests were fixed around dec 23, we skipped the breakages to give us time to catch up without holding up the world
21:51:21 <armax> dtroyer: reason why I asked is that I did this https://review.openstack.org/#/c/391188/ not too late ago
21:51:54 <armax> and if the functional jobs do exercise devstack, like I have seen with changes to the openstackclient patches
21:52:00 <armax> we would catch geniune failures
21:52:48 <dtroyer> IIUC, we were not catching them early enough, ie before an OSC release
21:53:28 <dtroyer> as soon as we can remove the recently added functioanl skips we'll do another OSC release
21:54:02 <armax> dtroyer: is there anything that the neutron team can do to help this process?
21:55:17 * armax looks a the clock and realized the meeting is about to be over
21:55:19 <kevinbenton> oh, i forgot i have a brief announcement
21:55:20 <dtroyer> just work through the SDK compatibility issues with us…
21:55:28 <kevinbenton> (if we have a sec)
21:55:29 <dtroyer> my apologies for the mess
21:55:30 <armax> kevinbenton: shoot
21:55:50 <armax> dtroyer: please feel free to tag me on any change you want us to review
21:55:53 <armax> or abhiraut
21:56:05 <kevinbenton> rally team has been getting several neutron scenarios submitted to their repo and they were hoping for people to take a look at them
21:56:07 <abhiraut> dtroyer : hit me up too
21:56:09 <kevinbenton> to make sure they are sane
21:56:10 <armax> if there’s anything specific you want us to own, please let us know
21:56:12 <kevinbenton> from the neutron team
21:56:26 <armax> kevinbenton: ack
21:56:30 <armax> kevinbenton: tag me?
21:56:46 <armax> kevinbenton and you and I can have a look at them?
21:56:55 <armax> ok
21:56:58 <kevinbenton> andreykurilin: do you have a rally neutron dashboard for us to pass around?
21:57:05 <kevinbenton> andreykurilin: for revies
21:57:07 <kevinbenton> reviews*
21:57:13 <armax> moving on to the next topic
21:57:23 <armax> #topic Neutron-lib, planned refactoring and other impacts
21:57:43 <armax> boden: you want to mention briefly the discussion we had today?
21:58:15 <armax> on my end I wanted to make sure ihar’s change https://review.openstack.org/#/q/topic:advertise-mtu-removal would get some limelight during this meeting
21:58:54 <andreykurilin> kevinbenton: I didn't create separate board yet(with quick link), but I have a "search term" - http://paste.openstack.org/show/594332/
21:59:05 <boden> armax: I’m not sure it’s that brief, maybe we should wait till next week
21:59:11 <armax> boden: ok
21:59:28 <armax> so with one minute left, I suppose we’ll have to wrap it up here
21:59:45 <armax> goodbye everybody!
21:59:57 <armax> thanks for joining
21:59:58 <mlavalle> Thanks!
21:59:59 <bzhao_> by
22:00:02 <armax> @#endmeeting
22:00:07 <dasanind> Thanks!!
22:00:09 <manjeets> thanks
22:00:17 <bzhao_> :)
22:00:17 <princenana> thanks
22:00:32 <njohnston> o/
22:00:34 <dasm> \o
22:00:36 <njohnston> #endmeeting
22:00:46 <njohnston> you had an at-sign preceding it
22:00:49 <njohnston> armax
22:01:18 <johnsom> armax has to do it
22:01:28 <dasm> #endmeeting