14:00:33 <jlibosva> #startmeeting networking
14:00:33 <openstack> Meeting started Tue Nov 22 14:00:33 2016 UTC and is due to finish in 60 minutes.  The chair is jlibosva. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:35 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:37 <openstack> The meeting name has been set to 'networking'
14:00:40 <jlibosva> Hello everybody!
14:00:44 <akamyshnikova> hi
14:00:45 <Sam-I-Am> hello
14:00:48 <dasanind_> hi
14:00:49 <korzen> Hi
14:00:50 <HenryG> o/
14:00:52 <boden> howdy
14:00:59 <ltomasbo> hi
14:01:02 <jlibosva> #topic Announcements
14:01:03 <ihrachys> o/
14:01:05 <mlavalle> o/
14:01:09 <hichihara> hi
14:01:18 <dalvarezs> \o
14:01:20 <jlibosva> The Liberty EOL is approaching fast.
14:01:27 <electrocucaracha> o/
14:01:37 <jlibosva> stable/liberty branches will be removed some time next week
14:01:42 <ihrachys> I think liberty is CVE only and I haven't heard of any CVEs lately
14:01:56 <reedip_> hi
14:02:07 <ajo> o/
14:02:12 <jlibosva> Which means all patches in stable/liberty branch that won't get merged will be abandoned.
14:02:35 <ajo> and those at this moment can only be CVE related
14:02:40 <njohnston> o/
14:02:49 <dasm> o/
14:02:49 <jlibosva> correct
14:03:00 <jlibosva> With regarding to Stadium, there were actions taken based on the assessments
14:03:11 <jlibosva> Some projects were retired
14:03:24 <jlibosva> Specifically vpnaas, l2gw and onos: https://review.openstack.org/#/c/392010/2
14:03:50 <jlibosva> There is still an ongoing review of assessment summary: https://review.openstack.org/#/c/389397/
14:04:01 <jlibosva> Please, have a look
14:04:23 <jlibosva> And if you want to protest, now is your chance :)
14:04:42 <jlibosva> on the review ofc ;)
14:04:56 <jlibosva> Next announcement is
14:05:00 <jlibosva> We're out from Ocata-1 milestone, yay!
14:05:18 <ajo> [#    ]
14:05:29 <jlibosva> anybody has anything else to announce?
14:05:52 <ihrachys> PTG registration is open
14:06:01 <ihrachys> I hope folks will join us in Atlanta ;)
14:06:13 <jlibosva> ihrachys: thanks!
14:06:24 <jlibosva> I have a feeling I'll be thanking you often today :)
14:06:56 <jlibosva> Please register and come :)
14:07:10 <jlibosva> anything else I forgot?
14:07:40 * jlibosva enjoys the silence
14:07:54 <jlibosva> so since we're out from ocata-1
14:07:57 <jlibosva> we should now focus on
14:07:59 <jlibosva> #topic Blueprints
14:08:05 <jlibosva> #link https://launchpad.net/neutron/+milestone/ocata-2
14:08:11 <jlibosva> Is anybody blocked by anything with his work on BP?
14:08:16 <jlibosva> Any important patches that lack reviews?
14:09:05 <jlibosva> all good? any patches that have way too many reviews? :-P
14:09:15 <boden> BP = spec?
14:09:37 <jlibosva> boden: yes
14:09:44 <boden> I could use some eyes on https://review.openstack.org/#/c/308973/
14:10:07 <jlibosva> boden: so you're gonna take over that? good :) thanks
14:10:22 <electrocucaracha> well, I don't have a review but I have a critical bug to take a look
14:10:32 <ihrachys> electrocucaracha: there will be a bugs section I believe
14:10:37 <jlibosva> yep
14:10:51 <electrocucaracha> ihrachys: jlibosva ok, I'll wait
14:11:01 <jlibosva> electrocucaracha: you'll get the stage soon ;)
14:11:31 <jlibosva> so please anybody interested in healthcheck, please take a look at boden's spec
14:11:40 <jlibosva> you can also take a look, if you're not interested
14:12:02 <jlibosva> #topic OVO/no API downtime
14:12:16 <jlibosva> We have a patch on review that forbids contract migrations for Ocata
14:12:22 <jlibosva> #link https://review.openstack.org/#/c/400239/
14:12:47 <jlibosva> I recommend to take a look at the patch and remember to not use contract migrations now on
14:13:03 <ihrachys> yeah, I think armax wanted everyone to at least be aware that's happening :)
14:13:03 <jlibosva> ihrachys: anything else you want to update?
14:13:12 <jlibosva> korzen: ^^
14:13:15 <ihrachys> jlibosva: sure
14:13:40 <ihrachys> I have a spec describing online upgrades in some short way: https://review.openstack.org/386685
14:14:21 <ihrachys> as for OVO work, I am looking at the topic that should capture most patches
14:14:24 <ihrachys> #link https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/adopt-oslo-versioned-objects-for-db
14:14:51 <ihrachys> I believe there is nothing specific to point out here, most of patches are doing same thing for different models/objects.
14:15:34 <ihrachys> we need more review eyes on those, even if not +2
14:15:57 <ihrachys> there is also korzen's patch that documents objects usage in neutron scope: https://review.openstack.org/336518
14:16:12 <ihrachys> would be nice to see folks from outside the upgrades subteam to chime in
14:16:34 <korzen> yes, please review and after receiving couple of reviews I will update the docs
14:16:36 <ihrachys> your reviews would be valuable since the intent there is to make it more clear for those NOT involved in the OVO effort
14:16:51 <electrocucaracha> korzen wrote a document about the OVO in this patch https://review.openstack.org/#/c/336518/
14:17:14 <ihrachys> electrocucaracha: ...and that's the patch I linked, yes ;)
14:17:18 <ihrachys> jlibosva: I think that's it
14:17:20 <jlibosva> I encourage everyone to look at OVO patches, it's interesting and fun ;)
14:17:24 <jlibosva> ihrachys: thanks for update
14:17:25 * electrocucaracha didn't noticed about the ihrachys previous line
14:17:27 <ihrachys> jlibosva: but is it really?
14:17:33 <jlibosva> ihrachys: it IS
14:17:43 <jlibosva> ok, let's move on
14:17:49 <jlibosva> #topic Bugs and gate failures
14:17:57 <mlavalle> ihrachys, korzen, electrocucaracha: great that a doc is being put together about OVO. I'll take a look
14:18:08 <jlibosva> mlavalle: good :) thanks
14:18:14 <jlibosva> This is gonna be a rich topic
14:18:24 <jlibosva> Let's update on a gate status first
14:18:39 <jlibosva> Gate doesn't like us these days and refuses to cooperate. :(
14:18:54 <jlibosva> Currently there is a huuuge queue of jobs in gate queue
14:18:59 <jlibosva> I believe most of you have noticed
14:19:12 <jlibosva> There is a workaround fix for one of issues that is reported here: https://bugs.launchpad.net/cinder/+bug/1642111
14:19:12 <openstack> Launchpad bug 1642111 in devstack "create lvm volume from snapshot fails with "device-mapper: reload ioctl on (252:4) failed: Invalid argument"" [Critical,In progress] - Assigned to Matt Riedemann (mriedem)
14:19:18 <jlibosva> https://review.openstack.org/#/c/400465/
14:19:45 <jlibosva> It's been merged by now
14:19:53 <jlibosva> So let's hope it will improve our situation
14:20:01 <jlibosva> We also experienced some issues with rally jobs
14:20:01 <ihrachys> jlibosva: there is also rally thing?
14:20:06 <ihrachys> ok, nevermind
14:20:13 <jlibosva> ihrachys: oh, right, the rally thing ;)
14:20:22 * ihrachys hides
14:20:26 <jlibosva> Once https://review.openstack.org/#/c/400183 and https://review.openstack.org/#/c/399752 get merged, it will hopefully get even better
14:20:52 <ajo> don't recheck rally until that's merged
14:21:00 <ihrachys> I believe the last one is not strictly needed to fix gate
14:21:01 <jlibosva> good point
14:21:02 <ajo> save CO2 to earth
14:21:14 <jlibosva> and osic to patches
14:21:33 <ihrachys> it's a good patch reducing the breakage area for rally jobs but it's safe to recheck without it after other two patches are in
14:22:21 <jlibosva> as you can see, the python-heatclient one is close to be merged, all our prayers should now go to jobs running on this patch
14:22:38 <jlibosva> I also reported a bug that keeps failing on grenade multinode jobs
14:22:45 <jlibosva> #link https://bugs.launchpad.net/neutron/+bug/1643485
14:22:45 <openstack> Launchpad bug 1643485 in neutron "Stable Newton grenade multinode jobs fail with "Failed to connect to the host via ssh: Host key verification failed."" [Critical,New]
14:23:04 <jlibosva> It's likely an infra issue though, so far it lacks any movement
14:23:23 <jlibosva> If anybody has experience with infra and is willing to help, that would be really great
14:23:55 <ihrachys> I think infra folks were mentioning some image rebuild that should have fixed it?
14:23:55 <jlibosva> It blocks a bit our effort to move to Xenial on Newton branch, which is due to 6th Dec I believe
14:24:17 <ihrachys> I don't know if that rebuild happened, or that it fixed though.
14:24:34 <ajo> host verification failed?, it means the remote host has a different key in known_hostS?
14:24:37 <jlibosva> I haven't heard anything from infra folks, I'll try to reach out to them after the meeting as NA folks should be connected
14:24:37 <ajo> how can that be? :?
14:24:54 <ihrachys> ajo: well maybe ssh is broken on base ubuntu xenial images
14:25:05 <ajo> aha
14:25:12 <ihrachys> I think it affects just multinode
14:25:18 <jlibosva> yep, it seems so
14:25:36 <jlibosva> anybody observed any other gate issues?
14:25:39 <reedip_> I think maybe the ssh is not enabled on xenial by default
14:26:01 <ajo> reedip_ hmm, not probably that, otherwise it'd fail to connect at all
14:26:11 <reedip_> I installed xenial and had to install openssh-server  explicitly
14:26:17 <reedip_> ajo , ok ..
14:26:51 <jlibosva> reedip_: feel free to comment on the LP bug, I think any piece of information can become handy
14:26:52 <ihrachys> ajo: yea. also I believe in logs, some ssh connections before the failure work for the node
14:27:41 <ajo> yeah, and it works before:
14:27:42 <ajo> http://logs.openstack.org/42/399142/1/gate/gate-grenade-dsvm-multinode/a25f646/console.html#_2016-11-20_09_53_46_060807
14:28:03 <ajo> I wonder why doesn't it work on the grenad connection
14:28:27 <jlibosva> we can discuss that on the LP, I think deeper troubleshooting is required
14:28:36 <ajo> yes
14:28:52 <jlibosva> any other issues that are worth to be mentioned?
14:29:24 <jlibosva> k
14:29:26 <jlibosva> Let's move to bugs
14:29:36 <jlibosva> Last week was bug deputy electrocucaracha
14:29:40 <jlibosva> electrocucaracha: stage is yours now :)
14:30:10 <electrocucaracha> well, the only opened critical bug that I have without and assigned is this one https://bugs.launchpad.net/neutron/+bug/1643485
14:30:10 <openstack> Launchpad bug 1643485 in neutron "Stable Newton grenade multinode jobs fail with "Failed to connect to the host via ssh: Host key verification failed."" [Critical,New]
14:30:41 <jlibosva> it gains popularity, good :)
14:30:49 <electrocucaracha> I haven't be able to see any other patch having similar errors
14:32:09 <jlibosva> it looks like all newer patches to Newton branch are affected: https://review.openstack.org/#/q/project:openstack/neutron+branch:stable/newton+status:open
14:32:45 <ihrachys> yeap :(
14:32:55 <jlibosva> electrocucaracha: so beside this, no other bugs that are worth mentioning?
14:33:02 <electrocucaracha> there is another that could be potentially critical but we need more information from kolla guys https://bugs.launchpad.net/neutron/+bug/1643485
14:33:02 <openstack> Launchpad bug 1643485 in neutron "Stable Newton grenade multinode jobs fail with "Failed to connect to the host via ssh: Host key verification failed."" [Critical,New]
14:33:13 <jlibosva> electrocucaracha: that's the same one :)
14:33:22 <electrocucaracha> https://bugs.launchpad.net/kolla/+bug/1642303
14:33:22 <openstack> Launchpad bug 1642303 in neutron "neutron multiple external flat networks fails" [Undecided,Incomplete]
14:33:29 <electrocucaracha> sorry, wrong link
14:33:36 <electrocucaracha> but we still need more info
14:33:59 <jlibosva> electrocucaracha: does it affect kolla jobs only?
14:34:29 <electrocucaracha> jlibosva: it seems to
14:34:31 <jlibosva> k
14:35:04 <jlibosva> electrocucaracha: thanks for your service :) You can hand over the badge to kevinbenton who is a bug deputy for this week I believe
14:35:22 <electrocucaracha> all yours kevinbenton
14:35:24 <jlibosva> I don't see anybody for the next week
14:35:39 <jlibosva> Do we have any volunteers here?
14:36:23 <ihrachys> I can do it
14:36:39 <jlibosva> ihrachys: you're the winner! you were the first!
14:36:42 <jlibosva> Thanks :)
14:36:49 * ihrachys is so happy
14:37:08 <jlibosva> #info ihrachys is bug deputy for week starting Nov 28th
14:37:16 <jlibosva> ihrachys: are you gonna update wiki page?
14:37:30 <ihrachys> I will.
14:37:32 <jlibosva> good
14:37:52 <jlibosva> anybody wants to raise any other bug?
14:38:14 <haleyb> https://bugs.launchpad.net/neutron/+bug/161123 has seemed to resurface
14:38:14 <openstack> Launchpad bug 161123 in ubiquity (Ubuntu) "gnewsense installation crashed with exit code -11" [Undecided,Invalid]
14:38:23 <haleyb> hmm, that's not it
14:38:39 <haleyb> https://bugs.launchpad.net/neutron/+bug/1611237
14:38:39 <openstack> Launchpad bug 1611237 in neutron "Restart neutron-openvswitch-agent get ERROR "Switch connection timeout"" [High,In progress] - Assigned to IWAMOTO Toshihiro (iwamoto)
14:39:01 <haleyb> procps-ng issue we thought was fixed
14:39:04 <jlibosva> haleyb: thanks for raising it
14:39:07 <ihrachys> oh we have a fix, nice
14:39:47 <haleyb> ihrachys: we do?  i know iwamoto was looking last night when it came up
14:40:01 <ihrachys> haleyb: https://review.openstack.org/#/c/400581/
14:40:04 * haleyb looks at bug
14:40:23 <haleyb> nice!
14:41:02 <ihrachys> that said, I don't think it answers the question why 'ps' tool logs that warning
14:41:09 <ihrachys> but that's a good start
14:41:19 <haleyb> i think ps needs to catch SIGTERM as well
14:41:54 <ihrachys> I think they do catch it (the last time I checked their code they did have a huge state machine to catch different signals, even exotic)
14:42:13 <jlibosva> It seems the bug is actively being worked
14:42:16 <haleyb> the git repo doesn't have SIGTERM :(
14:42:34 <ihrachys> yeah I guess we can follow up in gerrit
14:42:36 <jlibosva> haleyb: thanks for bringing this up
14:42:43 <jlibosva> any other bugs?
14:42:49 <liuyulong> mlavalle, kevinbenton, hi guys, around? This bug: https://bugs.launchpad.net/neutron/+bug/1610045
14:42:49 <openstack> Launchpad bug 1610045 in neutron "API: floating IP updating with {} change its association" [Undecided,In progress] - Assigned to LIU Yulong (dragon889)
14:43:13 <jlibosva> liuyulong: how critical is that?
14:43:17 <mlavalle> liuyulong: I added a point in the open agenda at the end of the meeting
14:43:37 <jlibosva> ok, let's discuss that in open agenda then
14:43:39 <jlibosva> #topic Docs
14:43:43 <jlibosva> Anybody has any updates here?
14:44:02 <jlibosva> I saw on wikipage this link, it seems it's quite old though but I'll just leave it here :)
14:44:06 <jlibosva> #link  http://lists.openstack.org/pipermail/openstack-dev/2016-July/099012.html
14:44:16 <Sam-I-Am> howdy
14:44:19 <jlibosva> Sam-I-Am: hello
14:44:25 <jlibosva> Sam-I-Am: stage is yours
14:44:30 <Sam-I-Am> the routed networks patch merged
14:44:35 <Sam-I-Am> just need to backport it to newton
14:44:45 <Sam-I-Am> the vlan-aware patch is close to being ready
14:45:22 <Sam-I-Am> aside from that, not much going on
14:45:22 <jlibosva> good
14:45:26 <Sam-I-Am> keep on submitting patches :)
14:45:28 <jlibosva> Sam-I-Am: thanks for update
14:45:50 <jlibosva> I think we can move on
14:45:51 <korzen> a link to above?
14:45:57 <Sam-I-Am> one sec
14:46:07 <jlibosva> https://review.openstack.org/#/c/361776/
14:46:12 <jlibosva> trunk ^^
14:46:22 <korzen> ok thanks
14:46:30 <jlibosva> https://review.openstack.org/#/c/356013/
14:46:33 <jlibosva> routed networks ^^
14:46:41 <Sam-I-Am> heh
14:46:45 <Sam-I-Am> you had them quicker than me
14:46:52 <korzen> nice, thanks :)
14:46:52 <Sam-I-Am> 7am meeting and not caffeinated yet...
14:47:06 <jlibosva> Sam-I-Am: I'm getting nervous that we're short on time :D
14:47:10 <jlibosva> #topic Transition to OSC
14:47:22 <jlibosva> rtheis: do you want to give any updates if you're already here?
14:47:49 <ihrachys> hasn't he moved on? I think amotoki was to take over the effort?
14:48:04 <jlibosva> oh, didn't know
14:48:09 <jlibosva> Seems amotoki is not here
14:48:30 <jlibosva> I think we can jump to another topic if noone has anything to update on OSC
14:48:37 <reedip_> I think rtheis was working on some of the items
14:48:54 <ihrachys> jlibosva: +
14:48:58 <jlibosva> #topic Neutron-lib and planned neutron refactoring
14:49:11 <jlibosva> There are some patches that should possibly be highlighted, seems like most of them are already close to merge
14:49:15 <jlibosva> #link https://review.openstack.org/#/q/topic:plugin-directory
14:49:27 <jlibosva> HenryG: do you have any other updates?
14:49:51 <ihrachys> yeap folks, if you don't adopt your project to the change in time, your gates are going to break
14:50:43 <boden> More details on how we plan to roll changes out here: https://review.openstack.org/#/c/331338/   add your input soon if you have any please
14:51:09 <ihrachys> boden: I was very helpful and found a typo!
14:51:14 <boden> I can add a few neutron-lib things.. since I don’t see HenryG
14:51:36 <HenryG> sorry
14:51:38 * jlibosva thinks that HenryG just pretends he's not here, I was him sending patches couple of minutes back
14:51:45 <jlibosva> ah :)
14:51:56 <boden> HenryG: go ahead.. I was just pretending to be you
14:52:10 <HenryG> If we can agree on https://review.openstack.org/331338 that would be nice
14:53:12 <ihrachys> I am fine with the wording
14:53:22 <ihrachys> I would merge if others feel like that's good
14:53:22 <HenryG> Other than that we need reviews and more rehoming patches
14:53:47 <jlibosva> HenryG: boden thanks for updates and bringing up the patch
14:54:15 <HenryG> And spread the word to keep an eye on this link:
14:54:18 <HenryG> https://review.openstack.org/#/q/status:open+message:%22NeutronLibImpact%22
14:54:34 <jlibosva> #link https://review.openstack.org/#/q/status:open+message:%22NeutronLibImpact%22
14:54:59 <jlibosva> HenryG: thanks, do you have anything else?
14:55:09 <hichihara> https://review.openstack.org/#/q/status:open+project:openstack/neutron-lib+branch:master+topic:bp/neutron-in-tree-api-ref
14:55:18 <hichihara> api-ref haven't completed. Please update if you are Owner of their patches
14:55:41 <jlibosva> hichihara: thanks for bringing this up
14:55:52 <jlibosva> please folks have a look at those links
14:56:04 <jlibosva> we're running out of time, so
14:56:06 <jlibosva> #topic Floating IP update with empty request body
14:56:12 <mlavalle> hi
14:56:17 <liuyulong> hi
14:56:20 <jlibosva> hi
14:56:44 <mlavalle> Reviewing this patchset a couple of days ago: https://review.openstack.org/#/c/351487/
14:57:02 <liuyulong> If passed an empty `{}` data dict to the floating IP update API,
14:57:03 <liuyulong> the floating IP will be dissociated from its port.
14:57:19 <liuyulong> IMHO, that's the problem.
14:57:33 <jlibosva> #link https://review.openstack.org/#/c/351487/
14:57:59 <jlibosva> #link https://bugs.launchpad.net/neutron/+bug/1610045
14:57:59 <openstack> Launchpad bug 1610045 in neutron "API: floating IP updating with {} change its association" [Undecided,In progress] - Assigned to LIU Yulong (dragon889)
14:58:11 <ihrachys> liuyulong: strictly speaking, that's an API change
14:58:23 <liuyulong> Neutron client use `{port_id: null}` to dissociate it.
14:58:27 <liuyulong> ihrachys, yes
14:58:30 <mlavalle> there is also a patchset that was merged by kevinbenton recently to preserve th current behavior
14:58:50 <mlavalle> #link https://review.openstack.org/#/c/353804/
14:59:15 <mlavalle> So I think that before changing the API behavior, we need to reach an community agreement
14:59:28 <ihrachys> right. I think it's better to keep current behaviour. I don't see why it's critical to change it now.
14:59:37 <mlavalle> so my goal today is to bring this issue to the attention of the team
14:59:47 <hichihara> ihrachys: +1
14:59:48 <jlibosva> mlavalle: thanks
14:59:54 <jlibosva> maybe the discussion can continue on LP
14:59:57 <ihrachys> if we would have a way to evolve API (microversions?) we could consider the change
15:00:17 <ihrachys> time!
15:00:26 <jlibosva> we're out of time
15:00:32 <liuyulong> ok, move to LP
15:00:43 <jlibosva> we can also continue discussing this topic on IRC channel
15:00:48 <jlibosva> thanks everyone for stopping bye
15:00:57 * ihrachys stops damned bye
15:00:57 <jlibosva> s/bye/by
15:01:00 <jlibosva> bye
15:01:01 <electrocucaracha> thanks
15:01:03 <jlibosva> #endmeetng
15:01:06 <mlavalle> bye
15:01:06 <jlibosva> #endmeeting