14:00:33 #startmeeting networking 14:00:33 Meeting started Tue Nov 22 14:00:33 2016 UTC and is due to finish in 60 minutes. The chair is jlibosva. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:35 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:37 The meeting name has been set to 'networking' 14:00:40 Hello everybody! 14:00:44 hi 14:00:45 hello 14:00:48 hi 14:00:49 Hi 14:00:50 o/ 14:00:52 howdy 14:00:59 hi 14:01:02 #topic Announcements 14:01:03 o/ 14:01:05 o/ 14:01:09 hi 14:01:18 \o 14:01:20 The Liberty EOL is approaching fast. 14:01:27 o/ 14:01:37 stable/liberty branches will be removed some time next week 14:01:42 I think liberty is CVE only and I haven't heard of any CVEs lately 14:01:56 hi 14:02:07 o/ 14:02:12 Which means all patches in stable/liberty branch that won't get merged will be abandoned. 14:02:35 and those at this moment can only be CVE related 14:02:40 o/ 14:02:49 o/ 14:02:49 correct 14:03:00 With regarding to Stadium, there were actions taken based on the assessments 14:03:11 Some projects were retired 14:03:24 Specifically vpnaas, l2gw and onos: https://review.openstack.org/#/c/392010/2 14:03:50 There is still an ongoing review of assessment summary: https://review.openstack.org/#/c/389397/ 14:04:01 Please, have a look 14:04:23 And if you want to protest, now is your chance :) 14:04:42 on the review ofc ;) 14:04:56 Next announcement is 14:05:00 We're out from Ocata-1 milestone, yay! 14:05:18 [#  ] 14:05:29 anybody has anything else to announce? 14:05:52 PTG registration is open 14:06:01 I hope folks will join us in Atlanta ;) 14:06:13 ihrachys: thanks! 14:06:24 I have a feeling I'll be thanking you often today :) 14:06:56 Please register and come :) 14:07:10 anything else I forgot? 14:07:40 * jlibosva enjoys the silence 14:07:54 so since we're out from ocata-1 14:07:57 we should now focus on 14:07:59 #topic Blueprints 14:08:05 #link https://launchpad.net/neutron/+milestone/ocata-2 14:08:11 Is anybody blocked by anything with his work on BP? 14:08:16 Any important patches that lack reviews? 14:09:05 all good? any patches that have way too many reviews? :-P 14:09:15 BP = spec? 14:09:37 boden: yes 14:09:44 I could use some eyes on https://review.openstack.org/#/c/308973/ 14:10:07 boden: so you're gonna take over that? good :) thanks 14:10:22 well, I don't have a review but I have a critical bug to take a look 14:10:32 electrocucaracha: there will be a bugs section I believe 14:10:37 yep 14:10:51 ihrachys: jlibosva ok, I'll wait 14:11:01 electrocucaracha: you'll get the stage soon ;) 14:11:31 so please anybody interested in healthcheck, please take a look at boden's spec 14:11:40 you can also take a look, if you're not interested 14:12:02 #topic OVO/no API downtime 14:12:16 We have a patch on review that forbids contract migrations for Ocata 14:12:22 #link https://review.openstack.org/#/c/400239/ 14:12:47 I recommend to take a look at the patch and remember to not use contract migrations now on 14:13:03 yeah, I think armax wanted everyone to at least be aware that's happening :) 14:13:03 ihrachys: anything else you want to update? 14:13:12 korzen: ^^ 14:13:15 jlibosva: sure 14:13:40 I have a spec describing online upgrades in some short way: https://review.openstack.org/386685 14:14:21 as for OVO work, I am looking at the topic that should capture most patches 14:14:24 #link https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/adopt-oslo-versioned-objects-for-db 14:14:51 I believe there is nothing specific to point out here, most of patches are doing same thing for different models/objects. 14:15:34 we need more review eyes on those, even if not +2 14:15:57 there is also korzen's patch that documents objects usage in neutron scope: https://review.openstack.org/336518 14:16:12 would be nice to see folks from outside the upgrades subteam to chime in 14:16:34 yes, please review and after receiving couple of reviews I will update the docs 14:16:36 your reviews would be valuable since the intent there is to make it more clear for those NOT involved in the OVO effort 14:16:51 korzen wrote a document about the OVO in this patch https://review.openstack.org/#/c/336518/ 14:17:14 electrocucaracha: ...and that's the patch I linked, yes ;) 14:17:18 jlibosva: I think that's it 14:17:20 I encourage everyone to look at OVO patches, it's interesting and fun ;) 14:17:24 ihrachys: thanks for update 14:17:25 * electrocucaracha didn't noticed about the ihrachys previous line 14:17:27 jlibosva: but is it really? 14:17:33 ihrachys: it IS 14:17:43 ok, let's move on 14:17:49 #topic Bugs and gate failures 14:17:57 ihrachys, korzen, electrocucaracha: great that a doc is being put together about OVO. I'll take a look 14:18:08 mlavalle: good :) thanks 14:18:14 This is gonna be a rich topic 14:18:24 Let's update on a gate status first 14:18:39 Gate doesn't like us these days and refuses to cooperate. :( 14:18:54 Currently there is a huuuge queue of jobs in gate queue 14:18:59 I believe most of you have noticed 14:19:12 There is a workaround fix for one of issues that is reported here: https://bugs.launchpad.net/cinder/+bug/1642111 14:19:12 Launchpad bug 1642111 in devstack "create lvm volume from snapshot fails with "device-mapper: reload ioctl on (252:4) failed: Invalid argument"" [Critical,In progress] - Assigned to Matt Riedemann (mriedem) 14:19:18 https://review.openstack.org/#/c/400465/ 14:19:45 It's been merged by now 14:19:53 So let's hope it will improve our situation 14:20:01 We also experienced some issues with rally jobs 14:20:01 jlibosva: there is also rally thing? 14:20:06 ok, nevermind 14:20:13 ihrachys: oh, right, the rally thing ;) 14:20:22 * ihrachys hides 14:20:26 Once https://review.openstack.org/#/c/400183 and https://review.openstack.org/#/c/399752 get merged, it will hopefully get even better 14:20:52 don't recheck rally until that's merged 14:21:00 I believe the last one is not strictly needed to fix gate 14:21:01 good point 14:21:02 save CO2 to earth 14:21:14 and osic to patches 14:21:33 it's a good patch reducing the breakage area for rally jobs but it's safe to recheck without it after other two patches are in 14:22:21 as you can see, the python-heatclient one is close to be merged, all our prayers should now go to jobs running on this patch 14:22:38 I also reported a bug that keeps failing on grenade multinode jobs 14:22:45 #link https://bugs.launchpad.net/neutron/+bug/1643485 14:22:45 Launchpad bug 1643485 in neutron "Stable Newton grenade multinode jobs fail with "Failed to connect to the host via ssh: Host key verification failed."" [Critical,New] 14:23:04 It's likely an infra issue though, so far it lacks any movement 14:23:23 If anybody has experience with infra and is willing to help, that would be really great 14:23:55 I think infra folks were mentioning some image rebuild that should have fixed it? 14:23:55 It blocks a bit our effort to move to Xenial on Newton branch, which is due to 6th Dec I believe 14:24:17 I don't know if that rebuild happened, or that it fixed though. 14:24:34 host verification failed?, it means the remote host has a different key in known_hostS? 14:24:37 I haven't heard anything from infra folks, I'll try to reach out to them after the meeting as NA folks should be connected 14:24:37 how can that be? :? 14:24:54 ajo: well maybe ssh is broken on base ubuntu xenial images 14:25:05 aha 14:25:12 I think it affects just multinode 14:25:18 yep, it seems so 14:25:36 anybody observed any other gate issues? 14:25:39 I think maybe the ssh is not enabled on xenial by default 14:26:01 reedip_ hmm, not probably that, otherwise it'd fail to connect at all 14:26:11 I installed xenial and had to install openssh-server explicitly 14:26:17 ajo , ok .. 14:26:51 reedip_: feel free to comment on the LP bug, I think any piece of information can become handy 14:26:52 ajo: yea. also I believe in logs, some ssh connections before the failure work for the node 14:27:41 yeah, and it works before: 14:27:42 http://logs.openstack.org/42/399142/1/gate/gate-grenade-dsvm-multinode/a25f646/console.html#_2016-11-20_09_53_46_060807 14:28:03 I wonder why doesn't it work on the grenad connection 14:28:27 we can discuss that on the LP, I think deeper troubleshooting is required 14:28:36 yes 14:28:52 any other issues that are worth to be mentioned? 14:29:24 k 14:29:26 Let's move to bugs 14:29:36 Last week was bug deputy electrocucaracha 14:29:40 electrocucaracha: stage is yours now :) 14:30:10 well, the only opened critical bug that I have without and assigned is this one https://bugs.launchpad.net/neutron/+bug/1643485 14:30:10 Launchpad bug 1643485 in neutron "Stable Newton grenade multinode jobs fail with "Failed to connect to the host via ssh: Host key verification failed."" [Critical,New] 14:30:41 it gains popularity, good :) 14:30:49 I haven't be able to see any other patch having similar errors 14:32:09 it looks like all newer patches to Newton branch are affected: https://review.openstack.org/#/q/project:openstack/neutron+branch:stable/newton+status:open 14:32:45 yeap :( 14:32:55 electrocucaracha: so beside this, no other bugs that are worth mentioning? 14:33:02 there is another that could be potentially critical but we need more information from kolla guys https://bugs.launchpad.net/neutron/+bug/1643485 14:33:02 Launchpad bug 1643485 in neutron "Stable Newton grenade multinode jobs fail with "Failed to connect to the host via ssh: Host key verification failed."" [Critical,New] 14:33:13 electrocucaracha: that's the same one :) 14:33:22 https://bugs.launchpad.net/kolla/+bug/1642303 14:33:22 Launchpad bug 1642303 in neutron "neutron multiple external flat networks fails" [Undecided,Incomplete] 14:33:29 sorry, wrong link 14:33:36 but we still need more info 14:33:59 electrocucaracha: does it affect kolla jobs only? 14:34:29 jlibosva: it seems to 14:34:31 k 14:35:04 electrocucaracha: thanks for your service :) You can hand over the badge to kevinbenton who is a bug deputy for this week I believe 14:35:22 all yours kevinbenton 14:35:24 I don't see anybody for the next week 14:35:39 Do we have any volunteers here? 14:36:23 I can do it 14:36:39 ihrachys: you're the winner! you were the first! 14:36:42 Thanks :) 14:36:49 * ihrachys is so happy 14:37:08 #info ihrachys is bug deputy for week starting Nov 28th 14:37:16 ihrachys: are you gonna update wiki page? 14:37:30 I will. 14:37:32 good 14:37:52 anybody wants to raise any other bug? 14:38:14 https://bugs.launchpad.net/neutron/+bug/161123 has seemed to resurface 14:38:14 Launchpad bug 161123 in ubiquity (Ubuntu) "gnewsense installation crashed with exit code -11" [Undecided,Invalid] 14:38:23 hmm, that's not it 14:38:39 https://bugs.launchpad.net/neutron/+bug/1611237 14:38:39 Launchpad bug 1611237 in neutron "Restart neutron-openvswitch-agent get ERROR "Switch connection timeout"" [High,In progress] - Assigned to IWAMOTO Toshihiro (iwamoto) 14:39:01 procps-ng issue we thought was fixed 14:39:04 haleyb: thanks for raising it 14:39:07 oh we have a fix, nice 14:39:47 ihrachys: we do? i know iwamoto was looking last night when it came up 14:40:01 haleyb: https://review.openstack.org/#/c/400581/ 14:40:04 * haleyb looks at bug 14:40:23 nice! 14:41:02 that said, I don't think it answers the question why 'ps' tool logs that warning 14:41:09 but that's a good start 14:41:19 i think ps needs to catch SIGTERM as well 14:41:54 I think they do catch it (the last time I checked their code they did have a huge state machine to catch different signals, even exotic) 14:42:13 It seems the bug is actively being worked 14:42:16 the git repo doesn't have SIGTERM :( 14:42:34 yeah I guess we can follow up in gerrit 14:42:36 haleyb: thanks for bringing this up 14:42:43 any other bugs? 14:42:49 mlavalle, kevinbenton, hi guys, around? This bug: https://bugs.launchpad.net/neutron/+bug/1610045 14:42:49 Launchpad bug 1610045 in neutron "API: floating IP updating with {} change its association" [Undecided,In progress] - Assigned to LIU Yulong (dragon889) 14:43:13 liuyulong: how critical is that? 14:43:17 liuyulong: I added a point in the open agenda at the end of the meeting 14:43:37 ok, let's discuss that in open agenda then 14:43:39 #topic Docs 14:43:43 Anybody has any updates here? 14:44:02 I saw on wikipage this link, it seems it's quite old though but I'll just leave it here :) 14:44:06 #link http://lists.openstack.org/pipermail/openstack-dev/2016-July/099012.html 14:44:16 howdy 14:44:19 Sam-I-Am: hello 14:44:25 Sam-I-Am: stage is yours 14:44:30 the routed networks patch merged 14:44:35 just need to backport it to newton 14:44:45 the vlan-aware patch is close to being ready 14:45:22 aside from that, not much going on 14:45:22 good 14:45:26 keep on submitting patches :) 14:45:28 Sam-I-Am: thanks for update 14:45:50 I think we can move on 14:45:51 a link to above? 14:45:57 one sec 14:46:07 https://review.openstack.org/#/c/361776/ 14:46:12 trunk ^^ 14:46:22 ok thanks 14:46:30 https://review.openstack.org/#/c/356013/ 14:46:33 routed networks ^^ 14:46:41 heh 14:46:45 you had them quicker than me 14:46:52 nice, thanks :) 14:46:52 7am meeting and not caffeinated yet... 14:47:06 Sam-I-Am: I'm getting nervous that we're short on time :D 14:47:10 #topic Transition to OSC 14:47:22 rtheis: do you want to give any updates if you're already here? 14:47:49 hasn't he moved on? I think amotoki was to take over the effort? 14:48:04 oh, didn't know 14:48:09 Seems amotoki is not here 14:48:30 I think we can jump to another topic if noone has anything to update on OSC 14:48:37 I think rtheis was working on some of the items 14:48:54 jlibosva: + 14:48:58 #topic Neutron-lib and planned neutron refactoring 14:49:11 There are some patches that should possibly be highlighted, seems like most of them are already close to merge 14:49:15 #link https://review.openstack.org/#/q/topic:plugin-directory 14:49:27 HenryG: do you have any other updates? 14:49:51 yeap folks, if you don't adopt your project to the change in time, your gates are going to break 14:50:43 More details on how we plan to roll changes out here: https://review.openstack.org/#/c/331338/ add your input soon if you have any please 14:51:09 boden: I was very helpful and found a typo! 14:51:14 I can add a few neutron-lib things.. since I don’t see HenryG 14:51:36 sorry 14:51:38 * jlibosva thinks that HenryG just pretends he's not here, I was him sending patches couple of minutes back 14:51:45 ah :) 14:51:56 HenryG: go ahead.. I was just pretending to be you 14:52:10 If we can agree on https://review.openstack.org/331338 that would be nice 14:53:12 I am fine with the wording 14:53:22 I would merge if others feel like that's good 14:53:22 Other than that we need reviews and more rehoming patches 14:53:47 HenryG: boden thanks for updates and bringing up the patch 14:54:15 And spread the word to keep an eye on this link: 14:54:18 https://review.openstack.org/#/q/status:open+message:%22NeutronLibImpact%22 14:54:34 #link https://review.openstack.org/#/q/status:open+message:%22NeutronLibImpact%22 14:54:59 HenryG: thanks, do you have anything else? 14:55:09 https://review.openstack.org/#/q/status:open+project:openstack/neutron-lib+branch:master+topic:bp/neutron-in-tree-api-ref 14:55:18 api-ref haven't completed. Please update if you are Owner of their patches 14:55:41 hichihara: thanks for bringing this up 14:55:52 please folks have a look at those links 14:56:04 we're running out of time, so 14:56:06 #topic Floating IP update with empty request body 14:56:12 hi 14:56:17 hi 14:56:20 hi 14:56:44 Reviewing this patchset a couple of days ago: https://review.openstack.org/#/c/351487/ 14:57:02 If passed an empty `{}` data dict to the floating IP update API, 14:57:03 the floating IP will be dissociated from its port. 14:57:19 IMHO, that's the problem. 14:57:33 #link https://review.openstack.org/#/c/351487/ 14:57:59 #link https://bugs.launchpad.net/neutron/+bug/1610045 14:57:59 Launchpad bug 1610045 in neutron "API: floating IP updating with {} change its association" [Undecided,In progress] - Assigned to LIU Yulong (dragon889) 14:58:11 liuyulong: strictly speaking, that's an API change 14:58:23 Neutron client use `{port_id: null}` to dissociate it. 14:58:27 ihrachys, yes 14:58:30 there is also a patchset that was merged by kevinbenton recently to preserve th current behavior 14:58:50 #link https://review.openstack.org/#/c/353804/ 14:59:15 So I think that before changing the API behavior, we need to reach an community agreement 14:59:28 right. I think it's better to keep current behaviour. I don't see why it's critical to change it now. 14:59:37 so my goal today is to bring this issue to the attention of the team 14:59:47 ihrachys: +1 14:59:48 mlavalle: thanks 14:59:54 maybe the discussion can continue on LP 14:59:57 if we would have a way to evolve API (microversions?) we could consider the change 15:00:17 time! 15:00:26 we're out of time 15:00:32 ok, move to LP 15:00:43 we can also continue discussing this topic on IRC channel 15:00:48 thanks everyone for stopping bye 15:00:57 * ihrachys stops damned bye 15:00:57 s/bye/by 15:01:00 bye 15:01:01 thanks 15:01:03 #endmeetng 15:01:06 bye 15:01:06 #endmeeting