14:00:55 #startmeeting networking 14:00:56 Meeting started Tue Feb 24 14:00:55 2015 UTC and is due to finish in 60 minutes. The chair is mestery. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:59 The meeting name has been set to 'networking' 14:01:09 #link https://wiki.openstack.org/wiki/Network/Meetings Agenda 14:01:15 #topic Announcements 14:01:27 We continue our march towards the Kilo release. 14:01:32 Dates of importance: 14:01:33 #info Feature Proposal Freeze: March 5 14:01:41 #info Feature, String, and Dependency Freeze: March 19 14:01:45 #info RCs: April 9-23 14:01:50 #info Kilo release: April 30 14:02:01 o/ 14:02:01 #link https://wiki.openstack.org/wiki/Kilo_Release_Schedule 14:02:02 o/ 14:02:16 Any questions on the upcoming Kilo dates? 14:02:58 I encourage people to continue focusing on reviewing code targeted for Kilo-3 14:03:11 #link https://launchpad.net/neutron/+milestone/kilo-3 14:03:16 aloha folks 14:03:23 Any other announcements from anyone? 14:03:26 o/ 14:03:31 O/ 14:03:34 o/ 14:03:42 OK, lets move on now 14:03:47 #topic Bugs 14:03:50 enikanorov__: Hi! 14:03:54 mestery: hi 14:04:04 not much news on bugs side 14:04:05 https://bugs.launchpad.net/neutron/+bug/1421232 14:04:07 Launchpad bug 1421232 in neutron "Restarting neutron openvswitch while having broadcast/multicast traffic going into br-tun makes a broadcast storm over the tunnel network" [Critical,Fix committed] - Assigned to Miguel Angel Ajo (mangelajo) 14:04:16 this one was critical and now committed 14:04:27 https://bugs.launchpad.net/neutron/+bug/1323658 14:04:28 Launchpad bug 1323658 in OpenStack Compute (nova) "Nova resize/restart results in guest ending up in inconsistent state with Neutron" [Critical,Confirmed] 14:04:42 and this still requires someone to dig in 14:05:08 enikanorov__: Is that a new one? Looking now 14:05:34 not really, it has been hanging in critical for a few weeks now 14:05:44 mestery: suspect it's still another version of "the something went wrong with networking" bug 14:05:47 enikanorov__: Yeah, I moved it out of Kilo-1 and no one has signed up for it yet 14:05:52 salv-orlando: It is indeed 14:06:08 salv-orlando: Your last comments on it indicated it was a tempest issue not a neutron issue 14:06:11 wait 14:06:12 salv-orlando: that's what most of us get salary for 14:06:14 that was a different bug 14:06:53 enikanorov__: are you paid for making neutron now work? 14:06:56 jogo asked me to help get someone assigned, I hadn't been able to help with that yet 14:07:02 Or as long as neutron doesn't work, we have job? 14:07:07 so thanks for bringing it up enikanorov__ 14:07:13 salv-orlando: somewhat in between 14:07:14 anteaya: I'll see if I can look into it a bit after the meeting 14:07:21 mestery: thank you 14:07:36 ok, the last one i'd like to mention is 14:07:44 mestery, anteaya: I get the feeling this is going to require pairing between Nova and Neutron devs 14:07:46 I want to mention another one when you finish ;) 14:07:50 hi ;D 14:07:56 markmcclain: Highly likely ;) 14:07:57 https://bugs.launchpad.net/neutron/+bug/1359523 14:07:58 Launchpad bug 1359523 in neutron "Security group rules errorneously applied to all ports having same ip addresses in different networks" [High,In progress] - Assigned to shihanzhang (shihanzhang) 14:08:03 markmcclain: fair enough 14:08:09 this has a patch on review but apparently the author has abandoned it 14:08:18 i think this is really important to be fixed in K 14:08:21 hmmm 14:08:24 markmcclain: if we can get a neutron dev, we can ask jogo to help or help find a nova dev 14:08:35 enikanorov__: I can review that as I'm familiar with the SG code 14:08:43 ajo: Thank you! 14:09:00 ajo: i think we'll need someone to drive code through, not just review it... 14:09:14 enikanorov__, I can talk to hanzhang and drive it myself if he can't 14:09:24 ajo: wold be great, thanks 14:09:29 :-), sure, np 14:09:31 https://bugs.launchpad.net/neutron/+bug/1410982 14:09:32 Launchpad bug 1410982 in neutron "test_slaac_from_os frequent failures" [High,Confirmed] 14:09:39 This bug is a bit of a mess, 14:09:55 another bug that’s hurting is this bug #1424482 14:09:56 bug 1424482 in tempest "AttributeError: 'module' object has no attribute 'MismatchError'" [Undecided,In progress] https://launchpad.net/bugs/1424482 - Assigned to Armando Migliaccio (armando-migliaccio) 14:10:15 mtreinish: ^^^ 14:10:16 it was exacerbated by a patch I submitted, and the original problem I'm unsure if it's still happening 14:10:28 well but 124482 is assigned so I'm not worried ;) 14:10:46 :-) 14:11:23 any question on bugs? 14:11:24 for 1410982, I couldn't find current test_slaac_from_os failures not associated to other massive failures 14:11:27 ajo: Looks like 1410982 has a healthy discussion in there between you, carl_baldwin, armax, and others 14:11:35 so I'm unsure if the bug is valid anymore 14:11:40 but... 14:11:57 what I couldn't find it, doesn't mean it doesnt exist... may be I'm not proficient enough with kibana (that for sure) 14:12:15 ajo: I’ll look into it, I haven’t had the chance lately though 14:12:35 armax, thanks, if you could do it, I tried 2 times lately without finding the issue anymore, 14:12:37 btw, 14:13:02 I find the netlink interface errors sometimes in well passed tests 14:13:14 ajo: that's normal 14:13:34 it might because it is an API test which spins off some dhcp action as a side effect but does not validate it 14:13:50 salv-orlando, yes, I found reasonable explanations for that, 14:14:44 OK, thanks for the updates on bugs enikanorov__ and others! 14:14:50 Any other bugs the team shoudl discuss today? 14:15:13 If not, we'll move on to ... 14:15:16 #topic Docs 14:15:22 emagana: Hi! 14:15:30 mestery: Hello! 14:16:04 The first commit for the nova-net to neutron has been committed 14:16:08 emagana: Matt Kassawara put an item in "On Deman" which I think we shoudl discuss here, so after your update, lets discuss that one. 14:16:10 emagana: Cool! 14:16:29 I will let Sam-I-Am to comment 14:16:31 im here, but a little slow 14:16:46 Sam-I-Am: :) 14:17:04 #link https://review.openstack.org/#/c/155947/ 14:17:18 Sam-I-Am: Do you want to share about the networking guide 14:17:30 emagana: sure 14:18:08 its moving along. i'm trying to fiish the L3HA+OVS scenario... writing it in rst from the start 14:18:20 the docs crew has been trying to get rst working for the networking guide 14:18:43 Sam-I-Am: Sounds cool - Add me as a reviewer. Also, you can use information from: https://review.openstack.org/#/c/150918/ 14:18:55 once the build system works, we'll try to get the scenarios uploaded into the official repo 14:19:15 i am looking for l3ha SMEs :) 14:19:25 Sam-I-Am: Start with amuller :) 14:19:34 thats what i just found out :) 14:19:44 we're writing instructions for both linuxbridge and ovs 14:20:14 The rest of the .rst links are on the wiki if anyone is interested in the current work! 14:20:24 emagana: Awesome, thanks! 14:20:27 i think thats it for the networking guide. i'll probably try to get more feedback at mid-cycle in a few weeks. 14:21:50 mestery: want me to go on about the other thing? 14:21:51 OK, thanks for the docs updates folks! 14:21:53 Thanks Sam-I-Am! 14:21:56 Sam-I-Am: Yes, lets move there. 14:22:28 i added an on-demand topic about confusion around the location of ovs/linuxbridge agent configuration after ML2 deprecated the plugins 14:23:17 the install guide moved the agent config into ml2_conf.ini 14:23:50 oh 14:23:54 I think it's wrong 14:24:06 however, i see periodic bugs filed by mostly distro packagers who insist on ising ovs_neutron_plugin.ini and linuxbridge_conf.ini still 14:24:08 there is still ovs_neutron_plugin.ini file 14:24:19 yes, there is, but it is not for a plugin anymore 14:24:20 though the name is bad, it's still reused by the agent 14:24:27 this causes lots o confusion with newbies 14:24:33 we can't just rename the file for backwards compat reasons 14:24:44 yes, we had bugs reported in RDO for that 14:24:53 but we stand our ground and use the file for the agent 14:24:53 ihrachyshka: why can't we rename the file for new installations? 14:24:57 why should the agent config be in the plugin ini file? 14:24:59 or sym link? 14:25:02 since that's where we maintain those conf values in the tree 14:25:27 markmcclain, we can. though it opens doors to upgrade issues. we would need to read from both files 14:25:41 ideally, agents would read from conf-dir, not conf-file 14:25:53 the linuxbridge_conf.ini file is a little less misleading, but if we do continue to use separate files, they should have better names (suggestions on the wiki) 14:25:56 and we would have more freedom with the file names 14:26:07 ihrachyshka: yes.. conf dir would be a bit cleaner except that we have overlapping options in some agents 14:26:12 also, it would help in the world of *aas repos split from main repo 14:26:25 ihrachyshka: that said I'm all for conf dir :) 14:26:28 f.e. before kilo, we read fwaas conf by l3 agent 14:26:50 but now that we have no guarantees to have fwaas conf installed, we can't 14:26:59 conf-dir would solve that 14:27:28 Sam-I-Am, do you see how to handle upgrade for existing users without grenade hook? 14:27:47 we could obviously rename the file, putting burden on deployers/packagers 14:28:22 me being lazy, I vote for leaving as-is, but if people see it a huge problem, ok, let's rename in start of the next cycle 14:28:25 not now 14:29:03 ihrachyshka: iirc, neutron just concatenates a bunch of config files when it loads 14:29:11 so short story, admin guide is currently wrong 14:29:19 you can include anything in the init scripts 14:29:27 Sam-I-Am, you need to pass all conf files as --conf-file arguments 14:29:32 yes 14:29:44 ihrachyshka: what's the difference between we change them now and keep them now in this cycle? 14:29:45 Sam-I-Am, right, but I'm not sure it plays nice with non-existent files 14:30:10 touch the new file and tell people to migrate their config to it? 14:30:22 amotoki, packagers are not forced to apply yet another change to packaging this cycle which is already beyond normal in terms of disruptions 14:30:38 ihrachyshka: Fair point 14:30:42 i'm just trying to fix a repeat confusion problem 14:30:48 Sam-I-Am, yeah, everything is possible, but reading ml2 conf is just wrong in any case 14:30:49 ihrachyshka: fair enough. 14:31:10 maybe create another config file and let people decide when they move to it? 14:31:46 Sam-I-Am: I think that woudl lead to more confusion 14:31:50 + 14:31:51 in my understanding all of existing conf files are examples and all have default values, 14:32:04 i'm just bringing this up as a pain point 14:32:11 it should have been fixed a long time ago 14:32:56 iirc, ubuntu packages dont include an ovs_neutron_plugin.ini file unless you installl the defunct plugin 14:33:03 Sam-I-Am: It's a fair point, lets discuss this offline and see what we can do perhaps. 14:33:05 so i have to create it anyway 14:33:08 even though most distributions use them as an initial conf files, so I think moving them in upstream tree only affects packagers. 14:33:11 amotoki, not completely. afaik neutron.conf has state_path defined 14:33:28 Sam-I-Am, so that's something to fix in ubuntu packaging 14:33:29 i have to maintain an install guide for all distros :/ 14:33:45 ihrachyshka: right. i know we have some exceptions and hopefully they should be clean-up. 14:33:58 mestery: sounds like a good plan 14:34:20 OK, thanks for the discussion here Sam-I-Am! 14:34:28 Lets move on now. 14:34:38 #topic Plugin Decomposition Status Update 14:35:09 ihrachyshka has offered to provide an update from a packaging perspective here on the changes plugin decomposition will make for packagers. 14:35:12 ihrachyshka: Ready? 14:35:17 right 14:35:22 ihrachyshka: thanks! 14:35:43 so I'm packaging neutron for RDO/RHOSP. status on our side is that no vendor libraries are actually packaged. :) 14:35:57 lol 14:36:01 and that's partly because of no pypi releases 14:36:19 there is no clear message from upstream on what is considered to be ready 14:36:36 also no clear list that tracks the progress (there is google doc from armax, but that's all) 14:37:05 I saw more patches to decomp plugins coming in 14:37:12 ihrachyshka: what do you consider something to be ready? pypy release available? 14:37:16 ihrachyshka: I can give you perspective from the decomposed ODL driver. 14:37:22 it would be great to have some kind of a list of what's actually ready 14:37:26 ihrachyshka: We're working on a few last bugs before releasing to pypi the first version. 14:37:38 Unsure if others are also fixing bugs before releasing 14:37:43 ihrachyshka: Arista is ready 14:37:56 armax, right, I would consider pypi release as an indication of library being ready 14:38:00 Sukhdev: You've released to pypi? 14:38:09 mestery: yes 14:38:24 Sukhdev: Nice work! 14:38:32 What does "release to pypi" entail? We have 0.0.1 there. 14:38:44 cool. that would be great to have some periodic update to downstreams on the progress. 14:39:05 ihrachyshka: ok, so it sounds to me that your readiness checklist would be to determine whether for each tracked effort we have a pypi package supplied 14:39:06 #link https://pypi.python.org/pypi/networking_arista/2015.1.1.dev12 14:39:07 HenryG, well, at least something to package. it's better if it actually works. 14:39:15 armax has a list we can use that 14:39:22 is having dev in the release name compliant with semver? 14:39:38 HenryG: https://wiki.openstack.org/wiki/GerritJenkinsGit#Tagging_a_Release 14:39:49 #link https://wiki.openstack.org/wiki/GerritJenkinsGit#Tagging_a_Release 14:39:54 it would seem distros will need some mapping from neutron release (including stable branches) to specific library versions, right? 14:40:50 rkukura, I think there is one, per plugin requirements.txt 14:40:53 anteaya: this is the correct one - https://pypi.python.org/pypi/networking_arista/2015.1.1 14:40:53 rkukura: i think requirements.txt in plugins/drviers dir should declare it. 14:40:58 rkukura, they should point to proper versions 14:40:59 rkukura: the requirements files in the shim directories should have that. 14:41:06 super jinx 14:41:10 :) 14:41:15 lol 14:41:44 but one question is when we should update requirements in shim plugin dir. version cap? 14:41:57 amotoki, when we absolutely need it? 14:42:26 there is one question about vendor split. what do we do with upgrade for existing juno users? 14:42:28 ihrachyshka: ++ 14:42:31 so if a vendor pushes an update to fix a critical bug in their library, vendors should be able to package that update without a new version of neutron referencing the new version in requirements.txt, right? 14:42:43 rkukura, right 14:42:46 2nd vendors should be distros 14:42:46 rkukura: Yes, neutron doesn't reference the shims, it's the other way around 14:43:07 neutron contains the shims 14:43:15 mestery, though neutron packages may actually reference vendor libs for upgrade purpose 14:43:30 rkukura: replace shims with backends 14:43:37 mestery: ok 14:43:38 in my comment 14:44:11 ihrachyshka: Where do they reference the vendor libs? 14:44:15 so in kili, requirements.txt for a driver will reference a specific backend lib version, right? 14:44:17 on second thought, if a distro properly splits plugins now into separate packages, then it's fine to add dependencies to vendor libraries 14:44:19 kilo 14:44:39 mestery, a openstack-neutron-arista package should depend on vendor lib 14:44:46 ihrachyshka: Got it 14:44:49 ihrachyshka: Please look at Arista directory - there is a requirements.txt in the drivers directory and it is released to pypi - what else needs to be done for it to be picked up? 14:44:58 If the vendor updates the backend lib, no change to neutrron is likely needed, except the update to the version in requirements.txt. 14:45:36 Sukhdev, I think it's done your side, and packagers should start their job. plus maybe there should be some public status update from armax about plugins ready for that. 14:45:44 rkukura: right. if we have " amotoki: ++ 14:46:17 amotoki: ok 14:46:18 ihrachyshka: ah OK - so, time to bug armax :-) 14:46:21 ihrachyshka: let’s agree on the degrees of readiness and we can revise the doc\ 14:46:41 armax, 'it works' and 'it's published somewhere other than git'? 14:46:41 ugh, pinning may be a necessary evil, but i'd hate to see it become the norm. it means the library sucks at backwards compatibility. 14:46:42 ihrachyshka: but it’s probably best to take this offline 14:46:44 in my understanding requirements fiel in plugins/drives are just reference for packages and users. 14:46:48 armax++ 14:46:51 ihrachyshka: docs too 14:46:56 armax: ++ 14:47:06 ihrachyshka: CI, etc… :) 14:47:11 amotoki: I thought so too. 14:47:16 ihrachyshka: so it sounds to me we’re reinventing the wheel 14:47:25 as to what we need to document :) 14:47:48 less than 15mins left. do we have more items to discuss? 14:47:59 but let’s circle back so that we don’t lose focus on what you and distros really needs 14:48:01 amotoki: Yes, I propose we move on and continue this discussion later. 14:48:08 armax ihrachyshka: Sound ok? 14:48:13 test 14:48:32 mestery++ 14:48:41 #topic nova-network to neutron migration 14:48:47 anteaya: Want to provide an update for the group? 14:48:56 sure 14:49:01 anteaya: thank you! 14:49:13 #link https://review.openstack.org/#/q/topic:bp/migration-from-nova-net,n,z 14:49:27 so no movement on the spec this week, which is to be expected 14:49:40 the patch for the db migration works with flat networks 14:49:53 is anyone able to test that out and provide some feedback on the patch? 14:50:16 and obondarev is going to have a new patchset for the proxy patch by thursday 14:50:22 anteaya: I will as part of my tests for proxy patch 14:50:30 obondarev: awesome, thanks! 14:50:30 also we have agreed that we are refocusing on the L release now 14:50:44 so we won't be expecting any more merges for K 14:51:03 obondarev: great and hopefully we can get some others to test as well 14:51:07 the more the merrier 14:51:10 #info migration work focusing on Liberty release, no more merges expected for Kilo 14:51:15 I think that is if for me 14:51:22 anteaya: Thanks! Any questions from anyone? 14:51:40 OK lets move on 14:51:44 #topic neutron as default in devstack 14:51:51 sc68cal: You're working on this, want to provide an update for the group? 14:52:07 Sure 14:52:12 sc68cal: Thanks! 14:52:30 Currently I'm just doing some preliminary work on taking PUBLIC_INTERFACE and adding it to the OVS_PHSYICAL_BRIDGE 14:52:38 then updating the IP address of the bridge and fixing up the routing table 14:52:48 sc68cal: ++, nice! 14:52:55 Bash is not my first, second, or even third language, so please bear with me - it's pretty ugly 14:53:14 sc68cal: We'll grade you on a courve ;) 14:53:17 It's ugly even when you know it well 14:53:25 dougwig: +1 14:53:34 +1 :D 14:53:35 but I know the way forward - i've done it manually, now it's just automating it 14:54:02 keep an eye on this link https://review.openstack.org/158424 14:54:03 that 14:54:06 is it for me 14:54:31 sc68cal: Thanks! 14:54:45 #topic Open Discussion 14:54:50 A note from dims: Dims needs Project Ideas, Mentors for GSoC 2015 14:54:57 #link https://wiki.openstack.org/wiki/GSoC2015 14:55:03 Go and submit your project ideas! 14:55:29 Anything else this week from anyone? 14:55:32 final bits of this for this cycle: https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/agent-child-processes-status,n,z 14:55:36 thanks mestery 14:55:42 mkolesnik and I proposed a total change of the process monitor api 14:55:43 just a poll 14:55:55 we thought it would be good before finishing the cycle 14:56:01 I have been making it clear that driver testing isn't allowed in the gate 14:56:08 what is neutron's stance on that? 14:56:09 the previous one was totally coupled to ProcessManager... 14:56:10 ajo: Nice! 14:56:21 anteaya: In the Neutron gate? 14:56:26 is has become a topic of discussion in infra 14:56:30 and also, this is ready for review / merge (if we like it) : https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/refactor-iptables-firewall-driver,n,z 14:56:31 well the example is cinder 14:56:41 but anything we do for one project the rest want as well 14:56:48 so there will be spill over 14:56:51 anteaya: For the open source decomposed plugins, we're doing driver testing for their own stackforge repositories (at least for ODL and soon for OVN as well) 14:57:04 mestery++ 14:57:05 :-) 14:57:08 anteaya: But for vendor plugins, not so much. 14:57:23 do you want the opensource plugins in the gate? 14:57:53 context: http://lists.openstack.org/pipermail/openstack-dev/2015-February/057472.html 14:58:00 anteaya: Maybe for the neutron gate, it's possible down the road. 14:58:09 okay thanks 14:58:10 We'll likely discuss this in Vancouver 14:58:15 anteaya: Thanks for bringing it up here! 14:58:22 I was going with no, but if I'm wrong then speakk up 14:58:38 OK, anything else in the last 80 seconds? 14:58:52 If not, thanks for everyone's hard work on Kilo! 14:58:58 Lets finish strong! :) 14:59:01 The hard work is ahead of us... 14:59:06 thanks everyone , yes :) 14:59:06 amuller: :) 14:59:14 ciao 14:59:16 #endmeeting