| *** networkstatic has joined #openstack-neutron | 00:04 | |
| *** fatguylittlecoat has joined #openstack-neutron | 00:16 | |
| JoeJulian | Rather than having to assign one of my very hard to acquire ipv4 addresses to a host, can I just set up a port and have that port do (tcp) port forwarding to an internal address? I know I can use iptables within the netns for that router, but how would I make that persistent? | 00:38 |
|---|---|---|
| *** salv-orlando_ has joined #openstack-neutron | 00:40 | |
| *** salv-orlando has quit IRC | 00:40 | |
| *** salv-orlando_ is now known as salv-orlando | 00:40 | |
| *** dave_tucker is now known as dave_tucker_zzz | 00:44 | |
| *** Makdaam_ has quit IRC | 01:16 | |
| *** Makdaam has joined #openstack-neutron | 01:16 | |
| *** Longgeek has joined #openstack-neutron | 01:18 | |
| *** sbalukoff has quit IRC | 01:21 | |
| *** Longgeek has quit IRC | 01:25 | |
| *** morganfainberg|z has quit IRC | 01:39 | |
| *** morganfainberg|z has joined #openstack-neutron | 01:40 | |
| *** morganfainberg|z is now known as morganfainberg | 01:40 | |
| *** morganfainberg is now known as Guest52195 | 01:40 | |
| *** thuc has joined #openstack-neutron | 02:21 | |
| *** thuc has quit IRC | 02:32 | |
| *** thuc has joined #openstack-neutron | 02:33 | |
| *** thuc has quit IRC | 02:37 | |
| *** Longgeek has joined #openstack-neutron | 02:41 | |
| *** otherwiseguy has quit IRC | 02:51 | |
| *** Longgeek has quit IRC | 03:04 | |
| *** HenryG has joined #openstack-neutron | 03:07 | |
| *** networks_ has joined #openstack-neutron | 03:19 | |
| *** networkstatic has quit IRC | 03:22 | |
| *** fatguylittlecoat has quit IRC | 03:22 | |
| *** fatguylittlecoat has joined #openstack-neutron | 03:23 | |
| *** yamahata has quit IRC | 03:45 | |
| *** obondarev_ has joined #openstack-neutron | 04:04 | |
| *** Longgeek has joined #openstack-neutron | 04:05 | |
| *** Longgeek has quit IRC | 04:11 | |
| *** networkstatic has joined #openstack-neutron | 04:12 | |
| *** fatguylittleco-1 has joined #openstack-neutron | 04:13 | |
| *** networks_ has quit IRC | 04:15 | |
| *** fatguylittlecoat has quit IRC | 04:16 | |
| *** banix has joined #openstack-neutron | 04:16 | |
| *** obondarev_ has quit IRC | 04:28 | |
| *** obondarev_ has joined #openstack-neutron | 04:29 | |
| *** WackoRobie has quit IRC | 04:29 | |
| *** WackoRobie has joined #openstack-neutron | 04:29 | |
| *** obondarev_ has quit IRC | 04:45 | |
| *** WackoRobie has quit IRC | 04:52 | |
| *** WackoRobie has joined #openstack-neutron | 04:53 | |
| *** WackoRobie has quit IRC | 04:54 | |
| *** WackoRobie has joined #openstack-neutron | 04:55 | |
| *** yfried has quit IRC | 05:04 | |
| *** Longgeek has joined #openstack-neutron | 05:06 | |
| *** Longgeek has quit IRC | 05:11 | |
| *** WackoRobie has quit IRC | 05:15 | |
| *** WackoRobie has joined #openstack-neutron | 05:15 | |
| *** WackoRobie has quit IRC | 05:19 | |
| *** ijw has joined #openstack-neutron | 05:33 | |
| *** vkozhukalov has joined #openstack-neutron | 05:38 | |
| *** ijw has quit IRC | 05:38 | |
| *** banix has quit IRC | 05:51 | |
| *** Longgeek has joined #openstack-neutron | 05:52 | |
| *** yfried has joined #openstack-neutron | 05:53 | |
| *** Jabadia has quit IRC | 06:00 | |
| *** dguitarbite has joined #openstack-neutron | 06:05 | |
| *** Longgeek has quit IRC | 06:08 | |
| *** Longgeek has joined #openstack-neutron | 06:11 | |
| *** Longgeek has quit IRC | 06:16 | |
| *** nati_ueno has joined #openstack-neutron | 06:20 | |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Imported Translations from Transifex https://review.openstack.org/66501 | 06:20 |
| *** ashaikh has joined #openstack-neutron | 06:22 | |
| *** nati_uen_ has joined #openstack-neutron | 06:25 | |
| *** WackoRobie has joined #openstack-neutron | 06:26 | |
| *** nati_ueno has quit IRC | 06:28 | |
| *** WackoRobie has quit IRC | 06:31 | |
| *** Longgeek has joined #openstack-neutron | 06:33 | |
| *** Longgeek has quit IRC | 06:38 | |
| *** xuhanp has joined #openstack-neutron | 06:43 | |
| *** irenab has joined #openstack-neutron | 06:48 | |
| *** Jabadia has joined #openstack-neutron | 07:11 | |
| *** madmike has joined #openstack-neutron | 07:21 | |
| *** salv-orlando has quit IRC | 07:22 | |
| *** salv-orlando has joined #openstack-neutron | 07:22 | |
| *** mfink has quit IRC | 07:25 | |
| *** csd has quit IRC | 07:25 | |
| *** lifeless has quit IRC | 07:25 | |
| *** obondarev has quit IRC | 07:25 | |
| *** obondarev has joined #openstack-neutron | 07:26 | |
| *** Makdaam has quit IRC | 07:26 | |
| *** lari_ has quit IRC | 07:26 | |
| *** lari_ has joined #openstack-neutron | 07:26 | |
| *** lifeless has joined #openstack-neutron | 07:27 | |
| *** csd has joined #openstack-neutron | 07:27 | |
| *** Makdaam has joined #openstack-neutron | 07:28 | |
| *** Longgeek has joined #openstack-neutron | 07:34 | |
| *** Longgeek has quit IRC | 07:41 | |
| *** yfried has quit IRC | 07:41 | |
| *** Longgeek has joined #openstack-neutron | 07:44 | |
| *** Longgeek has quit IRC | 07:45 | |
| *** shashank_ has joined #openstack-neutron | 07:52 | |
| *** sbalukoff has joined #openstack-neutron | 07:52 | |
| *** yfried has joined #openstack-neutron | 07:58 | |
| *** julim has joined #openstack-neutron | 08:21 | |
| *** julim has quit IRC | 08:25 | |
| *** xuhanp has quit IRC | 08:25 | |
| *** shashank_ has quit IRC | 08:27 | |
| *** ashaikh has quit IRC | 08:35 | |
| lifeless | anyone here used openvswitch 2.0.1 w/ linux 3.11? got a really strange thing happening | 09:31 |
| lifeless | outbound gre packets are emitted correctly, but the inbound path seems horked | 09:32 |
| lifeless | ovs-appctl dpif/dump-flows br-tun has no flows at all | 09:32 |
| *** Longgeek has joined #openstack-neutron | 09:46 | |
| *** sbalukoff has quit IRC | 09:47 | |
| *** jlibosva has joined #openstack-neutron | 09:48 | |
| *** Longgeek has quit IRC | 09:51 | |
| *** Longgeek has joined #openstack-neutron | 10:18 | |
| openstackgerrit | gongysh proposed a change to openstack/neutron: fix allowedaddress pair update https://review.openstack.org/67698 | 10:21 |
| *** majopela has quit IRC | 10:26 | |
| *** majopela has joined #openstack-neutron | 10:41 | |
| *** emagana has joined #openstack-neutron | 10:50 | |
| *** networkstatic has quit IRC | 10:59 | |
| *** emagana has quit IRC | 11:02 | |
| *** thuc has joined #openstack-neutron | 11:04 | |
| *** thuc has quit IRC | 11:13 | |
| openstackgerrit | Sumit Naiksatam proposed a change to openstack/neutron: Test patch (do not review) https://review.openstack.org/65855 | 11:30 |
| *** amuller has joined #openstack-neutron | 11:39 | |
| *** matrohon has quit IRC | 11:40 | |
| *** matrohon_ has quit IRC | 11:40 | |
| openstackgerrit | Sean Dague proposed a change to openstack/neutron: strip off the newline from the url for logging https://review.openstack.org/67700 | 12:10 |
| *** obondarev_ has joined #openstack-neutron | 12:17 | |
| *** afazekas_ has joined #openstack-neutron | 12:57 | |
| *** majopela has quit IRC | 13:02 | |
| *** itzikb has joined #openstack-neutron | 13:06 | |
| openstackgerrit | Édouard Thuleau proposed a change to openstack/neutron: OVS lib defer apply doesn't handle concurrency https://review.openstack.org/63917 | 13:16 |
| *** salv-orlando has quit IRC | 13:33 | |
| *** WackoRobie has joined #openstack-neutron | 13:36 | |
| *** bvandenh has joined #openstack-neutron | 13:37 | |
| *** Jabadia_ has joined #openstack-neutron | 13:46 | |
| *** Jabadia has quit IRC | 13:46 | |
| *** amuller is now known as amuller_afk | 13:48 | |
| *** amuller_afk is now known as amuller | 13:48 | |
| *** obondarev_ has quit IRC | 14:06 | |
| *** WackoRobie has quit IRC | 14:08 | |
| *** WackoRobie has joined #openstack-neutron | 14:08 | |
| *** WackoRobie has quit IRC | 14:13 | |
| *** Jabadia_ has quit IRC | 14:14 | |
| *** Jabadia has joined #openstack-neutron | 14:15 | |
| openstackgerrit | Avishay Balderman proposed a change to openstack/neutron: LBaaS L7 model (WIP) https://review.openstack.org/61721 | 14:17 |
| *** Jabadia has quit IRC | 14:20 | |
| *** WackoRobie has joined #openstack-neutron | 14:34 | |
| *** irenab has quit IRC | 14:34 | |
| *** RajeshMohan has quit IRC | 14:34 | |
| *** RajeshMohan has joined #openstack-neutron | 14:36 | |
| *** beagles has quit IRC | 14:43 | |
| *** WackoRobie has quit IRC | 14:49 | |
| *** aymenfrikha has joined #openstack-neutron | 14:51 | |
| *** aymenfrikha has left #openstack-neutron | 14:51 | |
| *** amuller has joined #openstack-neutron | 14:52 | |
| *** aymenfrikha1 has joined #openstack-neutron | 14:53 | |
| enikanorov_ | mestery: Hi, i think https://review.openstack.org/#/c/66670/ is ready to get merged as we found agreement among cores. can you give +a? | 14:59 |
| *** fatguylittleco-1 has quit IRC | 15:00 | |
| *** yfried has quit IRC | 15:00 | |
| *** aymenfrikha1 has quit IRC | 15:07 | |
| *** crazylion_ has joined #openstack-neutron | 15:10 | |
| *** crazylion_ has quit IRC | 15:13 | |
| *** yfried has joined #openstack-neutron | 15:16 | |
| *** WackoRobie has joined #openstack-neutron | 15:16 | |
| *** WackoRobie has quit IRC | 15:22 | |
| *** banix has joined #openstack-neutron | 15:24 | |
| *** enikanorov__ has quit IRC | 15:41 | |
| *** enikanorov has joined #openstack-neutron | 15:41 | |
| *** obondarev has quit IRC | 15:41 | |
| *** obondarev has joined #openstack-neutron | 15:42 | |
| *** pcm_ has joined #openstack-neutron | 15:46 | |
| *** pcm_ has quit IRC | 15:46 | |
| *** pcm_ has joined #openstack-neutron | 15:47 | |
| *** alagalah__ has quit IRC | 15:58 | |
| *** yfried has quit IRC | 16:07 | |
| *** roeyc has joined #openstack-neutron | 16:28 | |
| *** WackoRobie has joined #openstack-neutron | 16:33 | |
| *** banix has quit IRC | 16:37 | |
| *** WackoRobie has quit IRC | 16:37 | |
| *** clev has joined #openstack-neutron | 16:47 | |
| *** amuller has quit IRC | 16:48 | |
| *** shashank_ has joined #openstack-neutron | 17:03 | |
| *** itzikb has quit IRC | 17:06 | |
| *** Longgeek has quit IRC | 17:09 | |
| *** yfried has joined #openstack-neutron | 17:16 | |
| *** sdague has quit IRC | 17:30 | |
| *** sdague has joined #openstack-neutron | 17:30 | |
| *** nati_uen_ has quit IRC | 17:35 | |
| *** roeyc has quit IRC | 17:49 | |
| *** pcm_ has quit IRC | 17:49 | |
| HenryG | enikanorov_: ping | 17:52 |
| enikanorov_ | HenryG: hi | 17:52 |
| HenryG | enikanorov_: Can you take a quick look at the jenkins python27 log for https://review.openstack.org/63917 ? | 17:53 |
| HenryG | enikanorov_: here ... http://logs.openstack.org/17/63917/9/check/gate-neutron-python27/9624347/console.html | 17:53 |
| enikanorov_ | yeah, I've seen this. something strange | 17:54 |
| enikanorov_ | can't repro it locally | 17:54 |
| HenryG | enikanorov_: Why is test_parse_service_provider_opt_not_allowed_raises getting a KeyError? | 17:54 |
| enikanorov_ | it's not the test raising, but code in neutron/openstack/common/gettextutils.py raises when formatting exception message | 17:55 |
| enikanorov_ | the test itself expects exception to be thrown | 17:55 |
| *** dave_tucker_zzz is now known as dave_tucker | 17:55 | |
| HenryG | enikanorov_: yes, I see that. Is translation interfering somehow? | 17:55 |
| *** clev has quit IRC | 17:56 | |
| enikanorov_ | i don't know yet. I guess that might be some new package version affecting this, i don't know | 17:56 |
| enikanorov_ | for some reason not every patch is failing this test | 17:56 |
| HenryG | enikanorov_: ok, thanks. Good to know you are aware of it too. | 17:57 |
| enikanorov_ | i'm trying to think of how to catch it. | 17:57 |
| openstackgerrit | enikanorov proposed a change to openstack/neutron: LBaaS: Fix discrepancy between status and admin_state_up https://review.openstack.org/55032 | 17:58 |
| *** thuc has joined #openstack-neutron | 18:07 | |
| *** clev has joined #openstack-neutron | 18:08 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Process port_update notifications in the main agent loop https://review.openstack.org/61964 | 18:12 |
| *** salv-orlando has joined #openstack-neutron | 18:12 | |
| *** csd has quit IRC | 18:19 | |
| *** jlibosva has quit IRC | 18:19 | |
| openstackgerrit | enikanorov proposed a change to openstack/neutron: do not review https://review.openstack.org/67718 | 18:23 |
| *** ashaikh has joined #openstack-neutron | 18:25 | |
| HenryG | enikanorov_: I could repro it in my workspace | 18:27 |
| enikanorov_ | HenryG: how? | 18:29 |
| HenryG | enikanorov_: Here is what I did ... | 18:29 |
| HenryG | 1. git clone <neutron> | 18:29 |
| HenryG | 2. Edit requirements.txt and set psutil>=1.1.0 | 18:29 |
| HenryG | 3. git review -d 63917 | 18:30 |
| HenryG | 4. tox -e py27 | 18:30 |
| HenryG | enikanorov_: that's all | 18:30 |
| HenryG | enikanorov_: (actually steps 2 and 3 are swapped) | 18:32 |
| enikanorov_ | trying | 18:34 |
| enikanorov_ | why changing requirements? | 18:34 |
| HenryG | it's an issue with the latest pip version, let me find the bug/review for it ... | 18:34 |
| enikanorov_ | oh | 18:35 |
| *** nati_ueno has joined #openstack-neutron | 18:35 | |
| *** nati_ueno has quit IRC | 18:35 | |
| HenryG | enikanorov_: https://review.openstack.org/65209 | 18:35 |
| *** nati_ueno has joined #openstack-neutron | 18:36 | |
| HenryG | enikanorov_: versions psutil<1.1.0 are not on PyPI | 18:36 |
| HenryG | enikanorov_: I don't think psutil has anything to do with the svc_type error | 18:38 |
| enikanorov_ | ah, so i need to update pip | 18:38 |
| enikanorov_ | that's the issue | 18:38 |
| HenryG | enikanorov_: well, I think tox installs the latest pip inside the venv | 18:38 |
| enikanorov_ | right... | 18:38 |
| enikanorov_ | mine is 1.5 | 18:38 |
| HenryG | enikanorov_: start with a new clone of neutron so the venv gets the latest of all packages | 18:39 |
| enikanorov_ | i did that already, deleting .tox/py27 | 18:39 |
| HenryG | enikanorov_: that should work too | 18:39 |
| enikanorov_ | i think it made me a fresh venv | 18:39 |
| HenryG | enikanorov_: maybe also delete .testrepository to make sure | 18:40 |
| enikanorov_ | do you run full testsuite? | 18:40 |
| enikanorov_ | or only test_provider_configuration? | 18:40 |
| HenryG | enikanorov_: yes, the full | 18:41 |
| HenryG | I tried to emulate the gate | 18:41 |
| HenryG | enikanorov_: after that, when I run just the single test, it does not fail. :( | 18:43 |
| enikanorov_ | i see | 18:43 |
| enikanorov_ | running full test suite will take too much time... | 18:43 |
| enikanorov_ | ok, i'll try that | 18:43 |
| *** banix has joined #openstack-neutron | 18:46 | |
| *** clev has quit IRC | 18:51 | |
| *** thuc has quit IRC | 18:52 | |
| *** banix has quit IRC | 19:02 | |
| *** thuc has joined #openstack-neutron | 19:08 | |
| *** thuc has quit IRC | 19:11 | |
| *** thuc has joined #openstack-neutron | 19:11 | |
| *** thuc has quit IRC | 19:15 | |
| *** Jabadia has joined #openstack-neutron | 19:18 | |
| *** Jabadia has quit IRC | 19:20 | |
| *** thuc has joined #openstack-neutron | 19:20 | |
| *** Jabadia has joined #openstack-neutron | 19:20 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Avoid re-wiring ports unnecessarily https://review.openstack.org/63558 | 19:21 |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Improve handling of security group updates https://review.openstack.org/63100 | 19:21 |
| *** nati_uen_ has joined #openstack-neutron | 19:24 | |
| *** Longgeek has joined #openstack-neutron | 19:25 | |
| *** nati_ueno has quit IRC | 19:27 | |
| *** Longgeek has quit IRC | 19:32 | |
| *** dave_tucker is now known as dave_tucker_zzz | 19:37 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: Avoid processing ports which are not yet ready https://review.openstack.org/65838 | 19:52 |
| *** irenab has joined #openstack-neutron | 19:55 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: This is a multi-commit patch for testing. Do not review https://review.openstack.org/64185 | 19:57 |
| *** vkozhukalov has quit IRC | 19:58 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack/neutron: This is a multi-commit patch for testing. Do not review https://review.openstack.org/57420 | 19:59 |
| *** jroovers has joined #openstack-neutron | 20:04 | |
| *** salv-orlando has quit IRC | 20:15 | |
| *** afazekas_ has quit IRC | 20:20 | |
| *** thuc has quit IRC | 20:26 | |
| *** thuc has joined #openstack-neutron | 20:27 | |
| openstackgerrit | Sean Dague proposed a change to openstack/neutron: Sync with commit-id: 9d529dd324d234d7aeaa3e6b4d3ab961f177e2ed https://review.openstack.org/67537 | 20:30 |
| *** thuc has quit IRC | 20:32 | |
| HenryG | sdague: Will this ^ help with errors like http://logs.openstack.org/17/63917/9/check/gate-neutron-python27/9624347/console.html#_2014-01-19_13_36_44_126 | 20:49 |
| sdague | HenryG: that was me removing a change from the queue, because the overall neutron job failure rate is too high to put neutron changes in the gate right now | 20:51 |
| sdague | isolated jobs are failing 70% of the time, the regular job 45% | 20:51 |
| HenryG | sdague: also seeing plain py26 and py27 unit tests failing in gettextutils somewhere | 20:53 |
| lifeless | anteaya: hey; you've been chatting to the neutron folk for a bit now; who would be a good person to ask about openvswitch weirdness? | 20:56 |
| *** irenab has quit IRC | 20:57 | |
| *** zzelle has joined #openstack-neutron | 20:59 | |
| enikanorov_ | HenryG: apparently Salvatore has a patch for that | 20:59 |
| enikanorov_ | for that failure | 20:59 |
| enikanorov_ | HenryG: it's a sync from a fix that has been merged to oslo: https://review.openstack.org/#/c/67537/ | 21:00 |
| *** dave_tucker_zzz is now known as dave_tucker | 21:07 | |
| openstackgerrit | Sean Dague proposed a change to openstack/python-neutronclient: Remove unused imports https://review.openstack.org/64592 | 21:19 |
| openstackgerrit | Sean Dague proposed a change to openstack/python-neutronclient: Updated help text for 'subnet-list' command https://review.openstack.org/54261 | 21:20 |
| openstackgerrit | Sean Dague proposed a change to openstack/python-neutronclient: Remove unused imports https://review.openstack.org/64592 | 21:20 |
| openstackgerrit | Sean Dague proposed a change to openstack/python-neutronclient: Adding weight column to Neutron lb member list CLI https://review.openstack.org/65766 | 21:21 |
| *** salv-orlando has joined #openstack-neutron | 21:25 | |
| *** dave_tucker is now known as dave_tucker_zzz | 21:26 | |
| *** dave_tucker_zzz is now known as dave_tucker | 21:27 | |
| *** banix has joined #openstack-neutron | 21:29 | |
| *** Longgeek has joined #openstack-neutron | 21:31 | |
| *** bjornar has joined #openstack-neutron | 21:33 | |
| *** bjornar has quit IRC | 21:38 | |
| *** Longgeek has quit IRC | 21:40 | |
| *** WackoRobie has joined #openstack-neutron | 22:08 | |
| HenryG | lifeless: for OVS questions try mestery | 22:12 |
| lifeless | mestery: oh hai! | 22:13 |
| lifeless | HenryG: thanks | 22:13 |
| lifeless | mestery: I'm trying to get someone with insight into how to debug http://lists.openstack.org/pipermail/openstack-operators/2014-January/003893.html | 22:13 |
| *** salv-orlando has quit IRC | 22:20 | |
| *** bjornar has joined #openstack-neutron | 22:27 | |
| *** WackoRobie has quit IRC | 22:30 | |
| *** banix has quit IRC | 22:38 | |
| *** Longgeek has joined #openstack-neutron | 22:39 | |
| *** salv-orlando has joined #openstack-neutron | 22:39 | |
| salv-orlando | People of Neutron - are you seeing an increased rate of bug 1254890 or did I find a unique way to trigger it with every patch I push? | 22:42 |
| *** Longgeek has quit IRC | 22:44 | |
| salv-orlando | lifeless: could it be when doing tunnelling some extra header is being added in latest OVS version triggering fragmentation, which with GRE is not hardware-assisted? | 22:48 |
| salv-orlando | This could be analysis sniffing the packet before it enters br-tun and then again just before it enters the eth interfacee. | 22:49 |
| lifeless | salv-orlando: I filed a bug actually | 22:49 |
| salv-orlando | Seen that | 22:49 |
| salv-orlando | Bug 1270646 | 22:49 |
| lifeless | salv-orlando: there are two distinct problems; there is a complete tunnel failure with the dkms datapath | 22:49 |
| lifeless | salv-orlando: and with multiple different versions we have this mtuish issue | 22:50 |
| salv-orlando | ah ok. I though from the mailing list the issue with the packet not making it to br-tun was solved. | 22:50 |
| lifeless | salv-orlando: which is the bug I field | 22:50 |
| lifeless | *filed* | 22:50 |
| lifeless | salv-orlando: the packet not arriving was workeda round by using the kernel 3.11 datapath rather than the dkms datapath | 22:50 |
| lifeless | salv-orlando: so I filed the performance issue with neutron since it still happens in ovs 2; and I filed the tunnel being broken w/dkms datapath with ubuntu | 22:51 |
| salv-orlando | gotcha. I was then lazily discarding them as a "non-openstack" issue. But yes, it;s still an issue. | 22:51 |
| lifeless | salv-orlando: so - the performance issue - yes, could be an extra header | 22:51 |
| lifeless | salv-orlando: in http://openvswitch.org/releases/NEWS-2.0.0 | 22:51 |
| lifeless | v1.10.0 - 01 May 2013 | 22:51 |
| lifeless | - Inheritance of the Don't Fragment bit in IP tunnels (df_inherit) | 22:51 |
| lifeless | is no longer supported. | 22:51 |
| lifeless | - Path MTU discovery is no longer supported. | 22:52 |
| lifeless | salv-orlando: I haven't tried 1.9 | 22:52 |
| *** WackoRobie has joined #openstack-neutron | 22:53 | |
| salv-orlando | I mean if scaling the MTU back to 1440 worked for you we could do that as well. It's just that I would like to understand why that happens. Can you confirm it is related to different endpoints using different ovs releases? or does the issue occur even with both endpoints using the same version of ovs? | 22:54 |
| lifeless | both endpoints are running the same version of ovs | 22:54 |
| lifeless | test matrix: | 22:55 |
| lifeless | 1.10.2 - 1.10.2 - happens | 22:55 |
| lifeless | 2.0.1 - 2.0.1 - happens | 22:55 |
| lifeless | I only had mixed versions while debugging the tunnel failure during the 2.0 upgrade | 22:56 |
| *** WackoRobie has quit IRC | 23:01 | |
| lifeless | salv-orlando: I haven't touched the 'ovs_use_veth' setting that james page talks about | 23:04 |
| salv-orlando | that setting is just for how the vifs get plugged. But he might actually be thinking of the issue with br-int not connecting to br-tun | 23:05 |
| lifeless | salv-orlando: so he reported directional performance issues, and it being fixed when he toggled that setting | 23:06 |
| salv-orlando | because it originally used patch ports which were at some point removed from the kernel module | 23:06 |
| *** banix has joined #openstack-neutron | 23:06 | |
| lifeless | Message-ID: <524D387A.3080802@ubuntu.com> | 23:06 |
| lifeless | Date: Thu, 03 Oct 2013 10:27:22 +0100 | 23:06 |
| lifeless | anyhow, just putting it out there :) | 23:06 |
| salv-orlando | lifeless: oh well, I'll need to look into that. I've just looked at the patch which suppressed MTU path discovery in openvswitch. It seems it has an effect on how the MTU for a packet going over the tunnel is set: http://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=commit;h=85b53b3110a1c97c4eaaf647adb8ab8afbeba0e7 | 23:08 |
| lifeless | salv-orlando: yeah | 23:09 |
| lifeless | salv-orlando: hah, in fact - http://www.gossamer-threads.com/lists/openstack/dev/32350#32350 | 23:11 |
| salv-orlando | yeah and indeed I now seem to recall to have seen that patch | 23:12 |
| salv-orlando | for lowering MTU as a DHCP option | 23:12 |
| lifeless | horrible workaround | 23:13 |
| lifeless | I mean, I can understand the pragmatic side of it | 23:14 |
| lifeless | but breaking end to end IP feedback is a bad idea | 23:14 |
| lifeless | the network stack *expects* to be able to discover the width of the plumbing dynamically | 23:14 |
| salv-orlando | The change has surely not been merged. I have found a change (abandoned) for disabling MTU path discovery also when running versions of OVS < 1.9 | 23:18 |
| salv-orlando | Which will probably just cause this issue to propagate to older releases as well | 23:18 |
| lifeless | haha, this is interesting | 23:19 |
| lifeless | when I disable gro, the higher mtu still 'works' | 23:19 |
| lifeless | but we still see a completely broken pmtu : | 23:20 |
| lifeless | 19: 91.189.88.141 140.132ms reached | 23:20 |
| lifeless | Resume: pmtu 65535 hops 19 back 50 | 23:20 |
| salv-orlando | lifeless: if you disable GRE, the packets are just being sent down the eth interface direct. PMTU=65535 is interesting. Do you get the same value even with GRE? | 23:21 |
| lifeless | salv-orlando: GRO, not GRE | 23:21 |
| salv-orlando | ah sorry | 23:21 |
| lifeless | generic receive offload | 23:21 |
| salv-orlando | yep I miread | 23:21 |
| lifeless | darragh said that gro + ovs works badly; I wanted to rule out a foot gun | 23:22 |
| lifeless | AFAICT it works fine as long as you're not fragmenting the hell outta things ;) | 23:22 |
| salv-orlando | lifeless: would you believe that the first result of a google query for GRO and GRE is a server fault question from the same author of the patch which disabled Path MTU discovery in OVS? | 23:24 |
| *** bjornar has quit IRC | 23:26 | |
| salv-orlando | I think it's more a GRO w/GRE issue. Perhaps GRO keeps building up packets received from the wire and sends a larger than expected packet up to the stack which then triggers de-fragmentation in software. But I'm just guessing at this stage. | 23:27 |
| salv-orlando | All of my lab equipment are VMs running on openstack clouds… so I don't have any way to test this I am afraid | 23:28 |
| salv-orlando | But I'd trust Darragh on low-level networking issues :) | 23:28 |
| lifeless | salv-orlando: I would | 23:30 |
| lifeless | both trust darragn and believe the author | 23:30 |
| lifeless | the pmtu being /wrong/ though is a fairly big signal to me | 23:30 |
| lifeless | there's no way that the optimal behaviour can occur with that | 23:31 |
| salv-orlando | lifeless: as unfortunately I am pretty much guessing at the moment, I don't know how useful I am; last thing I can say if that, for any reason, the OVS tunnel port really receives a 65535 bytes packet, then you would see a lot of software-processed fragmenting as the OVS VIF ports on br-int have a much lower MTU. But then if you disable GRO, and do not observe anymore the performance issue while still seeing a 65K PMTU... | 23:34 |
| salv-orlando | I really have no answer | 23:34 |
| lifeless | fair enough | 23:36 |
| lifeless | the mtu on the neutron network node physical port is 1500 | 23:36 |
| lifeless | so its not possible to have a 65Kpmtu :) | 23:36 |
| lifeless | whats darragh's irc nicname? | 23:40 |
| openstackgerrit | Sean Dague proposed a change to openstack/neutron: turn off eventlet.wsgi debug https://review.openstack.org/67737 | 23:40 |
| lifeless | salv-orlando: also, if you want real hardware to test on, we have that in the tripleo test cloud; you could step up as a tripleo-cd-admin and get baremetal access | 23:40 |
| *** Longgeek has joined #openstack-neutron | 23:43 | |
| salv-orlando | lifeless: regarding the MTU I too think 65K is probably just a number coming out of nowhere. | 23:43 |
| salv-orlando | do you think it could be worth forwarding the question to the ovs mailing list? | 23:44 |
| lifeless | salv-orlando: I believe it's the highest value that a pmtu in ipv4 can be | 23:44 |
| lifeless | ovs mailing list? likely, yes | 23:44 |
| lifeless | but perhaps arosen will have ideas | 23:44 |
| lifeless | being nicera and all :) | 23:44 |
| salv-orlando | lifless: agreed. I think the ovs folks have a lot more knowledge than us on the idea... | 23:44 |
| salv-orlando | well, arosen is as nicira as me ;) | 23:44 |
| salv-orlando | the only difference is that he has the ovs guys in the same building | 23:45 |
| *** Longgeek has quit IRC | 23:49 | |
| lifeless | salv-orlando: oh, *facepalm* - sorry! | 23:50 |
| *** yfujioka has joined #openstack-neutron | 23:50 | |
| * lifeless extricates foot from fingers | 23:50 | |
| lifeless | salv-orlando: just shows how often I actually notice affiliation | 23:51 |
| *** zzelle has quit IRC | 23:53 | |
| salv-orlando | The reason why I'm asking to involve someone from the ovs side is that I think we should first be 100% sure of what's going on the network side before pushing any patch to Neutron. | 23:54 |
| lifeless | salv-orlando: agreed | 23:58 |
| lifeless | salv-orlando: my main concern is that at the moment there is a foot note in an install manual saying 'do X to make it work at all' | 23:59 |
| lifeless | and thats pretty bonkers | 23:59 |
| salv-orlando | lifeless: just out of curiosity, are GRE endpoints IPv4? | 23:59 |
| lifeless | even *if* mtu lowering is the right thing (it's not :P), it shouldn't require manual setup, since neutron has enough info to detect the configuration | 23:59 |
| lifeless | salv-orlando: yes | 23:59 |
| lifeless | salv-orlando: 10Gbps NICs | 23:59 |
| lifeless | mellanoxes | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!