| *** asselin has joined #openstack-neutron | 00:00 | |
| nati_ueno | mestery: yeah! Thanks | 00:00 |
|---|---|---|
| * mestery goes to refactor in hopes of getting nati_ueno's +2. | 00:00 | |
| *** alexpilotti has joined #openstack-neutron | 00:02 | |
| rkukura | salv-orlando: I think there were docimpact flags on the related commits, but we definitly should plan to update the portbindings API doc | 00:04 |
| *** nplanel_ has joined #openstack-neutron | 00:06 | |
| kevinbenton | salv-orlando: still there? | 00:06 |
| salv-orlando | yes but not for long | 00:06 |
| salv-orlando | mestery: and I thought I was the pedant one | 00:07 |
| mestery | salv-orlando: :P | 00:07 |
| mestery | salv-orlando: Trying to quickly spin a new version to address nati_ueno's issues. | 00:07 |
| *** banix has joined #openstack-neutron | 00:07 | |
| *** thuc has joined #openstack-neutron | 00:08 | |
| kevinbenton | salv-orlando: sorry, UT was beaking | 00:11 |
| kevinbenton | breaking | 00:11 |
| *** djoreilly has quit IRC | 00:11 | |
| kevinbenton | salv-orlando: see if https://review.openstack.org/#/c/83190/ will work | 00:11 |
| rkukura | kevinbenton: Are you introducing a new key in binding:vif_details now? | 00:12 |
| salv-orlando | kevinbenton: might, but I'll have to defer checks in the morning. Also people might not like the new key. | 00:12 |
| salv-orlando | since it's a problem for a single plugin go for whatever you get consensus on. | 00:13 |
| kevinbenton | rkukura: yes, if a plugin doesn't want hybrid it would specify it explicitly with the new key | 00:13 |
| salv-orlando | if the nsx plugin and a few others are going to run with increased latency or are broken it's now their problem | 00:13 |
| salv-orlando | ok, goodnight. | 00:14 |
| kevinbenton | salv-orlando: this approach gives them a way to fix it though | 00:14 |
| kevinbenton | salv-orlando | 00:14 |
| kevinbenton | ok | 00:14 |
| rkukura | kevinbenton: Is their a neutron-side patch adding this key? | 00:14 |
| kevinbenton | good night | 00:14 |
| kevinbenton | rkukura: it will be up to the plugins using direct plugs to add it | 00:14 |
| kevinbenton | rkukura: i suppose we could add the constant in a patch | 00:15 |
| *** armitage81 has quit IRC | 00:15 | |
| *** armitage81 has joined #openstack-neutron | 00:15 | |
| rkukura | kevinbenton: That’s what I was thinking | 00:15 |
| *** thuc has quit IRC | 00:17 | |
| rkukura | kevinbenton: Just realized you are not using ‘port-filter’ at all any more! | 00:17 |
| rkukura | What led to that decision? | 00:18 |
| kevinbenton | rkukura: yeah, salv-orlando pointed out that port-filter is to imply that security filtering is happening | 00:18 |
| kevinbenton | rkukura: but a plugin like NSX does do security filtering, even though it doesn't use the hybrid plugging strategy | 00:18 |
| *** matsuhashi has joined #openstack-neutron | 00:19 | |
| *** Trozz has joined #openstack-neutron | 00:19 | |
| kevinbenton | rkukura: so they would report port_filter is True and nova would incorrectly use the hybrid mode | 00:19 |
| *** Trozz has left #openstack-neutron | 00:20 | |
| beagles | kevinbenton: this is an annoying stupid nit, but how do you feel about taking neutron out of that function name in the vif model? | 00:21 |
| markmcclain | HenryG: I'll take a look in the morning | 00:21 |
| kevinbenton | beagles: sure. i think i have a bug anyway here | 00:21 |
| beagles | kevinbenton, that being said, I don't know what makes a better name... dammit, I'm going to shut up... too many cooks in the kitchen. | 00:23 |
| beagles | :) | 00:23 |
| openstackgerrit | Kyle Mestery proposed a change to openstack/neutron: Correct OVS VXLAN version check https://review.openstack.org/82931 | 00:24 |
| mestery | nati_ueno: ^^^ | 00:24 |
| * nati_ueno taking a look | 00:24 | |
| kevinbenton | nati_ueno: hold for a few mins | 00:25 |
| nati_ueno | kevinbenton: yep. I can hold here few hours | 00:25 |
| banix | kevinbenton: If I understand it correctly now by default the hybrid plugging is used; woudn't be better to require hybrid plugging only if it is asked explicitly? Just wondering. | 00:27 |
| kevinbenton | banix: i think ones that want direct are in the minority | 00:29 |
| banix | kevinbenton: and the majority right now get the hybrid plugging by default? Just trying to understand this. Not implying otherwise. | 00:30 |
| markmcclain | no there are plugins that want direct plugging | 00:31 |
| banix | I am just questioning what should be the deafult. kevinbenton says majority want hybrid and I was asking if those which require hybrid get that now by default. If that is the case having "hybrid" as the default option makes sense; otherwise, it should be the other way around. | 00:34 |
| mestery | nati_ueno: :P | 00:35 |
| *** ale has joined #openstack-neutron | 00:36 | |
| markmcclain | banix: direct has been the default for sometime | 00:38 |
| markmcclain | so changing this late in the game is something we should generally avoid | 00:38 |
| kevinbenton | markmcclain: ok, should i go towards direct as default then? | 00:38 |
| banix | markmcclain: thanks for clarification. That was my understanding but was not sure. | 00:40 |
| rkukura | kevinbenton: Does that mean we need a neutron patch to have the ML2 agent mechanism drivers (and maybe some plugins) pass the new key in binding:vif_details? | 00:40 |
| kevinbenton | rkukura: yeah | 00:40 |
| *** thuc has joined #openstack-neutron | 00:40 | |
| rkukura | kevinbenton: I have no objection to that | 00:41 |
| markmcclain | so we're going to need to pair 3 patches to validate this works? | 00:41 |
| kevinbenton | markmcclain: what's the 3rd patch? | 00:42 |
| rkukura | The neutron patch should be safe to merge before any of the others, since it just adds the new item to binding:vif_details. | 00:42 |
| markmcclain | nova+neutron+tempest | 00:42 |
| *** thuc has quit IRC | 00:42 | |
| kevinbenton | what needs to change in tempest? | 00:42 |
| markmcclain | we have a test that has been written to prevent a regression | 00:42 |
| *** thuc has joined #openstack-neutron | 00:43 | |
| * mestery ^5s markmcclain. | 00:44 | |
| mestery | Welcome back markmcclain! | 00:44 |
| markmcclain | mestery: thanks! | 00:45 |
| markmcclain | kevinbenton: this is the tempest change: https://review.openstack.org/#/c/62702/ | 00:45 |
| *** beagles has quit IRC | 00:46 | |
| *** thuc has quit IRC | 00:47 | |
| *** yamamoto has left #openstack-neutron | 00:50 | |
| *** ale has quit IRC | 00:50 | |
| *** mestery has quit IRC | 00:51 | |
| *** rkukura has quit IRC | 00:56 | |
| *** alexpilotti has quit IRC | 00:57 | |
| *** markmcclain has quit IRC | 00:59 | |
| *** nplanel_ has quit IRC | 01:01 | |
| *** enikanorov has joined #openstack-neutron | 01:04 | |
| kevinbenton | rkukura: still need to use the cap filter | 01:10 |
| *** SumitNaiksatam has quit IRC | 01:11 | |
| *** fouxm_ has joined #openstack-neutron | 01:11 | |
| *** _cjones_ has quit IRC | 01:11 | |
| *** matrohon has quit IRC | 01:12 | |
| *** enikanorov_ has quit IRC | 01:12 | |
| *** fouxm has quit IRC | 01:12 | |
| *** geekinutah has quit IRC | 01:12 | |
| *** matsuhashi has quit IRC | 01:12 | |
| *** nati_ueno has quit IRC | 01:12 | |
| *** harlowja has quit IRC | 01:12 | |
| *** otherwiseguy has quit IRC | 01:12 | |
| *** amotoki has quit IRC | 01:12 | |
| *** Matt2 has quit IRC | 01:12 | |
| *** zigo has quit IRC | 01:12 | |
| *** salv-orlando has quit IRC | 01:12 | |
| *** rm_work has quit IRC | 01:12 | |
| *** thuc has joined #openstack-neutron | 01:17 | |
| *** harlowja has joined #openstack-neutron | 01:17 | |
| *** thuc_ has joined #openstack-neutron | 01:17 | |
| *** claudiub has quit IRC | 01:18 | |
| *** geekinutah has joined #openstack-neutron | 01:20 | |
| *** thuc has quit IRC | 01:21 | |
| *** matrohon has joined #openstack-neutron | 01:22 | |
| *** tomoe_ has joined #openstack-neutron | 01:28 | |
| *** networkstatic has quit IRC | 01:34 | |
| kevinbenton | rkukura, markmcclain, nati_ueno: https://review.openstack.org/#/c/83190/6/ | 01:40 |
| kevinbenton | beagles: i ended up leaving neutron in | 01:41 |
| kevinbenton | beagles: since that variable is indicating that it is neutron that's filtering | 01:42 |
| *** xuhanp has joined #openstack-neutron | 01:42 | |
| *** baoli has joined #openstack-neutron | 01:45 | |
| *** SumitNaiksatam has joined #openstack-neutron | 01:47 | |
| *** SumitNaiksatam has quit IRC | 01:47 | |
| *** _cjones_ has joined #openstack-neutron | 01:47 | |
| *** SumitNaiksatam has joined #openstack-neutron | 01:48 | |
| *** thuc_ has quit IRC | 01:48 | |
| *** matsuhashi has joined #openstack-neutron | 01:49 | |
| *** nati_ueno has joined #openstack-neutron | 01:49 | |
| *** amotoki has joined #openstack-neutron | 01:49 | |
| *** Matt2 has joined #openstack-neutron | 01:49 | |
| *** zigo has joined #openstack-neutron | 01:49 | |
| *** salv-orlando has joined #openstack-neutron | 01:49 | |
| *** rm_work has joined #openstack-neutron | 01:49 | |
| *** thuc has joined #openstack-neutron | 01:49 | |
| *** SumitNaiksatam has left #openstack-neutron | 01:50 | |
| *** thuc has quit IRC | 01:53 | |
| *** xianghui has joined #openstack-neutron | 01:57 | |
| *** _cjones_ has quit IRC | 01:58 | |
| *** dguitarbite has joined #openstack-neutron | 02:00 | |
| *** Jianyong has joined #openstack-neutron | 02:01 | |
| *** baoli has quit IRC | 02:01 | |
| *** Jianyong has left #openstack-neutron | 02:02 | |
| *** Longgeek has joined #openstack-neutron | 02:05 | |
| *** flwang has quit IRC | 02:07 | |
| *** singhs has quit IRC | 02:09 | |
| *** armax has quit IRC | 02:10 | |
| nati_ueno | kevinbenton: how's going? | 02:16 |
| kevinbenton | https://review.openstack.org/#/c/83190/6/ | 02:16 |
| kevinbenton | nati_ueno: ^^ | 02:17 |
| nati_ueno | kevinbenton: yep | 02:17 |
| nati_ueno | kevinbenton: so what's conclusion with Salvatore? | 02:17 |
| kevinbenton | nati_ueno: this is the direction salvatore wanted. it will allow plugins like NSX to indicate that that support filtering but still allow direct plugging | 02:19 |
| *** harlowja is now known as harlowja_away | 02:19 | |
| nati_ueno | kevinbenton: gocha | 02:27 |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Disable XML tests on Py26 https://review.openstack.org/81865 | 02:27 |
| nati_ueno | kevinbenton: so we will have nova side fix? | 02:28 |
| nati_ueno | kevinbenton: sorry. neutron side? | 02:29 |
| kevinbenton | nati_ueno: yeah, about to upload | 02:29 |
| nati_ueno | kevinbenton: nice | 02:29 |
| nati_ueno | kevinbenton: Thank you for your taking this one | 02:30 |
| *** crc32 has quit IRC | 02:30 | |
| *** lifeless has joined #openstack-neutron | 02:30 | |
| *** lifeless has quit IRC | 02:35 | |
| *** s51itxsyc has joined #openstack-neutron | 02:36 | |
| *** harlowja_away is now known as harlowja | 02:37 | |
| *** s51itxsyc has quit IRC | 02:37 | |
| *** matsuhashi has quit IRC | 02:37 | |
| *** matsuhashi has joined #openstack-neutron | 02:38 | |
| *** HenryG has quit IRC | 02:46 | |
| *** armitage81 has quit IRC | 02:46 | |
| *** devlaps has quit IRC | 02:46 | |
| *** matsuhashi has quit IRC | 02:47 | |
| *** thuc has joined #openstack-neutron | 02:48 | |
| *** thuc has joined #openstack-neutron | 02:49 | |
| *** marun has quit IRC | 02:49 | |
| *** marun has joined #openstack-neutron | 02:50 | |
| *** lifeless has joined #openstack-neutron | 02:53 | |
| *** flwang has joined #openstack-neutron | 02:54 | |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 02:58 |
| kevinbenton | nati_ueno: http://review.openstack.org/83280 | 02:58 |
| nati_ueno | kevinbenton: Thanks | 03:03 |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 03:10 |
| *** ramishra has joined #openstack-neutron | 03:11 | |
| kevinbenton | nati_ueno: this devstack patch should be testing both | 03:11 |
| kevinbenton | https://review.openstack.org/#/c/83281/ | 03:11 |
| nati_ueno | kevinbenton: nice | 03:12 |
| kevinbenton | testing all three* | 03:12 |
| kevinbenton | the tempest one as well that enables the negative test | 03:12 |
| kevinbenton | nati_ueno: whoops, i missed your comment about the openvswitch plugin | 03:15 |
| *** tomoe_ has quit IRC | 03:15 | |
| *** lollipop has joined #openstack-neutron | 03:16 | |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 03:21 |
| openstackgerrit | A change was merged to openstack/neutron: Subnets should be set as lazy='join' https://review.openstack.org/83213 | 03:22 |
| nati_ueno | kevinbenton: OK let's wait Jenkins says OK for https://review.openstack.org/#/c/83280/3 | 03:23 |
| nati_ueno | any other core around? | 03:24 |
| *** thuc_ has joined #openstack-neutron | 03:32 | |
| openstackgerrit | Ian Wienand proposed a change to openstack/neutron: Record and log reason for dhcp agent resync https://review.openstack.org/81173 | 03:35 |
| *** ramishra_ has joined #openstack-neutron | 03:35 | |
| *** thuc has quit IRC | 03:35 | |
| *** thuc_ has quit IRC | 03:36 | |
| *** ramishra has quit IRC | 03:37 | |
| *** amotoki has quit IRC | 03:37 | |
| *** devlaps has joined #openstack-neutron | 03:40 | |
| *** networkstatic has joined #openstack-neutron | 03:44 | |
| *** harlowja is now known as harlowja_away | 03:47 | |
| *** matsuhashi has joined #openstack-neutron | 03:48 | |
| *** yamahata has joined #openstack-neutron | 03:51 | |
| *** chandan_kumar has joined #openstack-neutron | 03:52 | |
| *** ramishra_ has quit IRC | 03:57 | |
| *** ramishra has joined #openstack-neutron | 03:57 | |
| *** BuSerD has quit IRC | 04:04 | |
| *** yamahata has quit IRC | 04:05 | |
| *** thuc has joined #openstack-neutron | 04:06 | |
| *** banix has quit IRC | 04:14 | |
| *** ale has joined #openstack-neutron | 04:16 | |
| *** tomoe_ has joined #openstack-neutron | 04:18 | |
| *** chandan_kumar has quit IRC | 04:34 | |
| *** thuc_ has joined #openstack-neutron | 04:36 | |
| *** saju_m has joined #openstack-neutron | 04:38 | |
| *** thuc has quit IRC | 04:40 | |
| *** thuc_ has quit IRC | 04:40 | |
| *** ramishra has quit IRC | 04:44 | |
| *** matsuhashi has quit IRC | 04:45 | |
| *** qiuyu_ has quit IRC | 04:46 | |
| *** qiuyu has joined #openstack-neutron | 04:46 | |
| *** rkukura has joined #openstack-neutron | 04:48 | |
| *** matsuhas_ has joined #openstack-neutron | 04:49 | |
| enikanorov | jaypipes: thanks for the comments! | 04:49 |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 04:54 |
| *** sacharya has joined #openstack-neutron | 04:57 | |
| openstackgerrit | Akihiro Motoki proposed a change to openstack/neutron: Remove auto-generation of db schema from models at startup https://review.openstack.org/40296 | 05:00 |
| rkukura | kevinbenton: ping | 05:00 |
| kevinbenton | rkukura: pong | 05:00 |
| *** marun has quit IRC | 05:01 | |
| rkukura | In the very latest version of your neutron-side patch, I’m concerned that the openvswitch plugin cannot depend on the agent-only firewall_driver config being available in neutron-server. | 05:02 |
| WormMan | 5 cores processing is not any faster than 3 | 05:03 |
| rkukura | I had responded to the comment in the previous version that suggested that change, but you pushed your update before I finished my review. | 05:03 |
| WormMan | oops | 05:03 |
| rkukura | kevinbenton: I’m pretty sure things like puppet modules that configure openvswitch-agent, setting firewall_driver, are not typically run on the controller nodes where neutron-server runs but the L2 agent does not. | 05:04 |
| kevinbenton | rkukura: right, maybe i'll just leave that for a follow-up patch then and hardcode to hybrid for now? | 05:05 |
| rkukura | kevinbenton: That’s what I’d recommend | 05:05 |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 05:06 |
| openstackgerrit | Akihiro Motoki proposed a change to openstack/neutron: Remove auto-generation of db schema from models at startup https://review.openstack.org/40296 | 05:12 |
| kevinbenton | rkukura: with regard to the constant name, i didn't use VIF_DETAIL because CAP_PORT_FILTER didn't use it either | 05:12 |
| *** matsuhas_ has quit IRC | 05:12 | |
| rkukura | kevinbenton: that’s fine - we can establish a convention later when doing the fine grained VIF security stuff | 05:13 |
| kevinbenton | rkukura: ok. do you want me to fix the whitespace now since it's quite right now anyway? | 05:13 |
| rkukura | kevinbenton: not unless you need to change something else | 05:14 |
| kevinbenton | rkukura: not at the moment. i'll wait for more reviews in the morning then | 05:14 |
| rkukura | I’ll probably +2 as soon as jenkins finishes | 05:14 |
| *** pradipta_away is now known as pradipta | 05:19 | |
| *** dguitarbite has quit IRC | 05:22 | |
| *** yfried has quit IRC | 05:22 | |
| *** sacharya has quit IRC | 05:23 | |
| *** Jabadia has joined #openstack-neutron | 05:28 | |
| *** matsuhashi has joined #openstack-neutron | 05:29 | |
| *** Longgeek has quit IRC | 05:30 | |
| *** chandan_kumar has joined #openstack-neutron | 05:30 | |
| *** tomoe__ has joined #openstack-neutron | 05:31 | |
| *** sridhar has joined #openstack-neutron | 05:32 | |
| *** amotoki has joined #openstack-neutron | 05:32 | |
| *** tomoe_ has quit IRC | 05:35 | |
| *** itzikb1 has quit IRC | 05:35 | |
| *** linuxgeek_ has joined #openstack-neutron | 05:40 | |
| *** networkstatic has quit IRC | 05:44 | |
| *** networkstatic has joined #openstack-neutron | 05:44 | |
| * nati_ueno I'm still here | 05:47 | |
| linuxgeek_ | hi, in icehouse3 when i launch an instance it fails with error TRACE oslo.messaging._executors.base File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 51, in get_token | 05:53 |
| linuxgeek_ | TRACE oslo.messaging._executors.base TypeError: string indices must be integers, not str | 05:53 |
| linuxgeek_ | ERROR oslo.messaging._drivers.common [-] Returning exception string indices must be integers, not str to caller | 05:53 |
| linuxgeek_ | anyone seen this? | 05:54 |
| *** devlaps has quit IRC | 05:55 | |
| amotoki | kevinbenton: ping | 05:59 |
| kevinbenton | amotoki: pong | 05:59 |
| *** Jabadia has quit IRC | 05:59 | |
| amotoki | kevinbenton: i see your patch adding ovs-hybrid-flag. there are some plugin which need this flag. | 06:00 |
| *** devlaps has joined #openstack-neutron | 06:00 | |
| amotoki | kevinbenton: can we use this patch as a hub to support the flag? or should we add it in another patch? | 06:00 |
| amotoki | i am just back and would like to know the status of the situation. | 06:00 |
| kevinbenton | amotoki: do you know which plugins will need this? | 06:01 |
| amotoki | kevinbenton: for example, nec and ryu plugins need this. | 06:02 |
| kevinbenton | amotoki: is that it? | 06:03 |
| kevinbenton | amotoki: i can add them now | 06:03 |
| amotoki | kevinbenton: here is the partial list: https://bugs.launchpad.net/neutron/+bug/1297469/comments/6 | 06:04 |
| *** lollipop has quit IRC | 06:04 | |
| *** lollipop has joined #openstack-neutron | 06:05 | |
| amotoki | kevinbenton: thanks. | 06:05 |
| rkukura | grepping for VIF_TYPE_OVS will show which plugins might and ML2 drivers need this, although some of these don’t | 06:05 |
| rkukura | s/might and ML2 drivers/and ML2 drivers might/ | 06:06 |
| *** saju_m has quit IRC | 06:07 | |
| *** Longgeek has joined #openstack-neutron | 06:07 | |
| amotoki | yes. grepping VIF_TYPE_OVS works, but we need information about hybrid one is required from plugin maintainers. | 06:08 |
| amotoki | AFAIK, ovs, ryu, bsn, nec, mech-ofa -- hybrid is required, mech_odl, vmware -- not needed | 06:10 |
| *** rotbeard has joined #openstack-neutron | 06:10 | |
| *** dave_tucker is now known as dave_tucker_zzz | 06:13 | |
| rkukura | amotoki: Right - only a subset of the VIF_TYPE_OVS plugins/drivers need hybrid | 06:14 |
| kevinbenton | amotoki: you want me to do mech_ofagent as well? | 06:15 |
| *** skraynev_afk is now known as skraynev | 06:16 | |
| rkukura | kevinbenton: I suspect the cisco n1kv plugin need this, but not sure | 06:17 |
| amotoki | kevinbenton: Either is okay, but i heard so. | 06:17 |
| kevinbenton | rkukura: should i wait for someone to chime in? | 06:17 |
| amotoki | For n1kv, henryG said he will ask n1kv team. | 06:18 |
| kevinbenton | okay, no more then for tonight. not worth spending much more time on this if the nova team doesn't accept the main solution | 06:19 |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 06:19 |
| rkukura | kevinbenton: I’d leave out n1kv for now - doesn’t seem to implement SG extension | 06:19 |
| kevinbenton | rkukura: ok | 06:20 |
| kevinbenton | amotoki: nec, ryu, mech_ofagent added | 06:20 |
| amotoki | kevinbenton: we cannot clean up this topic at once. we need to add OVS_HYBRID_PLUG support in multple patches. your is the first round :) | 06:20 |
| amotoki | kevinbenton: really thanks for working on it! | 06:20 |
| kevinbenton | amotoki: no problem | 06:21 |
| kevinbenton | this devstack patch is testing all 3 | 06:25 |
| kevinbenton | https://review.openstack.org/#/c/83281/ | 06:25 |
| kevinbenton | looks like it's going to pass jenkins, one job remaining in check queue | 06:25 |
| *** tomoe__ has quit IRC | 06:28 | |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Imported Translations from Transifex https://review.openstack.org/83301 | 06:29 |
| *** marun has joined #openstack-neutron | 06:29 | |
| *** russellb has quit IRC | 06:32 | |
| *** yamahata has joined #openstack-neutron | 06:39 | |
| *** gdubreui has quit IRC | 06:40 | |
| *** tomoe_ has joined #openstack-neutron | 06:41 | |
| openstackgerrit | Kevin Benton proposed a change to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 06:41 |
| *** yfried has joined #openstack-neutron | 06:46 | |
| *** marun is now known as marun_afk | 06:46 | |
| *** saju_m has joined #openstack-neutron | 06:50 | |
| *** dfarrell07 has quit IRC | 06:57 | |
| *** irenab has joined #openstack-neutron | 06:58 | |
| *** dguitarbite has joined #openstack-neutron | 07:01 | |
| *** afazekas has joined #openstack-neutron | 07:04 | |
| *** mrsnivvel has joined #openstack-neutron | 07:04 | |
| *** lollipop has quit IRC | 07:04 | |
| *** ramishra has joined #openstack-neutron | 07:05 | |
| *** lollipop has joined #openstack-neutron | 07:07 | |
| *** devlaps has quit IRC | 07:16 | |
| *** pnavarro has joined #openstack-neutron | 07:19 | |
| *** dguitarbite_ has joined #openstack-neutron | 07:19 | |
| *** dguitarbite has quit IRC | 07:19 | |
| *** ramishra has quit IRC | 07:22 | |
| *** Jabadia has joined #openstack-neutron | 07:22 | |
| *** ramishra has joined #openstack-neutron | 07:22 | |
| *** Jabadia has quit IRC | 07:23 | |
| openstackgerrit | Xu Han Peng proposed a change to openstack/neutron: Permit ICMPv6 RAs only from known routers https://review.openstack.org/72252 | 07:23 |
| *** Jabadia has joined #openstack-neutron | 07:23 | |
| *** djoreilly has joined #openstack-neutron | 07:23 | |
| *** ramishra has quit IRC | 07:27 | |
| *** ale has quit IRC | 07:30 | |
| *** ale has joined #openstack-neutron | 07:30 | |
| * nati_ueno waiting jenkins get happy with 83280 | 07:32 | |
| *** sridhar has quit IRC | 07:33 | |
| *** jprovazn has joined #openstack-neutron | 07:36 | |
| *** ale has quit IRC | 07:37 | |
| *** pnavarro has quit IRC | 07:45 | |
| *** jistr has joined #openstack-neutron | 07:47 | |
| *** russellb has joined #openstack-neutron | 07:47 | |
| *** ramishra has joined #openstack-neutron | 07:49 | |
| *** _cjones_ has joined #openstack-neutron | 07:51 | |
| *** evgenyf has joined #openstack-neutron | 07:54 | |
| *** _cjones_ has quit IRC | 07:55 | |
| *** jlibosva has joined #openstack-neutron | 07:58 | |
| *** humbolt has joined #openstack-neutron | 08:00 | |
| *** luqas has joined #openstack-neutron | 08:11 | |
| *** luqas has quit IRC | 08:13 | |
| *** dave_tucker_zzz is now known as dave_tucker | 08:15 | |
| *** amuller has joined #openstack-neutron | 08:17 | |
| *** flwang has quit IRC | 08:19 | |
| *** luqas has joined #openstack-neutron | 08:21 | |
| *** roeyc has joined #openstack-neutron | 08:21 | |
| kevinbenton | nati_ueno: jenkins has spoken! | 08:22 |
| *** dguitarbite has joined #openstack-neutron | 08:23 | |
| *** jgallard has joined #openstack-neutron | 08:24 | |
| *** dguitarbite_ has quit IRC | 08:26 | |
| *** flwang has joined #openstack-neutron | 08:28 | |
| nati_ueno | kevinbenton: approved! | 08:29 |
| *** Longgeek has quit IRC | 08:29 | |
| * nati_ueno so now, we need nova side only.. | 08:30 | |
| kevinbenton | nati_ueno: thanks. i'm off for the night. talk to you tomorrow | 08:31 |
| nati_ueno | kevinbenton: Thanks you for your hard work!!! | 08:31 |
| *** matsuhashi has quit IRC | 08:33 | |
| irenab | nati_ueno: hi | 08:36 |
| *** thuc has joined #openstack-neutron | 08:37 | |
| nati_ueno | irenab: hi | 08:37 |
| irenab | nati_ueno: Can you please take a look at https://review.openstack.org/#/c/82729/. This is a fix for the issue we discussed few days ago | 08:39 |
| nati_ueno | irenab: sure! | 08:40 |
| *** roeyc has quit IRC | 08:41 | |
| irenab | nati_ueno: thanks | 08:41 |
| *** thuc has quit IRC | 08:41 | |
| *** Longgeek has joined #openstack-neutron | 08:43 | |
| *** saju_m has quit IRC | 08:43 | |
| *** ramishra has quit IRC | 08:43 | |
| salv-orlando | nati_ueno: you still around? | 08:44 |
| nati_ueno | salv-orlando: yes | 08:44 |
| salv-orlando | we would need now to do a combined test of 83190, and 83280 | 08:44 |
| *** roeyc has joined #openstack-neutron | 08:44 | |
| salv-orlando | possibly with tempest patch #62702 | 08:44 |
| *** ramishra has joined #openstack-neutron | 08:44 | |
| salv-orlando | if you have still energies for update devstack patch 83190 we can give it a spin | 08:44 |
| nati_ueno | kevinbenton: has patch | 08:44 |
| salv-orlando | all right | 08:45 |
| nati_ueno | salv-orlando: let me see.. | 08:45 |
| salv-orlando | so 83190 is to discard? | 08:45 |
| salv-orlando | https://review.openstack.org/#/c/83281/ | 08:45 |
| nati_ueno | salv-orlando: https://review.openstack.org/#/c/83281/ | 08:45 |
| nati_ueno | salv-orlando: no Kevin fixed 83190 as you wish | 08:46 |
| salv-orlando | I don't know why did he feel the need for duplicating the tempest patch, but maybe that's not important | 08:46 |
| nati_ueno | salv-orlando: ya, may be we won't have hisown :P | 08:46 |
| nati_ueno | oh we need to update patch set id on 83281/ | 08:47 |
| salv-orlando | uhm. yes. If you are already in bed I can do that. I just woke up ;) | 08:48 |
| nati_ueno | salv-orlando: Thanks!! | 08:48 |
| salv-orlando | well, over an hour ago | 08:48 |
| *** ramishra has quit IRC | 08:49 | |
| nati_ueno | salv-orlando: Actually, I should driver 101 from now.. little bit scary drive | 08:49 |
| nati_ueno | s/driver/drive/ | 08:49 |
| salv-orlando | so you stayed in a office the whole time? | 08:50 |
| nati_ueno | salv-orlando: yeah | 08:50 |
| salv-orlando | Well, get some coffee and then jump in your car then | 08:51 |
| *** ygbo has joined #openstack-neutron | 08:52 | |
| nati_ueno | salv-orlando: Thanks | 08:52 |
| *** saju_m has joined #openstack-neutron | 08:55 | |
| *** pnavarro has joined #openstack-neutron | 08:57 | |
| *** Longgeek has quit IRC | 08:57 | |
| *** flwang has quit IRC | 08:58 | |
| openstackgerrit | enikanorov proposed a change to openstack/neutron: Proof of Concept flavor framework https://review.openstack.org/83055 | 09:00 |
| enikanorov | salv-orlando: hi! If you have some spare time and interest, could you take a look at ^^^? | 09:02 |
| salv-orlando | cool, have you already spoken to markmcclain for potential icehouse inclusion? | 09:02 |
| salv-orlando | or is it just to have a look at ti now? | 09:02 |
| enikanorov | hope you're kidding :) | 09:03 |
| enikanorov | sure not for Icehouse | 09:03 |
| enikanorov | also it's a PoC | 09:03 |
| enikanorov | so... look only if you are bored with some usefull code :) | 09:04 |
| salv-orlando | ok then… I will look at it after we've cut RC1. It's being tough getting bug count to 0 | 09:04 |
| enikanorov | sure np! | 09:05 |
| *** safchain has joined #openstack-neutron | 09:08 | |
| *** leseb has joined #openstack-neutron | 09:13 | |
| *** saju_m has quit IRC | 09:16 | |
| *** pcm_ has quit IRC | 09:17 | |
| *** ale has joined #openstack-neutron | 09:17 | |
| *** Longgeek has joined #openstack-neutron | 09:26 | |
| *** saju_m has joined #openstack-neutron | 09:27 | |
| *** jpich has joined #openstack-neutron | 09:31 | |
| openstackgerrit | A change was merged to openstack/neutron: Adds OVS_HYBRID_PLUG flag to portbindings https://review.openstack.org/83280 | 09:34 |
| *** bvandenh has joined #openstack-neutron | 09:37 | |
| *** enikanorov_ has joined #openstack-neutron | 09:38 | |
| *** ale___ has joined #openstack-neutron | 09:38 | |
| *** enikanorov_ has quit IRC | 09:38 | |
| *** jistr has quit IRC | 09:38 | |
| *** ale has quit IRC | 09:40 | |
| openstackgerrit | Xu Han Peng proposed a change to openstack/python-neutronclient: Create new IPv6 attributes for Subnets by client https://review.openstack.org/75871 | 09:42 |
| *** bashok has joined #openstack-neutron | 09:43 | |
| *** leseb has quit IRC | 09:43 | |
| *** Longgeek has quit IRC | 09:43 | |
| *** matsuhashi has joined #openstack-neutron | 09:45 | |
| *** yamamoto has joined #openstack-neutron | 09:48 | |
| *** Longgeek has joined #openstack-neutron | 09:51 | |
| amotoki | nati_ueno: salv-orlando: thanks for coordinating the secgroup issue. We have the great progress in a day :-) | 09:56 |
| salv-orlando | amotoki: I've not been coordinating. Mostly being an a**hole to other people, because that's what I do better :p | 09:57 |
| amotoki | when I was back, the situation significantly changed for better direction. the community is really super :-) | 09:59 |
| *** amotoki has quit IRC | 10:00 | |
| *** ale___ has quit IRC | 10:01 | |
| *** sphoorti has joined #openstack-neutron | 10:01 | |
| *** jp_at_hp has joined #openstack-neutron | 10:02 | |
| *** ale___ has joined #openstack-neutron | 10:03 | |
| *** jistr has joined #openstack-neutron | 10:04 | |
| *** sphoorti has quit IRC | 10:07 | |
| *** tomoe_ has quit IRC | 10:07 | |
| *** sphoorti has joined #openstack-neutron | 10:09 | |
| *** yamamoto has quit IRC | 10:11 | |
| *** leseb has joined #openstack-neutron | 10:12 | |
| *** amuller has quit IRC | 10:15 | |
| *** ajo_ has joined #openstack-neutron | 10:15 | |
| *** ajo_ has quit IRC | 10:17 | |
| *** sphoorti has quit IRC | 10:17 | |
| *** sphoorti has joined #openstack-neutron | 10:17 | |
| *** ajo_ has joined #openstack-neutron | 10:18 | |
| *** xuhanp has quit IRC | 10:18 | |
| *** b3nt_pin has joined #openstack-neutron | 10:18 | |
| *** b3nt_pin is now known as beagles | 10:18 | |
| *** ajo_ has quit IRC | 10:19 | |
| akamyshnikova | salv-orlando, Hello, can you find some time to continue our yesterday discussion about NULL on vlan_id? I've post answer to your comment at my change https://review.openstack.org/82089 | 10:19 |
| *** jgallard has quit IRC | 10:23 | |
| *** saju_m has quit IRC | 10:24 | |
| *** mwagner_lap has quit IRC | 10:26 | |
| *** luqas has quit IRC | 10:26 | |
| *** EmilienM has quit IRC | 10:28 | |
| *** EmilienM has joined #openstack-neutron | 10:28 | |
| *** mwagner_notHere has quit IRC | 10:28 | |
| *** overlayer has joined #openstack-neutron | 10:30 | |
| *** sphoorti has quit IRC | 10:30 | |
| *** sphoorti has joined #openstack-neutron | 10:31 | |
| *** ale___ has quit IRC | 10:36 | |
| *** ale___ has joined #openstack-neutron | 10:37 | |
| *** bvandenh has quit IRC | 10:40 | |
| *** sphoorti has quit IRC | 10:44 | |
| *** sphoorti has joined #openstack-neutron | 10:45 | |
| *** sphoorti has quit IRC | 10:46 | |
| *** ale___ has quit IRC | 10:48 | |
| *** sphoorti has joined #openstack-neutron | 10:48 | |
| *** amuller has joined #openstack-neutron | 10:51 | |
| *** sphoorti has quit IRC | 10:51 | |
| *** flwang has joined #openstack-neutron | 10:52 | |
| *** sphoorti has joined #openstack-neutron | 10:52 | |
| *** bvandenh has joined #openstack-neutron | 10:53 | |
| *** sphoorti has quit IRC | 10:54 | |
| *** sphoorti has joined #openstack-neutron | 10:55 | |
| *** sphoorti has quit IRC | 10:56 | |
| *** sphoorti has joined #openstack-neutron | 10:56 | |
| *** luqas has joined #openstack-neutron | 10:58 | |
| *** networkstatic is now known as networkstatic_zZ | 10:58 | |
| *** sphoorti has quit IRC | 11:04 | |
| *** linuxgeek_ has quit IRC | 11:04 | |
| *** nati_ueno has quit IRC | 11:04 | |
| *** xianghui_ has joined #openstack-neutron | 11:04 | |
| *** sphoorti has joined #openstack-neutron | 11:05 | |
| *** xianghui has quit IRC | 11:06 | |
| *** matsuhashi has quit IRC | 11:07 | |
| *** yamamoto has joined #openstack-neutron | 11:07 | |
| *** yamahata has quit IRC | 11:07 | |
| *** yamamoto has quit IRC | 11:08 | |
| *** saju_m has joined #openstack-neutron | 11:14 | |
| *** leseb has quit IRC | 11:19 | |
| *** morganfainberg is now known as morganfainberg_Z | 11:19 | |
| *** leseb has joined #openstack-neutron | 11:19 | |
| *** sphoorti has quit IRC | 11:21 | |
| *** leseb has quit IRC | 11:24 | |
| *** sphoorti has joined #openstack-neutron | 11:24 | |
| *** xianghui_ has quit IRC | 11:25 | |
| *** nati_ueno has joined #openstack-neutron | 11:28 | |
| *** pnavarro has quit IRC | 11:28 | |
| openstackgerrit | Evgeny Fedoruk proposed a change to openstack/neutron: Cancelling thread start while unit tests running https://review.openstack.org/81323 | 11:29 |
| *** matsuhashi has joined #openstack-neutron | 11:32 | |
| openstackgerrit | A change was merged to openstack/neutron: LBaaS: make device driver decide whether to deploy instance https://review.openstack.org/82514 | 11:33 |
| *** saju_m has quit IRC | 11:33 | |
| *** sphoorti has quit IRC | 11:34 | |
| *** pcm_ has joined #openstack-neutron | 11:34 | |
| *** sphoorti has joined #openstack-neutron | 11:35 | |
| *** saju_m has joined #openstack-neutron | 11:35 | |
| openstackgerrit | Oleg Bondarev proposed a change to openstack/neutron: LBaaS: haproxy driver should respect vip/pool admin state https://review.openstack.org/82749 | 11:37 |
| *** HenryG has joined #openstack-neutron | 11:39 | |
| *** dguitarbite has quit IRC | 11:39 | |
| *** mwagner_lap has joined #openstack-neutron | 11:42 | |
| *** sphoorti has quit IRC | 11:43 | |
| *** matsuhashi has quit IRC | 11:45 | |
| *** sphoorti has joined #openstack-neutron | 11:46 | |
| *** Longgeek has quit IRC | 11:46 | |
| *** sphoorti has quit IRC | 11:51 | |
| *** sphoorti has joined #openstack-neutron | 11:52 | |
| *** aryan has quit IRC | 11:53 | |
| *** baoli has joined #openstack-neutron | 11:54 | |
| *** sphoorti has quit IRC | 11:55 | |
| *** baoli has quit IRC | 11:56 | |
| *** baoli has joined #openstack-neutron | 11:56 | |
| *** pcm_ has quit IRC | 11:57 | |
| *** jgallard has joined #openstack-neutron | 11:57 | |
| *** mwagner_ has joined #openstack-neutron | 11:59 | |
| *** afazekas has quit IRC | 12:01 | |
| *** matsuhashi has joined #openstack-neutron | 12:05 | |
| *** sphoorti has joined #openstack-neutron | 12:05 | |
| *** sphoorti has quit IRC | 12:08 | |
| *** pnavarro has joined #openstack-neutron | 12:09 | |
| *** sballe has quit IRC | 12:09 | |
| *** alexpilotti has joined #openstack-neutron | 12:11 | |
| *** sphoorti has joined #openstack-neutron | 12:12 | |
| *** sphoorti has quit IRC | 12:14 | |
| *** pcm_ has joined #openstack-neutron | 12:14 | |
| *** leseb has joined #openstack-neutron | 12:14 | |
| *** sphoorti has joined #openstack-neutron | 12:15 | |
| *** afazekas has joined #openstack-neutron | 12:15 | |
| *** alexpilotti has quit IRC | 12:17 | |
| openstackgerrit | Oleg Bondarev proposed a change to openstack/neutron: LBaaS: haproxy driver should respect vip/pool admin state https://review.openstack.org/82749 | 12:19 |
| *** leseb has quit IRC | 12:19 | |
| openstackgerrit | Oleg Bondarev proposed a change to openstack/neutron: Delete disassociated floating ips on external network deletion https://review.openstack.org/53364 | 12:22 |
| *** lollipop has quit IRC | 12:25 | |
| *** sphoorti has quit IRC | 12:25 | |
| *** mestery has joined #openstack-neutron | 12:25 | |
| *** sballe has joined #openstack-neutron | 12:25 | |
| *** mestery has quit IRC | 12:26 | |
| *** sphoorti has joined #openstack-neutron | 12:26 | |
| *** mestery has joined #openstack-neutron | 12:26 | |
| *** sphoorti has quit IRC | 12:38 | |
| *** sphoorti has joined #openstack-neutron | 12:45 | |
| openstackgerrit | Sylvain Afchain proposed a change to openstack/neutron: L2 pop regular mac learning as parameter https://review.openstack.org/83053 | 12:45 |
| *** matsuhashi has quit IRC | 12:45 | |
| *** matsuhashi has joined #openstack-neutron | 12:45 | |
| *** leseb has joined #openstack-neutron | 12:47 | |
| *** matsuhashi has quit IRC | 12:50 | |
| *** luqas has quit IRC | 12:51 | |
| *** Jabadia has quit IRC | 12:51 | |
| *** sphoorti has quit IRC | 12:51 | |
| *** jecarey has quit IRC | 12:52 | |
| *** dims_ has quit IRC | 12:52 | |
| *** Jabadia has joined #openstack-neutron | 12:54 | |
| *** Jabadia has quit IRC | 12:54 | |
| *** Jabadia has joined #openstack-neutron | 12:54 | |
| *** Jabadia has quit IRC | 12:55 | |
| *** Jabadia has joined #openstack-neutron | 12:55 | |
| *** sphoorti has joined #openstack-neutron | 12:56 | |
| *** ramishra has joined #openstack-neutron | 12:58 | |
| *** xuhanp has joined #openstack-neutron | 12:58 | |
| *** sphoorti has quit IRC | 12:59 | |
| *** thuc has joined #openstack-neutron | 13:01 | |
| *** pradipta is now known as pradipta_away | 13:01 | |
| *** sphoorti has joined #openstack-neutron | 13:01 | |
| *** sphoorti has quit IRC | 13:02 | |
| *** thuc_ has joined #openstack-neutron | 13:02 | |
| *** sphoorti has joined #openstack-neutron | 13:03 | |
| *** sphoorti has quit IRC | 13:03 | |
| *** Jianyong has joined #openstack-neutron | 13:03 | |
| *** sphoorti has joined #openstack-neutron | 13:03 | |
| *** Jianyong has left #openstack-neutron | 13:04 | |
| *** nplanel_ has joined #openstack-neutron | 13:04 | |
| *** dims_ has joined #openstack-neutron | 13:05 | |
| *** _sweston_ has quit IRC | 13:05 | |
| *** thuc has quit IRC | 13:06 | |
| *** sphoorti has joined #openstack-neutron | 13:06 | |
| *** markmcclain has joined #openstack-neutron | 13:08 | |
| *** lker has joined #openstack-neutron | 13:09 | |
| *** pcm_ has quit IRC | 13:12 | |
| *** changbl has quit IRC | 13:17 | |
| *** matsuhashi has joined #openstack-neutron | 13:18 | |
| *** julim has joined #openstack-neutron | 13:18 | |
| *** alexpilotti has joined #openstack-neutron | 13:21 | |
| *** sphoorti has quit IRC | 13:23 | |
| ihrachys | markmcclain: here? can we consider https://review.openstack.org/#/c/82786/ for stable neutron before release? one of my patches was successfully merged to stable today in the first go, so probably the issue with gate doesn't completely block new merges | 13:25 |
| *** matsuhashi has quit IRC | 13:25 | |
| markmcclain | ihrachys: approved | 13:25 |
| *** matsuhashi has joined #openstack-neutron | 13:26 | |
| *** jistr has quit IRC | 13:26 | |
| *** matsuhashi has quit IRC | 13:26 | |
| markmcclain | ihrachys: it doesn't block them so much as they randomly fail enough to cause a backup in the gate | 13:26 |
| *** irenab has quit IRC | 13:26 | |
| *** matsuhashi has joined #openstack-neutron | 13:26 | |
| *** luqas has joined #openstack-neutron | 13:28 | |
| openstackgerrit | vaibhav_kale proposed a change to openstack/neutron: Changed the message line of RouterInUse class https://review.openstack.org/83374 | 13:29 |
| ihrachys | markmcclain: the gate is quite empty at the moment (6 tasks in all the queues, only 3 in neutron one), so hopefully possible benefit is greater than the risk of backup | 13:30 |
| *** dave_tucker is now known as dave_tucker_zzz | 13:30 | |
| ihrachys | markmcclain: ...and thanks for approval :) | 13:31 |
| *** thuc has joined #openstack-neutron | 13:33 | |
| *** xianghui has joined #openstack-neutron | 13:34 | |
| *** dave_tucker_zzz is now known as dave_tucker | 13:34 | |
| *** chandankumar_ has joined #openstack-neutron | 13:35 | |
| *** krtaylor has quit IRC | 13:35 | |
| *** thuc_ has quit IRC | 13:36 | |
| *** sphoorti has joined #openstack-neutron | 13:36 | |
| *** sphoorti has quit IRC | 13:36 | |
| *** yamahata has joined #openstack-neutron | 13:36 | |
| *** sphoorti has joined #openstack-neutron | 13:38 | |
| *** skraynev is now known as skraynev_afk | 13:38 | |
| *** tvardeman has joined #openstack-neutron | 13:39 | |
| *** sphoorti has quit IRC | 13:39 | |
| *** lker has quit IRC | 13:40 | |
| *** ramishra has quit IRC | 13:43 | |
| *** thuc_ has joined #openstack-neutron | 13:43 | |
| *** jistr has joined #openstack-neutron | 13:44 | |
| *** thuc has quit IRC | 13:46 | |
| *** linuxgeek_ has joined #openstack-neutron | 13:48 | |
| *** thuc_ has quit IRC | 13:48 | |
| *** Jabadia has quit IRC | 13:48 | |
| *** thedodd has joined #openstack-neutron | 13:48 | |
| openstackgerrit | Kyle Mestery proposed a change to openstack/neutron: Correct OVS VXLAN version check https://review.openstack.org/82931 | 13:49 |
| mestery | salv-orlando: I think I finally addressed all of nati_ueno's pedant comments, can I beg for one more review on this one from you? https://review.openstack.org/#/c/82931/ | 13:50 |
| *** rkukura has quit IRC | 13:50 | |
| *** sphoorti has joined #openstack-neutron | 13:52 | |
| *** mestery has quit IRC | 13:55 | |
| sphoorti | markmcclain: hello :) | 13:57 |
| *** nplanel_ has quit IRC | 13:58 | |
| markmcclain | sphoorti: hi | 13:59 |
| sphoorti | markmcclain: I made those changes as per your comments . Could you please have a look at them when free? | 14:00 |
| markmcclain | sphoorti: yes I'll take a look | 14:01 |
| sphoorti | thank you markmcclain | 14:01 |
| *** zigo has quit IRC | 14:01 | |
| *** zigo has joined #openstack-neutron | 14:03 | |
| *** krtaylor has joined #openstack-neutron | 14:06 | |
| *** markwash has joined #openstack-neutron | 14:06 | |
| *** banix has joined #openstack-neutron | 14:06 | |
| *** mordred has quit IRC | 14:08 | |
| *** yfried has quit IRC | 14:08 | |
| *** jorgem has joined #openstack-neutron | 14:10 | |
| *** thuc has joined #openstack-neutron | 14:11 | |
| *** gmurphy has joined #openstack-neutron | 14:13 | |
| openstackgerrit | Aaron Rosen proposed a change to openstack/neutron: Prevent cross plugging router ports from other tenants https://review.openstack.org/83391 | 14:19 |
| *** blogan has joined #openstack-neutron | 14:22 | |
| *** aryan has joined #openstack-neutron | 14:23 | |
| *** sphoorti has quit IRC | 14:23 | |
| *** roeyc has quit IRC | 14:25 | |
| *** amotoki has joined #openstack-neutron | 14:26 | |
| *** dims_ is now known as dims | 14:28 | |
| *** dims is now known as Guest68499 | 14:28 | |
| *** Guest68499 has quit IRC | 14:29 | |
| *** dims_ has joined #openstack-neutron | 14:29 | |
| *** carl_baldwin has joined #openstack-neutron | 14:31 | |
| *** jgrimm has joined #openstack-neutron | 14:36 | |
| *** sphoorti has joined #openstack-neutron | 14:37 | |
| *** jecarey has joined #openstack-neutron | 14:37 | |
| *** leseb_ has joined #openstack-neutron | 14:38 | |
| *** thuc has quit IRC | 14:38 | |
| *** thuc has joined #openstack-neutron | 14:39 | |
| *** leseb has quit IRC | 14:42 | |
| *** sweston has joined #openstack-neutron | 14:43 | |
| *** jobewan has joined #openstack-neutron | 14:43 | |
| *** thuc has quit IRC | 14:43 | |
| *** baffle has joined #openstack-neutron | 14:45 | |
| *** ramishra has joined #openstack-neutron | 14:46 | |
| *** chandankumar_ has quit IRC | 14:46 | |
| *** pcm_ has joined #openstack-neutron | 14:46 | |
| *** devlaps has joined #openstack-neutron | 14:51 | |
| *** jobewan has quit IRC | 14:51 | |
| *** jobewan has joined #openstack-neutron | 14:52 | |
| *** chandankumar_ has joined #openstack-neutron | 14:53 | |
| salv-orlando | kevinbenton: if you're not around I can see whether unit tests can be added to https://review.openstack.org/#/c/83190 | 14:54 |
| kevinbenton | salv-orlando: i will do it. i was just waiting for some sign from the nova side that this approach is okay | 14:55 |
| salv-orlando | I think they will ask for more unit tests anyway. | 14:56 |
| *** pcm_ has quit IRC | 14:56 | |
| *** pcm_ has joined #openstack-neutron | 14:56 | |
| *** packet has quit IRC | 14:57 | |
| *** otherwiseguy has joined #openstack-neutron | 14:58 | |
| *** thuc has joined #openstack-neutron | 15:00 | |
| *** mestery has joined #openstack-neutron | 15:01 | |
| *** networkstatic_zZ has quit IRC | 15:01 | |
| blogan | enikanorov: glossary addition, virtualized appliance | 15:02 |
| enikanorov__ | markmcclain: still need you eyes on https://review.openstack.org/#/c/81537/ | 15:02 |
| enikanorov__ | blogan: ok | 15:02 |
| *** julim has quit IRC | 15:02 | |
| blogan | thanks | 15:03 |
| *** julim has joined #openstack-neutron | 15:04 | |
| openstackgerrit | Sylvain Afchain proposed a change to openstack/neutron: L2 pop regular mac learning as parameter https://review.openstack.org/83053 | 15:06 |
| *** ramishra has quit IRC | 15:06 | |
| *** jorgem has quit IRC | 15:06 | |
| *** jorgem has joined #openstack-neutron | 15:07 | |
| *** rkukura has joined #openstack-neutron | 15:08 | |
| mestery | salv-orlando amotoki: Thanks for the +2s and +A. | 15:09 |
| salv-orlando | np | 15:09 |
| * mestery goes back to looking at the OVS function he's working on at the OVS Hackathon. | 15:09 | |
| amotoki | mestery: np. it is a good improvement :) | 15:10 |
| *** ramishra has joined #openstack-neutron | 15:10 | |
| mestery | amotoki: Yes! And it also closes a critical bug as well. :) | 15:10 |
| enikanorov__ | OVS hackathon, cool | 15:12 |
| enikanorov__ | is in c or c++? | 15:13 |
| *** ramishra has quit IRC | 15:14 | |
| markmcclain | enikanorov__: looking at that review why are we not addressing the bug the lbaas is calling existing from within a scoped namespace? | 15:16 |
| *** yfried has joined #openstack-neutron | 15:16 | |
| *** matsuhashi has quit IRC | 15:18 | |
| enikanorov__ | markmcclain: it's common method garbage_collect_namespace | 15:18 |
| *** pnavarro has quit IRC | 15:18 | |
| enikanorov__ | markmcclain: from ip_lib | 15:18 |
| enikanorov__ | and the previous fix has introduced regression there | 15:18 |
| markmcclain | ok that is not explained anywhere in the commit message or the fix | 15:21 |
| markmcclain | this fixes a broken garbage_collection | 15:21 |
| enikanorov__ | yep, i can add this to commit message | 15:22 |
| enikanorov__ | can you please remove -2 for now? | 15:22 |
| *** chandankumar_ has quit IRC | 15:24 | |
| *** safchain has quit IRC | 15:28 | |
| *** markwash has quit IRC | 15:28 | |
| *** evgenyf has quit IRC | 15:29 | |
| markmcclain | also added comment about the execute call itself | 15:32 |
| markmcclain | enikanorov__: should be using the one local to the class | 15:32 |
| ihrachys | markmcclain: can we get this 'full sync' from oslo in icehouse: https://review.openstack.org/#/c/80998/ ? Or now that we're about to release rc1, will I need to do specific 'cherry-picks' now? | 15:33 |
| *** _cjones_ has joined #openstack-neutron | 15:39 | |
| *** sacharya has joined #openstack-neutron | 15:41 | |
| *** luqas has quit IRC | 15:41 | |
| *** _cjones_ has quit IRC | 15:42 | |
| *** _cjones_ has joined #openstack-neutron | 15:42 | |
| openstackgerrit | Thierry Carrez proposed a change to openstack/neutron: Open Juno development https://review.openstack.org/83434 | 15:43 |
| *** thuc_ has joined #openstack-neutron | 15:43 | |
| *** saju_m has quit IRC | 15:44 | |
| *** luqas has joined #openstack-neutron | 15:44 | |
| amotoki | ihrachys: cherry-pick is recommended. I talked with markmcclain about the same topic. It is the last moment of the release cycle. We need to sync only modules with specific bugs. | 15:45 |
| mestery | enikanorov__: OVS is written in C, though there are also python bindings. :) | 15:45 |
| ihrachys | amotoki: I synced specific modules only | 15:45 |
| ihrachys | amotoki: but should I cherry pick *from those modules*? | 15:46 |
| *** bashok has quit IRC | 15:46 | |
| ihrachys | amotoki: and btw markmcclain approved the patch, and it was already sent to gate | 15:46 |
| *** evgenyf has joined #openstack-neutron | 15:47 | |
| *** thuc has quit IRC | 15:47 | |
| *** otherwiseguy has quit IRC | 15:47 | |
| amotoki | ihrachys: rpc sync patch? | 15:47 |
| ihrachys | amotoki: yes | 15:47 |
| ihrachys | amotoki: it syncs rpc + gettextutils (as a dep) | 15:48 |
| *** thuc_ has quit IRC | 15:48 | |
| amotoki | ihrachys: ah... full sync limited to related modules sounds reasonable to me. | 15:48 |
| ihrachys | amotoki: in stable branches, we cherry-pick specific changes from specifc modules | 15:48 |
| ihrachys | amotoki: that's why I asked | 15:48 |
| *** asadoughi has quit IRC | 15:50 | |
| openstackgerrit | Yves-Gwenael Bourhis proposed a change to openstack/neutron: Make dnsmasq aware of all names https://review.openstack.org/52930 | 15:51 |
| *** marun_afk has quit IRC | 15:53 | |
| *** amir1 has joined #openstack-neutron | 15:55 | |
| *** amir1 is now known as asadoughi | 15:55 | |
| *** rotbeard has quit IRC | 15:55 | |
| *** zhipeng has joined #openstack-neutron | 15:55 | |
| kevinbenton | salv-orlando: new patch https://review.openstack.org/#/c/83190/ | 15:56 |
| *** xuhanp has quit IRC | 15:56 | |
| salv-orlando | thanks kevinbenton | 15:56 |
| *** asadoughi has quit IRC | 15:56 | |
| *** asadoughi has joined #openstack-neutron | 15:57 | |
| *** sacharya has quit IRC | 15:57 | |
| *** otherwiseguy has joined #openstack-neutron | 15:59 | |
| *** changbl has joined #openstack-neutron | 16:00 | |
| *** tvardeman has quit IRC | 16:00 | |
| *** mlavalle has joined #openstack-neutron | 16:00 | |
| *** tvardeman has joined #openstack-neutron | 16:01 | |
| *** claudiub_ has joined #openstack-neutron | 16:03 | |
| *** amuller has quit IRC | 16:03 | |
| *** SumitNaiksatam has joined #openstack-neutron | 16:04 | |
| *** xianghui has quit IRC | 16:04 | |
| *** evgenyf has quit IRC | 16:05 | |
| *** zhipeng has quit IRC | 16:10 | |
| *** dims_ has quit IRC | 16:10 | |
| *** emagana has joined #openstack-neutron | 16:10 | |
| *** zhipeng has joined #openstack-neutron | 16:11 | |
| openstackgerrit | enikanorov proposed a change to openstack/neutron: Fix namespace exist() method https://review.openstack.org/81537 | 16:12 |
| *** sphoorti has quit IRC | 16:13 | |
| *** sphoorti has joined #openstack-neutron | 16:13 | |
| jlibosva | markmcclain: hi, can I ask you please for review on https://review.openstack.org/#/c/83008/ ? | 16:15 |
| jlibosva | it makes grenade pass with neutron - https://jenkins04.openstack.org/job/check-grenade-dsvm/6435/ | 16:15 |
| *** sphoorti has quit IRC | 16:18 | |
| flwang | greetings | 16:20 |
| *** sphoorti has joined #openstack-neutron | 16:21 | |
| *** sphoorti has quit IRC | 16:21 | |
| *** thuc_ has joined #openstack-neutron | 16:21 | |
| *** carl_baldwin_ has joined #openstack-neutron | 16:21 | |
| *** thuc_ has quit IRC | 16:22 | |
| *** sphoorti has joined #openstack-neutron | 16:22 | |
| markmcclain | jlibosva: looking | 16:22 |
| * jlibosva thanking | 16:22 | |
| *** thuc_ has joined #openstack-neutron | 16:22 | |
| flwang | is there any way to know the public and private traffic? | 16:23 |
| flwang | is it possible to tag different network and then handle the notifications in ceilometer? | 16:24 |
| *** nati_ueno has quit IRC | 16:24 | |
| *** carl_baldwin has quit IRC | 16:25 | |
| *** carl_baldwin_ is now known as carl_baldwin | 16:25 | |
| markmcclain | flwang: for metering? | 16:25 |
| flwang | markmcclain: yes | 16:25 |
| *** dims_ has joined #openstack-neutron | 16:25 | |
| markmcclain | flwang: look at the metering extension for l3 | 16:26 |
| flwang | markmcclain: from the end user perspective, it would be great to distinguish public and private traffic (communication between VMs in the same availability zone and tenant) | 16:26 |
| flwang | markmcclain: could you please show me a link? I'm not a networking guy :) | 16:27 |
| *** afazekas has quit IRC | 16:27 | |
| openstackgerrit | A change was merged to openstack/neutron: Import request_id middleware bug fix from oslo https://review.openstack.org/83080 | 16:28 |
| *** yfried has quit IRC | 16:28 | |
| openstackgerrit | A change was merged to openstack/neutron: Correct OVS VXLAN version check https://review.openstack.org/82931 | 16:28 |
| markmcclain | flwang: http://docs.openstack.org/api/openstack-network/2.0/content/metering-ext.html | 16:29 |
| flwang | markmcclain: cool, thanks a lot | 16:29 |
| *** sweston has quit IRC | 16:31 | |
| *** yfried has joined #openstack-neutron | 16:31 | |
| *** sphoorti has quit IRC | 16:32 | |
| *** sphoorti has joined #openstack-neutron | 16:33 | |
| *** sphoorti has quit IRC | 16:33 | |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Updated from global requirements https://review.openstack.org/83474 | 16:34 |
| *** sbalukoff has quit IRC | 16:34 | |
| *** sphoorti has joined #openstack-neutron | 16:34 | |
| *** sacharya has joined #openstack-neutron | 16:36 | |
| flwang | markmcclain: so IIUC, on Neutron side, we just need to create metering label and rule for both public and private network, is it? | 16:38 |
| flwang | then the notifications sent by Neutron will include the metering label, right? | 16:39 |
| *** singhs has joined #openstack-neutron | 16:39 | |
| *** marun has joined #openstack-neutron | 16:40 | |
| markmcclain | flwang: if you're running the script to report on stats | 16:41 |
| *** zigo has quit IRC | 16:41 | |
| flwang | markmcclain: ^the script^ you mentioned is the metering extension for l3, right? | 16:42 |
| *** sphoorti has quit IRC | 16:43 | |
| markmcclain | neutron-metering-agent | 16:43 |
| *** sphoorti has joined #openstack-neutron | 16:44 | |
| *** sphoorti has quit IRC | 16:44 | |
| flwang | markmcclain: got it. so simple! | 16:45 |
| flwang | markmcclain: then on the Ceilometer side, we can just use the metering label to distinguish the traffic of public or private network, correct? | 16:46 |
| markmcclain | yes | 16:47 |
| markmcclain | a few deployers are using this extension to differentiate billing for intra datacenter traffic vs traffic that exits the DC | 16:47 |
| flwang | markmcclain: glad to know that. thank you so much! | 16:48 |
| *** jlibosva has quit IRC | 16:49 | |
| *** leseb_ has quit IRC | 16:49 | |
| *** leseb has joined #openstack-neutron | 16:50 | |
| openstackgerrit | A change was merged to openstack/neutron: VPNaaS support for VPN service admin state change and reporting https://review.openstack.org/81124 | 16:53 |
| *** yfried has quit IRC | 16:54 | |
| openstackgerrit | A change was merged to openstack/neutron: nec plugin: allow to delete resource with ERROR status https://review.openstack.org/82143 | 16:54 |
| *** yfried_ has joined #openstack-neutron | 16:54 | |
| *** leseb has quit IRC | 16:54 | |
| *** carl_baldwin has quit IRC | 16:54 | |
| *** zhipeng has quit IRC | 16:55 | |
| *** yfried_ is now known as yfried | 16:56 | |
| *** carl_baldwin has joined #openstack-neutron | 16:57 | |
| *** carl_baldwin has quit IRC | 16:58 | |
| *** otherwiseguy has quit IRC | 16:59 | |
| *** SumitNaiksatam has quit IRC | 17:00 | |
| *** zhipeng has joined #openstack-neutron | 17:01 | |
| *** dguitarbite has joined #openstack-neutron | 17:01 | |
| *** harlowja_away is now known as harlowja | 17:08 | |
| *** leseb has joined #openstack-neutron | 17:09 | |
| *** thedodd has quit IRC | 17:09 | |
| *** alexpilotti_ has joined #openstack-neutron | 17:12 | |
| *** afazekas has joined #openstack-neutron | 17:13 | |
| *** marun has quit IRC | 17:15 | |
| *** alexpilotti has quit IRC | 17:15 | |
| *** alexpilotti_ is now known as alexpilotti | 17:15 | |
| *** nati_ueno has joined #openstack-neutron | 17:15 | |
| nati_ueno | hi | 17:17 |
| *** ale___ has joined #openstack-neutron | 17:17 | |
| *** zhipeng has quit IRC | 17:18 | |
| *** hemanthravi has joined #openstack-neutron | 17:18 | |
| jogo | markmcclain: https://review.openstack.org/#/c/83055/2/neutron/db/flavors_db.py | 17:18 |
| openstackgerrit | Phil Day proposed a change to openstack/python-neutronclient: Allow user ID for authentication https://review.openstack.org/78596 | 17:18 |
| *** ygbo has quit IRC | 17:19 | |
| jogo | apparently some neutron folks are using bad copyrights | 17:19 |
| jogo | heh and you use author tags | 17:19 |
| *** bvandenh has quit IRC | 17:20 | |
| *** armitage81 has joined #openstack-neutron | 17:21 | |
| *** armitage81 has quit IRC | 17:21 | |
| *** armitage81 has joined #openstack-neutron | 17:21 | |
| *** singhs has quit IRC | 17:24 | |
| *** singhs has joined #openstack-neutron | 17:24 | |
| *** SumitNaiksatam has joined #openstack-neutron | 17:25 | |
| *** leseb has quit IRC | 17:26 | |
| *** morganfainberg_Z is now known as morganfainberg | 17:26 | |
| *** HenryG_ has joined #openstack-neutron | 17:28 | |
| kevinbenton | salv-orlando, nachi_ueno, markmcclain: devstack tests came back for sec groups and it looks good | 17:28 |
| kevinbenton | https://review.openstack.org/#/c/83281/ | 17:28 |
| *** bada has quit IRC | 17:28 | |
| nati_ueno | kevinbenton: woot | 17:29 |
| nati_ueno | russellb: around? | 17:29 |
| claudiub_ | rkukura: hello | 17:30 |
| claudiub_ | rkukura: if you have time today, to look at the commits we talked about yesterday, i'd be most grateful. :D | 17:31 |
| markmcclain | jogo: ugh on these days we'll file headers sorted | 17:31 |
| markmcclain | fortunately the code is WIP | 17:31 |
| *** dims_ has quit IRC | 17:32 | |
| claudiub_ | rkukura: here are the commits: https://review.openstack.org/#/c/83222/1 https://review.openstack.org/#/c/83223/ | 17:32 |
| *** banix has quit IRC | 17:34 | |
| *** zhipeng has joined #openstack-neutron | 17:34 | |
| jogo | markmcclain: yeah, I grep shows there are a lot of copyright headers in the code that are assigned to the foundation | 17:34 |
| jogo | markmcclain: also IMHO you should drop all author tags as well. but that is a seperate issue | 17:34 |
| jogo | markmcclain: so the only part of copyright header that is needed is the apache2 section not who owns it | 17:35 |
| *** networkstatic has joined #openstack-neutron | 17:35 | |
| rkukura | claudiub_: Will try | 17:36 |
| claudiub_ | rkukura: thanks. :) | 17:37 |
| *** luqas has quit IRC | 17:37 | |
| *** HenryG has quit IRC | 17:37 | |
| *** zigo has joined #openstack-neutron | 17:39 | |
| *** claudiub_ has quit IRC | 17:41 | |
| kevinbenton | salv-orlando: ping | 17:41 |
| russellb | nati_ueno: seemed there was some question yesterday about which patch we should really go with ... | 17:41 |
| russellb | nati_ueno: what's the current status? | 17:42 |
| nati_ueno | russellb: OK we decides fix both of neutron and nova. The neutron side patch is already meregd. | 17:42 |
| nati_ueno | russellb: https://review.openstack.org/#/c/83190/7 <-- This is nova side | 17:42 |
| russellb | what's the neutron side for reference | 17:43 |
| kevinbenton | russellb: i think we've mostly settled that this is the preferred approach as it doesn't enforce hybrid mode on plugins that don't want it or deployments that don't want firewalls | 17:43 |
| nati_ueno | russellb: This is a tempest run result with this patch + security group negative test https://review.openstack.org/#/c/83281/ | 17:43 |
| kevinbenton | russellb: https://review.openstack.org/#/c/83280/ | 17:43 |
| kevinbenton | neutron patch ^^ | 17:44 |
| *** mwagner_ has quit IRC | 17:44 | |
| *** banix has joined #openstack-neutron | 17:44 | |
| yfried | salv-orlando: I wonder - is it possible to have an api command that's for debug purposes only? | 17:44 |
| nati_ueno | russellb: In order to remove side effects, we take the way to add vif_hybrid_plugging | 17:45 |
| russellb | tested successfully? | 17:45 |
| nati_ueno | russellb: yes ->> https://review.openstack.org/#/c/83281/ | 17:45 |
| yfried | salv-orlando: like a request for neutron to dump all namespace data from all nodes | 17:45 |
| nati_ueno | russellb: 83281 is a devstack patch run with nova patch + security group negative test | 17:45 |
| kevinbenton | 83281 is a devstack patch that references the negative test case enabled in tempest and this nova patch | 17:45 |
| nati_ueno | russellb: 83281 passed Jenkins | 17:46 |
| *** marun has joined #openstack-neutron | 17:46 | |
| yfried | marun: ^^ | 17:46 |
| marun | yfried: Missed that, can you say again? | 17:47 |
| *** dims_ has joined #openstack-neutron | 17:47 | |
| yfried | marun: I wonder - is it possible to have an api command that's for debug purposes only? | 17:48 |
| yfried | marun: like a request for neutron to dump all namespace data from all nodes | 17:48 |
| marun | yfried: Anything is possible :) | 17:48 |
| yfried | marun: btw - didn't forget my promise to draft test plans - I'm just over-swamped atm | 17:48 |
| marun | yfried: A use case would be helpful in determining whether it was worth doing. | 17:49 |
| marun | yfried: No worries, busy busy time. | 17:49 |
| yfried | marun: tempset has a debug method that dumps all ip-netns data, but it's only for AIO | 17:50 |
| russellb | neat | 17:50 |
| russellb | that's a cool hack to get testing on cross project patches | 17:50 |
| russellb | don't think i'd seen someone do that before | 17:50 |
| marun | yfried: I think having 'health check' support from the agents would certainly be useful. | 17:50 |
| yfried | marun: define health-check | 17:50 |
| marun | russellb: the devstack trick? | 17:50 |
| yfried | marun: problem is - all node specifc operations are outside tempest scope | 17:51 |
| kevinbenton | russellb: saw aarosen do it | 17:51 |
| russellb | cool | 17:51 |
| russellb | i like it | 17:51 |
| marun | yfried: What is the use case though? I.e why is all that info useful? | 17:51 |
| nati_ueno | russellb: yeah, this make life eary :) | 17:52 |
| salv-orlando | yfried: you can propose a blueprint for operator debugging if you wish. I see some uses for it | 17:52 |
| *** marun has quit IRC | 17:52 | |
| kevinbenton | russellb: do you have any other code change suggestions or should I update the commit message now? | 17:52 |
| russellb | kevinbenton: one more tiny thing, one sec | 17:52 |
| *** marun has joined #openstack-neutron | 17:52 | |
| yfried | marun: for network related failures, you want to get the network data dumped (as the actual nodes are scrapped post test) | 17:53 |
| salv-orlando | pretty much that amounts to let jenkins do what we normally do in our environment | 17:53 |
| russellb | kevinbenton: k updated | 17:53 |
| salv-orlando | the only downside is that there will be a stale patch, but anyone of us has tons of them, so it probably doesn't matter | 17:53 |
| marun | yfried: the use case is important | 17:53 |
| *** carl_baldwin has joined #openstack-neutron | 17:53 | |
| marun | yfried: An sdn controller already provides a good deal of the necessary introspection, and reproducing that in neutron is of questionable value | 17:54 |
| *** mlavalle has quit IRC | 17:54 | |
| yfried | marun: "sdn controller"? | 17:55 |
| *** marun_ has joined #openstack-neutron | 17:56 | |
| kevinbenton | russellb: patch updated | 17:56 |
| yfried | marun_: sdn controller? | 17:57 |
| *** dims_ has quit IRC | 17:57 | |
| russellb | kevinbenton: k, +2 | 17:57 |
| marun_ | yfried: I'd recommend reading up on it. | 17:57 |
| marun_ | yfried: Lpeer and amuller could provide pointers | 17:58 |
| *** marun has quit IRC | 17:59 | |
| *** marun_ is now known as marun | 17:59 | |
| yfried | is it a dependency for neutron? can it be used with to provide data for tempest failures at the gate? if not, i don't see how this solves the problem | 17:59 |
| kevinbenton | salv-orlando: with regards to your comment about the unit test for no filters with neutron, I believe that is already covered. _assertTypeAndMacEquals checks the filter count | 17:59 |
| marun | yfried: Essentially, though, neutron's open source option can configure the data plane but doesn't provide much in the way of visibility | 17:59 |
| salv-orlando | ok sounds good to me then | 17:59 |
| marun | yfried: A proper sdn controller remedies that - | 17:59 |
| marun | Brb | 18:00 |
| *** pcm_ has quit IRC | 18:00 | |
| *** linuxgeek_ has quit IRC | 18:00 | |
| *** _tziOm has quit IRC | 18:00 | |
| *** linuxgeek_ has joined #openstack-neutron | 18:00 | |
| kevinbenton | russellb: thanks, do we need to find another nova core to check it out, or will you approve? | 18:01 |
| russellb | kevinbenton: need another | 18:01 |
| *** pcm_ has joined #openstack-neutron | 18:01 | |
| russellb | i asked in -nova | 18:01 |
| russellb | seeing if someone jumps on it | 18:01 |
| kevinbenton | russellb: ok, thanks | 18:01 |
| russellb | if not, i'll poke people directly later today | 18:01 |
| *** thuc has joined #openstack-neutron | 18:01 | |
| nati_ueno | russellb: Thanks! | 18:03 |
| marun | salv-orlando: Btw, hope to have wip later today for trapping undesired eventlet yield | 18:03 |
| russellb | thanks to all of you for working together to get this knocked out | 18:03 |
| russellb | much appreciated | 18:03 |
| salv-orlando | marun: ok great | 18:04 |
| *** thuc_ has quit IRC | 18:04 | |
| *** afazekas has quit IRC | 18:05 | |
| *** overlayer has quit IRC | 18:06 | |
| *** jistr has quit IRC | 18:07 | |
| *** marun_ has joined #openstack-neutron | 18:08 | |
| *** bjornar has joined #openstack-neutron | 18:08 | |
| *** marun has quit IRC | 18:08 | |
| *** marun_ is now known as marun | 18:08 | |
| *** marun has quit IRC | 18:08 | |
| *** thedodd has joined #openstack-neutron | 18:09 | |
| *** marun has joined #openstack-neutron | 18:09 | |
| *** marun has quit IRC | 18:11 | |
| *** dims has joined #openstack-neutron | 18:11 | |
| *** baoli has quit IRC | 18:16 | |
| *** marun has joined #openstack-neutron | 18:19 | |
| *** jgallard has quit IRC | 18:21 | |
| *** marun has quit IRC | 18:21 | |
| *** _cjones__ has joined #openstack-neutron | 18:26 | |
| *** _cjones_ has quit IRC | 18:29 | |
| *** markwash has joined #openstack-neutron | 18:30 | |
| *** markwash has quit IRC | 18:30 | |
| *** markwash has joined #openstack-neutron | 18:31 | |
| *** jpich has quit IRC | 18:32 | |
| *** afazekas has joined #openstack-neutron | 18:32 | |
| *** _cjones__ has quit IRC | 18:33 | |
| openstackgerrit | enikanorov proposed a change to openstack/neutron: Fix namespace exist() method https://review.openstack.org/81537 | 18:34 |
| *** _cjones__ has joined #openstack-neutron | 18:35 | |
| *** _cjones__ has quit IRC | 18:35 | |
| *** _cjones__ has joined #openstack-neutron | 18:36 | |
| *** drjones has joined #openstack-neutron | 18:36 | |
| *** drjones is now known as _cjones__ | 18:37 | |
| *** thuc has quit IRC | 18:37 | |
| *** thuc has joined #openstack-neutron | 18:38 | |
| *** _cjones__ has quit IRC | 18:38 | |
| *** drjones has joined #openstack-neutron | 18:38 | |
| *** drjones is now known as _cjones__ | 18:38 | |
| *** _cjones__ is now known as _cjones_ | 18:39 | |
| *** marun has joined #openstack-neutron | 18:40 | |
| *** jprovazn has quit IRC | 18:42 | |
| *** thuc has quit IRC | 18:42 | |
| *** yfried has quit IRC | 18:51 | |
| *** yfried has joined #openstack-neutron | 18:51 | |
| *** nplanel has quit IRC | 18:54 | |
| *** tvardeman has quit IRC | 18:55 | |
| *** TrevorV has joined #openstack-neutron | 18:55 | |
| *** ale___ has quit IRC | 18:58 | |
| *** sacharya has quit IRC | 19:00 | |
| *** ale has joined #openstack-neutron | 19:02 | |
| *** sbalukoff has joined #openstack-neutron | 19:03 | |
| *** jp_at_hp has quit IRC | 19:04 | |
| *** ale has quit IRC | 19:04 | |
| *** sacharya has joined #openstack-neutron | 19:05 | |
| *** terrylhowe has joined #openstack-neutron | 19:08 | |
| *** terrylhowe has left #openstack-neutron | 19:08 | |
| *** jorgem has quit IRC | 19:18 | |
| *** jorgem1 has joined #openstack-neutron | 19:18 | |
| *** jorgem1 is now known as jorgem | 19:18 | |
| *** amotoki has quit IRC | 19:19 | |
| *** otherwiseguy has joined #openstack-neutron | 19:19 | |
| *** hemanthravi has quit IRC | 19:22 | |
| *** markwash has quit IRC | 19:24 | |
| *** tziOm has joined #openstack-neutron | 19:26 | |
| *** sacharya has quit IRC | 19:28 | |
| *** jorgem1 has joined #openstack-neutron | 19:30 | |
| *** jorgem has quit IRC | 19:30 | |
| *** thuc has joined #openstack-neutron | 19:33 | |
| *** thuc has quit IRC | 19:34 | |
| *** thuc has joined #openstack-neutron | 19:34 | |
| *** BuSerD has joined #openstack-neutron | 19:36 | |
| openstackgerrit | rcurran proposed a change to openstack/neutron: ML2 Cisco Nexus MD: Support portchannel interfaces https://review.openstack.org/83546 | 19:39 |
| openstackgerrit | rcurran proposed a change to openstack/neutron: ML2 Cisco Nexus MD: Support portchannel interfaces https://review.openstack.org/83546 | 19:42 |
| *** beagles is now known as beagles_brb | 19:43 | |
| *** nplanel has joined #openstack-neutron | 19:49 | |
| *** dave_tucker is now known as dave_tucker_zzz | 19:50 | |
| *** dfarrell07 has joined #openstack-neutron | 19:51 | |
| *** zzelle has joined #openstack-neutron | 19:51 | |
| *** emagana has quit IRC | 19:55 | |
| *** dguitarbite has quit IRC | 19:56 | |
| *** rkukura has quit IRC | 19:59 | |
| *** baoli has joined #openstack-neutron | 19:59 | |
| *** baoli has quit IRC | 20:00 | |
| *** baoli has joined #openstack-neutron | 20:00 | |
| *** gdubreui has joined #openstack-neutron | 20:01 | |
| *** HenryG_ has quit IRC | 20:03 | |
| *** HenryG has joined #openstack-neutron | 20:03 | |
| *** harlowja is now known as harlowja_away | 20:03 | |
| *** beagles_brb is now known as beagles | 20:09 | |
| *** HenryG has quit IRC | 20:10 | |
| *** pcm_ has quit IRC | 20:11 | |
| *** afazekas has quit IRC | 20:12 | |
| *** sacharya has joined #openstack-neutron | 20:14 | |
| *** sacharya has quit IRC | 20:16 | |
| *** sacharya has joined #openstack-neutron | 20:17 | |
| *** TrevorV has quit IRC | 20:17 | |
| *** networkstatic has quit IRC | 20:18 | |
| *** packet has joined #openstack-neutron | 20:19 | |
| *** tvardeman has joined #openstack-neutron | 20:20 | |
| *** leseb has joined #openstack-neutron | 20:21 | |
| *** julim has quit IRC | 20:29 | |
| *** jorgem has joined #openstack-neutron | 20:30 | |
| *** jorgem1 has quit IRC | 20:30 | |
| *** manishg has joined #openstack-neutron | 20:31 | |
| *** jorgem has quit IRC | 20:31 | |
| *** jorgem1 has joined #openstack-neutron | 20:31 | |
| *** jorgem1 is now known as jorgem | 20:31 | |
| *** leseb has quit IRC | 20:34 | |
| *** harlowja_away is now known as harlowja | 20:35 | |
| *** EmilienM has quit IRC | 20:42 | |
| *** ale has joined #openstack-neutron | 20:42 | |
| *** EmilienM has joined #openstack-neutron | 20:43 | |
| *** ale___ has joined #openstack-neutron | 20:46 | |
| *** ale has quit IRC | 20:46 | |
| *** ale___ has quit IRC | 20:47 | |
| *** ale has joined #openstack-neutron | 20:47 | |
| *** leseb has joined #openstack-neutron | 20:49 | |
| *** mlavalle has joined #openstack-neutron | 20:49 | |
| *** mlavalle has quit IRC | 20:49 | |
| *** ale___ has joined #openstack-neutron | 20:50 | |
| *** sacharya1 has joined #openstack-neutron | 20:50 | |
| *** ale has quit IRC | 20:52 | |
| *** sacharya has quit IRC | 20:53 | |
| *** mlavalle has joined #openstack-neutron | 20:53 | |
| *** thedodd has quit IRC | 21:09 | |
| *** mwagner_lap is now known as mwagner_bbl | 21:11 | |
| *** pnavarro has joined #openstack-neutron | 21:15 | |
| *** mwagner_bbl has quit IRC | 21:16 | |
| openstackgerrit | A change was merged to openstack/neutron: Make dnsmasq aware of all names https://review.openstack.org/52930 | 21:18 |
| *** leseb has quit IRC | 21:21 | |
| *** jorgem has quit IRC | 21:22 | |
| *** jorgem has joined #openstack-neutron | 21:25 | |
| *** tvardeman has quit IRC | 21:26 | |
| *** jorgem1 has joined #openstack-neutron | 21:27 | |
| *** dims has quit IRC | 21:28 | |
| *** jorgem has quit IRC | 21:29 | |
| *** jorgem has joined #openstack-neutron | 21:34 | |
| *** jorgem1 has quit IRC | 21:35 | |
| *** djoreilly has quit IRC | 21:40 | |
| *** leseb has joined #openstack-neutron | 21:41 | |
| *** dims has joined #openstack-neutron | 21:43 | |
| *** markwash has joined #openstack-neutron | 21:43 | |
| *** sungju has joined #openstack-neutron | 22:04 | |
| *** yamahata has quit IRC | 22:06 | |
| *** changbl has quit IRC | 22:07 | |
| *** ale___ has quit IRC | 22:07 | |
| *** rkukura has joined #openstack-neutron | 22:13 | |
| *** networkstatic has joined #openstack-neutron | 22:14 | |
| *** pcm_ has joined #openstack-neutron | 22:19 | |
| *** alexpilotti_ has joined #openstack-neutron | 22:20 | |
| *** alexpilotti has quit IRC | 22:22 | |
| *** alexpilotti_ is now known as alexpilotti | 22:22 | |
| *** manishg has quit IRC | 22:22 | |
| *** leseb has quit IRC | 22:23 | |
| *** sungju has quit IRC | 22:24 | |
| *** pcm_ has quit IRC | 22:26 | |
| *** pcm_ has joined #openstack-neutron | 22:27 | |
| *** bjornar has quit IRC | 22:27 | |
| marun | comstud: ping | 22:28 |
| comstud | marun: pong | 22:28 |
| marun | comstud: I'm afraid I didn't get a chance to work on 'noyield' until today | 22:29 |
| marun | comstud: It looks trivial if I'm not missing something. | 22:29 |
| marun | mock.patch('eventlet.hubs.hub.BaseHub.switch') | 22:29 |
| marun | with this mocked out, no yielding is possible | 22:29 |
| marun | (I must be missing something though) | 22:29 |
| kevinbenton | marun: https://review.openstack.org/#/c/81709/4/neutron/openstack/common/lockutils.py | 22:30 |
| comstud | that stops yields to the hub. but there's a couple of direct yields from one greenthread to another | 22:30 |
| comstud | in the queue code | 22:30 |
| *** baoli has quit IRC | 22:30 | |
| comstud | So yes, that covers probably 99% of the cases | 22:30 |
| comstud | i doubt you'd use anything that would use Queue while in a transaction | 22:31 |
| comstud | heh | 22:31 |
| marun | comstud: hmmm, will look... | 22:31 |
| comstud | though note.. | 22:32 |
| comstud | you have to mock the object, not the class, right? | 22:32 |
| comstud | ie, get_hub().switch | 22:32 |
| *** beagles has quit IRC | 22:32 | |
| comstud | which will be the current Hub object | 22:32 |
| marun | comstud: I don't think it matters | 22:32 |
| marun | comstud: er | 22:33 |
| marun | comstud: question - why would it matter? | 22:33 |
| comstud | If you mock the class, you'd have to do it before it's instantiated | 22:33 |
| marun | comstud: I don't think that's true with mock | 22:33 |
| comstud | maybe | 22:33 |
| marun | comstud: normal monkey patching, you're definitely correct | 22:33 |
| comstud | maybe there's some magic I'm not aware of | 22:33 |
| marun | comstud: the mocking libraries are pretty smart about it | 22:33 |
| comstud | I have tried it :) | 22:34 |
| marun | comstud: but it doesn't hurt to be careful | 22:34 |
| comstud | er | 22:34 |
| comstud | I haven't tried it | 22:34 |
| marun | comstud: I'm not even sure if using mock is a good idea. | 22:34 |
| marun | comstud: might be better to use less magic | 22:34 |
| comstud | I'd probably just monkey patch manually | 22:34 |
| comstud | the get_hub() object | 22:34 |
| kevinbenton | marun: it would be the only thing that brought the mock library into deployments :-) | 22:34 |
| comstud | using mock outside of the tests seems... wrong | 22:34 |
| comstud | hehe | 22:35 |
| comstud | Here's the other case I was referring to: | 22:35 |
| comstud | https://github.com/eventlet/eventlet/blob/master/eventlet/queue.py#L101 | 22:35 |
| marun | and what type is self.greenlet? | 22:35 |
| comstud | greenlet.greenlet | 22:35 |
| comstud | (or it may be a eventlet.greenthread.Greenthread which is subclassed from greenlet.greenlet) | 22:36 |
| comstud | Here's one place that Waiter.switch() is called: https://github.com/eventlet/eventlet/blob/master/eventlet/queue.py#L240 | 22:36 |
| comstud | But there are other examples after that too | 22:37 |
| comstud | The queue code bypasses the hub and directly switches to a waiting greenthread if necessary | 22:37 |
| marun | is it only the queue code that does that? | 22:37 |
| marun | and where does rlock fit in? | 22:37 |
| comstud | i'm checking | 22:37 |
| comstud | threading.RLock is replaced by eventlet.semaphore.Semaphore | 22:38 |
| comstud | which I think just yields to the hub | 22:38 |
| comstud | yeah | 22:38 |
| comstud | don't let this: https://github.com/eventlet/eventlet/blob/master/eventlet/semaphore.py#L121 | 22:39 |
| comstud | fool you | 22:39 |
| comstud | that method is only called while in the hub context after switching to the hub | 22:39 |
| marun | ah, ok | 22:39 |
| comstud | ( https://github.com/eventlet/eventlet/blob/master/eventlet/semaphore.py#L115 ) | 22:39 |
| comstud | i'm checking for other direct greenthread switches | 22:39 |
| marun | so in the case of the queue, what about patching the queue such that calls with block=False are changed to block=True (or maybe raise an error)? | 22:40 |
| marun | it would appear that both get and put would be affected, since they both have _nowait variants | 22:40 |
| *** b3nt_pin has joined #openstack-neutron | 22:41 | |
| comstud | you could break any code that relied on those to not block | 22:41 |
| *** mlavalle has quit IRC | 22:41 | |
| comstud | :) | 22:41 |
| marun | comstud: I think we want that, though | 22:41 |
| marun | comstud: anything that yields when a lock is held is dangerous | 22:42 |
| marun | comstud: -> raise an error I mean. changing the behavior, as you say, could have negative consequences | 22:42 |
| comstud | someone with code such as: | 22:42 |
| comstud | try: | 22:42 |
| *** b3nt_pin is now known as beagles | 22:43 | |
| comstud | while True: | 22:43 |
| *** banix has quit IRC | 22:43 | |
| comstud | item = queue.get(block=False) | 22:43 |
| comstud | except Queue.Empty: | 22:43 |
| comstud | .. | 22:43 |
| ajo | project stats for , .. days | 22:43 |
| ajo | number of days not allowed | 22:43 |
| comstud | would break | 22:43 |
| marun | comstud: right, but we can't allow yielding | 22:43 |
| comstud | because get() would all of the sudden block and never raise Empty | 22:43 |
| marun | comstud: so we want it to break. | 22:43 |
| marun | comstud: but raising an exception woudl be the preferred way | 22:43 |
| comstud | well, you'd get a lock up in that case | 22:44 |
| comstud | which might be worse | 22:44 |
| comstud | heh | 22:44 |
| marun | comstud: rather than changing the blocking behaviour | 22:44 |
| comstud | yeah you could do that... this is starting to feel a little hacky | 22:44 |
| comstud | i mean | 22:44 |
| marun | comstud: it is entirely | 22:44 |
| comstud | what you're essentially doing.. | 22:44 |
| *** tchaypo is now known as jammyjam | 22:44 | |
| marun | comstud: but the alternative is more hacky | 22:44 |
| comstud | is the exact same thing that the semaphore does | 22:44 |
| comstud | :) | 22:45 |
| comstud | it keeps from switching greenthreads | 22:45 |
| marun | comstud: not exactly | 22:45 |
| marun | comstud: or wait, what? | 22:45 |
| comstud | Your goal here is to not allow greenthread switching while in the dbapi call right? | 22:45 |
| marun | arguably semaphore is worse... | 22:46 |
| comstud | That's exactly what the semaphore decorator does | 22:46 |
| marun | 2 greenthreads, both wanting to enter block A | 22:46 |
| marun | 1 other unrelated greenthread | 22:46 |
| comstud | 2 greenthreads don't ever run at the same time. | 22:46 |
| marun | comstud: bear with me, please | 22:46 |
| comstud | sure go ahead | 22:46 |
| marun | so, greenthread #1 gets semaphore, yields | 22:46 |
| marun | greenthread #2 can't run, waiting on semaphore | 22:46 |
| marun | greenthread #3, runs for an arbitrary length of time | 22:47 |
| comstud | yeah, gotcha. | 22:47 |
| marun | greenthread #1 is waiting for control back while #3 runs | 22:47 |
| marun | so the lock gets held longer than it should | 22:47 |
| marun | This 'hacky' alternative would allow #1 to complete and give up the lock faster | 22:47 |
| comstud | well it doesn't matter if the lock was held or not | 22:47 |
| marun | noyield | 22:47 |
| comstud | the problem is that it still allowed a switch | 22:47 |
| marun | the db lock i'm talking about | 22:47 |
| comstud | so I see the problem | 22:47 |
| marun | under the semaphore | 22:48 |
| comstud | Ok gotcha | 22:48 |
| comstud | yep | 22:48 |
| *** mwagner_bbl has joined #openstack-neutron | 22:48 | |
| marun | so, given that use case, do you think it makes sense to pursue the alternative or is the semaphore still preferable? | 22:48 |
| marun | My concern is that by introducing semaphores without preventing yields we are making it possible for other deadlocks to occur... | 22:49 |
| marun | whack-a-mole | 22:49 |
| comstud | ultimately you really shouldn't have another greenthread run for a long period of time | 22:49 |
| marun | if we can prevent yielding it might nip the problem in the bud. | 22:49 |
| *** ale has joined #openstack-neutron | 22:49 | |
| comstud | That would actually cause a lot more problems... | 22:49 |
| kevinbenton | marun: yep. already happened in that patch with our big switch plugin :-) | 22:49 |
| comstud | like an extremely slow service | 22:49 |
| comstud | :) | 22:50 |
| *** jobewan has quit IRC | 22:50 | |
| marun | comstud: the other + of adding this noyield everywhere we use db transactions would be tracking slow transactions | 22:50 |
| marun | comstud: like 'slow query' | 22:50 |
| marun | comstud: only at the sqla level | 22:50 |
| comstud | nod | 22:50 |
| marun | I don't think we have any visibility on this right now. | 22:50 |
| *** b3nt_pin has joined #openstack-neutron | 22:51 | |
| marun | in fact, maybe it makes sense to add the instrumentation first, and then think about preventing yields? | 22:51 |
| marun | that should be more acceptable from a reviewer perspective and then preventing yields could be system-wide with only changes to the wrapper function | 22:51 |
| marun | thoughts? | 22:51 |
| marun | markmcclain, salv-orlando ^^ | 22:52 |
| comstud | Well, let me talk about the endgame here first | 22:52 |
| *** blogan has quit IRC | 22:52 | |
| marun | ok | 22:52 |
| comstud | which is that DBAPI calls are actually done in Threads | 22:52 |
| comstud | (when we have eventlet fixed or an alternative that isn't buggy with threads) | 22:53 |
| marun | comstud: but does that imply only the calls to the server? | 22:53 |
| comstud | You can potentially have the same problem, I think.. in that python could decide to switch Threads in the middle of the transaction | 22:53 |
| comstud | It doesn't imply that, no | 22:53 |
| *** beagles has quit IRC | 22:54 | |
| comstud | you can't conceivably do that... | 22:54 |
| marun | comstud: that seems just as broken as today | 22:54 |
| marun | comstud: why is that a win? | 22:54 |
| comstud | because you can actually do more than 1 DB call at once | 22:54 |
| comstud | and DB calls aren't completely serial | 22:54 |
| comstud | right now they lock up the whole process | 22:55 |
| *** ale___ has joined #openstack-neutron | 22:55 | |
| marun | comstud: uh | 22:55 |
| comstud | with Threads the I/O happens without the GIL locked | 22:55 |
| * marun is boggled | 22:55 | |
| comstud | so it's a win for performance, but doesn't address your current problem | 22:55 |
| comstud | ok | 22:55 |
| rkukura | marun, comstud: kevinbenton and I were thinking replacing with_lockmode(‘update’) with with_lockmode(‘update_nowait’) in a loop that yields might be a nice solution to the deadlock issue | 22:56 |
| comstud | let me know which part I can explain better | 22:56 |
| *** jorgem has quit IRC | 22:56 | |
| marun | comstud: I guess I'm confused as to how any solution that involves db locks can safely yield. | 22:57 |
| * salv-orlando caught up on reading conversation. most things make sense. | 22:57 | |
| comstud | Well, I've not gotten to addressing that problem yet | 22:58 |
| *** ale has quit IRC | 22:58 | |
| comstud | :) | 22:58 |
| comstud | I'm stating a direction that we need to go in general in openstack with respect to DB calls | 22:58 |
| comstud | in solving the problem that all DB calls block the whole process right now | 22:58 |
| marun | salv-orlando: the question was whether it makes sense to add a transaction wrapper that can, at first, track duration of transactions | 22:58 |
| comstud | because eventlet can't wrap the I/O with the server | 22:58 |
| comstud | assuming mysql and mysqldb as the sql-a module | 22:58 |
| salv-orlando | I've read the last 30 minutes, I think I'm up to date now | 22:58 |
| marun | salv-orlando: and maybe be extended to provide additional details (like locks held) and maybe even prevent yields in the future | 22:59 |
| marun | salv-orlando: if the first part seems valuable, we can implement without regard to the second, which may or may not be valuable | 22:59 |
| *** ale___ has quit IRC | 22:59 | |
| marun | comstud: fair enough | 23:00 |
| comstud | So, yielding is only a problem if you get a 'Deadlock' error | 23:00 |
| marun | comstud: I guess the thing I'm struggling to decide is whether code can avoid deadlocks (by figuring out how not to yield in transaction) or whether deadlocks are an acceptable fact of life and can be mitigated (ala rkukura's update_nowait loop) | 23:00 |
| comstud | And apparently there's alternative queries that can be made to solve that | 23:01 |
| marun | 'alternative queries' | 23:01 |
| marun | ? | 23:01 |
| comstud | I don't know enough to expand much.. this is just what I've been told | 23:01 |
| comstud | things like: | 23:02 |
| comstud | foo = SELECT query.. | 23:02 |
| comstud | if foo exists: | 23:02 |
| comstud | UPDATE foo | 23:02 |
| comstud | else: | 23:02 |
| comstud | CREATE foo | 23:02 |
| comstud | can be done in 1 query in mysql | 23:02 |
| *** pcm_ has quit IRC | 23:02 | |
| comstud | that is just "UPDATE it... but create it if it doesn't exist" | 23:03 |
| marun | comstud: in pg too? | 23:03 |
| comstud | as a dumb example. | 23:03 |
| marun | comstud: :p | 23:03 |
| comstud | but I don't think it's in pg, no :) ANd I think that's the problem hehe.. | 23:03 |
| comstud | but anyway | 23:03 |
| * marun shakes fist at pg | 23:03 | |
| marun | comstud: I'm not sure we have much room for that type of optimization | 23:03 |
| comstud | I think deadlocks are somewhat a fact of life | 23:03 |
| comstud | and that's why we retry them in nova | 23:03 |
| marun | comstud: the problem tends to be cross-table updates where locking is required | 23:03 |
| rkukura | We need to avoid deadlocks, but we cannot avoid waiting for locks (like update locks) to be released. We need to bound how much time we spend waiting for a lock to be released without yielding to other greenthreads, or else we get deadlock. | 23:03 |
| comstud | I don't think we can reasonably determine what's going to cause them. | 23:03 |
| marun | rkukura: -> avoid yields | 23:04 |
| comstud | And by disallowing yields... you're removing parallelism (at least when using Threads) | 23:04 |
| rkukura | yield where you have to, but nowhere else | 23:04 |
| comstud | (because I think the only way to disallow yield with Thread would be to use your own lock :) | 23:04 |
| marun | comstud: we should be avoiding parallelism in transactions, though | 23:04 |
| comstud | like all of our db calls use transactions | 23:05 |
| marun | comstud, rkukura: consider the case of multiple processes... | 23:05 |
| comstud | so yeah, that's another great example | 23:05 |
| comstud | you can't really avoid yielding there :) | 23:05 |
| marun | holding locks is costlier than serialization | 23:05 |
| marun | when multiple processes are involved | 23:05 |
| comstud | unless you want to create a lock shared across processes | 23:05 |
| marun | i'm talking db locks | 23:05 |
| marun | so it implicitly is shared | 23:05 |
| comstud | yeah | 23:05 |
| comstud | I'm talking about.. say... you run 2 neutrons | 23:06 |
| rkukura | locking rows is statically unlikely to collide so has much less contention that serializing | 23:06 |
| marun | what i'm saying is that to really scale we have to be multiprocess so serializing greengreads/threads at the cost of in-process parallization is preferable to the alternative | 23:06 |
| comstud | both can be doing DB calls in parallel | 23:06 |
| comstud | linux kernel could yield 1 process to another in the middle of a transaction | 23:06 |
| comstud | how are you going to stop that? | 23:06 |
| comstud | :) | 23:06 |
| marun | comstud: pin the processes to cores | 23:06 |
| marun | :p | 23:06 |
| comstud | I think the answer is that you don't stop yields and find a better alternative | 23:07 |
| marun | (i'm talking out of my ass0 | 23:07 |
| comstud | haha | 23:07 |
| kevinbenton | across multiple processes is okay though, because a thread failing to yield for DB lock doesn't effect the other process, right? | 23:07 |
| comstud | again, I think this is another reason why we have the retry decorator in nova | 23:07 |
| comstud | It's more the multi-process case | 23:07 |
| marun | comstud: it's a smelly thing | 23:07 |
| comstud | i'm not sure it is | 23:07 |
| comstud | mysql says... | 23:07 |
| marun | comstud: i can't help but feel that we're missing something | 23:08 |
| comstud | Deadlock: Retry the transaction | 23:08 |
| *** b3nt_pin is now known as beagles | 23:08 | |
| comstud | :) | 23:08 |
| comstud | It's in the error message!! | 23:08 |
| comstud | ;) | 23:08 |
| beagles | do we have row level locking available across all db backends | 23:08 |
| rkukura | My suggestion to use with_lockmode(‘update_nowait’) is intended to allow other greenthreads to do useful work during the time that with_lockmode(‘update’) would be blocked. If there is no other work to do, the process just spins trying to lock the row. | 23:08 |
| marun | beagles: yes | 23:08 |
| beagles | interesting | 23:08 |
| comstud | here | 23:08 |
| comstud | marun: it's more argument for retry: | 23:08 |
| comstud | "Both deadlocks and lock wait timeouts are normal on busy servers and it is necessary for applications to be aware that they may happen and handle them by retrying." | 23:08 |
| comstud | http://dev.mysql.com/doc/refman/5.0/en/innodb-error-handling.html | 23:09 |
| beagles | I am accustomed to the always having to anticipate a deadlock exception | 23:09 |
| beagles | and deal | 23:09 |
| comstud | nod | 23:09 |
| *** sacharya1 has quit IRC | 23:10 | |
| marun | beagles, comstud: I can see the logic of accepting deadlocks when we're not doing stupid things like yielding in the middle of a transaction. | 23:10 |
| kevinbenton | the errors we get aren't usually "Deadlock: retry transaction" though | 23:10 |
| marun | the smelly part is not in expecting deadlocks | 23:10 |
| kevinbenton | that implies mysql understands it's a deadlock | 23:10 |
| marun | it's in stopping what we're doing for potentially stupid reasons | 23:10 |
| kevinbenton | which it doesn't in our case | 23:10 |
| beagles | "InnoDB uses automatic row-level locking. You can get deadlocks even in the case of transactions that just insert or delete a single row. That is because these operations are not really “atomic”; they automatically set locks on the (possibly several) index records of the row inserted or deleted. " | 23:10 |
| beagles | fwiw | 23:10 |
| marun | retry-on-deadlock -> not bad | 23:11 |
| comstud | marun: Think multi-process. You're going to be doing 2 DB calls at once no matter what | 23:11 |
| beagles | actually just see https://dev.mysql.com/doc/refman/5.0/en/innodb-deadlocks.html.. moving on | 23:11 |
| marun | retry-on-deadlock + yield -> smells | 23:11 |
| comstud | that's what this mysql error page is talking about really | 23:11 |
| beagles | marun, k | 23:11 |
| comstud | 'under high load, deadlocks are going to happen no matter what' | 23:11 |
| comstud | Now I'm sure that means some parallelism is required for that | 23:12 |
| marun | comstud: you don't need to convince me. see: retry-on-deadlock -> not bad | 23:12 |
| comstud | But what are you going to do then... | 23:12 |
| comstud | limit yourself to 1 process? | 23:12 |
| comstud | heheh ok | 23:12 |
| marun | comstud: it's the yielding that widens the window for deadlock, and I wonder if that could be avoided | 23:12 |
| marun | comstud: it has nothing to do with multiprocess | 23:12 |
| comstud | i argue that it doesn't | 23:12 |
| kevinbenton | those are mysql deadlocks it's referring to, which are fine because they are detected right away | 23:12 |
| comstud | Well | 23:13 |
| kevinbenton | the lock wait timeout is the eventlet/mysql deadlock, which takes a painfully long time (relatively) to notice | 23:13 |
| comstud | I argue that it doesn't any more than multiprocess does | 23:13 |
| comstud | yeah | 23:13 |
| comstud | the lock wait timeout is a larger problem | 23:13 |
| marun | comstud: except that linux is waaaaaaay fing smarter in scheduling | 23:13 |
| marun | comstud: come on, eventlet is coorperative! | 23:13 |
| comstud | hehe | 23:13 |
| *** markmcclain has quit IRC | 23:14 | |
| marun | comstud: we wouldn't expect a process to not get a slice, what, millions of times/second? | 23:14 |
| marun | comstud: anyway, i get your point that maybe it doesn't matter | 23:14 |
| comstud | :) | 23:14 |
| comstud | Just for you... | 23:14 |
| marun | and that workarounds now might conflict with switches from threads | 23:14 |
| comstud | I'm going to add an option to disable switches in my eventlet replacement. | 23:14 |
| marun | from -> threads | 23:15 |
| marun | :) | 23:15 |
| marun | from -> to *sigh* | 23:15 |
| comstud | but yeah, I think eventually any code you try to patch for eventlet.. | 23:15 |
| marun | comstud: so, after all that... | 23:15 |
| comstud | will end up going away because we'll be using real Threads | 23:16 |
| comstud | in which case, there will only be 1 greenthread in that Thread.. yours | 23:16 |
| marun | comstud: is there value at least in tracking transaction duration (at least when locking is involved)? | 23:16 |
| comstud | Sure, stats are good | 23:16 |
| marun | when are we going to be using 'real threads' anyway? | 23:16 |
| comstud | I don't know | 23:16 |
| comstud | when either | 23:16 |
| comstud | 1) We get a fix from eventlet | 23:16 |
| comstud | 2) We replace eventlet | 23:17 |
| comstud | (with something Thread safe) | 23:17 |
| comstud | #1 has a potential patch that needs testing | 23:17 |
| marun | rkukura: so, I'm going to work on adding instrumentation to all transactions. wrapper method/helper/whatever. it should be compatible with whatever we end up doing, and allow easier identification of problem cod | 23:17 |
| marun | e | 23:17 |
| comstud | but it's very new experimental code | 23:17 |
| marun | comstud: so, 'someday' :) | 23:17 |
| comstud | #2 I'm almost there with, also, but has the same issue, I guess. | 23:18 |
| comstud | hehe yes | 23:18 |
| comstud | soonish I hope | 23:18 |
| marun | comstud: so, plan of action... | 23:18 |
| marun | comstud: wrap all transactions in neutron so they can be instrumented | 23:18 |
| marun | comstud: then experiment with halting yielding (maybe only when locks are used) | 23:18 |
| *** alexpilotti has quit IRC | 23:18 | |
| marun | comstud: if we end up adopting a noyield, we can easily pull it out - only have to change the transaction wrapper | 23:19 |
| marun | comstud: make sense? | 23:19 |
| *** rm_work is now known as rm_work|away | 23:19 | |
| openstackgerrit | A change was merged to openstack/neutron: Prevent cross plugging router ports from other tenants https://review.openstack.org/83391 | 23:19 |
| comstud | yeah, makes sense | 23:20 |
| comstud | i'm not gonna stop you from trying no-yield :) | 23:20 |
| comstud | 1) i'm curious the results | 23:20 |
| comstud | 2) i don't have a large influence over neutron ;) | 23:21 |
| rkukura | marun: I’m not opposed you your plan, but am not very confident that preventing yields really resolves the deadlock. | 23:21 |
| comstud | but we'd probably not bother with that in nova | 23:21 |
| marun | comstud, rkukura: the plan is really instrumentation-focused | 23:21 |
| comstud | except maybe stats, which are always useful | 23:21 |
| comstud | nod | 23:21 |
| marun | rkukura: I agree with you | 23:22 |
| marun | rkukura: noyield won't prevent deadlocks across processes | 23:22 |
| rkukura | I think focusing on preventing the long-term socket wait inside the mysql C library is what’s needed | 23:22 |
| marun | rkukura: only within a process | 23:22 |
| marun | rkukura: but it won't be expensive | 23:22 |
| marun | rkukura: once we have instrumentation, the cost to insert noyield is trivial | 23:22 |
| rkukura | marun: I’m OK with the instrumentation | 23:22 |
| marun | (and the cost to remove it) | 23:22 |
| rkukura | and I completely agree unecessary yields inside transactions should be eliminated | 23:22 |
| marun | comstud: and maybe the transaction wrapper is where we insert retry-on-deadlock? | 23:23 |
| rkukura | but if a thread needs to lock a row thats already locked, we need to yield instead of blocking | 23:23 |
| marun | rkukura: right | 23:23 |
| marun | rkukura: the point is blocking on everything but db locking | 23:24 |
| comstud | marun: yeah, I think so | 23:24 |
| marun | comstud: is there a reason the nova decorator is applied on functions rather than on transactions? | 23:24 |
| marun | comstud: and should we pursue the same strategy in neutron? | 23:25 |
| comstud | because most of our functions immediately grab a session and start a transaction | 23:25 |
| comstud | We can only add it on the DB API call if the whole thing is inside a transaction | 23:25 |
| comstud | otherwise you might be re-trying something that didn't get rolled back in the transaction | 23:25 |
| comstud | marun: I haven't look at neutron db code, so I can't answer... | 23:26 |
| comstud | if you're building a transaction manager, it should go there | 23:26 |
| comstud | i mean | 23:27 |
| comstud | we're using mysql in a way such that there are implicit transactions | 23:27 |
| comstud | in nova anyway | 23:27 |
| comstud | although there can be subtransactions | 23:27 |
| comstud | but you'll notice at the top of our methods with the retry decorator... | 23:28 |
| comstud | the very first thing is typically with session.begin(): | 23:28 |
| marun | rkukura: is there a reason we aren't doing the same in neutron? | 23:28 |
| comstud | and all code is under that | 23:28 |
| *** jgrimm has quit IRC | 23:28 | |
| *** packet has quit IRC | 23:29 | |
| marun | comstud: ok, it might make sense to do the same | 23:30 |
| marun | comstud: I was pretty opposed to retry-on-deadlock when I saw it, and now…*sigh* | 23:30 |
| comstud | I should have dug out this mysql error handling refernece sooner | 23:30 |
| comstud | :) | 23:30 |
| marun | comstud: With my luck I'll end up giving the next questionable idea I see the benefit of the doubt, to my detriment. | 23:31 |
| comstud | I'd somewhat forgotten that it tells you right in the error message to retry | 23:31 |
| comstud | haha | 23:31 |
| comstud | i'll buy you a beer if it makes you feel better | 23:31 |
| marun | :) | 23:32 |
| comstud | or your choice of beverage | 23:32 |
| marun | rkukura, kevinbenton: thoughts on a generic retry-on-deadlock decorator vs explicit update_nowait? | 23:33 |
| comstud | it kinda seems like that maybe only does anything on pg | 23:34 |
| comstud | maybe not | 23:34 |
| marun | comstud: you mean the decorator? | 23:34 |
| comstud | update_nowait | 23:34 |
| marun | arg | 23:35 |
| marun | i think you're right | 23:35 |
| marun | rkukura, kevinbenton ^^ | 23:35 |
| marun | non-blocking queries are _not_ supported in mysql | 23:36 |
| comstud | I don't see any NOWAIT option in mysql docs | 23:36 |
| salv-orlando | thanks for this very informative discussion… just to make sure I'm following. Are we talking about a retry decorator for lock wait timeouts? | 23:36 |
| comstud | for SELECT FOR UPDATE | 23:36 |
| marun | salv-orlando: yes | 23:36 |
| comstud | for deadlocks probably more than the lock wait timeouts | 23:36 |
| comstud | but really for both | 23:36 |
| comstud | the lock wait timeouts are a special problem tho, because it takes 60s by default to timeout | 23:37 |
| marun | salv-orlando: if we keep beating our heads against the wall, eventually we're bound to break through! | 23:37 |
| comstud | meanwhile your whole process is locked up | 23:37 |
| salv-orlando | comstud: exactly what I was saying (50 sec by default) | 23:37 |
| marun | comstud: unless we yield ;) | 23:37 |
| comstud | ok, 50 | 23:37 |
| marun | comstud: oh wait | 23:37 |
| marun | comstud: C :( | 23:37 |
| comstud | can't yield.. unless you're using Threads | 23:37 |
| comstud | which is another reason why we need Threads | 23:37 |
| comstud | :) | 23:37 |
| comstud | yep | 23:37 |
| marun | ugh | 23:37 |
| salv-orlando | marun: It would not be the first time I break a wall only to find out there's an abyss on the other side... | 23:37 |
| marun | so nowait still might be useful | 23:37 |
| comstud | or you switch to make sql-a use the pure python mysql module | 23:38 |
| comstud | and trade for slowness | 23:38 |
| marun | sorry, noyield | 23:38 |
| marun | comstud: hmm. are there metrics on the tradeoff? | 23:38 |
| marun | comstud: under load, i can imagine lock contention being worse | 23:38 |
| comstud | i don't have any handy | 23:38 |
| marun | comstud: but that's a guess | 23:38 |
| *** rkukura has quit IRC | 23:38 | |
| marun | comstud: based on 50s timeouts! | 23:39 |
| marun | that's terrible | 23:39 |
| *** Sukhdev has joined #openstack-neutron | 23:39 | |
| *** ale___ has joined #openstack-neutron | 23:39 | |
| comstud | so | 23:39 |
| marun | comstud: so, thinking of the timeouts, avoiding them is only possible with semaphores? :( | 23:39 |
| comstud | those are a particular problem with the actual queries, I think | 23:39 |
| comstud | moreso than immediate deadlock error | 23:39 |
| comstud | marun: different sql queries I think is the only solution | 23:40 |
| comstud | remember I keep going back to the multi-process case | 23:40 |
| comstud | I think the solution should cover it as well, if there is any | 23:40 |
| marun | comstud: no problem, we'll just change our entire schema! | 23:40 |
| comstud | haha | 23:40 |
| comstud | yeah, no problem. | 23:40 |
| openstackgerrit | Carl Baldwin proposed a change to openstack/neutron: Delete duplicate external devices in router namespace https://review.openstack.org/83624 | 23:40 |
| marun | definitely agree with the multi-process | 23:40 |
| Sukhdev | mestery: ping | 23:40 |
| *** rkukura has joined #openstack-neutron | 23:40 | |
| *** ale___ has quit IRC | 23:41 | |
| comstud | I've got to bail out for now | 23:41 |
| salv-orlando | comstud: thanks a lot for joining the conversation | 23:41 |
| marun | comstud: thank you for the conversation, informative as always :) | 23:41 |
| comstud | ping me tomorrow if anything else comes up :) | 23:41 |
| marun | will do | 23:41 |
| comstud | no problem | 23:41 |
| rkukura | comstud: thanks! | 23:41 |
| comstud | sure! | 23:41 |
| comstud | outti | 23:41 |
| * comstud & | 23:41 | |
| marun | rkukura: it looks like non-blocking queries aren't supported in mysql | 23:42 |
| marun | rkukura: pg only | 23:42 |
| marun | rkukura: or did I miss something? | 23:42 |
| *** gdubreui has quit IRC | 23:42 | |
| rkukura | marun: I’ll talk to dkehn about that - he knows mysql internals | 23:42 |
| marun | rkukura: http://docs.sqlalchemy.org/en/rel_0_9/orm/query.html | 23:43 |
| marun | rkukura: sqla 0.9 shows update_nowait only supported by pg and oracle backends | 23:44 |
| *** _cerberus has joined #openstack-neutron | 23:44 | |
| salv-orlando | there's that (and keep in mind more DB backends might be official supported in the future), and that there in some cases you really want the full lock (thinking IPAM for instance). | 23:44 |
| salv-orlando | I mean I like the idea of preventing yields. | 23:44 |
| marun | salv-orlando: what do you mean 'full lock'? | 23:45 |
| salv-orlando | the thing which is not the with_lockmode('update_nowait') | 23:45 |
| salv-orlando | so with_lockmode('update') | 23:45 |
| salv-orlando | the blocking query | 23:45 |
| marun | the intention was to use update_nowait in a loop | 23:46 |
| rkukura | marun: looks like you are right re mysql | 23:46 |
| marun | rkukura: :( | 23:46 |
| marun | rkukura: comstud pointed that out | 23:46 |
| marun | rkukura: I wish it weren't so - waiting for lock timeout in the C code is something we'd best avoid | 23:46 |
| marun | rkukura: no way to yield or do anything useful while that's happening, meanwhile requests pile up | 23:47 |
| *** sacharya has joined #openstack-neutron | 23:47 | |
| marun | rkukura: semaphore is the only surefire way to avoid, though we might be able to tune the timeout | 23:47 |
| marun | (or just make the damn transactions faster) | 23:47 |
| rkukura | wonder if there is a way to use a very short timeout to simulate nowait with mysql | 23:48 |
| marun | rkukura: it probably makes sense to do so | 23:48 |
| marun | rkukura: i think the default timeout is 50s | 23:48 |
| salv-orlando | honestly, I still like more the idea of preventing the yield. | 23:48 |
| marun | rkukura: if it were way shorter than looping to acquire would make sense | 23:48 |
| marun | salv-orlando: haven't given up on it :) | 23:48 |
| marun | salv-orlando: just covering the angles | 23:48 |
| mestery | Sukhdev: pong | 23:48 |
| rkukura | I’m wondering if we could insert some SQL into the query forcing a 10ms timeout or somthing like that, then loop/yield | 23:49 |
| marun | salv-orlando: honestly, it may or may not be viable. If we instrument all the transactions at least the cost of experimentation will be low | 23:49 |
| beagles | marun, salv-orlando for my own clarity... what yield are you referring to? | 23:49 |
| marun | rkukura: definitely worth investigating | 23:49 |
| salv-orlando | the way our db access work is that it's serialized anyway at the end of the day. So if people don't do a lot of non-db stuff within a transaction I suspect performance impact won't be a lot | 23:49 |
| mestery | Sukhdev: About to shutdown for a bit, I'll be back online later. | 23:50 |
| salv-orlando | marun: the funny thing is that the lenght of a transaction might actually be impacted by yields! | 23:50 |
| salv-orlando | beagles: to cut a long story short, here's my summary | 23:50 |
| beagles | unless it is not a one-line answer... that is. | 23:50 |
| salv-orlando | coming... | 23:51 |
| salv-orlando | a greenthread starts a transaction and get a db lock (select for update). Then from within the transaction it yields (a log statement would trigger that). Control is given to another thread which open a transaction and tries to retrieve the lock held by the other thread. | 23:51 |
| salv-orlando | The 2nd thread blocks there and does not yield because greenthreads are dumb after all. | 23:52 |
| salv-orlando | result: 2nd thread fails for "lock wait timeout" | 23:52 |
| * beagles nods | 23:52 | |
| kevinbenton | beagles: here it is shown through the power of boxes with lines! | 23:53 |
| kevinbenton | beagles: https://docs.google.com/drawings/d/13A2x4AWbf8zmzeGApUmYVlBrW8CMTPFTCBGSP_nTzDA/edit?usp=sharing | 23:53 |
| salv-orlando | so marun is thinking in mocking eventlet from within transactions in order to avoid this | 23:53 |
| beagles | cool... | 23:53 |
| *** mestery has quit IRC | 23:53 | |
| marun | salv-orlando: will minimize the time that locks are held, but won't prevent cross-process deadlock | 23:53 |
| marun | salv-orlando: retry-on-deadlock decorator from nova is probably required regardless to cover that case (despite my initial reservations) | 23:54 |
| salv-orlando | marun: on this I agree with comstud that the deadlock is a different problem | 23:54 |
| beagles | regarding the yield thing.. the mocking... that's pretty radical :) | 23:54 |
| salv-orlando | yes, you would still need the retry decorator | 23:54 |
| marun | beagles: well, call it 'patching' instead ;) | 23:54 |
| beagles | I'm familiar with the implicit yield on i/o | 23:54 |
| salv-orlando | beagles: I see it as working around what is at the end of the day an eventlet issue | 23:54 |
| marun | the basic intent is to nip the current trend in deadlocking errors in the bud, since fixing one area seems to cause problems in another.... | 23:55 |
| *** rkukura has quit IRC | 23:56 | |
| *** pnavarro has quit IRC | 23:56 | |
| *** cgoncalves has quit IRC | 23:56 | |
| *** linuxgeek_ has quit IRC | 23:57 | |
| *** cgoncalves has joined #openstack-neutron | 23:57 | |
| *** otherwiseguy has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!