| *** pcm_ has quit IRC | 00:00 | |
| *** mlavalle has quit IRC | 00:01 | |
| anteaya | good night | 00:02 |
|---|---|---|
| *** carl_baldwin has joined #openstack-neutron | 00:06 | |
| *** carl_baldwin has quit IRC | 00:10 | |
| *** banix has quit IRC | 00:13 | |
| openstackgerrit | A change was merged to openstack/neutron: move rpc_setup to the last step of __init__ https://review.openstack.org/60805 | 00:25 |
| *** x86brandon has joined #openstack-neutron | 00:31 | |
| openstackgerrit | A change was merged to openstack/neutron: Add X-Tenant-ID to metadata request https://review.openstack.org/61439 | 00:33 |
| *** dims has joined #openstack-neutron | 00:43 | |
| *** yamahata_ has joined #openstack-neutron | 00:45 | |
| *** yamahata_ has quit IRC | 00:59 | |
| *** yamahata_ has joined #openstack-neutron | 01:00 | |
| *** banix has joined #openstack-neutron | 01:03 | |
| *** SumitNaiksatam has quit IRC | 01:05 | |
| *** salv-orlando_ has joined #openstack-neutron | 01:24 | |
| *** salv-orlando has quit IRC | 01:27 | |
| *** salv-orlando has joined #openstack-neutron | 01:27 | |
| *** salv-orlando_ has quit IRC | 01:28 | |
| *** ljjjustin has joined #openstack-neutron | 01:30 | |
| *** salv-orlando_ has joined #openstack-neutron | 01:33 | |
| *** salv-orlando_ has quit IRC | 01:34 | |
| *** salv-orlando__ has joined #openstack-neutron | 01:34 | |
| *** salv-orlando has quit IRC | 01:36 | |
| *** salv-orlando__ is now known as salv-orlando | 01:36 | |
| *** SumitNaiksatam has joined #openstack-neutron | 01:38 | |
| *** Jianyong has joined #openstack-neutron | 01:46 | |
| anteaya | mtreinish: nati_ueno posted a reply to your comment on https://review.openstack.org/#/c/61234/ | 01:48 |
| *** csd has quit IRC | 01:48 | |
| *** csd has joined #openstack-neutron | 01:50 | |
| *** gongysh has joined #openstack-neutron | 01:55 | |
| anteaya | mtreinish: thanks | 01:56 |
| anteaya | going offline now, see y'all later | 02:03 |
| *** csd has quit IRC | 02:05 | |
| *** vkozhukalov has joined #openstack-neutron | 02:38 | |
| *** Gu_______ has joined #openstack-neutron | 02:42 | |
| *** Gu_______ has quit IRC | 02:47 | |
| *** hxgqh1987 has joined #openstack-neutron | 02:48 | |
| *** suresh12 has quit IRC | 02:52 | |
| *** suresh12 has joined #openstack-neutron | 02:53 | |
| *** julim has quit IRC | 02:57 | |
| *** hxgqh1987 has quit IRC | 02:59 | |
| gongysh | sumitnaiksatam: ping | 03:01 |
| SumitNaiksatam | gongysh: hi | 03:02 |
| gongysh | sumitnaiksatam: https://review.openstack.org/#/c/61341/ can u help to review it, I got +2 already. | 03:02 |
| *** hxgqh1987 has joined #openstack-neutron | 03:02 | |
| SumitNaiksatam | yeah sure | 03:03 |
| SumitNaiksatam | i did see it | 03:03 |
| SumitNaiksatam | i was not completely sure what tun_br.defer_apply_on does :-) | 03:04 |
| *** SushilKM has joined #openstack-neutron | 03:06 | |
| gongysh | sumitnaiksatam: it defers the ovs-ofctl calls, store add-flow into a list and defer_apply_off will call add-flows in one shot. | 03:06 |
| SumitNaiksatam | yeah | 03:07 |
| *** suresh12 has quit IRC | 03:10 | |
| *** rpodolyaka has quit IRC | 03:12 | |
| SumitNaiksatam | gongysh: thinking about the test | 03:13 |
| *** suresh12 has joined #openstack-neutron | 03:13 | |
| *** rpodolyaka has joined #openstack-neutron | 03:13 | |
| SumitNaiksatam | gongysh: does it really test that all the flows are added to the list? | 03:13 |
| SumitNaiksatam | gongysh: i guess in the test we are just checking if we get the calls in the order | 03:15 |
| gongysh | <SumitNaiksatam>: a good question, I will see if I can improve it | 03:23 |
| SumitNaiksatam | gongysh: thats fine, i responded to the review | 03:23 |
| *** alagalah has joined #openstack-neutron | 03:24 | |
| *** suresh12 has quit IRC | 03:37 | |
| gongysh | nati_ueno: ping | 03:43 |
| *** marun has joined #openstack-neutron | 04:06 | |
| openstackgerrit | A change was merged to openstack/neutron: Updated from global requirements https://review.openstack.org/60974 | 04:14 |
| *** SushilKM has quit IRC | 04:17 | |
| *** hxgqh1987 has quit IRC | 04:22 | |
| *** banix has quit IRC | 04:27 | |
| *** chandankumar has joined #openstack-neutron | 04:36 | |
| *** suresh12 has joined #openstack-neutron | 04:56 | |
| *** yfried has quit IRC | 04:58 | |
| *** nati_ueno has quit IRC | 05:03 | |
| *** suresh12 has quit IRC | 05:08 | |
| *** yongli_away is now known as yongli | 05:14 | |
| *** garyk has quit IRC | 05:14 | |
| *** yfried has joined #openstack-neutron | 05:25 | |
| *** angryjesters has joined #openstack-neutron | 05:33 | |
| *** enikanorov_ has quit IRC | 05:37 | |
| *** enikanorov has joined #openstack-neutron | 05:37 | |
| *** AndreyGrebenniko has quit IRC | 05:41 | |
| *** AndreyGrebenniko has joined #openstack-neutron | 05:41 | |
| *** otherwiseguy has quit IRC | 05:50 | |
| *** gdubreui has quit IRC | 05:51 | |
| *** hxgqh1987 has joined #openstack-neutron | 05:53 | |
| *** suresh12 has joined #openstack-neutron | 05:54 | |
| *** garyk has joined #openstack-neutron | 05:56 | |
| *** alagalah has left #openstack-neutron | 06:03 | |
| *** suresh12 has quit IRC | 06:15 | |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/neutron: Fix the migration adding a UC to agents table https://review.openstack.org/61663 | 06:28 |
| *** suresh12 has joined #openstack-neutron | 06:32 | |
| openstackgerrit | Jenkins proposed a change to openstack/neutron: Imported Translations from Transifex https://review.openstack.org/61325 | 06:34 |
| *** SushilKM has joined #openstack-neutron | 06:38 | |
| gongysh | enikanorov: ping | 06:39 |
| *** alex_klimov has joined #openstack-neutron | 06:44 | |
| *** lifeless has quit IRC | 06:48 | |
| *** vkozhukalov has quit IRC | 06:51 | |
| *** jlibosva has joined #openstack-neutron | 06:52 | |
| *** yongli has quit IRC | 07:06 | |
| gongysh | rkukura: hi | 07:08 |
| openstackgerrit | gongysh proposed a change to openstack/neutron: synchronize ovs agent's rpc handlers and main thread https://review.openstack.org/61667 | 07:12 |
| *** gongysh_ has joined #openstack-neutron | 07:13 | |
| *** jianingy_afk is now known as jianingy | 07:13 | |
| *** lifeless has joined #openstack-neutron | 07:15 | |
| *** gongysh has quit IRC | 07:16 | |
| *** gongysh_ has quit IRC | 07:16 | |
| *** gongysh has joined #openstack-neutron | 07:16 | |
| gongysh | rkukura: ping | 07:16 |
| gongysh | salv-orlando: ping | 07:27 |
| *** Mierdin has joined #openstack-neutron | 07:31 | |
| *** amritanshu_RnD has joined #openstack-neutron | 07:45 | |
| *** Mierdin has left #openstack-neutron | 07:51 | |
| *** nati_ueno has joined #openstack-neutron | 07:52 | |
| *** ljjjustin has quit IRC | 08:02 | |
| *** Jianyong has quit IRC | 08:02 | |
| *** gongysh has quit IRC | 08:03 | |
| *** gongysh has joined #openstack-neutron | 08:05 | |
| *** Jianyong has joined #openstack-neutron | 08:06 | |
| *** safchain has joined #openstack-neutron | 08:08 | |
| *** mengxd has joined #openstack-neutron | 08:10 | |
| openstackgerrit | Maru Newby proposed a change to openstack/neutron: Send DHCP notifications regardless of agent status https://review.openstack.org/61168 | 08:10 |
| openstackgerrit | A change was merged to openstack/python-neutronclient: Pin Sphinx to <1.2 in test-requirements https://review.openstack.org/61553 | 08:13 |
| *** enikanorov_ has joined #openstack-neutron | 08:18 | |
| *** suresh12 has quit IRC | 08:21 | |
| *** yongli has joined #openstack-neutron | 08:22 | |
| *** vkozhukalov has joined #openstack-neutron | 08:28 | |
| openstackgerrit | Oleg Bondarev proposed a change to openstack/neutron: LBaaS: fix handling pending create/update members and health monitors https://review.openstack.org/61673 | 08:29 |
| *** fandikurnia01 has joined #openstack-neutron | 08:45 | |
| fandikurnia01 | Hi all | 08:45 |
| openstackgerrit | gongysh proposed a change to openstack/neutron: add ml2 plugin into agent model migration scripts https://review.openstack.org/61677 | 08:46 |
| *** ygbo_ has joined #openstack-neutron | 08:46 | |
| *** ygbo has joined #openstack-neutron | 08:46 | |
| *** ygbo has quit IRC | 08:46 | |
| fandikurnia01 | does openstack can be using multiple vlan /trunks for floating ip with many subnet | 08:46 |
| fandikurnia01 | ? | 08:46 |
| *** mengxd has quit IRC | 08:48 | |
| *** afazekas has joined #openstack-neutron | 08:51 | |
| *** fouxm has joined #openstack-neutron | 08:56 | |
| *** jistr has joined #openstack-neutron | 09:03 | |
| *** jpich has joined #openstack-neutron | 09:05 | |
| *** salv-orlando has quit IRC | 09:05 | |
| *** ekarlso has quit IRC | 09:17 | |
| *** ekarlso has joined #openstack-neutron | 09:17 | |
| enikanorov_ | gongysh: Hi. could you please take a look at jenkins build failure for neutronclient? https://review.openstack.org/#/c/61561/ | 09:17 |
| enikanorov_ | gate-python-neutronclient-docs fails for some reason, while locally it passes | 09:18 |
| enikanorov_ | i can't find out what could be wrong with my patch | 09:18 |
| gongysh | enikanorov_ , how about just recheck or wait for some time, for example tomorrow :) the ci is getting wrong now and then. | 09:22 |
| *** jorisroovers has joined #openstack-neutron | 09:23 | |
| gongysh | enikanorov_: would you like to approve https://review.openstack.org/#/c/61341/ which has got 2 +2 already. | 09:23 |
| enikanorov_ | gongysh: i'm not a core :) | 09:24 |
| enikanorov_ | i can review and put +1 though | 09:24 |
| gongysh | enikanorov_ thanks | 09:24 |
| gongysh | nati_ueno: ping | 09:25 |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/neutron: Fix the migration adding a UC to agents table https://review.openstack.org/61663 | 09:26 |
| rpodolyaka | gongysh: hey! Looks like the mistake in the migration script I made uncovered issues with migrations not applying for ML2 plugin. I rebased my patch on yours, should be fixed now... | 09:27 |
| *** jorisroovers has quit IRC | 09:27 | |
| openstackgerrit | Sylvain Afchain proposed a change to openstack/neutron: Fix Metering doesn't respect the l3 agent binding https://review.openstack.org/60016 | 09:28 |
| fandikurnia01 | hi there, | 09:28 |
| gongysh | rpodolyaka: '*' means all plugins | 09:29 |
| gongysh | rpodolyaka: so do u need https://review.openstack.org/61663 patch? | 09:29 |
| rpodolyaka | gongysh: yes | 09:29 |
| rpodolyaka | gongysh: otherwise bigswitch/plumgrid/etc are broken | 09:30 |
| rpodolyaka | gongysh: they don't create agents table | 09:30 |
| rpodolyaka | gongysh: on * ... yeah, I know... should have been more attentive :( | 09:30 |
| gongysh | rpodolyaka: got it. thanks | 09:31 |
| rpodolyaka | gongysh: but the question is... how neutron works in devstack/tripleo now with ML2 plugin? without your patch agents migrations shoudn't have been applied, right? | 09:31 |
| *** suresh12 has joined #openstack-neutron | 09:31 | |
| gongysh | rpodolyaka: can you reproduce that bug I filed? I am also wondering how CI passed with your constraint patch. | 09:34 |
| rpodolyaka | gongysh: on passing CI - I thought we were testing only OVS/ML2 pair for which agents migrations must have been enabled | 09:34 |
| rpodolyaka | gongysh: so my migration should have applied to those just fine | 09:35 |
| rpodolyaka | gongysh: but it's broken for the plugins which are not using agents | 09:35 |
| * rpodolyaka running devstack | 09:35 | |
| *** marun has quit IRC | 09:36 | |
| *** suresh12 has quit IRC | 09:36 | |
| gongysh | rpodolyaka: I got the bug for OVS/ML2 in devstack. | 09:36 |
| *** SushilKM has quit IRC | 09:37 | |
| *** marun has joined #openstack-neutron | 09:37 | |
| rpodolyaka | gongysh: then I'm totally confused :( if we are running migrations scripts in devstack in order to install neutron, we must have spotted the bug when running CI tests | 09:37 |
| gongysh | rpodolyaka: yes, that is the problem. So I am asking if you can reproduce it in by running devstack. | 09:39 |
| rpodolyaka | gongysh: running stack.sh now | 09:40 |
| *** SushilKM has joined #openstack-neutron | 09:43 | |
| rpodolyaka | gongysh: http://paste.openstack.org/show/54870/ | 09:43 |
| rpodolyaka | gongysh: so no errors | 09:43 |
| rpodolyaka | gongysh: this is my patch NOT rebased on master | 09:44 |
| rpodolyaka | gongysh: let me try master now | 09:44 |
| gongysh | rpodolyaka: yes, please run on master | 09:45 |
| *** jp_at_hp has joined #openstack-neutron | 09:55 | |
| rpodolyaka | gongysh: looks like enikanorov found the problem. https://github.com/openstack/neutron/blob/master/neutron/db/migration/alembic_migrations/versions/511471cc46b_agent_ext_model_supp.py#L38 | 09:57 |
| rpodolyaka | gongysh: agents table is created for LBaaS service plugin | 09:58 |
| rpodolyaka | gongysh: I have LBaaS enabled in localrc, and AFAIK it's true for jenkins nodes too | 09:58 |
| gongysh | rpodolyaka: any way, the ml2 plugin is later introduced than agent model, for ml2, we need it fixed. | 10:02 |
| gongysh | rpodolyaka: yes, my localrc has no lbaas enabled. | 10:02 |
| rpodolyaka | gongysh: sure | 10:03 |
| *** chandankumar_ has joined #openstack-neutron | 10:05 | |
| *** zhhuabj has quit IRC | 10:05 | |
| *** chandankumar has quit IRC | 10:08 | |
| *** gongysh has quit IRC | 10:10 | |
| *** AlexF has joined #openstack-neutron | 10:14 | |
| openstackgerrit | Isaku Yamahata proposed a change to openstack/neutron: ml2: gre, vxlan type driver can leak segment_id https://review.openstack.org/61694 | 10:22 |
| *** AlexF has quit IRC | 10:28 | |
| *** salv-orlando has joined #openstack-neutron | 10:34 | |
| *** zhhuabj has joined #openstack-neutron | 10:35 | |
| *** fandikurnia01 has quit IRC | 10:41 | |
| *** salv-orlando has quit IRC | 10:42 | |
| *** bashok has joined #openstack-neutron | 10:43 | |
| *** heyongli has joined #openstack-neutron | 10:46 | |
| *** Jianyong has left #openstack-neutron | 10:51 | |
| openstackgerrit | Oleg Bondarev proposed a change to openstack/neutron: LBaaS: fix handling pending create/update members and health monitors https://review.openstack.org/61673 | 10:55 |
| *** hxgqh1987 has quit IRC | 10:55 | |
| *** yamahata_ has quit IRC | 10:58 | |
| openstackgerrit | Oleg Bondarev proposed a change to openstack/neutron: LBaaS: check for associations before deleting health monitor https://review.openstack.org/53821 | 11:01 |
| *** djoreilly has joined #openstack-neutron | 11:09 | |
| *** pcm_ has joined #openstack-neutron | 11:31 | |
| *** fouxm has quit IRC | 11:33 | |
| *** rossella_s has joined #openstack-neutron | 11:34 | |
| *** pcm_ has quit IRC | 11:36 | |
| *** pcm_ has joined #openstack-neutron | 11:37 | |
| *** salv-orlando has joined #openstack-neutron | 11:43 | |
| *** salv-orlando has quit IRC | 11:43 | |
| *** rkukura has quit IRC | 11:56 | |
| *** hxgqh1987 has joined #openstack-neutron | 11:59 | |
| *** gdubreui has joined #openstack-neutron | 12:00 | |
| *** gdubreui has quit IRC | 12:07 | |
| *** enikanorov has quit IRC | 12:24 | |
| *** enikanorov has joined #openstack-neutron | 12:24 | |
| *** hxgqh1987 has quit IRC | 12:26 | |
| *** hxgqh1987 has joined #openstack-neutron | 12:29 | |
| *** dims has quit IRC | 12:41 | |
| *** dims has joined #openstack-neutron | 12:41 | |
| openstackgerrit | Avishay Balderman proposed a change to openstack/neutron: LBaaS L7 model (WIP) https://review.openstack.org/61721 | 12:45 |
| *** yamahata_ has joined #openstack-neutron | 12:55 | |
| *** kruskakli has joined #openstack-neutron | 12:59 | |
| *** fouxm has joined #openstack-neutron | 13:02 | |
| *** yamahata_ has quit IRC | 13:02 | |
| *** yamahata_ has joined #openstack-neutron | 13:04 | |
| *** yamahata_ has quit IRC | 13:09 | |
| *** dims has quit IRC | 13:17 | |
| *** amuller has joined #openstack-neutron | 13:17 | |
| *** Abhishek_ has joined #openstack-neutron | 13:17 | |
| *** dims has joined #openstack-neutron | 13:19 | |
| *** yamahata_ has joined #openstack-neutron | 13:24 | |
| *** enikanorov__ has joined #openstack-neutron | 13:26 | |
| openstackgerrit | Sushil Kumar proposed a change to openstack/neutron: Updates tox.ini to use new features https://review.openstack.org/60825 | 13:29 |
| *** chandankumar_ has quit IRC | 13:29 | |
| *** chandankumar has joined #openstack-neutron | 13:30 | |
| *** yamahata_ has quit IRC | 13:35 | |
| *** enikanorov__ has quit IRC | 13:50 | |
| *** yamahata_ has joined #openstack-neutron | 13:54 | |
| openstackgerrit | Jianing Yang proposed a change to openstack/neutron: Implement basic functionalities for port forwarding https://review.openstack.org/60512 | 13:56 |
| *** yamahata_ has quit IRC | 13:57 | |
| *** yamahata_ has joined #openstack-neutron | 13:57 | |
| *** yamahata_ has quit IRC | 13:58 | |
| *** yamahata_ has joined #openstack-neutron | 13:59 | |
| *** hxgqh1987 has quit IRC | 14:00 | |
| *** enikanorov__ has joined #openstack-neutron | 14:00 | |
| *** enikanorov__ has joined #openstack-neutron | 14:01 | |
| *** julim has joined #openstack-neutron | 14:09 | |
| *** hxgqh1987 has joined #openstack-neutron | 14:10 | |
| *** jianingy is now known as jianingy_afk | 14:12 | |
| *** yamahata_ has quit IRC | 14:12 | |
| *** yamahata_ has joined #openstack-neutron | 14:13 | |
| *** bashok has quit IRC | 14:15 | |
| *** jprovazn has joined #openstack-neutron | 14:16 | |
| *** changbl has quit IRC | 14:19 | |
| *** rkukura has joined #openstack-neutron | 14:24 | |
| *** hxgqh1987 has quit IRC | 14:26 | |
| *** carl_baldwin has joined #openstack-neutron | 14:29 | |
| *** otherwiseguy has joined #openstack-neutron | 14:33 | |
| *** hxgqh1987 has joined #openstack-neutron | 14:33 | |
| *** SushilKM has quit IRC | 14:35 | |
| *** banix has joined #openstack-neutron | 14:37 | |
| *** heyongli has quit IRC | 14:40 | |
| anteaya | I recommend you read this post: http://lists.openstack.org/pipermail/openstack-dev/2013-December/022065.html | 14:41 |
| anteaya | seems that https://bugs.launchpad.net/tempest/+bug/1253896 is indeed a neutron issue | 14:41 |
| anteaya | has anyone seen salvatore today? | 14:42 |
| *** aveiga has joined #openstack-neutron | 14:42 | |
| anteaya | he is the neutron person with the most experience with bug 1253896 | 14:42 |
| *** hxgqh1987 has quit IRC | 14:42 | |
| *** alex_klimov has quit IRC | 14:46 | |
| *** hxgqh1987 has joined #openstack-neutron | 14:46 | |
| *** alagalah has joined #openstack-neutron | 14:51 | |
| *** enikanorov__ has quit IRC | 14:53 | |
| *** hxgqh1987 has quit IRC | 14:55 | |
| openstackgerrit | Zang MingJie proposed a change to openstack/neutron: Modular L2 TypeDriver refact (ovs-agent) https://review.openstack.org/61761 | 14:56 |
| openstackgerrit | Zang MingJie proposed a change to openstack/neutron: Modular L2 TypeDriver refact (service) https://review.openstack.org/37893 | 14:56 |
| *** jecarey has joined #openstack-neutron | 14:58 | |
| *** jistr has quit IRC | 14:58 | |
| *** jistr has joined #openstack-neutron | 15:02 | |
| anteaya | sdague jog0 e-r is stating this patch is hitting 1253896 https://review.openstack.org/#/c/60825/4 | 15:05 |
| anteaya | can you help me find the location of the Error reading SSH protocol banner error? so far, I can't find it | 15:06 |
| anteaya | I can find Create member 30fab5ef-fffd-4452-8be4-8145e1f78016 failed on device driver haproxy_ns | 15:07 |
| anteaya | 2013-12-12 14:18:48.744 6250 TRACE neutron.services.loadbalancer.drivers.haproxy.agent_manager Traceback (most recent call last): | 15:07 |
| anteaya | in lbaas.txt | 15:07 |
| *** clev has joined #openstack-neutron | 15:09 | |
| sdague | anteaya: more 1253896.yaml | 15:11 |
| sdague | query: > | 15:11 |
| sdague | message:"SSHTimeout: Connection to the" AND message:"via SSH timed | 15:11 |
| sdague | out." | 15:11 |
| sdague | AND filename:"console.html" | 15:11 |
| sdague | http://logs.openstack.org/25/60825/4/check/check-tempest-dsvm-neutron/fb72815/console.html#_2013-12-12_14_38_42_337 | 15:11 |
| sdague | is the triggering line | 15:11 |
| openstackgerrit | Mathieu Rohon proposed a change to openstack/neutron: Send fdb remove message when a port is migrated https://review.openstack.org/61767 | 15:11 |
| roaet | Any neutron cores willing to approve this patch? It has been waiting since september https://review.openstack.org/#/c/44942/ | 15:11 |
| anteaya | roaet: no approvals right now | 15:12 |
| roaet | Oh? | 15:12 |
| anteaya | we need to get two pip install -U patches in first | 15:13 |
| jog0 | anteaya: http://logs.openstack.org/25/60825/4/check/check-tempest-dsvm-neutron/fb72815/testr_results.html.gz | 15:13 |
| * roaet nods | 15:13 | |
| anteaya | see mailing list post | 15:13 |
| anteaya | http://lists.openstack.org/pipermail/openstack-dev/2013-December/022065.html | 15:13 |
| * roaet can't keep up with the ML | 15:13 | |
| anteaya | roaet: thanks, yes I know that is why I am trying to help | 15:13 |
| anteaya | if you want to review those patches, that would speed things along | 15:14 |
| anteaya | sdague jog0 thanks, I guess I am used to finding the actual error output in the title of the bug report or somewhere near the top | 15:15 |
| anteaya | jog0: so last week we were in agreement that this wasn't neutron and sdague believes neutron is playing a large role | 15:16 |
| anteaya | who wants to help me fill in the blanks so I can act effectively | 15:16 |
| anteaya | like with knowledge | 15:16 |
| jog0 | anteaya: yeah so last week I don't think it was as neutron bug | 15:17 |
| jog0 | if you look at the graph for that bug, there was a recent uptick in hits | 15:17 |
| anteaya | roaet: I screwed up my mailing list post: https://review.openstack.org/#/c/61508/ is the other one | 15:19 |
| anteaya | jog0: okay thanks, I will look | 15:19 |
| roaet | anteaya: if our packages aren't version locked could the default action of upgrading cause issues? | 15:20 |
| * roaet wishes all requirements were version locked | 15:20 | |
| *** alagalah has left #openstack-neutron | 15:21 | |
| *** carl_baldwin has quit IRC | 15:21 | |
| anteaya | roaet: I don't know | 15:22 |
| anteaya | do you have specific packages that you are concerned about? | 15:23 |
| *** carl_baldwin has joined #openstack-neutron | 15:23 | |
| roaet | anteaya: no specific ones. all of them that aren't pinned. I don't have a solution for 'getting new features' other than letting things upgrade magically, but I dislike the risk involved with that. | 15:24 |
| *** carl_baldwin has quit IRC | 15:24 | |
| roaet | Maybe a version review board or something. I prefer human intervention when it comes to bumping versions of dependencies. | 15:25 |
| *** jgrimm has joined #openstack-neutron | 15:25 | |
| anteaya | jog0 sdague am I missing something? did the fingerprint for bug 1253896 change at any point? | 15:25 |
| anteaya | roaet: I can understand that | 15:26 |
| *** AlexF has joined #openstack-neutron | 15:26 | |
| anteaya | this patch is to help folks with their local env | 15:26 |
| anteaya | you can pull the patch down and work with it locally noting any packaging issues you encounter in a comment on the patch | 15:27 |
| anteaya | that would help actually | 15:27 |
| jog0 | anteaya: the fingerprint is a relativly generic one | 15:28 |
| jog0 | that can be triggered by multiple root causes | 15:28 |
| jog0 | so we just cought a second bug in that one I think | 15:28 |
| jog0 | that or somehow the tests or config changed to make this worse | 15:29 |
| anteaya | could we possibly separate them out identifying individual causes? | 15:31 |
| anteaya | to me Error reading SSH protocol banner is different than the haproxy_ns failure | 15:32 |
| anteaya | unless they are the same and I just don't know it | 15:33 |
| jog0 | haproxy_ns? | 15:41 |
| anteaya | that was the error I saw | 15:42 |
| jog0 | anteaya: in short yes we could seperate them as long as we have two fingerprints | 15:42 |
| jog0 | anteaya: link | 15:42 |
| *** garyk has quit IRC | 15:42 | |
| anteaya | http://logs.openstack.org/25/60825/4/check/check-tempest-dsvm-neutron/fb72815/logs/screen-q-lbaas.txt.gz#_2013-12-12_14_18_45_707 | 15:43 |
| anteaya | I never saw an error mentioning banner anywhere in the logs I looked at | 15:43 |
| anteaya | the e-r fingerprint for 1253896 doesn't mention banner, but the bug report states it in the title | 15:45 |
| jog0 | so the q-lbaas error may be unrelated | 15:45 |
| anteaya | hmmmm | 15:45 |
| jog0 | anteaya: ahh so the ssh bug has changed a little bit over time Ithink | 15:45 |
| anteaya | unrelated to 1253896? | 15:45 |
| jog0 | which is to say its confusing | 15:45 |
| anteaya | yes | 15:45 |
| jog0 | anteaya: not 100% convinced its unrelated but I can't see how it is related | 15:46 |
| anteaya | hard to for me to follow up with the person fixing it if I don't have a clear understanding at the very least, of the fingerprint | 15:46 |
| anteaya | jog0: fair enough | 15:46 |
| anteaya | anything else I can do to dig deeper into this? | 15:46 |
| anteaya | to clarify if we do have just one bug or if this is multiple bugs being attributed to the same bug number? | 15:47 |
| jog0 | anteaya: so the fingerpint now is | 15:49 |
| jog0 | message:"SSHTimeout: Connection to the" AND message:"via SSH timed out." AND filename:"console.html" | 15:49 |
| jog0 | that means we are basing this bug on the output of the tempest run | 15:49 |
| jog0 | if we can find a line in neutron logs that allign with this failure then we can get a better bug classification on this | 15:49 |
| jog0 | anteaya: also this started happening a lot in the past few days so go back to the commit logs and see if anything is suspicious | 15:50 |
| *** vkozhukalov has quit IRC | 15:51 | |
| *** carl_baldwin has joined #openstack-neutron | 15:53 | |
| anteaya | I will do my best | 15:54 |
| anteaya | so this might be a tempest commit or a neutron commit, yes? | 15:55 |
| jog0 | anteaya: my guess is: neutron, tempest, nova, devstack-gate | 15:55 |
| jog0 | looking at neutron now | 15:55 |
| jog0 | 'git log --no-merges --since=3.days' | 15:55 |
| anteaya | k | 15:58 |
| anteaya | just in terms of my groking the elastic-recheck goodness this error "Timed out waiting for thing 5de6f55a-7f53-4c4e-b69e-3c16d1988d2c to become ACTIVE | 15:58 |
| anteaya | " | 15:58 |
| anteaya | triggered https://bugs.launchpad.net/nova/+bug/1257626 | 15:59 |
| *** SushilKM has joined #openstack-neutron | 15:59 | |
| anteaya | but I can't see the mention of RPC | 15:59 |
| jog0 | http://logs.openstack.org/45/61745/1/check/check-tempest-dsvm-neutron/3d82336/logs/screen-q-svc.txt.gz?level=TRACE | 15:59 |
| anteaya | not in here: http://logs.openstack.org/08/61508/1/check/gate-tempest-dsvm-large-ops/775535b/testr_results.html.gz | 15:59 |
| jog0 | anteaya: I can only look at one bug at a time | 16:00 |
| jog0 | I am still on 1253896 | 16:00 |
| jog0 | so the q-svc log above ^ | 16:00 |
| jog0 | anteaya: thats a lot of stacktraces | 16:00 |
| jog0 | don't know which ones are normal or not | 16:00 |
| *** otherwiseguy has quit IRC | 16:00 | |
| jog0 | but fixing those will make this bug a little clearer | 16:00 |
| *** otherwiseguy has joined #openstack-neutron | 16:01 | |
| anteaya | jog0: yeah sorry | 16:01 |
| anteaya | fixing the stacktraces for the testr_results for 61508 | 16:02 |
| anteaya | sorry 61745 | 16:02 |
| anteaya | jog0: 61745 is a heat patch: https://review.openstack.org/#/c/61745 | 16:03 |
| anteaya | sorry fixing it for q-svc | 16:03 |
| jog0 | http://logstash.openstack.org/#eyJzZWFyY2giOiJcIiBTZXNzaW9uJ3MgdHJhbnNhY3Rpb24gaGFzIGJlZW4gcm9sbGVkIGJhY2sgZHVlIHRvIGEgcHJldmlvdXMgZXhjZXB0aW9uIGR1cmluZyBmbHVzaC4gXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzODY4NjQxOTE3NTV9 | 16:04 |
| jog0 | message:" Session's transaction has been rolled back due to a previous exception during flush. " | 16:04 |
| jog0 | that doesn't signify a failure but that is a new bug that be related | 16:04 |
| jog0 | heh thats haproxy | 16:05 |
| anteaya | okay | 16:06 |
| *** yamahata_ has quit IRC | 16:07 | |
| *** ywu has joined #openstack-neutron | 16:07 | |
| anteaya | jog0 sdague which test results in screen-q-svc.txt? http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/network | 16:08 |
| jog0 | anything that talks to neutron | 16:09 |
| *** mlavalle has joined #openstack-neutron | 16:09 | |
| jog0 | screen-q-svc is neutron server | 16:09 |
| anteaya | so if my mission is to fix the stacktraces for q-svc | 16:09 |
| jog0 | those are all stacktraces in neutron | 16:10 |
| anteaya | how do I begin my locating the file or files generating those stacktraces | 16:10 |
| *** dhellmann is now known as dhellmann_ | 16:10 | |
| anteaya | so I can id them for clean up? | 16:10 |
| anteaya | or are you saying file bugs for each of the errors that show up | 16:10 |
| jog0 | you don't really need to know what test generated them (but you can find that via tracing request-ids) | 16:11 |
| jog0 | yeah file a bug for each stacktrace | 16:11 |
| jog0 | http://logs.openstack.org/45/61745/1/check/check-tempest-dsvm-neutron/3d82336/logs/screen-q-svc.txt.gz?level=TRACE#_2013-12-12_14_34_40_310 | 16:11 |
| anteaya | thus eliminating the reason for the stacktrace being generated | 16:11 |
| jog0 | yeah, so this many stacktraces makes it very easy for a gate failing stacktrace to hide in the noise | 16:11 |
| anteaya | k | 16:12 |
| anteaya | so fir the link you gave me this is the error: MemberNotFound: Member 21f6ce73-fff8-4a7e-b798-527a4bb64b92 could not be found | 16:12 |
| anteaya | for | 16:12 |
| anteaya | yes? | 16:12 |
| jog0 | anteaya: yes, but the full stacktrace is the important part | 16:13 |
| *** amritanshu_RnD has quit IRC | 16:13 | |
| jog0 | the bug should say, loadbalancer is stacktracing during passing jobs (you would have to confirm that part) | 16:14 |
| jog0 | then include a logstash query | 16:14 |
| anteaya | how do I create a logstash query? | 16:15 |
| anteaya | I tried yesterday and failed miserably | 16:15 |
| anteaya | over a million hits for my query, very useless | 16:15 |
| *** markmcclain has joined #openstack-neutron | 16:16 | |
| jog0 | that one is hard | 16:16 |
| jog0 | for PoolMonitorAssociationNotFound: Monitor 26a39bb9-5a68-4fdd-853f-123e0f04125d is not associated with Pool 2bfcdb7a-8e5d-4bbc-8510-185916c863b1 | 16:16 |
| jog0 | you can do message:" PoolMonitorAssociationNotFound: Monitor 26a39bb9-5a68-4fdd-853f-123e0f04125d is not associated with Pool 2bfcdb7a-8e5d-4bbc-8510-185916c863b1 | 16:16 |
| jog0 | you can do message:" PoolMonitorAssociationNotFound" | 16:17 |
| anteaya | k | 16:17 |
| anteaya | let me try that | 16:17 |
| jog0 | message:"PoolMonitorAssociationNotFound" AND filename:"logs/screen-q-svc.txt" AND loglevel:"ERROR" | 16:18 |
| anteaya | ah even better | 16:18 |
| jog0 | if you click on a result, you get a list of attributes | 16:18 |
| jog0 | and if you click on the magnifying glass that gets added to the query | 16:19 |
| jog0 | FYI that query is from this patch I9fd90a1321611d202ef838681273081fa6c1686a | 16:19 |
| anteaya | http://bit.ly/18FVAnd | 16:19 |
| jog0 | the haproxy stuff is bad | 16:20 |
| jog0 | err stacktracey | 16:20 |
| anteaya | one thing at a time | 16:20 |
| jog0 | anteaya: I would search for last 7 days at least | 16:20 |
| anteaya | k | 16:20 |
| openstackgerrit | Avishay Balderman proposed a change to openstack/neutron: LBaaS L7 model (WIP) https://review.openstack.org/61721 | 16:21 |
| anteaya | http://bit.ly/1gsrsAx | 16:22 |
| *** changbl has joined #openstack-neutron | 16:22 | |
| anteaya | that one started two days ago | 16:22 |
| jog0 | anteaya: yup | 16:22 |
| anteaya | okay now let's address "if you click on a result, you get a list of attributes" | 16:22 |
| anteaya | build change looks like a good bit of info | 16:23 |
| anteaya | how does that help me | 16:23 |
| jog0 | things like build_patchset, build_uuid,filename | 16:23 |
| jog0 | etc | 16:23 |
| jog0 | you can click the magnifying glass in the action column to narrow your search | 16:23 |
| *** yfried has quit IRC | 16:24 | |
| anteaya | clicking | 16:24 |
| *** SushilKM has quit IRC | 16:25 | |
| markmcclain | anteaya: hi | 16:25 |
| anteaya | markmcclain: hello | 16:25 |
| anteaya | trying to sort where we are on bug https://bugs.launchpad.net/tempest/+bug/1253896 | 16:25 |
| anteaya | jog0 is helping | 16:25 |
| anteaya | can't reach salvatore yet | 16:26 |
| *** SushilKM has joined #openstack-neutron | 16:26 | |
| anteaya | 1253896 is openstack's worst bug right now | 16:26 |
| anteaya | would be great if we could make some progress on it today | 16:26 |
| anteaya | jog0: I'm still uncertain about what the magnifying glass does | 16:27 |
| anteaya | but we can move on | 16:27 |
| markmcclain | anteaya: when I spoke with salv-orlando he was still working on it | 16:28 |
| anteaya | great | 16:28 |
| anteaya | did he by change submit any kind of patch yet? | 16:28 |
| anteaya | or do you know when he will be online? | 16:28 |
| jog0 | anteaya: my brain is fried (its pat 5:30 here) | 16:29 |
| jog0 | look up a kibana tutorial | 16:29 |
| jog0 | that should help with logstash.openstack.org | 16:29 |
| jog0 | kibana is the UI on logstash.openstack.org | 16:29 |
| anteaya | jog0: thanks for all your help | 16:30 |
| anteaya | hope Barcelona is going well | 16:30 |
| anteaya | g'night | 16:30 |
| *** srao has joined #openstack-neutron | 16:32 | |
| *** markmcclain has quit IRC | 16:33 | |
| anteaya | obondarev: can you take a look at https://review.openstack.org/#/c/40381/ please? | 16:36 |
| srao | Noob using devstack. My bare metal server has eth0 connected to internet. eth2 connected to another blade. I want to run openstack as multi-node. How? Pls point to worksheets. I could only find single nic procedures for install. | 16:36 |
| anteaya | particularly looking at jog0's comment | 16:37 |
| anteaya | hi srao | 16:37 |
| anteaya | are you using devstack as a development environment? | 16:37 |
| *** amuller has quit IRC | 16:37 | |
| srao | @anteaya: Yes. | 16:37 |
| *** garyk has joined #openstack-neutron | 16:38 | |
| *** Abhishek_ has quit IRC | 16:38 | |
| anteaya | great | 16:38 |
| mlavalle | nati_ueno: sorry for last nigh…. as I said, I suspected I was missing something ;-( | 16:39 |
| anteaya | so what instructions are you following for your devstack installation? | 16:39 |
| anteaya | mlavalle: hey there | 16:39 |
| mlavalle | anteaya; hey | 16:39 |
| anteaya | mlavalle: have you had a chance to assess the results of nati_ueno's patch yet? | 16:39 |
| srao | I tried the devstack multinode procedure on the devstack.org homepage. | 16:40 |
| *** otherwiseguy has quit IRC | 16:40 | |
| mlavalle | not yet | 16:40 |
| anteaya | srao: can you post the link for that please | 16:40 |
| *** otherwiseguy has joined #openstack-neutron | 16:40 | |
| anteaya | mlavalle: okay let me know what you find once you have | 16:40 |
| mlavalle | give me a little bit | 16:40 |
| srao | http://devstack.org/guides/multinode-lab.html | 16:41 |
| anteaya | mlavalle: sure | 16:42 |
| anteaya | srao: what image are you using? | 16:43 |
| srao | ubuntu 12.04 on the server | 16:43 |
| anteaya | have you successfully installed devstack at any point in the past? | 16:43 |
| srao | yes. The default procedure works. | 16:45 |
| anteaya | great | 16:45 |
| *** Abhishek_ has joined #openstack-neutron | 16:45 | |
| anteaya | srao: have you been able to implement the instructions for the multi-node lab as outlined? | 16:47 |
| srao | Yes. But I loose internet connectivity | 16:48 |
| srao | when I specify eth2 instead of eth0 | 16:48 |
| *** SumitNaiksatam has quit IRC | 16:48 | |
| openstackgerrit | dekehn proposed a change to openstack/neutron: extra_dhcp_opt add checks for empty strings https://review.openstack.org/59858 | 16:49 |
| anteaya | srao: can you paste any output you have? paste.openstack.org unless you like something else | 16:49 |
| *** ekarlso has quit IRC | 16:50 | |
| *** jistr has quit IRC | 16:50 | |
| *** ekarlso has joined #openstack-neutron | 16:51 | |
| srao | @anateaya: What output do you need? There are so many screen windows spewing logs | 16:54 |
| *** tnadeau has joined #openstack-neutron | 16:57 | |
| mlavalle | anteaya: nati_ueno's batch fixed https://bugs.launchpad.net/tempest/+bug/1259282 | 16:58 |
| *** emagana has joined #openstack-neutron | 16:58 | |
| *** srao has left #openstack-neutron | 16:59 | |
| mlavalle | anteaya: so that sends us back to analyze what's triggering the next incarnation of https://bugs.launchpad.net/neutron/+bug/1251448 | 17:01 |
| anteaya | mlavalle: yay | 17:01 |
| anteaya | mlavalle: okay, any inital thoughts | 17:02 |
| mlavalle | anteaya; looking…. | 17:02 |
| anteaya | k | 17:02 |
| mestery | FYI: Neutron third-party testing meeting in #openstack-meeting-alt now (Agenda on the etherpad here: https://etherpad.openstack.org/p/multi-node-neutron-tempest) | 17:03 |
| mestery | For those interested. | 17:03 |
| *** safchain has quit IRC | 17:03 | |
| emagana | mestery: I do not see antone there! | 17:04 |
| emagana | anyone* | 17:04 |
| mestery | ??? | 17:04 |
| mestery | We're all talking now emagana :) | 17:04 |
| emagana | mine IRC client is making me a joke! | 17:04 |
| *** SumitNaiksatam has joined #openstack-neutron | 17:05 | |
| anteaya | I need a core to +A this patch: https://review.openstack.org/#/c/60825/ | 17:06 |
| anteaya | nati_ueno: are you about? | 17:06 |
| *** srao has joined #openstack-neutron | 17:08 | |
| *** afazekas has quit IRC | 17:10 | |
| *** amotoki has joined #openstack-neutron | 17:13 | |
| *** JunPark has joined #openstack-neutron | 17:13 | |
| *** rudrarugge has joined #openstack-neutron | 17:13 | |
| mestery | anteaya: I can +A that patch if you want | 17:15 |
| anteaya | yes please | 17:16 |
| mestery | anteaya: Done. | 17:16 |
| anteaya | thanks | 17:16 |
| *** yfried has joined #openstack-neutron | 17:16 | |
| *** jpark has joined #openstack-neutron | 17:17 | |
| mlavalle | anteaya: we now hit a problem with XML client….. that trips tempest.api.network.test_networks.BulkNetworkOpsXML.test_bulk_create_delete_network | 17:17 |
| mlavalle | which leaves networks alive and that, in turn, triggers other tests trying to attach instances to a default network when several networks are possible…. I'll triage it in more detail this afternoon | 17:17 |
| roaet | Are there currently ml2 issues that are breaking devstack that we know about? https://bugs.launchpad.net/devstack/+bug/1260411 | 17:18 |
| mestery | roaet: I saw a few reviews for DB migration issues with devstack, yes. | 17:18 |
| mestery | I reviewed them this morning. | 17:18 |
| mestery | Can't recall what htey are, in an IRC meeting ATM as well. | 17:18 |
| anteaya | mlavalle: awesome thanks | 17:18 |
| *** JunPark_ has joined #openstack-neutron | 17:19 | |
| anteaya | if you don't beat me too it, I will add your above comments to that bug report | 17:19 |
| *** JunPark has quit IRC | 17:21 | |
| *** garyk has quit IRC | 17:21 | |
| srao | @anteaya: How can I get neutron to create an external bridge EACH for physical interface present using dashboard? | 17:21 |
| srao | I should've sent that question to all audience :-) | 17:22 |
| *** jpark has quit IRC | 17:22 | |
| anteaya | using the dashboard? I am not sure horizon has the functionality | 17:22 |
| anteaya | we all read all the stuff | 17:23 |
| *** decede has quit IRC | 17:23 | |
| srao | ok. In that case, how do I get it done in neutron? Is there a write up that you can point me to? | 17:23 |
| anteaya | I am mostly just the greeter and clarifier of questions, other folks jump in once they have a better sense of what you are looking for | 17:23 |
| anteaya | not that I know of | 17:23 |
| anteaya | dtroyer is the devstack guy and is in #openstack-qa | 17:24 |
| anteaya | you might try talking directly to him to get your devstack up | 17:24 |
| anteaya | if this is for developement purposes | 17:24 |
| srao | Thanks. I'll try talking to him. | 17:25 |
| anteaya | sure | 17:25 |
| *** garyk has joined #openstack-neutron | 17:28 | |
| *** jaypipes has joined #openstack-neutron | 17:28 | |
| *** jaypipes has quit IRC | 17:28 | |
| *** AlexF has quit IRC | 17:29 | |
| *** nati_ueno has quit IRC | 17:34 | |
| *** djbkd has joined #openstack-neutron | 17:36 | |
| *** amotoki has quit IRC | 17:42 | |
| SumitNaiksatam | rudrarugge: for your 3rd part testing, did you need to put the test box on the public network? | 17:45 |
| rudrarugge | Yes we have a public server(s) | 17:45 |
| SumitNaiksatam | rudrarugge: ok thanks, so it seems like that would be the requirement? | 17:46 |
| rudrarugge | SumitNaiksatam: We launch VMs on these and then run the tests | 17:46 |
| rudrarugge | SumitNaiksatam: yes it seems like that | 17:46 |
| *** emagana has quit IRC | 17:46 | |
| rudrarugge | SumitNaiksatam: now we are trying to plug in a Jenkins and patch filter framework | 17:46 |
| SumitNaiksatam | rudrarugge: great, hope we can also catch up soon :-P | 17:47 |
| rudrarugge | SumitNaiksatam: surely - need to work on the fwaas also from my side :) | 17:47 |
| openstackgerrit | Evgeny Fedoruk proposed a change to openstack/neutron: Extending quota support for neutron LBaaS entities https://review.openstack.org/58720 | 17:47 |
| SumitNaiksatam | rudrarugge: yes, looking forward to it | 17:47 |
| *** decede has joined #openstack-neutron | 17:51 | |
| *** yfried has quit IRC | 17:51 | |
| *** rossella_s has quit IRC | 17:52 | |
| *** JunPark_ has quit IRC | 17:53 | |
| *** jpich has quit IRC | 17:54 | |
| *** nati_ueno has joined #openstack-neutron | 18:00 | |
| *** ygbo_ has quit IRC | 18:02 | |
| *** decede has quit IRC | 18:11 | |
| *** vkozhukalov has joined #openstack-neutron | 18:12 | |
| anteaya | nati_ueno: thanks for https://review.openstack.org/#/c/61234/ | 18:13 |
| anteaya | mlavalle: is working on the next bug now | 18:14 |
| nati_ueno | your welcome. Thank you for finding this bug | 18:14 |
| mlavalle | nati_ueno: sorry for last night | 18:14 |
| nati_ueno | mlavalle: ? last night? | 18:14 |
| mlavalle | nati_ueno: yeah, my comments in the review… I knew I was missing something | 18:15 |
| nati_ueno | mlavalle: Totally no problem :) | 18:15 |
| mlavalle | nati_ueno: :-) | 18:15 |
| *** fouxm has quit IRC | 18:18 | |
| anteaya | I'm stepping out, time for some sunshine. I should be back in an hour and a half or so | 18:24 |
| anteaya | if savatore comes online if someone could ask him for an update on https://bugs.launchpad.net/tempest/+bug/1253896 that would be much appreciated | 18:25 |
| *** decede has joined #openstack-neutron | 18:25 | |
| anteaya | http://lists.openstack.org/pipermail/openstack-dev/2013-December/022052.html for context | 18:26 |
| *** suresh12 has joined #openstack-neutron | 18:27 | |
| *** salv-orlando has joined #openstack-neutron | 18:27 | |
| *** jhurlbert has joined #openstack-neutron | 18:29 | |
| *** decede has quit IRC | 18:32 | |
| *** jhurlbert has quit IRC | 18:41 | |
| *** jhurlbert has joined #openstack-neutron | 18:42 | |
| *** jhurlbert has quit IRC | 18:44 | |
| openstackgerrit | Zang MingJie proposed a change to openstack/neutron: Modular L2 TypeDriver refact (service) https://review.openstack.org/37893 | 18:44 |
| openstackgerrit | Zang MingJie proposed a change to openstack/neutron: Modular L2 TypeDriver refact (ovs-agent) https://review.openstack.org/61761 | 18:44 |
| *** nati_uen_ has joined #openstack-neutron | 18:46 | |
| *** decede has joined #openstack-neutron | 18:46 | |
| *** nati_ueno has quit IRC | 18:49 | |
| *** decede has quit IRC | 18:51 | |
| *** nati_uen_ has quit IRC | 18:54 | |
| *** nati_ueno has joined #openstack-neutron | 18:55 | |
| *** djbkd has quit IRC | 19:02 | |
| *** jgrimm has quit IRC | 19:03 | |
| *** Abhishek_ has quit IRC | 19:08 | |
| *** decede has joined #openstack-neutron | 19:09 | |
| *** mlavalle has quit IRC | 19:15 | |
| *** dims has quit IRC | 19:33 | |
| SumitNaiksatam | anteaya: is the embargo still on for approving (+A) patches? | 19:34 |
| *** dims has joined #openstack-neutron | 19:34 | |
| roaet | is +A the new +2? | 19:34 |
| *** alex_klimov has joined #openstack-neutron | 19:34 | |
| roaet | or are they not the same? | 19:35 |
| SumitNaiksatam | anteaya: looking at the gate queue it seems, it will be better to spare it for some time | 19:35 |
| SumitNaiksatam | roaet: second +2 is not automatic approval | 19:35 |
| roaet | ot it | 19:35 |
| roaet | +g | 19:35 |
| SumitNaiksatam | roaet: the approve button still has to be hit -> +A :-) | 19:35 |
| *** salv-orlando has quit IRC | 19:36 | |
| *** salv-orlando has joined #openstack-neutron | 19:36 | |
| *** SushilKM__ has joined #openstack-neutron | 19:43 | |
| *** SushilKM has quit IRC | 19:46 | |
| anteaya | SumitNaiksatam: let me check if the pip install -U patches have merged yet | 19:46 |
| anteaya | that is what we are waiting on | 19:46 |
| SumitNaiksatam | anteaya: i think that has gone through | 19:46 |
| SumitNaiksatam | anteaya: but the queue is pretty long | 19:46 |
| anteaya | https://review.openstack.org/#/c/60825/4 is still in the gate | 19:47 |
| anteaya | so is https://review.openstack.org/#/c/61508/ | 19:47 |
| SumitNaiksatam | ok | 19:47 |
| anteaya | once those two have merged we are good for approvals again | 19:47 |
| SumitNaiksatam | anteaya: so we wait | 19:47 |
| anteaya | thanks for checking SumitNaiksatam | 19:47 |
| anteaya | we do | 19:47 |
| SumitNaiksatam | anteaya: sure | 19:47 |
| *** jprovazn has quit IRC | 19:58 | |
| *** mlavalle has joined #openstack-neutron | 19:59 | |
| *** jlibosva has quit IRC | 19:59 | |
| *** markmcclain has joined #openstack-neutron | 20:07 | |
| jog0 | anteaya: any progress on https://bugs.launchpad.net/tempest/+bug/1253896 ? | 20:08 |
| anteaya | jog0: not so far | 20:11 |
| anteaya | salvatore seems to know the most about this bug and so far I have not be able to find him | 20:11 |
| anteaya | still working on getting a status update | 20:12 |
| anteaya | jog0: is the sun up where you are? | 20:12 |
| jog0 | anteaya: no its been down for a long time | 20:12 |
| anteaya | I also need to improve my kibana foo, as you pointed out | 20:12 |
| anteaya | jog0: are you still in Spain? | 20:12 |
| jog0 | anteaya: thanks, anyway this bug is bugging us | 20:13 |
| jog0 | anteaya: yup fly out in the AM | 20:13 |
| anteaya | jog0: no kidding | 20:13 |
| anteaya | it is definitely a priority for me too | 20:13 |
| jog0 | anteaya: thanks | 20:14 |
| jog0 | night | 20:14 |
| anteaya | my pleasure, thanks for all your help | 20:14 |
| anteaya | night | 20:14 |
| *** yfried has joined #openstack-neutron | 20:20 | |
| *** djoreilly has quit IRC | 20:21 | |
| *** otherwiseguy has quit IRC | 20:28 | |
| *** SushilKM__ has quit IRC | 20:33 | |
| anteaya | jog0: just for the sake of conversation, for some reason I come up empty searching for kibana tutorials | 20:39 |
| anteaya | the best material I got was from clarkb's patch: https://review.openstack.org/#/c/61298/4/README.rst | 20:40 |
| anteaya | point to lucene docs: http://lucene.apache.org/core/4_0_0/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#package_description | 20:40 |
| anteaya | so I'll start there | 20:40 |
| anteaya | though I did come up with some nice distraction on ikebana | 20:40 |
| anteaya | so pretty | 20:40 |
| *** otherwiseguy has joined #openstack-neutron | 20:44 | |
| *** Swami_ has joined #openstack-neutron | 21:02 | |
| *** rwsu has quit IRC | 21:05 | |
| *** ichihara has joined #openstack-neutron | 21:07 | |
| Swami_ | hi nachi | 21:10 |
| nati_ueno | Swami_: hi | 21:11 |
| SumitNaiksatam | Swami_ nati_ueno: hi | 21:11 |
| Swami_ | Ok, summit: mentioned that he will join | 21:11 |
| Swami_ | Hi all | 21:11 |
| SumitNaiksatam | Swami_: i am here :-) | 21:11 |
| Swami_ | Thanks for joining the meeting in a short notice | 21:11 |
| Swami_ | The agenda here is to discuss about the services when we run the L3-Service in a distributed mode versus the centralized mode | 21:11 |
| *** djbkd has joined #openstack-neutron | 21:12 | |
| Swami_ | There are some services such as VPN, which does not need to be distributed, but some services such as FW can be distributed. How are we going to achieve this | 21:12 |
| *** rkukura has quit IRC | 21:13 | |
| *** djbkd has quit IRC | 21:13 | |
| *** alex_klimov has quit IRC | 21:13 | |
| *** rkukura has joined #openstack-neutron | 21:13 | |
| SumitNaiksatam | Swami_: that's true, the distribution will create a complication for fwaas | 21:13 |
| *** rkukura has left #openstack-neutron | 21:13 | |
| SumitNaiksatam | Swami_: you have thoughts on this? | 21:13 |
| Swami_ | No, I am investigating it right now, and I wanted to have a cleaner approach. I initially was thinking to run all the services in a centralized router and have the basic L3 distributed, but as you mentioned that would be a problem for FW. | 21:14 |
| *** AlexF has joined #openstack-neutron | 21:14 | |
| Swami_ | Also if I think of distributing all the services to all nodes, that would be a problem for VPN. | 21:15 |
| SumitNaiksatam | Swami_: ok | 21:15 |
| Swami_ | Is there a way that we can identify the router as a core router and apply the services only to those core router and have the traffic forwarded to the right network | 21:16 |
| Swami_ | nachi: You mentioned in your email that we can make use of the "provider" flag. | 21:17 |
| nati_ueno | yes | 21:17 |
| Swami_ | But the provider flag will only determine what driver to use for the services. | 21:17 |
| nati_ueno | IMO, it is sufficient | 21:18 |
| nati_ueno | for VPN | 21:18 |
| nati_ueno | We can identify valid combination | 21:18 |
| SumitNaiksatam | Swami_: i am not very clear on the "core" router notion that you mentioned | 21:19 |
| Swami_ | sumit: I mentioned something similar to a service router rather than a normal router, so that we only apply the services on that router. | 21:19 |
| SumitNaiksatam | Swami_: ah ok | 21:19 |
| Swami_ | But still that seems to be same as the Network node router that we have today. | 21:19 |
| SumitNaiksatam | Swami_: so there will be a single services' router? | 21:20 |
| Swami_ | sumit: Yes, and that service router can be attached to any compute node that the tenant wants to add it to. | 21:20 |
| SumitNaiksatam | Swami_: okay, but then the choke point shifts to this service router | 21:21 |
| SumitNaiksatam | ? | 21:21 |
| Swami_ | Yes, exactly right.- it is no different than a Network node | 21:21 |
| SumitNaiksatam | Swami_: hmmm | 21:21 |
| SumitNaiksatam | Swami_: that said, it could be a starting point | 21:22 |
| Swami_ | But if we wanted to distribute the services, say for example do you want to run VPN on all the compute Nodes, just because you have a distributed router setup | 21:22 |
| Swami_ | If we have a hybrid model, where we have centralized Router and distributed router and the centralized router will host the centralized services and the distributed router will host the distributed services such as FW, then we may have to try to tie these two networks together through another internal gateway | 21:24 |
| SumitNaiksatam | Swami_: yeah | 21:24 |
| SumitNaiksatam | Swami_: i think we can incrementally work towards a more sophisticated model | 21:24 |
| Swami_ | sumit: Yes I agree with you that incremental approach to solve the services problem would be the right approach | 21:25 |
| SumitNaiksatam | Swami_: so with the "services'" router notion, there would two flavors of routers | 21:26 |
| SumitNaiksatam | ? | 21:26 |
| Swami_ | sumit: Running east-west traffic and having a centralized router ( Network Node) for all services will work, but still it would break the FW. | 21:26 |
| SumitNaiksatam | Swami_: you might need FW for east-west traffic as well | 21:27 |
| SumitNaiksatam | Swami_: so yeah | 21:27 |
| Swami_ | sumit: Yes, there should be a regular router providing L3 services and to the extent a Gateway service but any other services should be part of the service router | 21:27 |
| Swami_ | sumit: Yes, that's what I mentioned, even if we target the first use case the FW will break. | 21:27 |
| SumitNaiksatam | Swami_: is the current implementation of lbaas (with HA Proxy driver) affected by this at all? | 21:29 |
| Swami_ | sumit: I am not sure about the lbaas, that is the reason I include enikanorov, but he did not reply. | 21:30 |
| SumitNaiksatam | Swami_: my guess is that it won't be affected, but better to get confirmation | 21:30 |
| *** banix has quit IRC | 21:31 | |
| Swami_ | sumit: I think that it should also have some implications in distributing the HA Proxy. | 21:31 |
| Swami_ | But running it centralized should not cause any problem. | 21:31 |
| SumitNaiksatam | Swami_: hmmm may be…but would distributing the agent not still be independent of the router? | 21:32 |
| Swami_ | Sumit: In the FW case, if we plan to distribute the FW to all the routers in the node, will that be a major change, or the code should already support it. | 21:32 |
| SumitNaiksatam | Swami_: regarding FW, you mean send the same rules everywhere? | 21:32 |
| Swami_ | sumit: yes distributing the agent is independent of the router. But if the service is tied to the router, then we need to take care. | 21:33 |
| Swami_ | sumit: Yes | 21:33 |
| Swami_ | Will that cause any performance issues. | 21:33 |
| SumitNaiksatam | Swami_: i don't think that should be a big issue code wise | 21:33 |
| SumitNaiksatam | Swami_: performance yes, may be | 21:33 |
| SumitNaiksatam | Swami_: performance degrades linearly with the number of rules | 21:33 |
| Swami_ | But in this case the number of rules will not change. | 21:34 |
| SumitNaiksatam | Swami_: so if we can selectively distribute the rules, that would be ideal | 21:34 |
| SumitNaiksatam | Swami_: if apply the same rules at all points in the distributed router, the number of rules will not increase | 21:35 |
| Swami_ | sumit: Today when we configure a Firewall, the tenant create a Policy, apply the rules and then associates with a router or with a network | 21:35 |
| SumitNaiksatam | Swami_: i guess what i was thinking is that not all rules might be required at all the distributed points | 21:35 |
| nati_ueno | sorry i should go. let's me catch up log later | 21:35 |
| nati_ueno | TL! | 21:35 |
| *** nati_ueno has quit IRC | 21:36 | |
| SumitNaiksatam | Swami_: there is no association/insertion by the tenant | 21:36 |
| SumitNaiksatam | Swami_: the implementation automatically associates with a router | 21:36 |
| Swami_ | nachi: Thanks, I will talk to you later. | 21:36 |
| SumitNaiksatam | Swami_: thats the default and only mode | 21:36 |
| SumitNaiksatam | Swami_: we are trying to get away from there, by introducing the insertion context so that we can support different insertion modes | 21:37 |
| SumitNaiksatam | Swami_: including the "network" association you mention | 21:37 |
| Swami_ | sumit: So if the same router is created across the compute node, the rules will be automatically applied. | 21:37 |
| SumitNaiksatam | Swami_: yes | 21:37 |
| *** yfried has quit IRC | 21:38 | |
| Swami_ | Then theoretically the existing firewall implementation should work if we distribute the router | 21:38 |
| Swami_ | sumit: In other words, is there any dependency that the current Firewall should be only centralized. | 21:40 |
| SumitNaiksatam | Swami_: need to think a little more, but i think that should work | 21:41 |
| Swami_ | sumit: Ok give it thought and I will keep you in the loop, meanwhile I will also check | 21:42 |
| SumitNaiksatam | Swami_: in the current implementation, l3_agent extends the fwaas base (we follow the mixin approach) | 21:42 |
| Swami_ | So if we use the same L3 agent, it should not be a problem | 21:42 |
| SumitNaiksatam | Swami_: so if, fwaas is configured, it should kick in wherever the l3 agent is run | 21:42 |
| SumitNaiksatam | Swami_: yes | 21:42 |
| SumitNaiksatam | Swami_: and the fwaas base implementation loads the fwaas driver | 21:43 |
| Swami_ | There were two approaches that we were thinking, either to use the existing L3 agent to support distributed or else to create an new dvr_agent. | 21:43 |
| SumitNaiksatam | Swami_: yeah, hence letting you know how it works today | 21:43 |
| SumitNaiksatam | Swami_: if you plan to create a different agent, then the fwaas dependency will have to be accounted for | 21:44 |
| Swami_ | If we were to address the Hybrid model, with centralized and distributed, we might need two agents, otherwise, if we were to only make it either distributed or centralized, then we would go with the current L3-agent. | 21:44 |
| SumitNaiksatam | Swami_: we don't want to create a separate agent for fw | 21:44 |
| SumitNaiksatam | Swami_: yeah | 21:44 |
| Swami_ | sumit: Yes agreed | 21:44 |
| *** jecarey has quit IRC | 21:45 | |
| SumitNaiksatam | Swami_: for the perimeter firewall case, the firewall should be deployed at any point where the gateway resides | 21:45 |
| Swami_ | sumit: Is service chaining address for Icehouse? | 21:45 |
| SumitNaiksatam | Swami_: yes, its currently targeted | 21:45 |
| Swami_ | How will service chaining affect the services. | 21:46 |
| Swami_ | Will service chaining give us more flexibility in where to run the services and how to connect it to the internal private network. | 21:47 |
| SumitNaiksatam | Swami_: its not really that service chaining will affect the services' implementation | 21:47 |
| SumitNaiksatam | Swami_: we might have to tweak some things if we run into issues | 21:48 |
| *** yfried has joined #openstack-neutron | 21:48 | |
| *** marun has quit IRC | 21:48 | |
| SumitNaiksatam | Swami_: but more importantly, if services support the insertion context, then the entity that drives the chain implementation can insert each service based on that insertion context | 21:49 |
| Swami_ | Mainly service chaining helps the virtual appliance vendors and the hardware vendors to hook into their appliances to the required services. | 21:49 |
| SumitNaiksatam | Swami_: not necessarily, chaining does not make assumptions of virtual appliances or physical hardware | 21:49 |
| SumitNaiksatam | Swami_: sorry, i think thats what you were saying | 21:50 |
| Swami_ | In that case will it even make sense to distribute the services to all compute nodes? | 21:50 |
| SumitNaiksatam | Swami_: but that should be independent of chaining right? | 21:50 |
| SumitNaiksatam | Swami_: chaining should allow you to say, that i want a FW -> VPN | 21:51 |
| Swami_ | sumit: Yes my understanding is we have chaining and service insertion to add services where and when required. | 21:51 |
| SumitNaiksatam | Swami_: distribution might create implementation complications, for insertion/chaining | 21:52 |
| SumitNaiksatam | Swami_: however, conceptually it seems these should be independent | 21:52 |
| Swami_ | sumit: got it | 21:52 |
| SumitNaiksatam | Swami_: distributed aspect of the router is an implementation artifact (at least in my understanding) | 21:53 |
| Swami_ | sumit; are you working on the chaining and service insertion. | 21:53 |
| SumitNaiksatam | Swami_: yeah, along with other folks | 21:53 |
| Swami_ | sumit: Yes distributed router is completely implementation specific, but it has all these issues that we need to address. | 21:54 |
| SumitNaiksatam | Swami_: true…implementation is a challenge :-) | 21:54 |
| Swami_ | Do you have a weekly meeting for the chaining and service insertion, I would like to listen in. | 21:54 |
| SumitNaiksatam | Swami_: we were having earlier, but i suspended it so as not distract from the ongoing efforts around neutron gate stability and nova parity | 21:55 |
| SumitNaiksatam | Swami_: also, we have had so much discussion on this - so its actually time to put a patch up :-) | 21:56 |
| SumitNaiksatam | Swami_: else this will be analysis paralysis forever | 21:56 |
| Swami_ | sumit: I would take a look at the current FW code and see if there would be any other implications running it in a distributed way and then I will get back to you. | 21:56 |
| Swami_ | sumit: Thanks for your time, and I will discuss with Nachi on the VPN service. | 21:57 |
| SumitNaiksatam | Swami_: sure | 21:57 |
| Swami_ | Swami: I will continue to work on the distributed router implementation and probably put some use cases for the services. | 21:57 |
| SumitNaiksatam | Swami_: thanks for starting this discussion | 21:58 |
| Swami_ | sumit: Thanks and bye. Catch you later | 21:58 |
| SumitNaiksatam | Swami_: it will help for me to be sync with the distributed router discussion | 21:58 |
| SumitNaiksatam | Swami_: ttyl, bye | 21:58 |
| *** junpark_ has joined #openstack-neutron | 21:59 | |
| *** junpark_ has quit IRC | 21:59 | |
| *** AlexF has quit IRC | 22:00 | |
| Swami_ | sumit: sure I will include you in the loop | 22:00 |
| *** Swami_ has left #openstack-neutron | 22:00 | |
| *** changbl has quit IRC | 22:03 | |
| *** AlexF has joined #openstack-neutron | 22:03 | |
| *** junpark has joined #openstack-neutron | 22:03 | |
| *** julim_ has joined #openstack-neutron | 22:05 | |
| *** suresh12 has quit IRC | 22:05 | |
| *** suresh12_ has joined #openstack-neutron | 22:05 | |
| *** AlexF has quit IRC | 22:07 | |
| *** junpark has quit IRC | 22:07 | |
| *** djbkd has joined #openstack-neutron | 22:07 | |
| *** julim has quit IRC | 22:07 | |
| *** markmcclain has quit IRC | 22:07 | |
| anteaya | gate-tempest-dsvm-neutron is in very bad shape right now | 22:10 |
| anteaya | this bug is skyrocketed today | 22:11 |
| anteaya | does anyone have any chance to do some digging | 22:11 |
| anteaya | salv-orlando: you are online | 22:11 |
| anteaya | do you have any update on https://bugs.launchpad.net/tempest/+bug/1253896 | 22:12 |
| salv-orlando | I am | 22:12 |
| anteaya | thank you for being online | 22:12 |
| salv-orlando | I have unfortunately seen the rise in bug occurrence only a few hours ago | 22:12 |
| * anteaya nods | 22:12 | |
| anteaya | any thoughts at this point? | 22:12 |
| anteaya | can I round up any resources for you to help address this? | 22:13 |
| salv-orlando | I will try and root cause before bed; otherwise we can go for the usual "revert whatever is related approach" | 22:13 |
| *** banix has joined #openstack-neutron | 22:13 | |
| salv-orlando | marun has often helped. He should be UTC+8 these days. So perhaps he could be up in an hour or two | 22:13 |
| salv-orlando | I have no more than two hours before bed time | 22:13 |
| anteaya | understood | 22:14 |
| anteaya | anything you can do in that time is much apprciated | 22:14 |
| anteaya | and let me know where it stands so I can recruit other help | 22:14 |
| anteaya | any chance this patch might be playing a role? https://review.openstack.org/#/c/40381/ | 22:15 |
| *** vkozhukalov has quit IRC | 22:15 | |
| anteaya | it went in Dec. 10th | 22:15 |
| anteaya | jog0: mentioned that it is throwing many stacktraces in screen-q-svc.txt | 22:15 |
| anteaya | might be unrelated | 22:15 |
| anteaya | sdague: any additional information on 1253896? | 22:16 |
| salv-orlando | umm 40381 is not really related to timeout failures unless it prevents neutron from notifying agents | 22:17 |
| anteaya | d'oh he is in the qa meeting | 22:17 |
| anteaya | salv-orlando: okay, didn't think so | 22:17 |
| *** ichihara has left #openstack-neutron | 22:18 | |
| *** gdubreui has joined #openstack-neutron | 22:23 | |
| *** clev has quit IRC | 22:28 | |
| *** aveiga has quit IRC | 22:35 | |
| *** dave_tucker has joined #openstack-neutron | 22:40 | |
| *** banix has quit IRC | 22:42 | |
| *** rkukura has joined #openstack-neutron | 22:44 | |
| *** banix has joined #openstack-neutron | 22:49 | |
| anteaya | neutron tempest tests -- add link to relevant api to assist reviewers please, adding this in a comment is fine | 22:57 |
| *** rudrarug_ has joined #openstack-neutron | 23:00 | |
| *** clev has joined #openstack-neutron | 23:01 | |
| *** rudrarug_ has quit IRC | 23:01 | |
| *** dave_tucker has quit IRC | 23:01 | |
| *** rudrarugge has quit IRC | 23:02 | |
| *** pcm_ has quit IRC | 23:08 | |
| *** SumitNaiksatam has quit IRC | 23:09 | |
| *** dave_tucker has joined #openstack-neutron | 23:10 | |
| salv-orlando | anteaya: added a comment for bug 1253896. Still inconclusive so far. Hope the insights I found might be useful for someone else who can take over while I sleep. | 23:11 |
| anteaya | very good | 23:12 |
| anteaya | my thanks | 23:12 |
| anteaya | happy sleep | 23:12 |
| anteaya | need a recruit who knows the L3 agent and floating ips to continuing working on https://bugs.launchpad.net/neutron/+bug/1253896 | 23:14 |
| *** jgrimm has joined #openstack-neutron | 23:14 | |
| anteaya | mestery: any thoughts who would have that knowledge whom I could ping to work on this? | 23:15 |
| anteaya | do you? | 23:15 |
| *** rkukura has quit IRC | 23:16 | |
| mestery | anteaya: Looking | 23:18 |
| mestery | anteaya: I actually see this error in my own Tempest runs all the time too :( | 23:18 |
| anteaya | :( | 23:18 |
| mestery | anteaya: I would like to say I could work on this, but I am stretched fairly thin ATM. I can see about looking into this. | 23:19 |
| anteaya | mestery: awesome | 23:19 |
| mestery | anteaya: I may have someone on our team who can look at this, let me ping him and see. | 23:19 |
| anteaya | who else might you suggest as well | 23:19 |
| mestery | Will send email tonight once I get a response. | 23:19 |
| anteaya | this is openstack's top bug atm | 23:19 |
| mestery | anteaya: Understood. | 23:19 |
| anteaya | thanks | 23:19 |
| anteaya | mestery: also please update the bug report with any progress, and ask your team-mate to do the same | 23:24 |
| mestery | anteaya: I sent email, he should ping in channel if he is able to help, FYI. | 23:24 |
| anteaya | I'm going to keep recruiting and pestering until a solution is found | 23:24 |
| anteaya | awesome thank you | 23:25 |
| mestery | anteaya: Makes sense. I told him I could help but would need him to take the lead, so lets see. | 23:25 |
| anteaya | k | 23:25 |
| anteaya | I'll wander off and tend to dinner for a bit | 23:25 |
| * mestery wanders too | 23:25 | |
| *** suresh12_ has quit IRC | 23:27 | |
| *** suresh12 has joined #openstack-neutron | 23:27 | |
| *** djbkd has quit IRC | 23:30 | |
| *** suresh12 has quit IRC | 23:32 | |
| *** dave_tucker has quit IRC | 23:34 | |
| *** dave_tucker has joined #openstack-neutron | 23:36 | |
| *** suresh12 has joined #openstack-neutron | 23:38 | |
| *** alagalah has joined #openstack-neutron | 23:39 | |
| *** otherwiseguy has quit IRC | 23:47 | |
| *** yamahata_ has joined #openstack-neutron | 23:58 | |
| *** salv-orlando has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!