| opendevreview | liuyulong proposed openstack/neutron master: Add table for pps limitaion https://review.opendev.org/c/openstack/neutron/+/804213 | 03:50 |
|---|---|---|
| opendevreview | liuyulong proposed openstack/neutron master: Reuse the qos constants from neuron-lib https://review.opendev.org/c/openstack/neutron/+/804380 | 03:50 |
| opendevreview | liuyulong proposed openstack/neutron master: Make flow installation trunk size automatic adjustment https://review.opendev.org/c/openstack/neutron/+/765072 | 03:57 |
| opendevreview | Zhai Mengdong proposed openstack/neutron master: [OVN] Implement router gateway IP QoS https://review.opendev.org/c/openstack/neutron/+/749012 | 06:37 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Execute the quota reservation removal in an isolated DB txn https://review.opendev.org/c/openstack/neutron/+/809983 | 07:05 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/xena: Execute the quota reservation removal in an isolated DB txn https://review.opendev.org/c/openstack/neutron/+/811124 | 07:06 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [DVR] Check if SNAT iptables manager is initialized https://review.opendev.org/c/openstack/neutron/+/811318 | 07:11 |
| jpic | hi all, how to restore OVN database when restoring neutron from mysqldump? is there a command? | 07:15 |
| ralonsoh | jpic, that should be done by the sync workers | 07:15 |
| ralonsoh | those should sanitize the OVN DB according to the Neutron DB | 07:16 |
| hjensas | elvira: Hi, I want to try to reproduce https://bugs.launchpad.net/neutron/+bug/1944948, do you happen to have a good devstack configursation to set up neutron with ovn? | 07:25 |
| elvira | hjensas: sure! | 07:36 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena https://review.opendev.org/c/openstack/neutron-vpnaas/+/809795 | 07:45 |
| lajoskatona | slaweq, ralonsoh, obondarev: Hi, Could you please check these release related patches if You have few minutes: https://review.opendev.org/q/project:openstack/networking-odl+status:open+-age:2w ? | 07:47 |
| ralonsoh | lajoskatona, sure | 07:48 |
| slaweq | lajoskatona: sure | 07:48 |
| obondarev | lajoskatona, will do | 07:49 |
| slaweq | all done :) | 07:49 |
| ralonsoh | (I was late!) | 07:49 |
| obondarev | same for me :) | 07:50 |
| slaweq | lajoskatona: FYI, some time ago we added to our docs https://docs.openstack.org/neutron/latest/contributor/policies/code-reviews.html#neutron-code-review-practices some agreement that such mechanical patches like e.g. https://review.opendev.org/c/openstack/networking-odl/+/809087 can be merged with one +2 :) | 07:51 |
| lajoskatona | slaweq: ok, cool | 07:51 |
| opendevreview | Merged openstack/networking-odl master: Drop install_venv https://review.opendev.org/c/openstack/networking-odl/+/809908 | 08:12 |
| opendevreview | Merged openstack/networking-odl master: Update master for stable/xena https://review.opendev.org/c/openstack/networking-odl/+/809086 | 08:12 |
| opendevreview | Merged openstack/networking-odl master: Add Python3 yoga unit tests https://review.opendev.org/c/openstack/networking-odl/+/809087 | 08:23 |
| opendevreview | Yi Yang proposed openstack/neutron master: Use Openflow to implement DVR L3 https://review.opendev.org/c/openstack/neutron/+/805930 | 08:32 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Sanitize profile column of ml2_port_bindings table in the DB https://review.opendev.org/c/openstack/neutron/+/811411 | 09:01 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Enable QoS minimum packet rate rule for OVS backend https://review.opendev.org/c/openstack/neutron/+/805391 | 09:02 |
| opendevreview | Merged openstack/neutron stable/ussuri: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811343 | 09:06 |
| opendevreview | Merged openstack/neutron stable/victoria: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811342 | 09:06 |
| opendevreview | Merged openstack/neutron stable/wallaby: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811341 | 09:06 |
| opendevreview | Merged openstack/neutron stable/xena: Replace cirros 0.4.0 by 0.5.2 in ovn migration create-resources.sh.j2 https://review.opendev.org/c/openstack/neutron/+/811340 | 09:06 |
| seba | ralonsoh, could you have another look at the networksegments db constraints? I fixed the tests and added a release note to it. https://review.opendev.org/c/openstack/neutron/+/788714 | 09:11 |
| ralonsoh | seba, ok | 09:12 |
| opendevreview | Hemanth N proposed openstack/neutron stable/wallaby: [DVR] Set arp entries only for single IPs given as allowed addr pair https://review.opendev.org/c/openstack/neutron/+/811497 | 09:14 |
| opendevreview | Hemanth N proposed openstack/neutron stable/victoria: [DVR] Set arp entries only for single IPs given as allowed addr pair https://review.opendev.org/c/openstack/neutron/+/811498 | 09:14 |
| seba | tnx :) | 09:14 |
| opendevreview | Hemanth N proposed openstack/neutron stable/ussuri: [DVR] Set arp entries only for single IPs given as allowed addr pair https://review.opendev.org/c/openstack/neutron/+/811499 | 09:14 |
| ralonsoh | seba, we are now in Yoga | 09:23 |
| ralonsoh | you need to move the alembic migration from Xena to Yoga | 09:24 |
| seba | will do | 09:24 |
| opendevreview | Sebastian Lohff proposed openstack/neutron master: Fix dynamic segment allocation race condition https://review.opendev.org/c/openstack/neutron/+/788714 | 09:30 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Sanitize profile column of ml2_port_bindings table in the DB https://review.opendev.org/c/openstack/neutron/+/811411 | 09:54 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Enable QoS minimum packet rate rule for OVS backend https://review.opendev.org/c/openstack/neutron/+/805391 | 09:54 |
| hjensas | elvira: Hi, see my comment on https://review.opendev.org/c/openstack/neutron/+/811435 and the bug. | 10:11 |
| elvira | sure, hjensas. Thanks | 10:14 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Add API extension for QoS minimum pps rule https://review.opendev.org/c/openstack/neutron/+/803045 | 10:28 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: ovs-agent: Report pkt processing info in heartbeat https://review.opendev.org/c/openstack/neutron/+/800444 | 10:28 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Report CUSTOM_VNIC_TYPE_ traits on Neutron agent RP https://review.opendev.org/c/openstack/neutron/+/800445 | 10:28 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Report pkt processing capacity on Neutron agent RP https://review.opendev.org/c/openstack/neutron/+/800446 | 10:28 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Add port-resource-request-groups extension https://review.opendev.org/c/openstack/neutron/+/805637 | 10:30 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Sanitize profile column of ml2_port_bindings table in the DB https://review.opendev.org/c/openstack/neutron/+/811411 | 10:30 |
| opendevreview | Przemyslaw Szczerbik proposed openstack/neutron master: Enable QoS minimum packet rate rule for OVS backend https://review.opendev.org/c/openstack/neutron/+/805391 | 10:30 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron-lib master: Add oneline_string validators https://review.opendev.org/c/openstack/neutron-lib/+/811708 | 10:35 |
| opendevreview | Harald Jensås proposed openstack/neutron master: [OVN] Allow IP allocation with different segments for OVN service ports https://review.opendev.org/c/openstack/neutron/+/811719 | 11:57 |
| gmann | lajoskatona: ralonsoh slaweq can you verify the neutron extensions for Xena in case i missed any https://review.opendev.org/c/openstack/devstack/+/811485 | 12:32 |
| lajoskatona | gmann: sure | 12:32 |
| lajoskatona | gmann: I checked the list, and some extension/feature from it now yet fully working, no tempest tests for them, but better to keep them out from this list | 12:41 |
| gmann | lajoskatona: I think keep it enable should not harm as their test can be added later in tempest or neutron tempest plugin and they start running on stable/xena too. like trunk case. | 12:45 |
| gmann | lajoskatona: or you think those will be risky to run on stable as per stability or so? | 12:45 |
| gmann | if so then we can remove for now and then add when we add test, that is also fine | 12:45 |
| lajoskatona | gmann: on xena the features are not working | 12:46 |
| gmann | lajoskatona: i see. then we should remove. please highlight those and I will remove. | 12:46 |
| opendevreview | Bernard Cafarelli proposed openstack/neutron stable/xena: [stable/xena] Drop -master jobs https://review.opendev.org/c/openstack/neutron/+/811187 | 12:49 |
| gmann | lajoskatona: also about if we want to enable trunk extensions so does testing on stable or not. please comment what neutron team thinks https://review.opendev.org/c/openstack/grenade/+/811512 | 12:51 |
| gmann | i am ok with both option | 12:51 |
| lajoskatona | gmann: I listed all which should not be in the xena list, and thanks for taking care of it | 12:55 |
| gmann | lajoskatona: great, thanks a lot. | 12:55 |
| ralonsoh | gmann, sure, let me check | 12:56 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [DVR] Check if SNAT iptables manager is initialized https://review.opendev.org/c/openstack/neutron/+/811318 | 13:53 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas master: req: Bump Mako to 1.0.7 https://review.opendev.org/c/openstack/neutron-vpnaas/+/811731 | 14:12 |
| opendevreview | Merged openstack/neutron stable/xena: Revert "update subport status when trunk/subport create/delete is triggerred" https://review.opendev.org/c/openstack/neutron/+/810975 | 14:14 |
| opendevreview | Merged openstack/neutron master: Replace "Inspector.from_engine()" with "sqlalchemy.inspect()" https://review.opendev.org/c/openstack/neutron/+/808103 | 14:14 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas master: req: Bump Mako to 1.0.7 https://review.opendev.org/c/openstack/neutron-vpnaas/+/811731 | 14:50 |
| spatel | ralonsoh Hi | 14:52 |
| ralonsoh | spatel, hi | 14:53 |
| spatel | ralonsoh I have question related DPDK performance. I have setup DPDK on one of my compute node to replace SRIOV | 14:55 |
| spatel | I am seeing very poor performance, like standard virtio | 14:56 |
| spatel | let me show you some numbers and the way i am doing load-testing | 14:56 |
| opendevreview | Merged openstack/networking-bgpvpn master: Update master for stable/xena https://review.opendev.org/c/openstack/networking-bgpvpn/+/809051 | 15:00 |
| opendevreview | Merged openstack/networking-bgpvpn master: Add Python3 yoga unit tests https://review.opendev.org/c/openstack/networking-bgpvpn/+/809052 | 15:08 |
| spatel | ralonsoh sorry for delay, here is the result - https://paste.opendev.org/show/809675/ | 15:14 |
| spatel | I can only pump 150kpps packets after that i am seeing packet loss start | 15:15 |
| spatel | I have configure vm with CPU pinning and numa, my flavor setting - hw:cpu_policy='dedicated', hw:cpu_sockets='2', hw:cpu_threads='2', hw:mem_page_size='large' | 15:17 |
| spatel | in internet people saying with dpdk you can hit 10mpps rate but how ? | 15:17 |
| spatel | i have little confusion related hugepages.. how does DPDK use them i believe i did correct setting for hugepage so could you validate it? | 15:20 |
| opendevreview | Merged openstack/networking-sfc master: Update master for stable/xena https://review.opendev.org/c/openstack/networking-sfc/+/809702 | 15:25 |
| opendevreview | Merged openstack/networking-sfc master: Add Python3 yoga unit tests https://review.opendev.org/c/openstack/networking-sfc/+/809703 | 15:35 |
| spatel | ralonsoh do you have any idea about that | 15:43 |
| ralonsoh | spatel, to be honest, I can't tell you, checking those logs, what is failing there | 15:45 |
| spatel | nothing failing but just poor transfer rate.. | 15:46 |
| ralonsoh | yes I know | 15:46 |
| spatel | do you think my VM is bottleneck here? | 15:46 |
| ralonsoh | did you have the FW enabled? | 15:46 |
| spatel | you default security group | 15:46 |
| spatel | i can check log of contrack on compute node and see | 15:47 |
| ralonsoh | another question is | 15:47 |
| ralonsoh | how are you bechmarking the traffic? | 15:47 |
| ralonsoh | from a VM to another host, right? | 15:48 |
| spatel | Yes - From another host to VM | 15:48 |
| ralonsoh | so the packets should cross a physical interface | 15:48 |
| spatel | Physical server -----------> DPDK vm | 15:48 |
| ralonsoh | how did you configured this physical interface? | 15:49 |
| spatel | its single 10G nic port configure for dpdk | 15:49 |
| ralonsoh | are you using VLAN? VXLAN? | 15:49 |
| spatel | VLAN | 15:49 |
| spatel | https://paste.opendev.org/show/809676/ | 15:49 |
| ralonsoh | ok so you have your physical bridge connected to br-int and the DPDK port | 15:50 |
| spatel | Yes | 15:51 |
| spatel | You can see here - https://paste.opendev.org/show/809675/ | 15:51 |
| ralonsoh | I don't know if the performance issue is in the DPDK TAP port or the DPDK NIC port | 15:51 |
| ralonsoh | you can also check VM-to-VM | 15:51 |
| ralonsoh | inside the same host | 15:51 |
| spatel | br-provider attach with dpdk-0 port and then br-provider ----patch----br-int | 15:51 |
| ralonsoh | that will put aside the physical NIC | 15:52 |
| spatel | hmm that is good idea VM-to-VM on same host | 15:52 |
| ralonsoh | check the FW (disable it) | 15:52 |
| spatel | But if i run same load-test on SRIOV vm then result is much better | 15:52 |
| ralonsoh | check the packet size | 15:52 |
| spatel | I am using 64 and 150 packet size | 15:52 |
| spatel | that is what we have in production workload for UDP traffic | 15:53 |
| ralonsoh | as I said, check the FW then | 15:53 |
| spatel | I can see dpdk created this file that means its using hugepage - /dev/hugepages/rtemap_0 | 15:53 |
| ralonsoh | I don't know if DPDK is properly configured or working fine | 15:53 |
| spatel | I am looking at iptables and not seeing any issue like connection table full etc.. | 15:54 |
| opendevreview | Merged openstack/networking-bagpipe master: Update master for stable/xena https://review.opendev.org/c/openstack/networking-bagpipe/+/809059 | 15:54 |
| opendevreview | Merged openstack/networking-bagpipe master: Add Python3 yoga unit tests https://review.opendev.org/c/openstack/networking-bagpipe/+/809062 | 15:54 |
| spatel | i can remove port-security and give it a try | 15:54 |
| ralonsoh | no no, not iptables | 15:54 |
| spatel | ? | 15:54 |
| ralonsoh | you can't use hybrid firewall | 15:54 |
| spatel | what FW are you talking about? | 15:54 |
| spatel | I am running OVN | 15:54 |
| ralonsoh | ah, I didn't know that | 15:55 |
| ralonsoh | I thought it was OVS | 15:55 |
| spatel | OVN + OVS-DPDK | 15:55 |
| spatel | does that matter ? | 15:56 |
| ralonsoh | it shouldn't | 15:56 |
| ralonsoh | I didn't test OVN+DPDK | 15:56 |
| spatel | how do you test your DPDK deployment for performance (lets forget about OVN) | 15:57 |
| spatel | i would like to know what people prefer to banchmark | 15:58 |
| ralonsoh | I used an IXIA traffic generator, with the different RFC tests implemented | 15:58 |
| spatel | and what do you use in VM (testpmd?) | 15:59 |
| ralonsoh | a dpdk traffic reflector | 15:59 |
| spatel | hmm but in real world it doesn't work like that right? | 16:00 |
| ralonsoh | inside the VM you need to use a DPDK application | 16:01 |
| ralonsoh | e.g.: TREX with DPDK | 16:01 |
| spatel | I am seeing kernel context switching going about 200k when i run simple load-test | 16:01 |
| spatel | if i am running haproxy in VM then how does it use dpdk ? | 16:01 |
| ralonsoh | you need to create hugepages inside the VM | 16:02 |
| ralonsoh | and assign the VM port to DPDK | 16:02 |
| spatel | my workload is all audio processing which use udp stream and need very high packet rate.. | 16:03 |
| ralonsoh | and create DPDK compatible app | 16:03 |
| spatel | i am creating VM with this flavor setting, doesn't that enough? hw:mem_page_size='large' | 16:03 |
| ralonsoh | packets inside the VM will use kernel space | 16:04 |
| spatel | yes i have noticed that | 16:04 |
| ralonsoh | then there you have your performance stopper | 16:05 |
| ralonsoh | with DPDK you need everything, including the traffic inside the VM, being handled by DPDK | 16:05 |
| ralonsoh | to have best performance | 16:05 |
| spatel | hmm | 16:05 |
| ralonsoh | in any case, I'm not an expert at all on this subject | 16:05 |
| spatel | how does hugepage play role in dpdk ? | 16:06 |
| spatel | you assign hugepage to dpdk but then how does VM use that to copy packet? | 16:06 |
| ralonsoh | no idea | 16:07 |
| spatel | some document saying you need to create hugepage and mount it and the tell libvirtd to use it but then what is this setting for in flavor hw:mem_page_size='large' | 16:07 |
| opendevreview | Balazs Gibizer proposed openstack/neutron master: Enable min pps tempest tests https://review.opendev.org/c/openstack/neutron/+/811746 | 16:08 |
| spatel | i didn't do any setting in libvirtd for hugepage | 16:08 |
| ralonsoh | you have info here https://docs.openstack.org/nova/pike/admin/huge-pages.html | 16:08 |
| spatel | I did that - mount -t hugetlbfs hugetlbfs /dev/hugepages | 16:10 |
| spatel | default dpdk use this path - /dev/hugepages | 16:10 |
| spatel | That is how i can see this - https://paste.opendev.org/show/809677/ | 16:11 |
| spatel | curious why VM directory is empty - /dev/hugepages/libvirt/qemu/1-instance-00000045/ | 16:12 |
| spatel | is that normal or something wrong there? | 16:12 |
| ralonsoh | tbh, no idea, this is Nova stuff | 16:13 |
| spatel | no problem.. so much moving piece so hard to understand :) | 16:14 |
| spatel | ralonsoh do you know nuetron support bonding for SRIOV VF inside VM? | 16:20 |
| spatel | if that work then i can go with sriov and bond my vf inside vm | 16:20 |
| ralonsoh | 1) neutron knows nothing about the VM network, this must be done inside the VM | 16:21 |
| ralonsoh | 2) bonding two VFs from the same PF will provide no speed gain (just in case) | 16:21 |
| spatel | I have plan to replace nic with dual port | 16:25 |
| spatel | so i will have two PF | 16:25 |
| spatel | currently i am running very large cloud with all sriov compute and because of sriov nature i don't have bonding :( | 16:26 |
| spatel | if one of my TOR switch pop out then whole rack go down.. | 16:26 |
| spatel | That is why i am exploring alternative to use DPDK so i will have dual redundancy | 16:27 |
| spatel | Does neutron support providing dual VF from two different PF then i can do some magic in VM to bond them and my all problem will get resolved | 16:28 |
| spatel | ralonsoh ^ | 16:28 |
| ralonsoh | now I don't remember how but you can specify the NIC to be used in a Neutron port | 16:29 |
| ralonsoh | but you should ask this in #nova | 16:29 |
| opendevreview | Balazs Gibizer proposed openstack/neutron master: Enable min pps tempest tests https://review.opendev.org/c/openstack/neutron/+/811746 | 16:30 |
| spatel | ralonsoh sure will ask there | 16:31 |
| spatel | i found this article and look like little complicated - https://www.redpill-linpro.com/techblog/2021/01/30/bonding-sriov-nics-with-openstack.html | 16:34 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [WIP][HA] Do not add initial state change delay in HA router https://review.opendev.org/c/openstack/neutron/+/811751 | 16:37 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas master: req: Bump some requirements https://review.opendev.org/c/openstack/neutron-vpnaas/+/811731 | 17:20 |
| opendevreview | Merged openstack/neutron master: Replace "tenant_id" with "project_id" in Quota engine https://review.opendev.org/c/openstack/neutron/+/805849 | 17:27 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Execute the quota reservation removal in an isolated DB txn https://review.opendev.org/c/openstack/neutron/+/809983 | 17:33 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas master: req: Bump some requirements https://review.opendev.org/c/openstack/neutron-vpnaas/+/811731 | 19:51 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!