| openstackgerrit | Matthew Kassawara proposed openstack/networking-ovn: Vagrant: Enable OVN commands to use a remote DB https://review.openstack.org/273227 | 00:03 |
|---|---|---|
| *** salv-orl_ has quit IRC | 00:15 | |
| *** manand has quit IRC | 00:19 | |
| *** azbiswas has joined #openstack-neutron-ovn | 00:24 | |
| *** dlundquist has quit IRC | 00:29 | |
| *** chenli has joined #openstack-neutron-ovn | 00:57 | |
| *** chenli has quit IRC | 01:08 | |
| *** Guest43685 has quit IRC | 01:32 | |
| *** yamamoto_ has joined #openstack-neutron-ovn | 01:45 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 01:47 | |
| *** manand has joined #openstack-neutron-ovn | 01:50 | |
| *** salv-orlando has quit IRC | 01:57 | |
| *** yamamoto_ has quit IRC | 02:03 | |
| *** yamamoto_ has joined #openstack-neutron-ovn | 02:04 | |
| *** yamamoto_ has quit IRC | 02:09 | |
| *** yamamoto_ has joined #openstack-neutron-ovn | 02:10 | |
| *** chenli has joined #openstack-neutron-ovn | 02:22 | |
| *** yamamoto_ has quit IRC | 02:24 | |
| *** s3wong has quit IRC | 02:46 | |
| *** jckasper has quit IRC | 03:07 | |
| *** jckasper has joined #openstack-neutron-ovn | 03:08 | |
| *** jckasper has quit IRC | 03:12 | |
| *** yamamoto has joined #openstack-neutron-ovn | 03:13 | |
| *** azbiswas has quit IRC | 03:33 | |
| *** azbiswas has joined #openstack-neutron-ovn | 03:35 | |
| *** azbiswas has quit IRC | 03:35 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 03:43 | |
| *** salv-orlando has quit IRC | 03:56 | |
| *** jckasper has joined #openstack-neutron-ovn | 03:57 | |
| *** roeyc has joined #openstack-neutron-ovn | 04:17 | |
| *** roeyc has quit IRC | 04:40 | |
| *** tongli has quit IRC | 05:00 | |
| *** roeyc has joined #openstack-neutron-ovn | 05:03 | |
| *** roeyc has quit IRC | 05:14 | |
| *** xianghui has quit IRC | 05:31 | |
| *** manand has quit IRC | 05:33 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 06:38 | |
| *** numans has joined #openstack-neutron-ovn | 06:40 | |
| *** salv-orlando has quit IRC | 06:49 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 07:26 | |
| *** chandrav has joined #openstack-neutron-ovn | 07:55 | |
| *** salv-orlando has quit IRC | 08:10 | |
| *** numans has quit IRC | 08:19 | |
| *** numans has joined #openstack-neutron-ovn | 08:20 | |
| openstackgerrit | Numan Siddique proposed openstack/networking-ovn: (WIP) Support port security API extension https://review.openstack.org/269219 | 08:32 |
| *** armax has quit IRC | 08:54 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 09:11 | |
| *** armax has joined #openstack-neutron-ovn | 09:23 | |
| *** chenli has quit IRC | 09:30 | |
| *** roeyc has joined #openstack-neutron-ovn | 09:36 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 10:06 | |
| *** salv-orlando has quit IRC | 10:09 | |
| *** roeyc has quit IRC | 10:20 | |
| *** roeyc has joined #openstack-neutron-ovn | 10:25 | |
| *** roeyc has quit IRC | 10:29 | |
| *** yamamoto has quit IRC | 10:30 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 11:19 | |
| *** armax has quit IRC | 11:34 | |
| *** salv-orl_ has quit IRC | 11:34 | |
| *** fzdarsky has quit IRC | 12:05 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 12:06 | |
| *** roeyc has joined #openstack-neutron-ovn | 12:11 | |
| *** rtheis has joined #openstack-neutron-ovn | 12:16 | |
| *** armax has joined #openstack-neutron-ovn | 12:17 | |
| *** fzdarsky has quit IRC | 12:18 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 12:21 | |
| *** armax has quit IRC | 12:28 | |
| *** ioni has joined #openstack-neutron-ovn | 12:44 | |
| *** yamamoto has joined #openstack-neutron-ovn | 13:19 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 13:34 | |
| *** fzdarsky has quit IRC | 13:41 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 13:42 | |
| *** yamamoto has quit IRC | 14:08 | |
| *** salv-orlando has quit IRC | 14:09 | |
| *** yamamoto has joined #openstack-neutron-ovn | 14:11 | |
| *** yamamoto has quit IRC | 14:12 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 14:26 | |
| *** salv-orlando has quit IRC | 14:29 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 14:29 | |
| *** manand has joined #openstack-neutron-ovn | 14:37 | |
| *** yamamoto has joined #openstack-neutron-ovn | 14:38 | |
| *** jckasper has quit IRC | 14:43 | |
| *** jckasper has joined #openstack-neutron-ovn | 14:43 | |
| *** jckasper has quit IRC | 14:48 | |
| *** fzdarsky has quit IRC | 14:48 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 14:49 | |
| *** jckasper has joined #openstack-neutron-ovn | 14:54 | |
| *** yamamoto has quit IRC | 14:56 | |
| *** dlundquist has joined #openstack-neutron-ovn | 14:56 | |
| *** chandrav has quit IRC | 15:04 | |
| *** yamamoto has joined #openstack-neutron-ovn | 15:08 | |
| openstackgerrit | Merged openstack/networking-ovn: Vagrant: Enable OVN commands to use a remote DB https://review.openstack.org/273227 | 15:15 |
| openstackgerrit | Kyle Mestery proposed openstack/networking-ovn: extensions: Add the "subnet_allocation" extension to OVN https://review.openstack.org/273618 | 15:22 |
| russellb | SO. | 15:27 |
| russellb | we're still getting a lot of failures about "ports still exist blah blah" | 15:27 |
| * mestery listens | 15:27 | |
| russellb | that's it. | 15:28 |
| russellb | (kidding i have more to say) | 15:28 |
| mestery | Heh :) | 15:28 |
| russellb | our delete_network() is quite simple. | 15:28 |
| mestery | Is something racey here? | 15:29 |
| russellb | I'm looking at delete_network() in ml2 and it's auto-deleting some ports in there | 15:29 |
| mestery | Oh wow | 15:29 |
| russellb | yes, something is, but ^ | 15:29 |
| mestery | So, I wonder if ML2 is doing that to be proactive in preventing race conditions? | 15:29 |
| gsagie | i think its suppose to delete the DHCP ports no? | 15:30 |
| russellb | yes, DHCP | 15:31 |
| gsagie | and in OVN they suppose to be garbage collected when the lswitch is deleted | 15:31 |
| russellb | db_base_plugin_v2 delete_network() deletes those ports automatically as well ... | 15:32 |
| russellb | so i'm not sure why ml2's plugin does it then | 15:32 |
| gsagie | russellb: from what i remember some time ago, the DHCP port delete was not sent to the plugin | 15:32 |
| gsagie | to core plugin | 15:32 |
| russellb | https://github.com/openstack/neutron/blob/master/neutron/db/db_base_plugin_v2.py#L366 | 15:34 |
| russellb | it auto deletes dhcp ports there | 15:34 |
| russellb | BUT it sets synchronize_session=False there | 15:34 |
| russellb | which *I think* means that the next query is still going to return the DHCP port if it existed, even though it's been marked for deletion already | 15:35 |
| *** fzdarsky has quit IRC | 15:35 | |
| * russellb boggles | 15:36 | |
| russellb | help, i'm stuck in sqlalchemy! i need an adult! | 15:38 |
| mestery | rofl | 15:38 |
| *** jckasper has quit IRC | 15:45 | |
| *** jckasper has joined #openstack-neutron-ovn | 15:46 | |
| *** dlundquist has quit IRC | 15:54 | |
| *** dlundquist has joined #openstack-neutron-ovn | 15:55 | |
| *** fzdarsky has joined #openstack-neutron-ovn | 15:57 | |
| *** numans has quit IRC | 15:58 | |
| *** dlundquist has quit IRC | 15:59 | |
| *** salv-orl_ has joined #openstack-neutron-ovn | 16:06 | |
| mestery | russellb: I know you're likely busy, but https://review.openstack.org/273618 gets OVN working with Kuryr again and is a 1-line change whcih gsagie has already +2'd ... | 16:08 |
| mestery | :) | 16:08 |
| * mestery hugs russellb | 16:09 | |
| mestery | Thanks my friend! | 16:09 |
| *** salv-orlando has quit IRC | 16:09 | |
| *** roeyc has quit IRC | 16:09 | |
| russellb | didn't i already +2 that? | 16:09 |
| mestery | Oh, maybe you did and I just missed it :) | 16:09 |
| russellb | yup. | 16:09 |
| mestery | So .... | 16:09 |
| mestery | With taht change and the Kuryr change gsagie merged this morning | 16:10 |
| mestery | I can succesfully create a network using docker and kuryr | 16:10 |
| mestery | Boot a container | 16:10 |
| mestery | Then boot a VM | 16:10 |
| mestery | And they talk to each other | 16:10 |
| mestery | BOOM | 16:10 |
| *** yamamoto has quit IRC | 16:12 | |
| *** yamamoto has joined #openstack-neutron-ovn | 16:12 | |
| russellb | hot | 16:12 |
| russellb | now someone integrate container-in-vm support with kuryr | 16:12 |
| russellb | http://docs.openstack.org/developer/networking-ovn/containers.html | 16:13 |
| russellb | :) | 16:13 |
| mestery | russellb: Aye :) | 16:15 |
| *** yamamoto has quit IRC | 16:16 | |
| *** yamamoto has joined #openstack-neutron-ovn | 16:17 | |
| Sam-I-Am | mestery: you get a chance to look at my larger vagrant patch? | 16:18 |
| mestery | Sam-I-Am: I have not yet sir | 16:18 |
| * mestery goes to look | 16:18 | |
| *** yamamoto has quit IRC | 16:19 | |
| *** yamamoto has joined #openstack-neutron-ovn | 16:20 | |
| *** roeyc has joined #openstack-neutron-ovn | 16:21 | |
| *** jckasper has quit IRC | 16:21 | |
| *** jckasper has joined #openstack-neutron-ovn | 16:22 | |
| *** jckasper has quit IRC | 16:24 | |
| *** jckasper has joined #openstack-neutron-ovn | 16:25 | |
| *** fzdarsky has quit IRC | 16:25 | |
| *** yamamoto has quit IRC | 16:36 | |
| *** yamamoto has joined #openstack-neutron-ovn | 16:37 | |
| *** yamamoto has quit IRC | 16:44 | |
| *** yamamoto has joined #openstack-neutron-ovn | 16:47 | |
| *** s3wong has joined #openstack-neutron-ovn | 16:50 | |
| *** yamamoto has quit IRC | 16:51 | |
| *** yamamoto has joined #openstack-neutron-ovn | 16:52 | |
| *** yamamoto has quit IRC | 16:59 | |
| *** yamamoto has joined #openstack-neutron-ovn | 17:04 | |
| *** yamamoto has quit IRC | 17:31 | |
| openstackgerrit | Richard Theis proposed openstack/networking-ovn: Update DevStack testing documentation https://review.openstack.org/273685 | 17:34 |
| *** roeyc has quit IRC | 17:34 | |
| *** regXboi has joined #openstack-neutron-ovn | 17:36 | |
| * regXboi wanders in... looking for russellb | 17:36 | |
| regXboi | russellb: ovsdb-server is single-threaded, yes? | 17:37 |
| *** numans has joined #openstack-neutron-ovn | 17:40 | |
| *** armax has joined #openstack-neutron-ovn | 17:46 | |
| *** salv-orl_ has quit IRC | 17:48 | |
| *** dlundquist has joined #openstack-neutron-ovn | 17:48 | |
| russellb | regXboi: I'm not sure | 17:58 |
| regXboi | russellb: actually, I answered myself - it looks like it is | 17:59 |
| russellb | ovs-vswitchd is multi-threaded, but that's it AFAIK | 18:02 |
| regXboi | yeah - and I think that leads to the linearity issues I've been seeing | 18:02 |
| regXboi | turns out ovsdb-server is pretty damn good w.r.t. inserts | 18:02 |
| regXboi | updates... OTOH | 18:03 |
| regXboi | trying to upload some graphs now | 18:03 |
| regXboi | all: graph of times to insert rows or insert/update rows in OVN_Northbound | 18:06 |
| regXboi | http://ibin.co/2V2VVrQYDKyI | 18:06 |
| regXboi | graph of times to update port status in OVN_Northbound: | 18:06 |
| regXboi | http://ibin.co/2V2Vjb9rVqUK | 18:06 |
| russellb | still looks pretty fast :) | 18:06 |
| regXboi | russellb: it *is* fast - but the slope is the problem - the level of scale we are going after means we really can't have *any* slope | 18:07 |
| russellb | ok so, Y axis is seconds i'm guessing | 18:08 |
| russellb | what's the x axis | 18:08 |
| regXboi | Y axis is in seconds | 18:08 |
| regXboi | X axis is "trial" | 18:08 |
| *** azbiswas_ has joined #openstack-neutron-ovn | 18:08 | |
| russellb | trial == ? | 18:08 |
| regXboi | yeah, that's not useful at all | 18:08 |
| regXboi | this is where it gets complicated | 18:08 |
| russellb | so here's a question for you ... are you sure ovsdb-server is the right thing to be analyzing right now? | 18:09 |
| russellb | it may be, i'm not sure. | 18:09 |
| regXboi | I'm running a scaling experiment and calculating the operation times during that experiment, so I can't really say (at this point) much more than on the left, I have 0 project/routers etc and on the right I have 400 | 18:09 |
| russellb | but it seems the most useful work we can do is to 1) find the bottlenecks (what falls over first), then 2) dig into that component to figure out the leading cause of what's slowing it down, 3) find ways to optimize it. | 18:09 |
| regXboi | that is a fair question - right now I'm trying to find the long poles in the tent | 18:10 |
| russellb | heh, so, at trial 2500 we have 400 projects/routers? heh | 18:10 |
| regXboi | yeah, we have something like 1600 ports at that point | 18:10 |
| russellb | ok | 18:10 |
| regXboi | so I'm to the point of the bottleneck being either IDL in python or ovsdb-server | 18:11 |
| regXboi | and I've not quite figured out how to nail down the amount of time IDL spends in python yet | 18:11 |
| russellb | vs blocking on network? | 18:12 |
| regXboi | yes vs blocking on network vs time in the server vs ... | 18:12 |
| russellb | btw all: OVN weekly meeting in 3 minutes | 18:12 |
| regXboi | yep - moving over there | 18:12 |
| *** zhouhan has joined #openstack-neutron-ovn | 18:16 | |
| *** chandrav has joined #openstack-neutron-ovn | 18:17 | |
| *** dlundquist has quit IRC | 18:25 | |
| *** azbiswas has joined #openstack-neutron-ovn | 18:28 | |
| *** azbiswas_ has quit IRC | 18:28 | |
| *** dlundquist has joined #openstack-neutron-ovn | 18:38 | |
| *** gangil has joined #openstack-neutron-ovn | 18:50 | |
| *** gangil has joined #openstack-neutron-ovn | 18:50 | |
| *** dlundquist1 has joined #openstack-neutron-ovn | 18:50 | |
| *** dlundquist has quit IRC | 18:52 | |
| *** dlundquist has joined #openstack-neutron-ovn | 19:11 | |
| *** roeyc has joined #openstack-neutron-ovn | 19:12 | |
| *** jckasper has quit IRC | 19:19 | |
| *** jckasper has joined #openstack-neutron-ovn | 19:21 | |
| regXboi | russellb: btw, I've got an update for that devstack patch and it looks like northd just doesn't log to console at all | 19:33 |
| russellb | weird | 19:33 |
| russellb | feel free to file a bug and i (or whoever) will figure it out | 19:33 |
| regXboi | will do - once I finish figuring out the timestamp stuff and pushing the new patch | 19:34 |
| *** salv-orlando has joined #openstack-neutron-ovn | 19:42 | |
| *** salv-orlando has quit IRC | 19:50 | |
| *** azbiswas has quit IRC | 19:51 | |
| *** azbiswas has joined #openstack-neutron-ovn | 19:51 | |
| *** azbiswas has quit IRC | 19:55 | |
| *** dlundquist has quit IRC | 20:01 | |
| *** azbiswas has joined #openstack-neutron-ovn | 20:01 | |
| *** allan_h has joined #openstack-neutron-ovn | 20:02 | |
| *** gangil has quit IRC | 20:03 | |
| *** s3wong has quit IRC | 20:04 | |
| *** allan_h has quit IRC | 20:06 | |
| *** allan_h has joined #openstack-neutron-ovn | 20:08 | |
| openstackgerrit | Merged openstack/networking-ovn: extensions: Add the "subnet_allocation" extension to OVN https://review.openstack.org/273618 | 20:09 |
| * regXboi tries stacking with a patch that (gasp) might even fix the timestamps | 20:09 | |
| *** azbiswas has quit IRC | 20:12 | |
| *** azbiswas_ has joined #openstack-neutron-ovn | 20:14 | |
| *** salv-orlando has joined #openstack-neutron-ovn | 20:17 | |
| regXboi | russellb: I'll answer you on the ML in more detail, but the tl;dr is there are multiple prongs to what's being looked at | 20:17 |
| russellb | sounds scary | 20:18 |
| regXboi | so while I'd say ovn-controller was the first long pole, ovsdb-server isn't too far behind | 20:18 |
| regXboi | I forget, did your controller optimization patches get merged? | 20:18 |
| russellb | no | 20:19 |
| russellb | I posted the first one as an RFC to the OVS dev list | 20:19 |
| *** azbiswas_ has quit IRC | 20:19 | |
| russellb | the most significant improvements we can make will take much more work | 20:19 |
| regXboi | ah, ok - fyi, removing --log-file from the ovn-controller start up make devstack sad | 20:19 |
| regXboi | :( | 20:19 |
| russellb | rawr | 20:19 |
| regXboi | so... I'm putting it back - I'm not going to argue double logging for the moment | 20:20 |
| russellb | when is devstack happy? | 20:20 |
| * regXboi notes there are so many decorative comments in this patch set | 20:20 | |
| *** allan_h has quit IRC | 20:22 | |
| *** allan_h has joined #openstack-neutron-ovn | 20:23 | |
| openstackgerrit | Ryan Moats proposed openstack/networking-ovn: Make OVS related logs multi-stack friendly https://review.openstack.org/272265 | 20:28 |
| *** salv-orlando has quit IRC | 20:30 | |
| regXboi | russellb: bug filed, but I'm not sure I got the process correct | 20:44 |
| russellb | thanks for filing one :) | 20:46 |
| *** gangil has joined #openstack-neutron-ovn | 20:48 | |
| *** gangil has joined #openstack-neutron-ovn | 20:48 | |
| rtheis | Is there a know tempest failure on tempest.api.network.test_routers_negative.RoutersNegativeTest? | 21:00 |
| rtheis | Unable to complete operation on network ... ports still in use ... | 21:01 |
| *** gangil has quit IRC | 21:02 | |
| *** gangil has joined #openstack-neutron-ovn | 21:02 | |
| *** gangil has joined #openstack-neutron-ovn | 21:02 | |
| *** dlundquist has joined #openstack-neutron-ovn | 21:02 | |
| *** azbiswas has joined #openstack-neutron-ovn | 21:03 | |
| russellb | rtheis: yes, it's a failure we're getting a lot right now, but not consistently | 21:09 |
| russellb | same error, test it happens in is all over | 21:10 |
| rtheis | russellb: thanks | 21:10 |
| russellb | np | 21:10 |
| *** salv-orlando has joined #openstack-neutron-ovn | 21:11 | |
| openstackgerrit | Ryan Moats proposed openstack/networking-ovn: Make OVS related logs multi-stack friendly https://review.openstack.org/272265 | 21:29 |
| russellb | regXboi: you may want to file that ovn-northd console bug against networking-ovn and tag it with "ovn-upstream" | 21:39 |
| russellb | there's not really an actively used bug tracker for ovs/ovn itself, reports just go to the discuss list | 21:39 |
| russellb | so i'm keeping some things in the networking-ovn tracker for now | 21:39 |
| regXboi | russellb: ah ok | 21:43 |
| *** salv-orl_ has joined #openstack-neutron-ovn | 22:06 | |
| *** salv-orlando has quit IRC | 22:08 | |
| openstackgerrit | Russell Bryant proposed openstack/networking-ovn: tempest debugging, ignore... https://review.openstack.org/269705 | 22:10 |
| openstackgerrit | Akihiro Motoki proposed openstack/networking-ovn: Update translation setup https://review.openstack.org/273788 | 22:13 |
| *** roeyc has quit IRC | 22:17 | |
| *** roeyc has joined #openstack-neutron-ovn | 22:18 | |
| *** rtheis has quit IRC | 22:20 | |
| *** dlundquist has quit IRC | 22:23 | |
| openstackgerrit | Russell Bryant proposed openstack/networking-ovn: tempest debugging, ignore .... https://review.openstack.org/273797 | 22:27 |
| *** dlundquist has joined #openstack-neutron-ovn | 22:27 | |
| *** jckasper has quit IRC | 22:31 | |
| *** jckasper has joined #openstack-neutron-ovn | 22:32 | |
| *** gangil has quit IRC | 22:35 | |
| *** gangil has joined #openstack-neutron-ovn | 22:42 | |
| *** gangil has joined #openstack-neutron-ovn | 22:42 | |
| *** regXboi has quit IRC | 23:03 | |
| *** yamamoto has joined #openstack-neutron-ovn | 23:06 | |
| *** roeyc has quit IRC | 23:13 | |
| *** yamamoto has quit IRC | 23:19 | |
| *** dlundquist has quit IRC | 23:27 | |
| *** gangil1 has joined #openstack-neutron-ovn | 23:27 | |
| *** gangil has quit IRC | 23:29 | |
| *** dlundquist has joined #openstack-neutron-ovn | 23:34 | |
| *** dlundquist has quit IRC | 23:37 | |
| *** dlundquist has joined #openstack-neutron-ovn | 23:47 | |
| *** dlundquist has quit IRC | 23:47 | |
| *** chandrav has quit IRC | 23:50 | |
| *** dlundquist has joined #openstack-neutron-ovn | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!