| *** yamamoto has quit IRC | 00:04 | |
| *** lseki has quit IRC | 00:06 | |
| *** mmethot_ has joined #openstack-meeting | 00:09 | |
| *** mmethot has quit IRC | 00:13 | |
| *** rbudden has joined #openstack-meeting | 00:13 | |
| *** mattw4 has quit IRC | 00:16 | |
| *** b1airo has quit IRC | 00:19 | |
| *** markvoelker has joined #openstack-meeting | 00:23 | |
| *** tetsuro has joined #openstack-meeting | 00:32 | |
| *** senrique_ has joined #openstack-meeting | 00:33 | |
| *** hongbin has joined #openstack-meeting | 00:34 | |
| *** jamesmcarthur_ has quit IRC | 00:47 | |
| *** armax has quit IRC | 00:50 | |
| *** yamamoto has joined #openstack-meeting | 00:52 | |
| *** ykatabam has joined #openstack-meeting | 00:52 | |
| *** tetsuro has quit IRC | 00:57 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:01 | |
| *** ricolin has joined #openstack-meeting | 01:02 | |
| *** tetsuro has joined #openstack-meeting | 01:02 | |
| *** jamesmcarthur has quit IRC | 01:04 | |
| *** carloss has quit IRC | 01:11 | |
| *** igordc has quit IRC | 01:13 | |
| *** igordc has joined #openstack-meeting | 01:15 | |
| *** ricolin_ has joined #openstack-meeting | 01:17 | |
| *** ricolin has quit IRC | 01:20 | |
| *** igordc has quit IRC | 01:23 | |
| *** pcaruana has quit IRC | 01:26 | |
| *** tetsuro has quit IRC | 01:28 | |
| *** gyee has quit IRC | 01:33 | |
| *** yamahata has quit IRC | 01:38 | |
| *** iyamahat_ has quit IRC | 01:38 | |
| *** markvoelker has quit IRC | 01:39 | |
| *** bobh has joined #openstack-meeting | 01:40 | |
| *** markvoelker has joined #openstack-meeting | 01:40 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:56 | |
| *** jamesmcarthur has quit IRC | 01:58 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:59 | |
| *** haleyb has quit IRC | 02:01 | |
| *** apetrich has quit IRC | 02:09 | |
| *** ykatabam has quit IRC | 02:18 | |
| *** jamesmcarthur has quit IRC | 02:22 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:22 | |
| *** jamesmcarthur has quit IRC | 02:25 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:26 | |
| *** ricolin_ is now known as ricolin | 02:27 | |
| *** senrique_ has quit IRC | 02:41 | |
| *** yamamoto has quit IRC | 02:44 | |
| *** yamamoto has joined #openstack-meeting | 02:45 | |
| *** igordc has joined #openstack-meeting | 02:49 | |
| *** ircuser-1 has quit IRC | 02:52 | |
| *** bobh has quit IRC | 02:53 | |
| *** jamesmcarthur has quit IRC | 02:56 | |
| *** tetsuro has joined #openstack-meeting | 02:58 | |
| *** ykatabam has joined #openstack-meeting | 03:07 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:12 | |
| *** markvoelker has quit IRC | 03:14 | |
| *** whoami-rajat has joined #openstack-meeting | 03:19 | |
| *** yamahata has joined #openstack-meeting | 03:20 | |
| *** tetsuro has quit IRC | 03:29 | |
| *** psachin has joined #openstack-meeting | 03:36 | |
| *** markvoelker has joined #openstack-meeting | 03:38 | |
| *** ianychoi_ has joined #openstack-meeting | 03:38 | |
| *** ianychoi has quit IRC | 03:42 | |
| *** markvoelker has quit IRC | 03:43 | |
| *** pcaruana has joined #openstack-meeting | 03:44 | |
| *** hongbin has quit IRC | 03:49 | |
| *** dviroel has quit IRC | 03:59 | |
| *** jamesmcarthur has quit IRC | 04:00 | |
| *** tetsuro has joined #openstack-meeting | 04:03 | |
| *** diablo_rojo has joined #openstack-meeting | 04:04 | |
| *** markvoelker has joined #openstack-meeting | 04:24 | |
| *** jamesmcarthur has joined #openstack-meeting | 04:30 | |
| *** markvoelker has quit IRC | 04:38 | |
| *** rbudden has quit IRC | 04:39 | |
| *** jamesmcarthur has quit IRC | 04:55 | |
| *** rbudden has joined #openstack-meeting | 04:58 | |
| *** jamesmcarthur has joined #openstack-meeting | 05:01 | |
| *** links has joined #openstack-meeting | 05:06 | |
| *** diablo_rojo has quit IRC | 05:11 | |
| *** Luzi has joined #openstack-meeting | 05:19 | |
| *** kopecmartin|off is now known as kopecmartin | 05:23 | |
| *** tetsuro has quit IRC | 05:30 | |
| *** igordc has quit IRC | 05:31 | |
| *** jamesmcarthur has quit IRC | 05:35 | |
| *** iyamahat has joined #openstack-meeting | 05:35 | |
| *** ociuhandu has joined #openstack-meeting | 05:49 | |
| *** ociuhandu has quit IRC | 05:53 | |
| *** jamesmcarthur has joined #openstack-meeting | 06:03 | |
| *** markvoelker has joined #openstack-meeting | 06:18 | |
| *** markvoelker has quit IRC | 06:22 | |
| *** tetsuro has joined #openstack-meeting | 06:24 | |
| *** apetrich has joined #openstack-meeting | 06:37 | |
| *** jamesmcarthur has quit IRC | 06:37 | |
| *** slaweq has joined #openstack-meeting | 06:48 | |
| *** yamamoto has quit IRC | 07:00 | |
| *** rcernin has quit IRC | 07:03 | |
| *** tssurya has joined #openstack-meeting | 07:08 | |
| *** yamamoto has joined #openstack-meeting | 07:09 | |
| *** ykatabam has quit IRC | 07:10 | |
| *** tesseract has joined #openstack-meeting | 07:26 | |
| *** ralonsoh has joined #openstack-meeting | 07:29 | |
| *** jamesmcarthur has joined #openstack-meeting | 07:35 | |
| *** lpetrut has joined #openstack-meeting | 07:38 | |
| *** ralonsoh has quit IRC | 07:54 | |
| *** jamesmcarthur has quit IRC | 08:08 | |
| *** Lucas_Gray has joined #openstack-meeting | 08:12 | |
| *** links has quit IRC | 08:12 | |
| *** mauro|call has quit IRC | 08:13 | |
| *** mauro|call has joined #openstack-meeting | 08:14 | |
| *** e0ne has joined #openstack-meeting | 08:18 | |
| *** panda has quit IRC | 08:22 | |
| *** links has joined #openstack-meeting | 08:23 | |
| *** panda has joined #openstack-meeting | 08:23 | |
| *** ralonsoh has joined #openstack-meeting | 08:26 | |
| *** tetsuro has quit IRC | 08:35 | |
| *** ralonsoh has quit IRC | 08:35 | |
| *** tetsuro has joined #openstack-meeting | 08:36 | |
| *** jamesmcarthur has joined #openstack-meeting | 08:42 | |
| *** ralonsoh has joined #openstack-meeting | 08:47 | |
| *** Lucas_Gray has quit IRC | 08:48 | |
| *** markvoelker has joined #openstack-meeting | 08:50 | |
| *** Lucas_Gray has joined #openstack-meeting | 08:53 | |
| *** markvoelker has quit IRC | 08:55 | |
| *** mauro|call has quit IRC | 08:58 | |
| *** mauro|call has joined #openstack-meeting | 09:00 | |
| *** Lucas_Gray has quit IRC | 09:01 | |
| *** Lucas_Gray has joined #openstack-meeting | 09:02 | |
| *** ociuhandu has joined #openstack-meeting | 09:03 | |
| *** ociuhandu has quit IRC | 09:13 | |
| *** mauro|call is now known as takamatsu | 09:14 | |
| *** ociuhandu has joined #openstack-meeting | 09:16 | |
| *** tetsuro has quit IRC | 09:19 | |
| *** ociuhandu has quit IRC | 09:25 | |
| *** ociuhandu has joined #openstack-meeting | 09:29 | |
| *** jamesmcarthur has quit IRC | 09:57 | |
| *** ricolin_ has joined #openstack-meeting | 10:01 | |
| *** zaneb has quit IRC | 10:03 | |
| *** zbitter has joined #openstack-meeting | 10:03 | |
| *** ricolin has quit IRC | 10:03 | |
| *** carloss has joined #openstack-meeting | 10:12 | |
| *** yamamoto has quit IRC | 10:18 | |
| *** ykatabam has joined #openstack-meeting | 10:39 | |
| *** yamamoto has joined #openstack-meeting | 10:41 | |
| *** yamamoto has quit IRC | 10:45 | |
| *** rfolco has joined #openstack-meeting | 10:54 | |
| *** rfolco|ruck has quit IRC | 10:54 | |
| *** yamamoto has joined #openstack-meeting | 10:59 | |
| *** ociuhandu has quit IRC | 11:07 | |
| *** Wryhder has joined #openstack-meeting | 11:16 | |
| *** Lucas_Gray has quit IRC | 11:17 | |
| *** Wryhder is now known as Lucas_Gray | 11:17 | |
| *** Lucas_Gray has quit IRC | 11:21 | |
| *** Lucas_Gray has joined #openstack-meeting | 11:31 | |
| *** raildo has joined #openstack-meeting | 11:38 | |
| *** jamesmcarthur has joined #openstack-meeting | 11:38 | |
| *** boxiang has quit IRC | 11:42 | |
| *** boxiang has joined #openstack-meeting | 11:42 | |
| *** markvoelker has joined #openstack-meeting | 11:49 | |
| *** ociuhandu has joined #openstack-meeting | 12:09 | |
| donnyd | So I have the compiled data on where the nodepool instances are headed on the public interwebs | 12:11 |
|---|---|---|
| *** markvoelker has quit IRC | 12:11 | |
| donnyd | Just trying to figure out how to present it in a way that doesn't cough up topologies | 12:11 |
| *** ociuhandu has quit IRC | 12:14 | |
| *** jamesmcarthur has quit IRC | 12:16 | |
| *** markvoelker has joined #openstack-meeting | 12:16 | |
| *** ociuhandu has joined #openstack-meeting | 12:19 | |
| *** ociuhandu has quit IRC | 12:22 | |
| *** ociuhandu has joined #openstack-meeting | 12:23 | |
| *** ykatabam has quit IRC | 12:27 | |
| *** kaisers has joined #openstack-meeting | 12:28 | |
| *** rfolco has quit IRC | 12:28 | |
| *** Lucas_Gray has quit IRC | 12:34 | |
| *** ociuhandu has quit IRC | 12:38 | |
| *** haleyb has joined #openstack-meeting | 12:38 | |
| *** ociuhandu has joined #openstack-meeting | 12:39 | |
| *** zbr has quit IRC | 12:46 | |
| *** ociuhandu has quit IRC | 12:46 | |
| *** ociuhandu has joined #openstack-meeting | 12:48 | |
| *** ricolin_ is now known as ricolin | 12:48 | |
| *** jamesmcarthur has joined #openstack-meeting | 12:49 | |
| *** zbr has joined #openstack-meeting | 13:02 | |
| *** Lucas_Gray has joined #openstack-meeting | 13:09 | |
| *** mriedem has joined #openstack-meeting | 13:10 | |
| *** lseki has joined #openstack-meeting | 13:12 | |
| *** iyamahat has quit IRC | 13:13 | |
| *** iyamahat_ has joined #openstack-meeting | 13:13 | |
| *** jamesmcarthur has quit IRC | 13:13 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:14 | |
| *** raub has quit IRC | 13:22 | |
| *** zbitter is now known as zaneb | 13:22 | |
| donnyd | Ok, I think I have it sanitized enough. Now should I send it somewhere? | 13:39 |
| *** jamesmcarthur has quit IRC | 13:44 | |
| *** raub has joined #openstack-meeting | 13:46 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:46 | |
| *** ociuhandu has quit IRC | 13:48 | |
| *** ociuhandu has joined #openstack-meeting | 13:49 | |
| *** Lucas_Gray has quit IRC | 13:49 | |
| *** electrofelix has quit IRC | 13:50 | |
| *** jamesmcarthur has quit IRC | 13:51 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:53 | |
| *** links has quit IRC | 13:55 | |
| *** Lucas_Gray has joined #openstack-meeting | 13:56 | |
| *** tesseract has quit IRC | 13:56 | |
| *** tesseract has joined #openstack-meeting | 13:57 | |
| *** whoami-rajat has quit IRC | 13:57 | |
| *** tesseract has quit IRC | 14:00 | |
| liuyulong | #startmeeting neutron_l3 | 14:00 |
| openstack | Meeting started Wed Aug 7 14:00:53 2019 UTC and is due to finish in 60 minutes. The chair is liuyulong. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: neutron_l3)" | 14:00 | |
| openstack | The meeting name has been set to 'neutron_l3' | 14:01 |
| haleyb | hi | 14:01 |
| liuyulong | #chair haleyb | 14:01 |
| openstack | Current chairs: haleyb liuyulong | 14:01 |
| liuyulong | hi | 14:01 |
| ralonsoh | hi | 14:01 |
| liuyulong | #topic Announcements | 14:01 |
| *** openstack changes topic to "Announcements (Meeting topic: neutron_l3)" | 14:01 | |
| *** tesseract has joined #openstack-meeting | 14:01 | |
| liuyulong | Unfortunately, the port forwarding topic was not accepted by the Summit Team. | 14:03 |
| *** ociuhandu has quit IRC | 14:03 | |
| liuyulong | So mlavalle and I will not share this in the Summit. : ) | 14:04 |
| liuyulong | Any other announcements? | 14:04 |
| liuyulong | OK, let's move on. | 14:05 |
| liuyulong | #topic Bugs | 14:05 |
| *** openstack changes topic to "Bugs (Meeting topic: neutron_l3)" | 14:05 | |
| *** ociuhandu has joined #openstack-meeting | 14:06 | |
| *** liuyulong_ has joined #openstack-meeting | 14:07 | |
| haleyb | there were some new bugs filed last week in the l3 space | 14:07 |
| *** slaweq has quit IRC | 14:07 | |
| liuyulong_ | Bad network connection... | 14:07 |
| liuyulong_ | #link https://wiki.openstack.org/wiki/Network/Meetings#Bug_deputy | 14:08 |
| haleyb | https://bugs.launchpad.net/neutron/+bug/1838699 | 14:08 |
| openstack | Launchpad bug 1838699 in neutron "Removing a subnet from DVR router also removes DVR MAC flows for other router on network" [High,Confirmed] | 14:08 |
| *** slaweq has joined #openstack-meeting | 14:08 | |
| haleyb | slaweq confirmed it, but it will need an owner | 14:08 |
| ralonsoh | My comment yesterday was not registered... | 14:08 |
| ralonsoh | in this bug | 14:09 |
| liuyulong_ | #link https://bugs.launchpad.net/neutron/+bug/1838697 | 14:09 |
| openstack | Launchpad bug 1838697 in neutron "DVR Mac conversion rules are only added for the first router a network is attached to" [Undecided,Incomplete] | 14:09 |
| liuyulong_ | this is also related. | 14:09 |
| ralonsoh | if you have more than one DVR, the match flows will be the same | 14:09 |
| ralonsoh | that means, when you deleted the flows, you'll delete all of them | 14:09 |
| haleyb | liuyulong_: ack, makes sense think they were both filed by same person | 14:09 |
| ralonsoh | (I'l write this comment again in the bug) | 14:10 |
| haleyb | ralonsoh: thanks | 14:10 |
| liuyulong_ | ralonsoh, so it is designed like that, it is a feature? | 14:10 |
| ralonsoh | I don't think so | 14:10 |
| ralonsoh | that means we can't have more than one DVR per host | 14:11 |
| ralonsoh | but I need confirmation | 14:11 |
| *** ociuhandu has quit IRC | 14:11 | |
| ralonsoh | I'll write the comment again in the bug | 14:11 |
| *** dviroel has joined #openstack-meeting | 14:11 | |
| liuyulong_ | OK, thank you, I read some code ealier, the ovs-agent will query the related flow by ports subnet. | 14:12 |
| liuyulong_ | If more than one dvr ports in same subnet, it will indeed delete once for all. | 14:12 |
| liuyulong_ | haleyb, I have bad network connection now, please take over the meeting chair. | 14:15 |
| haleyb | ack | 14:15 |
| haleyb | next bug, https://bugs.launchpad.net/neutron/+bug/1838793 | 14:15 |
| openstack | Launchpad bug 1838793 in neutron ""KeepalivedManagerTestCase" tests failing during namespace deletion" [High,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 14:15 |
| haleyb | https://review.opendev.org/#/c/674820/ was created - thanks ralonsoh | 14:15 |
| ralonsoh | I need to check the CI again | 14:16 |
| haleyb | ralonsoh: i've added myself to review so will look at next update | 14:16 |
| ralonsoh | thanks! | 14:16 |
| haleyb | next bug, https://bugs.launchpad.net/neutron/+bug/1838403 | 14:18 |
| openstack | Launchpad bug 1838403 in neutron "Asymmetric floating IP notifications" [Medium,New] | 14:18 |
| haleyb | i had triaged this last week and couldn't reproduce part of it. see now it was on queens, so perhaps part was fixed | 14:19 |
| haleyb | still needs owner to track down the other possible issued with notifications, if noone wants it i can take a look | 14:20 |
| liuyulong_ | How to "delete a router that still has fip"? | 14:20 |
| *** ociuhandu has joined #openstack-meeting | 14:20 | |
| haleyb | liuyulong_: right, it didn't work for me on master, but can't imagine it works on queens either | 14:20 |
| haleyb | liuyulong_: the part we need to investigate is what happens with a VM is destroyed - is the floating IP in one of the messages? his trace showed it wasn't | 14:21 |
| liuyulong_ | If the VM port is delete, l3_db will catch port_delete notification and release the FIP. | 14:22 |
| haleyb | liuyulong_: yes, but is that sending a notification? | 14:24 |
| *** ociuhandu has quit IRC | 14:25 | |
| * haleyb wonders how fast liuyulong_'s modem is :) (if anyone remembers what a modem is) | 14:25 | |
| ralonsoh | 28.8 bauds per sec | 14:26 |
| haleyb | zoom zoom | 14:26 |
| liuyulong_ | Subscrition is more accurate. | 14:26 |
| liuyulong_ | Subscription | 14:26 |
| haleyb | liuyulong_: oh, maybe they didn't subscribe to all the events? | 14:27 |
| *** Lucas_Gray has quit IRC | 14:27 | |
| liuyulong_ | https://github.com/openstack/neutron/blob/master/neutron/db/l3_db.py#L1848 | 14:27 |
| liuyulong_ | This line it is. | 14:28 |
| *** ykatabam has joined #openstack-meeting | 14:28 | |
| *** Lucas_Gray has joined #openstack-meeting | 14:29 | |
| haleyb | liuyulong_: there is nothing there regarding floating IP though, maybe i'm mis-understanding | 14:30 |
| *** Luzi has quit IRC | 14:30 | |
| liuyulong_ | https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L1941-L1943 | 14:30 |
| haleyb | liuyulong_: if i'm listening for FLOATINGIP events shouldn't i get one when a port with an associated floating IP is deleted? | 14:31 |
| haleyb | either way, please add a comment to the bug so maybe the submitter can track things down | 14:32 |
| liuyulong_ | I have no details about the FIP events now, but according to my experiences, the floating IP will finally get disassociated. | 14:33 |
| haleyb | there was one more new bug | 14:33 |
| haleyb | https://bugs.launchpad.net/neutron/+bug/1839004 | 14:33 |
| openstack | Launchpad bug 1839004 in neutron "Rocky DVR-SNAT seems missing entries for conntrack marking" [Undecided,Incomplete] | 14:33 |
| haleyb | don't know if tidwellr is here, but this looked like maybe a mis-configuration | 14:34 |
| * tidwellr is lurking | 14:34 | |
| haleyb | tidwellr: hi, and this involved dynamic-routing too, any thoughts based on last update? | 14:35 |
| tidwellr | there's still something to look into in that bug, supposedly there was an address scope mismatch and yet the API was reporting that it was finding a next-hop for the tenant subnet | 14:36 |
| tidwellr | that doesn't seem right | 14:36 |
| haleyb | tidwellr: it could be correct if snat was enabled though i think, but he had it disabled | 14:37 |
| *** senrique_ has joined #openstack-meeting | 14:38 | |
| tidwellr | you can see that the uplink subnet is in the null address scope | 14:40 |
| *** Wryhder has joined #openstack-meeting | 14:40 | |
| tidwellr | then, he shows neutron-dynamic-routing returning a next-hop for his tenant subnet | 14:40 |
| tidwellr | that should only happen if the inside and outside subnets are both in the same address scope | 14:41 |
| tidwellr | so | 14:41 |
| tidwellr | either the info in the bug report is inaccurate, or there is still a bug | 14:41 |
| *** Lucas_Gray has quit IRC | 14:41 | |
| *** Wryhder is now known as Lucas_Gray | 14:41 | |
| haleyb | so is that a bug in neutron or dynamic-routing? | 14:42 |
| tidwellr | assuming the steps in the bug report are accurate, yes | 14:42 |
| tidwellr | not sure how that would slip through the tests though, I would have to really look closely at that | 14:43 |
| haleyb | yes a bug in dynamic-routing? | 14:43 |
| tidwellr | yes | 14:44 |
| tidwellr | he ran "openstack bgp speaker list advertised routes" and it returned routes it shouldn't have had | 14:44 |
| haleyb | tidwellr: should we re-assign? as the scoping issue looks like user error | 14:44 |
| *** ociuhandu has joined #openstack-meeting | 14:45 | |
| *** ykatabam has quit IRC | 14:45 | |
| *** mattw4 has joined #openstack-meeting | 14:46 | |
| tidwellr | I'll leave a comment, then try to reproduce this myself. It seems pretty straight forward given his instructions in the bug report | 14:46 |
| *** altlogbot_2 has quit IRC | 14:46 | |
| haleyb | tidwellr: thanks | 14:46 |
| haleyb | liuyulong_: i didn't have any other new bugs, did you have old ones you wanted to talk about? or anyone else? | 14:46 |
| *** altlogbot_0 has joined #openstack-meeting | 14:47 | |
| liuyulong_ | Yes, I have | 14:47 |
| liuyulong_ | For the fix: https://review.opendev.org/#/c/673557/ and the https://bugs.launchpad.net/neutron/+bug/1834308 | 14:48 |
| openstack | Launchpad bug 1834308 in neutron "[DVR][DB] too many slow query during agent restart" [Medium,In progress] - Assigned to LIU Yulong (dragon889) | 14:48 |
| *** yamamoto has quit IRC | 14:48 | |
| liuyulong_ | It is well tested locally. It does not break DVR functions. But I still hope to see if more test result can come from our community. | 14:49 |
| liuyulong_ | The next is: https://bugs.launchpad.net/neutron/+bug/1828494 | 14:50 |
| openstack | Launchpad bug 1828494 in neutron "[RFE][L3] l3-agent should have its capacity" [Wishlist,In progress] - Assigned to LIU Yulong (dragon889) | 14:50 |
| liuyulong_ | I have one question, how many router do you guys think a network node can host? 100? 200? 1000? | 14:51 |
| ralonsoh | no idea | 14:51 |
| tidwellr | when I ran network nodes in production, we capped it at 250 | 14:51 |
| haleyb | liuyulong_: there is no easy answer for that | 14:51 |
| *** jamesmcarthur has quit IRC | 14:51 | |
| haleyb | i think the limiting factor always seems to be how long it takes to restart the agents | 14:52 |
| tidwellr | there are a lot of unique factors that go into that number we came up with though | 14:52 |
| tidwellr | and when I say we capped it, I mean we would add network nodes and rebalance routers to spread the load | 14:52 |
| liuyulong_ | I have one result, when the router reach 300+, the ovs-agent will never restart successfully. | 14:53 |
| tidwellr | that's consistent with what I've observed (anecdotally) | 14:54 |
| *** ianychoi_ is now known as ianychoi | 14:54 | |
| liuyulong_ | My env is 17 physical hosts for dvr_snat nodes, with 2700+ router, disable DHCP. | 14:54 |
| *** yamamoto has joined #openstack-meeting | 14:54 | |
| liuyulong_ | Every ovs-agent will host about 1700+ ports! | 14:54 |
| liuyulong_ | Yes, I've tested 400+ ports for a ovs-agent once, it is about 40+ mins to restart. | 14:55 |
| *** mattw4 has quit IRC | 14:55 | |
| liuyulong_ | Ovs-agent seems can be easily stuck in many code path.... | 14:57 |
| haleyb | that is too long of course, should it be on the performance sub-team's list ? | 14:58 |
| liuyulong_ | Should be, make sense | 15:00 |
| haleyb | liuyulong_: we're at time | 15:01 |
| liuyulong_ | OK | 15:01 |
| liuyulong_ | Let's end here. | 15:01 |
| ralonsoh | bye | 15:01 |
| liuyulong_ | This nick does not have that right. | 15:01 |
| liuyulong_ | haleyb, please end our meeting, thank you. | 15:01 |
| haleyb | #endmeeting | 15:02 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:02 | |
| openstack | Meeting ended Wed Aug 7 15:02:00 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:02 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-08-07-14.00.html | 15:02 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-08-07-14.00.txt | 15:02 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-08-07-14.00.log.html | 15:02 |
| *** mattw4 has joined #openstack-meeting | 15:02 | |
| *** liuyulong_ has quit IRC | 15:02 | |
| *** iyamahat_ has quit IRC | 15:04 | |
| *** mattw4 has quit IRC | 15:08 | |
| *** mattw4 has joined #openstack-meeting | 15:09 | |
| *** jamesmcarthur has joined #openstack-meeting | 15:18 | |
| *** ircuser-1 has joined #openstack-meeting | 15:19 | |
| *** yamamoto has quit IRC | 15:21 | |
| *** yamamoto has joined #openstack-meeting | 15:21 | |
| *** slaweq_ has joined #openstack-meeting | 15:26 | |
| *** slaweq has quit IRC | 15:29 | |
| *** davidsha has joined #openstack-meeting | 15:31 | |
| *** rosmaita has joined #openstack-meeting | 15:35 | |
| *** zhengMa has joined #openstack-meeting | 15:37 | |
| *** tosky has joined #openstack-meeting | 15:45 | |
| *** diablo_rojo has joined #openstack-meeting | 15:48 | |
| *** apetrich has quit IRC | 15:51 | |
| *** jamesmcarthur has quit IRC | 15:52 | |
| *** apetrich has joined #openstack-meeting | 15:53 | |
| *** armax has joined #openstack-meeting | 15:55 | |
| *** whoami-rajat has joined #openstack-meeting | 15:55 | |
| *** woojay has joined #openstack-meeting | 15:56 | |
| *** ralonsoh has quit IRC | 15:57 | |
| *** ralonsoh has joined #openstack-meeting | 15:58 | |
| *** tesseract has quit IRC | 15:58 | |
| *** jbadiapa has joined #openstack-meeting | 16:00 | |
| jungleboyj | #startmeeting cinder | 16:00 |
| openstack | Meeting started Wed Aug 7 16:00:39 2019 UTC and is due to finish in 60 minutes. The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: cinder)" | 16:00 | |
| whoami-rajat | Hi | 16:00 |
| openstack | The meeting name has been set to 'cinder' | 16:00 |
| jungleboyj | courtesy ping: jungleboyj whoami-rajat rajinir lseki carloss pots woojay erlon geguileo eharney rosmaita enriquetaso e0ne smcginnis davidsha walshh_ xyang hemna _hemna tosky | 16:00 |
| smcginnis | o/ | 16:00 |
| pots | o/ | 16:01 |
| e0ne | hi | 16:01 |
| rosmaita | o/ | 16:01 |
| eharney | hi | 16:01 |
| tosky | o/ | 16:01 |
| geguileo | hi! o/ | 16:01 |
| kaisers | hi | 16:01 |
| jungleboyj | @! | 16:01 |
| _pewp_ | jungleboyj (。･∀･)ﾉ | 16:01 |
| woojay | o/ | 16:01 |
| jungleboyj | This week's agenda: | 16:01 |
| xyang | hi | 16:01 |
| jungleboyj | #link https://etherpad.openstack.org/p/cinder-train-meetings | 16:01 |
| *** tesseract has joined #openstack-meeting | 16:02 | |
| jungleboyj | Looks like we have all the usual suspects here. | 16:03 |
| jungleboyj | So, we can get started. | 16:03 |
| jungleboyj | #topic announcements | 16:03 |
| *** openstack changes topic to "announcements (Meeting topic: cinder)" | 16:03 | |
| jungleboyj | We are just two weeks from our mid-cycle: https://etherpad.openstack.org/p/cinder-train-mid-cycle-planning | 16:03 |
| jungleboyj | #link https://etherpad.openstack.org/p/cinder-train-mid-cycle-planning | 16:04 |
| jungleboyj | Topics are still a little light. | 16:04 |
| jungleboyj | Glad to see we are back up to 5 people in physical attendance. | 16:04 |
| jungleboyj | Welcome woojay ! | 16:04 |
| woojay | thank you. | 16:04 |
| *** yamamoto has quit IRC | 16:05 | |
| whoami-rajat | jungleboyj: should we also add the feature progress discussion there. | 16:05 |
| lseki | hi | 16:05 |
| jungleboyj | So, please get your topics into the etherpad. | 16:05 |
| davidsha | Hey! | 16:05 |
| jungleboyj | whoami-rajat: Yes. I was going to work on going through our previous PTGs and pull todos and stuff out of there. | 16:05 |
| *** gyee has joined #openstack-meeting | 16:05 | |
| pots | pwd | 16:05 |
| jungleboyj | If you want to start a section on feature progress that would be great. | 16:06 |
| jungleboyj | pots: #/root | 16:06 |
| *** yamamoto has joined #openstack-meeting | 16:06 | |
| *** yamamoto has quit IRC | 16:06 | |
| e0ne | :) | 16:06 |
| whoami-rajat | jungleboyj: okay, great. | 16:06 |
| *** sfernand has joined #openstack-meeting | 16:06 | |
| jungleboyj | whoami-rajat: Thank you. | 16:06 |
| *** yamamoto has joined #openstack-meeting | 16:07 | |
| jungleboyj | I will be getting in to Raleigh early Monday so I should be in town before everyone if you need anything. | 16:07 |
| jungleboyj | I will add my cell phone to the etherpad. | 16:07 |
| *** igordc has joined #openstack-meeting | 16:07 | |
| *** iyamahat has joined #openstack-meeting | 16:08 | |
| jungleboyj | Ok. I think that is all on that for today. More next week. | 16:09 |
| jungleboyj | I have gone through all our CI systems and marked those that just aren't reporting as unsupported. | 16:09 |
| jungleboyj | #link https://review.opendev.org/#/q/status:open+project:openstack/cinder+branch:master+topic:ci_unsupported | 16:09 |
| *** tssurya has quit IRC | 16:10 | |
| jungleboyj | In a sign that the process works, Synology saw the patch and got things fixed. :-) | 16:10 |
| rosmaita | \o/ | 16:10 |
| jungleboyj | rosmaita: Indeed. | 16:11 |
| jungleboyj | Or they are working on fixing things at least. | 16:11 |
| *** yamamoto has quit IRC | 16:11 | |
| jungleboyj | More on CIs later. | 16:11 |
| jungleboyj | rosmaita: Had an announcement. | 16:11 |
| rosmaita | just a reminder of some upcoming deadlines | 16:12 |
| *** Lucas_Gray has quit IRC | 16:12 | |
| rosmaita | Train release of non-client libraries (os-brick, cinderlib) roughly 4 weeks away (week of 2 September) | 16:12 |
| rosmaita | M-3 is the week after that (week of 9 Sept) | 16:12 |
| rosmaita | M-3 marks the Train release of client libraries (python-cinderclient, python-brick-cinderclient-ext) | 16:12 |
| rosmaita | it is also the feature freeze | 16:12 |
| rosmaita | that's all | 16:13 |
| jungleboyj | rosmaita: How is patch throughput on those libraries? | 16:14 |
| jungleboyj | Do we need more attention to get patches in? | 16:14 |
| rosmaita | haven't really checked | 16:14 |
| jungleboyj | rosmaita: Ok, something we should look into before the mid-cycle so we can get reviews done if necessary. | 16:14 |
| rosmaita | right | 16:14 |
| jungleboyj | :-) | 16:15 |
| jungleboyj | I added it to the etherpad. | 16:15 |
| rosmaita | spot check of os-brick looks good | 16:15 |
| jungleboyj | Ok. Good. | 16:16 |
| jungleboyj | I thought I had looked a little while back and things looked ok. | 16:16 |
| jungleboyj | I think that is good for announcements. | 16:16 |
| jungleboyj | #topic Python 3rd Party CI requirement | 16:17 |
| *** openstack changes topic to "Python 3rd Party CI requirement (Meeting topic: cinder)" | 16:17 | |
| jungleboyj | So, despite our best efforts to communicate this new requirement. It appears that half our CIs haven't made a chance. | 16:17 |
| jungleboyj | #link https://etherpad.openstack.org/p/cinder-py3-ci-review | 16:17 |
| e0ne | :( | 16:17 |
| e0ne | unfortunately, it will be a hard requirement in the next release anyway | 16:18 |
| kaisers | just as a note: I'm one of the stragglers and now working on it | 16:18 |
| kaisers | (quobyte_ci) | 16:18 |
| jungleboyj | I sent out a note to all CI Maintainers and have not seen e-mail back. | 16:18 |
| jungleboyj | I do see that two maintainers have updated the etherpad as requested. | 16:19 |
| jungleboyj | kaisers: Thank you for the update. | 16:19 |
| jungleboyj | Also Hedvig is working on support. | 16:19 |
| jungleboyj | We have a couple CIs that need to be updated from py3.5 to 3.7 | 16:19 |
| jungleboyj | Not rushing to mark them unsupported as they are on 3.5 | 16:19 |
| jungleboyj | Thank you to rosmaita and pots for putting together some helpful information. | 16:20 |
| jungleboyj | #link https://wiki.openstack.org/wiki/Cinder/3rdParty-drivers-py3-update | 16:20 |
| jungleboyj | So, the question is what do we do now? | 16:20 |
| jungleboyj | Do I mark half out drivers as unsupported? | 16:20 |
| jungleboyj | *our | 16:21 |
| * jungleboyj doesn't really want to be the PTL that cut the project in half. :-) | 16:21 | |
| e0ne | IMO we need to do it | 16:21 |
| smcginnis | Better to do it now and give some warning than just having them removed in U because they can no longer work. | 16:21 |
| e0ne | because they won't work in U | 16:21 |
| smcginnis | e0ne: ++ | 16:21 |
| jungleboyj | e0ne: smcginnis ++ | 16:21 |
| eharney | just for context... why won't they work in U? | 16:21 |
| jungleboyj | Ok. | 16:21 |
| *** ociuhandu has quit IRC | 16:22 | |
| e0ne | eharney: U release will drop python2 support | 16:22 |
| eharney | ah | 16:22 |
| *** jamesmcarthur has joined #openstack-meeting | 16:22 | |
| *** ociuhandu has joined #openstack-meeting | 16:22 | |
| jungleboyj | The thing that scares me is that the vendors continue to be ignorant of this. | 16:22 |
| rosmaita | i wonder whether we leave them in for T but have an additional flag or something for the driver page | 16:22 |
| *** jbadiapa has quit IRC | 16:23 | |
| rosmaita | give just a bit more time to get them fully supported for U | 16:23 |
| *** ricolin_ has joined #openstack-meeting | 16:23 | |
| jungleboyj | We pull the drivers in V and then when customers update to that level in a couple years we have a crap storm on our hands. | 16:23 |
| e0ne | rosmaita: that's why we introduced 'unsupported' flag | 16:23 |
| smcginnis | They will be left in for T, they just will require an operator to explicitly declare in their config that they are aware they are running a driver that is not validated. | 16:24 |
| jungleboyj | rosmaita: I don't have a problem with adding a py3 support flag in the matrix so it is clear that is why it is being deprecated. | 16:24 |
| rosmaita | well, the thing is, if they are running T on py2, they should actually be supported, right? | 16:24 |
| smcginnis | But then if they don't in U, we haven't gone through a deprecation period to remove them. | 16:25 |
| pots | I didn't make any changes to my driver when I switched from 2.7 to 3.7, so it's conceivable that these other drivers won't all stop working when python2 is gone. | 16:25 |
| *** ricolin has quit IRC | 16:26 | |
| jungleboyj | :-) | 16:26 |
| smcginnis | pots: Yes, but no way to know that for sure and that is why we have the third party CI requirement we have had since ~icehouse. | 16:26 |
| eharney | have we heard anything about issues blocking CIs from moving to py3? or is it mostly just about making sure someone goes and does it? | 16:26 |
| * jungleboyj loves pots' optimism. | 16:26 | |
| smcginnis | Hedvig did find they have a code issue with py3.7, but likely something easy(ish) to fix. | 16:26 |
| jungleboyj | eharney: Based on pots it is about people actually paying attention and doing it. | 16:26 |
| jungleboyj | eharney or rosmaita: Do you have any strings as Red Hat to pull to try to get more attention to this? | 16:28 |
| jungleboyj | This is going to impact you guys as much as anyone. | 16:28 |
| eharney | jungleboyj: our string is that drivers will have to certify on python 3.6 for the Stein release of our product | 16:28 |
| *** ociuhandu has quit IRC | 16:28 | |
| *** markvoelker has quit IRC | 16:28 | |
| jungleboyj | eharney: So, they need to show support even before train. | 16:29 |
| *** ricolin_ is now known as ricolin | 16:29 | |
| *** psachin has quit IRC | 16:30 | |
| jungleboyj | So, some vendors will get hit by this for RHOSP16 ? | 16:30 |
| eharney | osp15 | 16:31 |
| jungleboyj | Oh. That is right. | 16:31 |
| jungleboyj | That is in beta right now. So, what testing has been done there? | 16:31 |
| eharney | i'm not sure it's gotten too far down the path of driver certification yet | 16:32 |
| jungleboyj | Ok. | 16:32 |
| jungleboyj | eharney: So, that isn't too helpful at this point. | 16:33 |
| pots | just curious, does anyone offer Cinder CI as a service, as an alternative to every vendor rolling their own? | 16:33 |
| smcginnis | :) | 16:33 |
| jungleboyj | Do you think you can find out a bit more before the mid-cycle eharney. | 16:33 |
| smcginnis | A few of us have joked about starting a business to do that for years. | 16:33 |
| jungleboyj | pots: Someone could be rich. | 16:33 |
| jungleboyj | pots: Looking to start a business? | 16:33 |
| eharney | jungleboyj: sure | 16:33 |
| e0ne | pots: only for opensource storages :) | 16:34 |
| jungleboyj | eharney: Thanks. | 16:34 |
| pots | well, this is so different from how we certify storage for Windows, VMware, etc. It might be easier for some companies to just write a check than to stand up a CI like this in house. | 16:35 |
| smcginnis | Yeah, I do think it's a legitimate business idea. | 16:35 |
| jungleboyj | pots: But then someone needs to have one of every storage backend. | 16:35 |
| e0ne | pots: such CIs require to have a hardware | 16:35 |
| jungleboyj | e0ne: ++ | 16:35 |
| pots | hardware is the easy part for the vendors | 16:36 |
| smcginnis | Just need a colo facility and customers to ship their hardware to you. | 16:36 |
| pots | I'm in NYC. I DO have free electricity in my apartment but... | 16:36 |
| smcginnis | Haha | 16:36 |
| jungleboyj | pots: Come join us at the mid-cycle and we can start a business plan. | 16:37 |
| smcginnis | Anyway, probably should get back to business. | 16:37 |
| jungleboyj | I am going Solar and to 1 GB fibre this fall. Not sure I have enough outlets. | 16:37 |
| jungleboyj | Yeah, back to business. | 16:37 |
| jungleboyj | So, it sounds like I should start pushing up the unsupported patches. | 16:37 |
| jungleboyj | Do we want to add a field to the support matrix? | 16:38 |
| e0ne | jungleboyj: +1 | 16:38 |
| rosmaita | so getting back to the 'unsupported' issue, is it possible to keep them supported in T but 'deprecated' due to lack of py3 CI? | 16:38 |
| *** markvoelker has joined #openstack-meeting | 16:38 | |
| jungleboyj | rosmaita: The previous answer was that that is what the unsupported flag is for. | 16:38 |
| *** baojg has quit IRC | 16:38 | |
| rosmaita | yeah, but it seems different semantically | 16:39 |
| rosmaita | though i haven't been around here very long, admittedly | 16:39 |
| jungleboyj | The point there is to get people's attention when they upgrade. The driver is about to not work and the operator should know that. | 16:39 |
| smcginnis | Well, if it was a requirement to run CI under py37, and they didn't update their CI, then that does meet the requirement that it is unsupported. | 16:39 |
| *** tesseract has quit IRC | 16:39 | |
| smcginnis | But I get the nuance. | 16:39 |
| *** ricolin has quit IRC | 16:40 | |
| rosmaita | i'm just worried about the sheer number of noncompliant drivers | 16:40 |
| jungleboyj | So, I guess that is an optics issue. | 16:40 |
| jungleboyj | Do we add a new flag/warning for T and hope that people do something about it. | 16:40 |
| whoami-rajat | jungleboyj: the drivers running py35 or py36 won't be marked unsupported right? | 16:41 |
| *** lpetrut has quit IRC | 16:41 | |
| jungleboyj | whoami-rajat: No, but I will be following up with them to get them to udpate it. | 16:41 |
| eharney | is there a doc/landing page somewhere for people to get all the info if they are suddenly finding a "your driver is unsupported." message from cinder? | 16:41 |
| jungleboyj | Well, we have the release note that goes with it explaining what it means. | 16:42 |
| *** markvoelker has quit IRC | 16:42 | |
| jungleboyj | We don't have a page explaining all that that I know of. Happy to add some documentation. | 16:42 |
| smcginnis | I think the release notes we include with that are pretty good - https://review.opendev.org/#/c/674642/1/releasenotes/notes/storpool-unsupported-11b221fed109535f.yaml | 16:43 |
| jungleboyj | That is probably a good thing to do. | 16:43 |
| *** markvoelker has joined #openstack-meeting | 16:44 | |
| *** baojg has joined #openstack-meeting | 16:44 | |
| *** baojg has quit IRC | 16:44 | |
| jungleboyj | Yeah, it tells them what to do. | 16:44 |
| jungleboyj | I think it would be good to have an operator facing page that explains why this happens thouhg. | 16:44 |
| *** ociuhandu has joined #openstack-meeting | 16:44 | |
| *** baojg has joined #openstack-meeting | 16:44 | |
| jungleboyj | Just now that we are likely to have more people running into this. | 16:44 |
| *** baojg has quit IRC | 16:45 | |
| jungleboyj | Want to make it clear that this is something they should follow up with the vendor on, not be mad at OpenStack. | 16:45 |
| *** baojg has joined #openstack-meeting | 16:45 | |
| *** baojg has quit IRC | 16:45 | |
| jungleboyj | So, I will look into doing that. | 16:46 |
| *** baojg has joined #openstack-meeting | 16:46 | |
| rosmaita | i guess my caveat to marking these unsupported would be the releasenote should be clear it's a py3 CI issue | 16:46 |
| *** baojg has quit IRC | 16:46 | |
| rosmaita | i think we want to distinguish people doing py2 CI vs. no CI at all | 16:46 |
| jungleboyj | rosmaita: Ok. Agreed. | 16:46 |
| jungleboyj | rosmaita: ++ | 16:46 |
| rosmaita | ok | 16:47 |
| *** baojg has joined #openstack-meeting | 16:47 | |
| *** baojg has quit IRC | 16:47 | |
| jungleboyj | Ok, so, make the reason clear in the release note. Improve the documentation on what it means to be unsupported. | 16:47 |
| *** baojg has joined #openstack-meeting | 16:48 | |
| *** baojg has quit IRC | 16:48 | |
| rosmaita | i can take a crack at the doc improvement, because i clearly don't have a good grasp of this at the moment | 16:48 |
| jungleboyj | Start submitting patches for this. Try to hold out until late in the release to merge the patches to give other people time to start fixing this. | 16:48 |
| jungleboyj | rosmaita: Ok. Sounds good. | 16:48 |
| jungleboyj | I can help you with that. | 16:48 |
| rosmaita | great | 16:48 |
| *** baojg has joined #openstack-meeting | 16:48 | |
| jungleboyj | Think we should do that in official documentation. Not a wiki. | 16:48 |
| *** baojg has quit IRC | 16:48 | |
| rosmaita | right, i think on the support matrix page (though we can discuss offline) | 16:49 |
| jungleboyj | Ok. Sounds good. | 16:49 |
| pots | Does OpenStack have a marketing person who should be aware of this? It seems like we should be reaching out to the vendors' marketing folks in case the right people didn't see Jay's e-mail. | 16:49 |
| rosmaita | pots: ++ | 16:49 |
| jungleboyj | pots: Ah, good question. | 16:49 |
| jungleboyj | Let me follow up on that with the foundation. | 16:50 |
| *** baojg has joined #openstack-meeting | 16:50 | |
| *** baojg has quit IRC | 16:50 | |
| *** ociuhandu has quit IRC | 16:50 | |
| *** baojg has joined #openstack-meeting | 16:51 | |
| *** igordc has quit IRC | 16:51 | |
| *** baojg has quit IRC | 16:51 | |
| jungleboyj | Wes. I will send an e-mail to him today. | 16:51 |
| jungleboyj | And Erin too maybe. | 16:51 |
| *** baojg has joined #openstack-meeting | 16:51 | |
| *** baojg has quit IRC | 16:51 | |
| smcginnis | I'm a little concerned that we are flipping this where the community needs to be the ones watching and taking care of vendor drivers when it should be the other way around. If a vendor has customers that want to use their storage in OpenStack, it's on them to stay involved and make sure that is available. | 16:52 |
| *** baojg has joined #openstack-meeting | 16:52 | |
| *** baojg has quit IRC | 16:52 | |
| e0ne | smcginnis: +1 | 16:53 |
| jungleboyj | smcginnis: I agree, but this is also at a point where I feel like this should be escalated. | 16:53 |
| rosmaita | smcginnis: ordinarily, i would agree with you, but when half the drivers are noncompliant, i think we need to look on our side too | 16:53 |
| *** baojg has joined #openstack-meeting | 16:53 | |
| jungleboyj | I don't want Cinder to be cut in half and have the foundation come back and say WTF? | 16:53 |
| *** baojg has quit IRC | 16:53 | |
| jungleboyj | rosmaita: ++ | 16:53 |
| jungleboyj | I mean, this makes me stick to my stomach. | 16:53 |
| rosmaita | sick, even | 16:54 |
| *** baojg has joined #openstack-meeting | 16:54 | |
| jungleboyj | Yeah, that too. Can't type while wretching | 16:54 |
| *** baojg has quit IRC | 16:54 | |
| rosmaita | :P | 16:54 |
| jungleboyj | This speaks to the health of the community. | 16:54 |
| jungleboyj | Anyway, I am going to reach out to the foundation and see what happens. | 16:54 |
| smcginnis | These vendors are not participating and making sure OpenStack is healthy. I'm just fine seeing them go away (and lose markey opportunities) if they don't care about the project. | 16:54 |
| *** baojg has joined #openstack-meeting | 16:54 | |
| *** baojg has quit IRC | 16:55 | |
| smcginnis | *marketing | 16:55 |
| jungleboyj | smcginnis: I know. | 16:55 |
| *** baojg has joined #openstack-meeting | 16:55 | |
| *** baojg has quit IRC | 16:55 | |
| jungleboyj | Anyway, I think we have a general direction here. Can talk more at the mid-cycle. | 16:55 |
| rosmaita | ++ | 16:55 |
| jungleboyj | There is one more topic on the schedule. | 16:56 |
| jungleboyj | #topic encryption effort update | 16:56 |
| *** openstack changes topic to "encryption effort update (Meeting topic: cinder)" | 16:56 | |
| jungleboyj | rosmaita: | 16:56 |
| *** baojg has joined #openstack-meeting | 16:56 | |
| *** diablo_rojo__ has joined #openstack-meeting | 16:56 | |
| rosmaita | proposed changes to os-brick will be along the lines of what was proposed as an oslo library | 16:56 |
| rosmaita | #link https://review.opendev.org/#/c/618754/ | 16:56 |
| *** baojg has quit IRC | 16:56 | |
| rosmaita | with the Train os-brick release a month away, i'm pessimistic that they'll make train | 16:56 |
| *** diablo_rojo has quit IRC | 16:56 | |
| rosmaita | i was thinking it might be a good idea to request a feature branch for this work? | 16:56 |
| rosmaita | or is that too much trouble? | 16:56 |
| *** baojg has joined #openstack-meeting | 16:57 | |
| *** baojg has quit IRC | 16:57 | |
| rosmaita | that's all i have | 16:57 |
| jungleboyj | Not sure how hard that is to do. | 16:57 |
| eharney | i think we usually get pretty far with a well-organized patch series | 16:57 |
| smcginnis | eharney: ++ | 16:57 |
| jungleboyj | eharney: ++ | 16:57 |
| eharney | not sure a feature branch would provide much benefit | 16:57 |
| *** baojg has joined #openstack-meeting | 16:57 | |
| smcginnis | From what I've heard, feature branches end up being mroe trouble than they're worth. | 16:58 |
| *** baojg has quit IRC | 16:58 | |
| rosmaita | ok, i will not suggest it, then | 16:58 |
| smcginnis | Their easy to make, just need to add it to the deliverable file in the releases repo. | 16:58 |
| smcginnis | *they're | 16:58 |
| smcginnis | Bleh | 16:58 |
| *** baojg has joined #openstack-meeting | 16:58 | |
| *** baojg has quit IRC | 16:58 | |
| jungleboyj | Ok. But if they are too much trouble lets not go that direction. | 16:59 |
| jungleboyj | Ok. So, I think that is it for this meeting. | 16:59 |
| jungleboyj | Thanks for the discussion everyone. | 16:59 |
| jungleboyj | #endmeeting | 16:59 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 16:59 | |
| openstack | Meeting ended Wed Aug 7 16:59:55 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:59 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-08-07-16.00.html | 16:59 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-08-07-16.00.txt | 17:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-08-07-16.00.log.html | 17:00 |
| *** tosky has left #openstack-meeting | 17:00 | |
| *** rosmaita has left #openstack-meeting | 17:01 | |
| *** ociuhandu has joined #openstack-meeting | 17:01 | |
| *** woojay has left #openstack-meeting | 17:01 | |
| *** zhengMa has quit IRC | 17:01 | |
| *** e0ne has quit IRC | 17:06 | |
| *** igordc has joined #openstack-meeting | 17:10 | |
| *** jamesmcarthur has quit IRC | 17:12 | |
| *** artom has quit IRC | 17:13 | |
| *** brinzhang_ has joined #openstack-meeting | 17:16 | |
| *** altlogbot_0 has quit IRC | 17:17 | |
| *** davidsha has quit IRC | 17:18 | |
| *** brinzhang has quit IRC | 17:19 | |
| *** altlogbot_3 has joined #openstack-meeting | 17:23 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:28 | |
| *** ociuhandu has quit IRC | 17:29 | |
| *** altlogbot_3 has quit IRC | 17:30 | |
| *** jamesmcarthur has quit IRC | 17:32 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:34 | |
| *** ekcs has joined #openstack-meeting | 17:35 | |
| *** altlogbot_0 has joined #openstack-meeting | 17:35 | |
| *** mattw4 has quit IRC | 17:38 | |
| *** ociuhandu has joined #openstack-meeting | 17:39 | |
| *** jamesmcarthur has quit IRC | 17:39 | |
| *** ociuhandu has quit IRC | 17:43 | |
| *** jamesmcarthur has joined #openstack-meeting | 17:49 | |
| *** jamesmcarthur has quit IRC | 18:00 | |
| *** kopecmartin is now known as kopecmartin|off | 18:00 | |
| *** jamesmcarthur has joined #openstack-meeting | 18:01 | |
| *** Vadmacs has joined #openstack-meeting | 18:05 | |
| *** diablo_rojo__ is now known as diablo_rojo | 18:13 | |
| *** artom has joined #openstack-meeting | 18:17 | |
| *** yamahata has quit IRC | 18:27 | |
| *** iyamahat has quit IRC | 18:27 | |
| *** Vadmacs has quit IRC | 18:32 | |
| *** mattw4 has joined #openstack-meeting | 18:33 | |
| *** iyamahat has joined #openstack-meeting | 18:42 | |
| *** mattw4 has quit IRC | 18:44 | |
| *** mattw4 has joined #openstack-meeting | 18:44 | |
| *** isq_ has left #openstack-meeting | 18:46 | |
| *** e0ne has joined #openstack-meeting | 18:49 | |
| diablo_rojo | Hello StoryBoard meeting attendees! We will be skipping our weekly meeting this week. If you have any questions/comments, please join us in #storyboard! | 18:52 |
| *** jamesmcarthur has quit IRC | 19:03 | |
| *** jamesmcarthur has joined #openstack-meeting | 19:16 | |
| *** jamesmcarthur has quit IRC | 19:35 | |
| *** lpetrut has joined #openstack-meeting | 19:35 | |
| *** jamesmcarthur has joined #openstack-meeting | 19:36 | |
| *** sfernand has quit IRC | 19:36 | |
| *** vesper11 has quit IRC | 19:37 | |
| *** e0ne has quit IRC | 19:46 | |
| *** markvoelker has quit IRC | 20:01 | |
| *** lpetrut has quit IRC | 20:01 | |
| *** markvoelker has joined #openstack-meeting | 20:03 | |
| *** whoami-rajat has quit IRC | 20:17 | |
| *** tdasilva has joined #openstack-meeting | 20:55 | |
| *** ralonsoh has quit IRC | 20:58 | |
| clayg | let's go!!! | 21:00 |
| timburke | #startmeeting swift | 21:00 |
| openstack | Meeting started Wed Aug 7 21:00:12 2019 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:00 |
| *** openstack changes topic to " (Meeting topic: swift)" | 21:00 | |
| openstack | The meeting name has been set to 'swift' | 21:00 |
| timburke | who's here for the swift meeting? | 21:00 |
| clayg | o/ | 21:00 |
| timburke | clayg, so excited! you beat the meeting start! | 21:00 |
| kota_ | o/ | 21:00 |
| kota_ | lol | 21:00 |
| clayg | 😁 | 21:00 |
| rledisez | o/ | 21:01 |
| tdasilva | o/ | 21:01 |
| mattoliverau | o/ | 21:01 |
| timburke | agenda's mostly a continuation of a lot of outstanding work | 21:01 |
| timburke | #link https://wiki.openstack.org/wiki/Meetings/Swift | 21:01 |
| timburke | #topic shanghai | 21:01 |
| *** openstack changes topic to "shanghai (Meeting topic: swift)" | 21:01 | |
| timburke | just a quick reminder that prices for the shanghai summit increase next week -- if you're going, you might want to buy now | 21:02 |
| timburke | (i'm not sure off-hand what the price will increase *to*, i just know it will increase) | 21:02 |
| timburke | #link https://www.openstack.org/summit/shanghai-2019/ | 21:02 |
| timburke | thanks everybody for decalring whether you'll be there or not on the etherpad -- it should definitely help the foundation plan for attendance | 21:02 |
| kota_ | and also timburke should respond to the TC how *Swift* PTG will be going? | 21:03 |
| *** yamamoto has joined #openstack-meeting | 21:03 | |
| timburke | yup -- on my list for this week :-) | 21:03 |
| kota_ | great | 21:03 |
| timburke | (attendance will probably be a little light, but then, we were expecting that) | 21:03 |
| timburke | on to updates! | 21:04 |
| timburke | #topic py3 | 21:04 |
| *** openstack changes topic to "py3 (Meeting topic: swift)" | 21:04 | |
| timburke | i've been working on finishing up the py3 func tests recently | 21:04 |
| timburke | there were a couple patches small enough that i didn't feel bad +A'ing myself | 21:04 |
| timburke | #link https://review.opendev.org/#/c/674878/ | 21:04 |
| timburke | #link https://review.opendev.org/#/c/674703/ | 21:04 |
| timburke | aside from that, the main advance was in starting on the s3 tests | 21:04 |
| timburke | #link https://review.opendev.org/#/c/674716/ | 21:04 |
| timburke | and i saw that mattoliverau's been doing some review, so thanks for that! | 21:04 |
| mattoliverau | Nps | 21:05 |
| timburke | i'm torn between wanting probe tests (because they'll exercise more of the system) or func tests (so we can start reducing the total number of gate jobs running) | 21:06 |
| timburke | but we're getting there on both fronts! | 21:06 |
| *** slaweq_ has quit IRC | 21:06 | |
| timburke | any questions on py3? | 21:06 |
| timburke | #topic lots of small files | 21:07 |
| *** openstack changes topic to "lots of small files (Meeting topic: swift)" | 21:07 | |
| *** diablo_rojo has quit IRC | 21:07 | |
| timburke | i'm guessing there hasn't been too much going on upstream here, with alecuyer still on vacation | 21:07 |
| *** diablo_rojo has joined #openstack-meeting | 21:08 | |
| kota_ | timburke: true | 21:08 |
| rledisez | timburke: yes | 21:08 |
| kota_ | nothing except I bought the PTG ticket. | 21:08 |
| *** yamamoto has quit IRC | 21:08 | |
| timburke | but we've been playing around with it at SwiftStack! we can get it to run, at least :-) | 21:08 |
| kota_ | +1 | 21:08 |
| timburke | is there anything else we should talk about for it? what are the next steps that we want to take? | 21:09 |
| rledisez | wow, that's something! somebody else make it running \o/ | 21:09 |
| timburke | rledisez, always good when we can get past "works on my machine" :-) | 21:09 |
| clayg | or "works in my public cloud" or w/e | 21:10 |
| rledisez | :) | 21:10 |
| clayg | but definately progress - it's gunna be a hoot | 21:10 |
| *** slaweq_ has joined #openstack-meeting | 21:11 | |
| timburke | hopefully i'll be able to give it more attention Real Soon Now | 21:11 |
| timburke | rledisez, as we find issues, where's a good place for us to collect them? | 21:12 |
| clayg | gerrit? with fixes? | 21:12 |
| rledisez | clayg: +1 | 21:12 |
| clayg | obviously if we don't know *how* to fix it a failing test and a ping in IRC would be a good start... probably NOT lp bugs | 21:12 |
| timburke | clayg, that's my usual way of dealing with issues, but sometimes people want something more trackable ;-P | 21:13 |
| clayg | but we could certainly do a etherpad or trello tracker if activity ramps up to the point that it's needed? | 21:13 |
| timburke | i was trying to remember if we had a trello board set up or anything | 21:13 |
| kota_ | trello might be good place | 21:13 |
| rledisez | and definitively a ping, because we are some commits ahead for some bugfix (I think it's on alecuyer todolist to upstream them asap) | 21:13 |
| kota_ | https://trello.com/b/xhNxrcLX/losf | 21:13 |
| kota_ | #link: https://trello.com/b/xhNxrcLX/losf | 21:13 |
| timburke | thanks kota_ | 21:13 |
| clayg | ^ oh, BOOM | 21:13 |
| timburke | love it. i think there's one thing i oughta write up (or as clayg suggested, just go ahead and fix) | 21:14 |
| timburke | #topic sharding | 21:15 |
| *** openstack changes topic to "sharding (Meeting topic: swift)" | 21:15 | |
| timburke | i know we've got mattoliverau's patches | 21:15 |
| mattoliverau | I haven't done much extra here | 21:15 |
| timburke | i put up one or two others -- not exactly to do with autosharding, but certainly to do with having a completed feature | 21:15 |
| mattoliverau | I've been meaning to do some more testing of the existing patches, but got caught up with work :( | 21:15 |
| mattoliverau | I saw! thanks | 21:16 |
| timburke | #link https://review.opendev.org/#/c/675014/ | 21:16 |
| mattoliverau | One tracking updated and some py3 fixes right? | 21:16 |
| timburke | in particular, to stop hammering the root container quite so much | 21:16 |
| mattoliverau | yeah thank'll be a good thing :) | 21:17 |
| mattoliverau | *that'll | 21:17 |
| timburke | oh, right -- the other thing wasn't a patch but a bug | 21:17 |
| timburke | #link https://bugs.launchpad.net/swift/+bug/1839355 | 21:17 |
| openstack | Launchpad bug 1839355 in OpenStack Object Storage (swift) "container-sharder should keep cleaving when there are no rows" [Undecided,New] | 21:17 |
| *** slaweq_ has quit IRC | 21:17 | |
| mattoliverau | oh, interesting | 21:17 |
| timburke | though i'm not entirely sure on the right resolution there. working through mostly-empty DBs quickly would be good, but i wonder if we could save the hassle of creating (and replicating) all those shard dbs | 21:18 |
| timburke | somethign to think about, anyway | 21:18 |
| mattoliverau | +1 | 21:19 |
| timburke | #topic symlinks and versioning | 21:19 |
| *** openstack changes topic to "symlinks and versioning (Meeting topic: swift)" | 21:19 | |
| clayg | the realization for me was that it can take a long time to shard an empty container just because of the cardinality of shard ranges (the handoff knows about ranges because of replication, but it still takes many cycles to say "yup, noting in that range, nothing in that one either; good enough for this cycle - see you next time!" ZZZzzzzz | 21:19 |
| timburke | yeah, that was about the point at which i wrote up the bug, shortly after doing the math to figure out how long it'll take to clear it | 21:20 |
| clayg | hardlinks to SLOs won't always have the slo_etag and symlink_bytes bits correct in the container listings -> https://review.opendev.org/#/c/633094/24/swift/common/middleware/symlink.py | 21:21 |
| timburke | not sure there's much we can do about that, though :-/ | 21:21 |
| clayg | would love to have some feedback on how we put that in the docs without sounding stupid... "fixing it" seems not worth it considering the cruft and time it would take to essentially just reupload the manifest for them | 21:22 |
| clayg | sometimes trying to make things better just makes things more complicated 🤷♂️ | 21:22 |
| timburke | on the plus side, neither will the SLO it's pointing at... | 21:22 |
| clayg | timburke: that's not a plus by any measure | 21:22 |
| clayg | it's just bad and now the badness will accretes | 21:23 |
| timburke | true enough. but we're *mostly* not making things actively worse? | 21:23 |
| *** markvoelker has quit IRC | 21:23 | |
| *** raildo has quit IRC | 21:23 | |
| clayg | I mean I think that paragraph that explains how hardlinks worse *does* make it actively worse - like the sphere of context where the problem forced users to have to think about it is bigger now | 21:24 |
| clayg | ... but on the whole I think the change is still a net gain - obviously I want hardlinks! | 21:24 |
| timburke | i'm still hopeful that the total number of SLOs written *with* the metadata will be orders of magnitude larger than the total number written *without*, given enough time | 21:24 |
| clayg | just rubber hit the road and this is best we've come up with yet | 21:24 |
| clayg | ... suggestions welcome 😁 | 21:24 |
| *** vesper11 has joined #openstack-meeting | 21:25 | |
| clayg | i'm rebasing symlink_versions now and mostly hoping i'm done with hardlinks (sans the amazing doco suggestions I'm going to get following this meeting) | 21:26 |
| clayg | I am also finding it annoying that you have to dig out the manifest etag to make a hardlink | 21:26 |
| timburke | clayg, is there anything the rest of us should be weighing in on? are you looking for consensus on what the listing should look like, say? | 21:26 |
| clayg | I'd like to either add support python-swiftclient's `stat` command to do query-params (?multipart-manifest=get) OR make *LO's always return the an X-Manifest-Etag | 21:27 |
| clayg | ... or both? preferences? opinions? | 21:27 |
| clayg | I would like as many people as possible to read the docs on the hardlink patch so they understand the feature | 21:27 |
| timburke | x-manifest-etag's not a bad idea... we've already got it loaded in our head anyway... | 21:27 |
| tdasilva | seems like the api change is better than the client change?? | 21:28 |
| clayg | if they want to play with `swift upload links -H 'content-length: 0' --object-name link.version -H 'x-symlink-target: test/big.version' -H 'x-symlink-target-etag: b89c395acda3886b823803b7dccb4765' - </dev/null` that's cool too, practically a review! | 21:28 |
| clayg | but just reading the docs and asking questions about the parts that seem weird would be SUPER helpful - maybe we missed something obvious | 21:29 |
| *** jamesmcarthur has quit IRC | 21:29 | |
| clayg | but even better - it adds documentation to the review - so later after we merge it and we're like "why didn't we do XYZ" we wrote that down somewhere | 21:29 |
| timburke | client change might be nice, too -- i could see some utility to being able to download the SLO manifest from the cli more easily | 21:29 |
| *** jamesmcarthur has joined #openstack-meeting | 21:29 | |
| *** macz has joined #openstack-meeting | 21:29 | |
| clayg | timburke: well there's also symlink=get - so yeah broad support in the cli for query params would be pretty sick | 21:30 |
| tdasilva | timburke: agree, was just thinking in terms of one versus the other, but both sound even better :) | 21:31 |
| clayg | but it sounds like everyone likes x-manifest-etag too - so that's probably even cheaper | 21:31 |
| timburke | clayg, do you want to add that as part of the hardlinks patch, or should i propose that separately? | 21:31 |
| *** senrique_ has quit IRC | 21:32 | |
| timburke | or you could do it. maybe you'd prefer i spend my time reviewing ;-) | 21:32 |
| clayg | no strong preference on my end - probably sufficiently orthogonal to warrent a seperate patch if for no other reason to keep patch count down | 21:32 |
| timburke | 👍 | 21:32 |
| clayg | well, I'm not going to do anything until I finish the rebase on symlink_versions and I'm out of time today... | 21:32 |
| timburke | i'll see about knocking that out real quick | 21:33 |
| clayg | so if while looking at hardlinks you were to "accidently" purpose a dependent patch that added x-manifest-etag I could +2 it in the morning ;) | 21:33 |
| timburke | anything else for symlinks and versioning? | 21:33 |
| *** jamesmcarthur has quit IRC | 21:34 | |
| clayg | not at this time - other non swiftstack cores - please take 10m to read the docs and throw some comments on there | 21:34 |
| clayg | happy to take questions | 21:34 |
| clayg | appreciate the feedback | 21:34 |
| kota_ | will do | 21:35 |
| kota_ | might take a time Friday on my time. | 21:35 |
| timburke | thanks kota_ | 21:35 |
| timburke | #topic open discussion | 21:35 |
| *** openstack changes topic to "open discussion (Meeting topic: swift)" | 21:35 | |
| timburke | anything else we ought to discuss? | 21:35 |
| timburke | fwiw i've been playing with different ways to get a feel for what's going on in swift, and i feel like i've come up with a decent dashboard, at least to prep for this meeting | 21:37 |
| timburke | #link https://review.opendev.org/#/dashboard/?title=The+Week+That+Was&foreach=-age:1week&Landed+(Server)=is:merged+project:openstack/swift&Landed+(Client)=is:merged+project:openstack/python-swiftclient&Active+(Server)=is:open+project:openstack/swift&Active+(Client)=is:open+project:openstack/python-swiftclient&Abandoned+(Server)=is:abandoned+project:openstack/swift&Abandoned+(Client)=is:abandoned+project:openstack/python-swiftclient | 21:37 |
| kota_ | looks good | 21:38 |
| *** markvoelker has joined #openstack-meeting | 21:38 | |
| timburke | i might need to propose fewer patches though -- i'm not sure i like seeing my name that much :-/ | 21:38 |
| mattoliverau | timburke: don't stop being you, your awesome! | 21:39 |
| *** jamesmcarthur has joined #openstack-meeting | 21:40 | |
| timburke | all right, seems like there isn't much more to talk about right now -- let's call it a little early and let kota_ and mattoliverau get breakfast ;-) | 21:40 |
| mattoliverau | And just to jump back a bit. I'll start reading the hardlink docs today. I have some long meetings this morning... I'll need something to read :P | 21:41 |
| mattoliverau | \o/ breakfast time! | 21:41 |
| timburke | thanks mattoliverau! | 21:41 |
| timburke | thank you all for coming, and thank you for working on swift! | 21:41 |
| timburke | #endmeeting | 21:41 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 21:41 | |
| openstack | Meeting ended Wed Aug 7 21:41:32 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:41 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-08-07-21.00.html | 21:41 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-08-07-21.00.txt | 21:41 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-08-07-21.00.log.html | 21:41 |
| *** zaitcev_ has joined #openstack-meeting | 21:43 | |
| *** zaitcev_ has left #openstack-meeting | 21:43 | |
| *** diablo_rojo has quit IRC | 21:46 | |
| *** markvoelker has quit IRC | 21:50 | |
| *** tidwellr has quit IRC | 22:02 | |
| *** diablo_rojo has joined #openstack-meeting | 22:05 | |
| *** ykatabam has joined #openstack-meeting | 22:08 | |
| *** diablo_rojo has quit IRC | 22:10 | |
| *** slaweq_ has joined #openstack-meeting | 22:11 | |
| *** rcernin has joined #openstack-meeting | 22:15 | |
| *** slaweq_ has quit IRC | 22:16 | |
| *** markvoelker has joined #openstack-meeting | 22:19 | |
| *** mriedem has quit IRC | 22:24 | |
| *** brinzhang has joined #openstack-meeting | 22:31 | |
| *** ykatabam has quit IRC | 22:33 | |
| *** brinzhang_ has quit IRC | 22:34 | |
| *** markvoelker has quit IRC | 22:38 | |
| *** jamesmcarthur has quit IRC | 22:41 | |
| *** jamesmcarthur has joined #openstack-meeting | 22:42 | |
| *** notmyname has quit IRC | 22:45 | |
| *** jamesmcarthur has quit IRC | 22:47 | |
| *** jamesmcarthur has joined #openstack-meeting | 22:50 | |
| *** markvoelker has joined #openstack-meeting | 22:53 | |
| *** macz has quit IRC | 22:53 | |
| *** jamesmcarthur has quit IRC | 22:57 | |
| *** tdasilva has quit IRC | 23:02 | |
| *** tdasilva has joined #openstack-meeting | 23:02 | |
| *** markvoelker has quit IRC | 23:08 | |
| *** yamamoto has joined #openstack-meeting | 23:08 | |
| *** slaweq_ has joined #openstack-meeting | 23:11 | |
| *** notmyname has joined #openstack-meeting | 23:15 | |
| *** slaweq_ has quit IRC | 23:16 | |
| *** jamesmcarthur has joined #openstack-meeting | 23:28 | |
| *** jamesmcarthur has quit IRC | 23:33 | |
| *** igordc has quit IRC | 23:35 | |
| *** markvoelker has joined #openstack-meeting | 23:38 | |
| *** markvoelker has quit IRC | 23:43 | |
| *** jamesmcarthur has joined #openstack-meeting | 23:43 | |
| *** jamesmcarthur has quit IRC | 23:46 | |
| *** tdasilva has quit IRC | 23:51 | |
| *** mattw4 has quit IRC | 23:51 | |
| *** tdasilva has joined #openstack-meeting | 23:52 | |
| *** tdasilva has quit IRC | 23:55 | |
| *** tdasilva has joined #openstack-meeting | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!