| *** lseki has quit IRC | 00:03 | |
| *** armax has quit IRC | 00:03 | |
| *** links has joined #openstack-meeting | 00:06 | |
| *** links has quit IRC | 00:06 | |
| *** yangjianfeng has joined #openstack-meeting | 00:14 | |
| *** yangjianfeng has quit IRC | 00:16 | |
| *** markvoelker has joined #openstack-meeting | 00:24 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:29 | |
| *** jamesmcarthur has quit IRC | 00:35 | |
| *** diablo_rojo has joined #openstack-meeting | 00:37 | |
| *** zhangchi has joined #openstack-meeting | 00:43 | |
| *** donghao has joined #openstack-meeting | 00:48 | |
| *** nitinuikey has joined #openstack-meeting | 00:52 | |
| *** nitinuikey has quit IRC | 00:52 | |
| *** gyee has quit IRC | 00:52 | |
| *** donghao has quit IRC | 00:53 | |
| *** mriedem has quit IRC | 01:00 | |
| zhangchi | #startmeeting tricircle | 01:00 |
|---|---|---|
| openstack | Meeting started Wed Sep 25 01:00:26 2019 UTC and is due to finish in 60 minutes. The chair is zhangchi. Information about MeetBot at http://wiki.debian.org/MeetBot. | 01:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 01:00 |
| *** openstack changes topic to " (Meeting topic: tricircle)" | 01:00 | |
| openstack | The meeting name has been set to 'tricircle' | 01:00 |
| zhangchi | Morning all | 01:00 |
| zhangchi | welcome back home to tricircle bio-weekly meeting | 01:01 |
| zhangchi | A new version will be published soon in a few days | 01:03 |
| *** markvoelker has quit IRC | 01:08 | |
| *** ekcs has quit IRC | 01:10 | |
| zhangchi | region_name has been added in kuryr for neutron_client retriving data in different regions | 01:15 |
| *** Liang__ has joined #openstack-meeting | 01:21 | |
| zhangchi | Any other issues | 01:28 |
| *** XSM_HUST has joined #openstack-meeting | 01:30 | |
| zhangchi | Hi XSM_HUST | 01:31 |
| XSM_HUST | hi | 01:31 |
| zhangchi | neutron_client in kuryr can retrieving data from different region now.How about your container guide manual | 01:32 |
| XSM_HUST | I'm sorry that I'm busy looking for a job recently. The progress of the project may not be pushed forward recently. | 01:34 |
| XSM_HUST | I will finish the container guide manual as soon as I finish my job-hunting. | 01:35 |
| *** armax has joined #openstack-meeting | 01:36 | |
| zhangchi | Yeah , Good luck for your job-hunting. It does not matter much as a new development circle will be started . Hope to see you then | 01:38 |
| *** diablo_rojo has quit IRC | 01:38 | |
| zhangchi | There has been a few exceptions for uni-test then as neutron master udpated. jobs listed here https://etherpad.openstack.org/p/tricircle-train-plan | 01:40 |
| zhangchi | Next meeting is 9th Oct Hope to see you then | 01:42 |
| *** armax has quit IRC | 01:42 | |
| zhangchi | Thanks for you attending XSM_HUST Good luck to you | 01:43 |
| XSM_HUST | Ok, thank you for your understanding. | 01:43 |
| zhangchi | I see. | 01:44 |
| *** XSM_HUST has quit IRC | 01:45 | |
| *** hongbin has joined #openstack-meeting | 01:49 | |
| zhangchi | Hi hongbin | 01:52 |
| zhangchi | See you all next time | 01:53 |
| zhangchi | Bye | 01:53 |
| zhangchi | #endmeeting | 01:53 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 01:53 | |
| openstack | Meeting ended Wed Sep 25 01:53:40 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 01:53 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tricircle/2019/tricircle.2019-09-25-01.00.html | 01:53 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tricircle/2019/tricircle.2019-09-25-01.00.txt | 01:53 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tricircle/2019/tricircle.2019-09-25-01.00.log.html | 01:53 |
| *** njohnston has quit IRC | 02:03 | |
| *** armax has joined #openstack-meeting | 02:05 | |
| *** apetrich has quit IRC | 02:10 | |
| *** ekcs has joined #openstack-meeting | 02:24 | |
| *** ekcs has quit IRC | 02:25 | |
| *** ekcs has joined #openstack-meeting | 02:26 | |
| *** markvoelker has joined #openstack-meeting | 02:35 | |
| *** hongbin has quit IRC | 02:36 | |
| *** igordc has quit IRC | 02:39 | |
| *** markvoelker has quit IRC | 02:40 | |
| *** ricolin has joined #openstack-meeting | 02:47 | |
| *** donghao has joined #openstack-meeting | 02:50 | |
| *** donghao has quit IRC | 02:54 | |
| *** rfolco has quit IRC | 03:09 | |
| *** ekcs has quit IRC | 03:09 | |
| *** whoami-rajat has joined #openstack-meeting | 03:11 | |
| *** ekcs has joined #openstack-meeting | 03:21 | |
| *** psachin has joined #openstack-meeting | 03:31 | |
| *** Liang__ has quit IRC | 03:34 | |
| *** boxiang has quit IRC | 04:16 | |
| *** zhubx has joined #openstack-meeting | 04:17 | |
| *** ociuhandu has joined #openstack-meeting | 04:30 | |
| *** ociuhandu has quit IRC | 04:35 | |
| *** enriquetaso has joined #openstack-meeting | 04:38 | |
| *** dviroel has quit IRC | 04:40 | |
| *** artom has quit IRC | 04:41 | |
| *** donghao has joined #openstack-meeting | 04:44 | |
| *** donghao has quit IRC | 04:49 | |
| *** belmoreira has joined #openstack-meeting | 04:51 | |
| *** enriquetaso has quit IRC | 04:57 | |
| *** belmoreira has quit IRC | 05:05 | |
| *** ekcs has joined #openstack-meeting | 05:07 | |
| *** ekcs has quit IRC | 05:07 | |
| *** seongsoocho has joined #openstack-meeting | 05:34 | |
| *** hyunsikyang__ has joined #openstack-meeting | 05:37 | |
| *** hyunsikyang has quit IRC | 05:40 | |
| *** zbr|ruck has quit IRC | 05:43 | |
| *** zbr has joined #openstack-meeting | 05:43 | |
| *** brault has joined #openstack-meeting | 05:46 | |
| *** radeks has joined #openstack-meeting | 06:06 | |
| *** e0ne has joined #openstack-meeting | 06:16 | |
| *** kopecmartin|off is now known as kopecmartin | 06:20 | |
| *** enriquetaso has joined #openstack-meeting | 06:32 | |
| *** belmoreira has joined #openstack-meeting | 06:35 | |
| *** pcaruana has joined #openstack-meeting | 06:42 | |
| *** eharney has joined #openstack-meeting | 06:46 | |
| *** rpittau|afk is now known as rpittau | 06:47 | |
| *** markvoelker has joined #openstack-meeting | 06:47 | |
| *** donghao has joined #openstack-meeting | 06:47 | |
| *** donghao has quit IRC | 06:52 | |
| *** markvoelker has quit IRC | 06:52 | |
| *** slaweq has joined #openstack-meeting | 06:55 | |
| *** e0ne has quit IRC | 06:56 | |
| *** tesseract has joined #openstack-meeting | 07:01 | |
| *** ykatabam has quit IRC | 07:15 | |
| *** brault has quit IRC | 07:21 | |
| *** apetrich has joined #openstack-meeting | 07:27 | |
| *** enriquetaso has quit IRC | 07:34 | |
| *** enriquetaso has joined #openstack-meeting | 07:35 | |
| *** zhubx has quit IRC | 07:38 | |
| *** zhubx has joined #openstack-meeting | 07:38 | |
| *** e0ne has joined #openstack-meeting | 07:39 | |
| *** yaawang has quit IRC | 07:47 | |
| *** yaawang has joined #openstack-meeting | 07:49 | |
| *** ralonsoh has joined #openstack-meeting | 07:51 | |
| *** rpittau is now known as rpittau|bbl | 08:01 | |
| *** ykatabam has joined #openstack-meeting | 08:03 | |
| *** moguimar has joined #openstack-meeting | 08:16 | |
| *** e0ne has quit IRC | 08:28 | |
| *** priteau has joined #openstack-meeting | 08:34 | |
| *** brault has joined #openstack-meeting | 08:39 | |
| *** zbr is now known as zbr|ruck | 08:40 | |
| *** brault has quit IRC | 08:43 | |
| *** markvoelker has joined #openstack-meeting | 08:48 | |
| *** donghao has joined #openstack-meeting | 08:49 | |
| *** markvoelker has quit IRC | 08:53 | |
| *** donghao has quit IRC | 08:54 | |
| *** markvoelker has joined #openstack-meeting | 08:56 | |
| *** markvoelker has quit IRC | 09:00 | |
| *** ttsiouts has joined #openstack-meeting | 09:12 | |
| *** brault has joined #openstack-meeting | 09:24 | |
| *** e0ne has joined #openstack-meeting | 09:31 | |
| *** electrofelix has joined #openstack-meeting | 09:35 | |
| *** ociuhandu has joined #openstack-meeting | 09:35 | |
| *** ociuhandu has quit IRC | 09:36 | |
| *** ociuhandu has joined #openstack-meeting | 09:38 | |
| *** ociuhandu has quit IRC | 09:39 | |
| *** brault has quit IRC | 09:43 | |
| *** bbowen has quit IRC | 09:46 | |
| *** bbowen has joined #openstack-meeting | 09:47 | |
| *** brault has joined #openstack-meeting | 09:51 | |
| *** rpittau|bbl is now known as rpittau | 09:57 | |
| *** ociuhandu has joined #openstack-meeting | 09:59 | |
| *** enriquetaso has quit IRC | 10:02 | |
| *** ociuhandu has quit IRC | 10:04 | |
| *** brault has quit IRC | 10:17 | |
| *** pcaruana has quit IRC | 10:23 | |
| *** rcernin has quit IRC | 10:23 | |
| *** enriquetaso has joined #openstack-meeting | 10:30 | |
| *** pcaruana has joined #openstack-meeting | 10:36 | |
| *** zhangchi has quit IRC | 10:49 | |
| *** donghao has joined #openstack-meeting | 10:51 | |
| *** artom has joined #openstack-meeting | 10:53 | |
| *** oneswig has joined #openstack-meeting | 10:55 | |
| oneswig | Hi All, I'm proposing to cancel today's meeting. I'm currently in a design workshop and can't give it due attention. | 10:56 |
| *** markvoelker has joined #openstack-meeting | 10:57 | |
| *** donghao has quit IRC | 10:57 | |
| *** ociuhandu has joined #openstack-meeting | 10:58 | |
| *** janders has joined #openstack-meeting | 11:00 | |
| oneswig | Apologies, I'm in a workshop currently, can't chair the meeting | 11:01 |
| *** ttsiouts has quit IRC | 11:01 | |
| *** markvoelker has quit IRC | 11:01 | |
| janders | no worries | 11:02 |
| *** b1airo has joined #openstack-meeting | 11:02 | |
| janders | hey Blair! :) | 11:02 |
| b1airo | Hey hey | 11:02 |
| b1airo | How goes it? | 11:02 |
| janders | fast! | 11:02 |
| oneswig | Hi b1airo janders | 11:02 |
| janders | we hit 100GB/s on the GPFS cluster I'm working on today | 11:03 |
| janders | 6 servers, 6 clients | 11:03 |
| oneswig | janders: sheesh | 11:03 |
| b1airo | Sounds like there's a story behind that claim janders ... | 11:03 |
| b1airo | Noice! | 11:03 |
| oneswig | Going for IO500? | 11:03 |
| oneswig | again? | 11:03 |
| janders | haha :) maybe | 11:03 |
| oneswig | I have to head off I'm afraid. | 11:03 |
| b1airo | :-( | 11:03 |
| oneswig | apologies, can't join you today. | 11:04 |
| oneswig | gotta go :-( | 11:04 |
| *** oneswig has quit IRC | 11:04 | |
| janders | we'll chat later | 11:04 |
| b1airo | Anyone else along this evening? | 11:04 |
| janders | Blair, do you have any experience with running IOPSy workloads on GPFS? | 11:04 |
| b1airo | I wouldn't say "much" | 11:06 |
| b1airo | Whatever it is probably works better inside a container though ;-) | 11:06 |
| janders | :D | 11:06 |
| janders | I haven't reached that part yet, but it's in the pipeline | 11:06 |
| b1airo | What are you seeing with IOPS? | 11:07 |
| janders | I'm seeing interesting IOPS performance profile - much of which is to do with the underlying hardware, but GPFS itself plays a role in this | 11:07 |
| janders | the NVMes this thing is build on can do 700k IOPS each | 11:07 |
| janders | but a single client can only see 70k against the entire fs | 11:07 |
| janders | but... | 11:07 |
| janders | it's 70k per client, no matter how many clients | 11:08 |
| janders | so on 6 we can go over 300k IOPS | 11:08 |
| janders | GPFS block size might have something to do with this (we figured 1M should be reasonable for a general purpose fs) | 11:09 |
| *** brault has joined #openstack-meeting | 11:09 | |
| janders | but yeah I find it interesting - it seems that the single client is IOPS constrained, the filesystem - much less so | 11:09 |
| b1airo | Doesn't scale similarly with higher I/O queue depths? | 11:09 |
| janders | I only tested on 32 queue depth and 16 threads | 11:10 |
| janders | (getting the fio file) | 11:10 |
| *** njohnston has joined #openstack-meeting | 11:11 | |
| janders | [global]bs=4kioengine=libaioiodepth=32size=100gdirect=1runtime=60filename=/gpfs/iops23[rand-read]rw=randreadstonewall[rand-write]rw=randwritestonewall | 11:11 |
| janders | oops | 11:11 |
| b1airo | I wouldn't be surprised if GPFS's client side metadata caching ended up adding some bottlenecks that need to be tuned away to realise full potential of such a backend | 11:11 |
| janders | very good insight - will discuss with the team tomorrow | 11:12 |
| janders | possible! | 11:12 |
| janders | I've seen it causing side effects in the past (not specifically this though) | 11:12 |
| b1airo | I can read that fio config fine btw | 11:12 |
| janders | another interesting lesson learned is... HDR200 heat issues | 11:12 |
| janders | we had some thermal shutdowns of the HCAs with some neighbouring 10GE adapters falling victim of the hotspots | 11:13 |
| *** brault has quit IRC | 11:13 | |
| janders | CX6 warning temperature is 105C and shutdown temperature is 125C | 11:13 |
| b1airo | Yikes | 11:13 |
| b1airo | What wattage are they pulling? | 11:14 |
| janders | We ended up taking plastic bits out of the server (these maximise memory cooling at the expense of NICs/HCAs) | 11:14 |
| janders | and fixing the fan speed to 80% | 11:14 |
| janders | running at around 70C now | 11:14 |
| janders | down from 90-120 | 11:14 |
| janders | good question.. I know how to check temperatures but not wattage - do you have a command handy? | 11:14 |
| b1airo | Ha, resourceful! | 11:14 |
| *** thgcorrea has joined #openstack-meeting | 11:16 | |
| b1airo | Not sure if you can ask the HCA itself, would expect to dig in the system management interfaces | 11:16 |
| janders | [ansible@s23 ~]$ ansible spectrum -a 'mget_temp -d mlx5_1's204-ib3 | SUCCESS | rc=0 >>68 s202-ib3 | SUCCESS | rc=0 >>70 s205-ib3 | SUCCESS | rc=0 >>66 s201-ib3 | SUCCESS | rc=0 >>70 s203-ib3 | SUCCESS | rc=0 >>64 s206-ib3 | SUCCESS | rc=0 >>70 [ansible@s23 ~]$ | 11:16 |
| b1airo | Nice | 11:17 |
| b1airo | Hmm, just remembered something I was going to investigate earliest today but didn't, so you're getting the question now :-) ... | 11:17 |
| janders | sure will try to help | 11:18 |
| b1airo | Any ideas if the memory cgroup can be configured to allow prioritised ... Hmmmm actually, think I just realised I should be looking at tuning OOM behaviour | 11:19 |
| b1airo | Sometimes "verbalisation" of the problem solves it :-D | 11:19 |
| janders | :D | 11:19 |
| janders | is OOM taking out things running in cgroups? | 11:20 |
| b1airo | I'll give you some context... | 11:20 |
| b1airo | We've got some hugemem nodes that are particularly popular among genomics users | 11:20 |
| b1airo | Thing is, they are generally pretty bad at estimating and optimising their memory requirements, and anyway, they're often dealing with algorithms and datasets that make it hard to predict correctly | 11:22 |
| *** ttsiouts has joined #openstack-meeting | 11:22 | |
| b1airo | But our resources of this kind are scarce | 11:23 |
| *** raildo has joined #openstack-meeting | 11:24 | |
| b1airo | I was thinking about whether there'd be a cgroup based way to achieve a sort of memory overcommit (as these users usually over-estimate their memory limits quite a lot to avoid surprises 2 days into an assembly) | 11:24 |
| b1airo | So we could let other workloads run and just ensure OOM priorities are set appropriately within that memory cgroup | 11:26 |
| janders | crazy idea - wondering if some clever form of swap could help? | 11:26 |
| janders | maybe combined with custom nice value.. | 11:27 |
| b1airo | Potentially, though hard to avoid swap activity hurting everyone on the node | 11:28 |
| janders | true but the nice value could differentiate that, perhaps | 11:28 |
| b1airo | niceness only impacts CPU time scheduling doesn't it? Wonder what the nice value for "yield on major fault that would cause demand pageout" ... :-) | 11:32 |
| janders | youre right (checked the doco) | 11:33 |
| b1airo | Anyway, I'll let you know if I (or more likely someone smarter that has the misfortune to work for me) come up with something | 11:33 |
| b1airo | You heading to eResearch Australasia this year? | 11:33 |
| janders | https://www.kernel.org/doc/Documentation/cgroup-v2.txt | 11:34 |
| janders | memory.* bits might be relevant | 11:34 |
| b1airo | Can get you a talk invite for a storage BoF if you're interested.. | 11:35 |
| b1airo | ARDC Infrastructure Summit is on too | 11:35 |
| janders | it would be nice, but with the current workload I don't think I will get approval to go anywhere before mid November unfortunately | 11:36 |
| janders | I need the GPFS stuff to be wrapped up by mid Oct and I need to have OpenStack running against it by end October | 11:37 |
| b1airo | "running against it"..? | 11:37 |
| b1airo | We talking Manila shares or something else entirely? | 11:38 |
| janders | this GPFS will be OpenStack's nova/glance/cinder backend - and parallel fs for the baremetal nodes, possibly | 11:38 |
| janders | but not sure if the fs part will be integrated with OpenStack yet - might be similar to Stig's BeeGFS approach | 11:39 |
| janders | with the way things are going, we'll likely be helping users set their stuff up with Ansible, so if we take root off them in the process that'll probably be okay | 11:40 |
| janders | gotta check where things are with Kerberos and GPFS | 11:40 |
| b1airo | Pretty sure that's a working thing | 11:41 |
| janders | will ask my GPFS guys about this | 11:42 |
| janders | would be supernice | 11:42 |
| *** markvoelker has joined #openstack-meeting | 11:42 | |
| b1airo | I guess alternatively you could have on-demand GPFS-NFS protocol node exports into private network overlays | 11:43 |
| janders | true | 11:43 |
| *** brault has joined #openstack-meeting | 11:43 | |
| janders | on a slightly different note, I noticed some interesting performance implications of using LSI expander cards | 11:43 |
| janders | to plug in all the NVMes, I need some of those in passthru | 11:43 |
| janders | I've got enough PCIe lanes for everything to be non blocking | 11:44 |
| janders | and with read, this is indeed the case | 11:44 |
| janders | with write, less so, there's a bit of a hit on bandwidth | 11:44 |
| b1airo | NVMe expanders this is? | 11:44 |
| janders | with IOPS, it's quite signigicant, in order of 70% | 11:44 |
| janders | 5e:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID Tri-Mode SAS3508 (rev 01) | 11:45 |
| janders | 86:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID Tri-Mode SAS3408 (rev 01) | 11:45 |
| b1airo | That's curious | 11:45 |
| janders | with IOPS, I can sort of understand cause at least it's symmetrical. Capped to 200k read/write, end of story | 11:46 |
| b1airo | They're not doing any silly caching are they? | 11:46 |
| janders | but with bandwidth - how come read is non blocking but write is? | 11:46 |
| janders | I was assured they are just dumb devices providing extra ports | 11:46 |
| janders | but will chat to my hardware guys about this | 11:46 |
| janders | 1) why read and write differ via the expander, when they do not differ without | 11:47 |
| janders | 2) what's on with the IOPS | 11:47 |
| janders | I suppose forcing NVMe to speak SAS might force them to communicate in sub-optimal ways | 11:47 |
| janders | but I do find this very curious | 11:48 |
| b1airo | I'll be interested to hear | 11:48 |
| janders | it doesn't hurt performance much as we're capped with HDR200 bandwidth anyway | 11:48 |
| b1airo | I'm fading, almost midnight here. Might have to bail | 11:48 |
| janders | (and similarly, there's a cap on IOPS somewhere and it's not the expander) | 11:48 |
| janders | I just realised I should benchmark IOPS on two NVMes hanging off the same expander... | 11:49 |
| janders | no worries - thank you for the chat! | 11:49 |
| janders | till next time | 11:49 |
| b1airo | Cheers and likewise! | 11:49 |
| *** brault has quit IRC | 11:50 | |
| *** belmoreira has quit IRC | 11:54 | |
| *** dviroel has joined #openstack-meeting | 11:56 | |
| *** belmoreira has joined #openstack-meeting | 11:58 | |
| *** yaawang has quit IRC | 12:03 | |
| *** janders has quit IRC | 12:06 | |
| *** rfolco has joined #openstack-meeting | 12:12 | |
| *** hyunsikyang has joined #openstack-meeting | 12:29 | |
| *** hyunsikyang__ has quit IRC | 12:32 | |
| *** mriedem has joined #openstack-meeting | 12:47 | |
| *** donghao has joined #openstack-meeting | 12:50 | |
| *** donghao has quit IRC | 12:54 | |
| *** lseki has joined #openstack-meeting | 12:55 | |
| *** psachin has quit IRC | 13:04 | |
| *** ociuhandu has quit IRC | 13:04 | |
| *** ociuhandu has joined #openstack-meeting | 13:07 | |
| *** ociuhandu has quit IRC | 13:11 | |
| *** belmoreira has quit IRC | 13:12 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:19 | |
| *** belmoreira has joined #openstack-meeting | 13:23 | |
| *** brault has joined #openstack-meeting | 13:24 | |
| *** rossella_s has joined #openstack-meeting | 13:24 | |
| *** ykatabam has quit IRC | 13:24 | |
| *** yaawang has joined #openstack-meeting | 13:26 | |
| *** jamesmcarthur has quit IRC | 13:27 | |
| *** brault has quit IRC | 13:28 | |
| *** ociuhandu has joined #openstack-meeting | 13:31 | |
| *** liuyulong has joined #openstack-meeting | 13:31 | |
| *** ociuhandu has quit IRC | 13:34 | |
| *** ociuhandu has joined #openstack-meeting | 13:35 | |
| *** yaawang has quit IRC | 13:37 | |
| *** ociuhandu has quit IRC | 13:37 | |
| *** ociuhandu has joined #openstack-meeting | 13:37 | |
| *** ociuhandu has quit IRC | 13:39 | |
| *** whoami-rajat has quit IRC | 13:39 | |
| *** ociuhandu has joined #openstack-meeting | 13:40 | |
| *** zhangchi has joined #openstack-meeting | 13:46 | |
| *** ociuhandu has quit IRC | 13:49 | |
| *** zhangchi has quit IRC | 13:52 | |
| *** slaweq has quit IRC | 13:54 | |
| *** slaweq has joined #openstack-meeting | 13:57 | |
| *** panda is now known as panda|bbl | 13:59 | |
| *** yaawang has joined #openstack-meeting | 13:59 | |
| liuyulong | tes | 14:00 |
| liuyulong | t | 14:00 |
| slaweq | hi | 14:00 |
| liuyulong | hi | 14:00 |
| liuyulong | #startmeeting neutron_l3 | 14:00 |
| openstack | Meeting started Wed Sep 25 14:00:19 2019 UTC and is due to finish in 60 minutes. The chair is liuyulong. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: neutron_l3)" | 14:00 | |
| openstack | The meeting name has been set to 'neutron_l3' | 14:00 |
| ralonsoh | hi | 14:00 |
| haleyb | hi | 14:01 |
| liuyulong | #chair haleyb | 14:01 |
| openstack | Current chairs: haleyb liuyulong | 14:01 |
| *** b1airo has quit IRC | 14:01 | |
| *** liuyulong__ has joined #openstack-meeting | 14:02 | |
| liuyulong | OooooK, let's ssstart. | 14:02 |
| liuyulong | #topic Announcements | 14:02 |
| *** openstack changes topic to "Announcements (Meeting topic: neutron_l3)" | 14:02 | |
| liuyulong | #link https://releases.openstack.org/train/index.html | 14:03 |
| liuyulong__ | #link https://releases.openstack.org/train/index.html | 14:04 |
| liuyulong | We have a Projects Release Date: 16 October, 2019 | 14:04 |
| liuyulong | it will come soon | 14:05 |
| liuyulong | And I received many RC release mail. | 14:05 |
| slaweq | yes, we already released rc1 for many stadium projects | 14:06 |
| *** brault has joined #openstack-meeting | 14:06 | |
| liuyulong | #link https://launchpad.net/neutron/+milestone/train-rc1 | 14:06 |
| liuyulong | This is our train-rc1 related topics. IMO, they are in good shape now, we could catch up the scheduler. | 14:08 |
| liuyulong | Next | 14:10 |
| liuyulong | #link https://etherpad.openstack.org/p/Shanghai-Neutron-Planning | 14:10 |
| liuyulong | Call for topics, summit is also coming soon, we will have 5 days meeting? | 14:11 |
| slaweq | ptg starts on Wednesday | 14:11 |
| slaweq | Monday - Wednesday is summit and Forum | 14:12 |
| slaweq | IIRC | 14:12 |
| liuyulong | Good, so 3 days, another thing is that we will also hold the meeting online in parallel? | 14:13 |
| slaweq | liuyulong: I sent You an email recently about that | 14:14 |
| slaweq | I will probably need Your help to organize some streaming | 14:14 |
| slaweq | but we can sync later with that | 14:14 |
| liuyulong | And mlavalle send us the mail to query if there will have a nice tool for those who can not go to Shanghai. | 14:15 |
| *** slaweq has quit IRC | 14:15 | |
| *** slaweq has joined #openstack-meeting | 14:16 | |
| slaweq | liuyulong: yes, but I will now take care of it instead of mlavalle | 14:16 |
| *** ociuhandu has joined #openstack-meeting | 14:16 | |
| liuyulong | Wechat is widely used in China. So I may want to give a try for our L3 team members to test some tools before the PTG. | 14:18 |
| slaweq | liuyulong: will You be available tomorrow morning my time? So afternoon Your time | 14:18 |
| slaweq | we can sync about it then | 14:18 |
| slaweq | or on Friday if You want | 14:18 |
| liuyulong | And a Proxy Server can also be another choice, so I wonder if there are any free server for us to user from the community? | 14:19 |
| liuyulong__ | Backup is coming... | 14:19 |
| liuyulong__ | Seems that connection is broken... | 14:20 |
| *** ociuhandu has quit IRC | 14:20 | |
| liuyulong__ | slaweq, sure, I will ping you tomorrow. Friday can also be fine. | 14:22 |
| slaweq | liuyulong__: great | 14:22 |
| slaweq | thx | 14:22 |
| liuyulong__ | All right, I have no more announcement. If you have please go ahead. | 14:23 |
| liuyulong__ | My irc connection is unstable. Let's speedup. | 14:25 |
| liuyulong__ | #topic Bugs | 14:25 |
| *** ociuhandu has joined #openstack-meeting | 14:25 | |
| *** liuyulong has quit IRC | 14:26 | |
| *** liuyulong has joined #openstack-meeting | 14:26 | |
| liuyulong | #topic Bugs | 14:27 |
| *** openstack changes topic to "Bugs (Meeting topic: neutron_l3)" | 14:27 | |
| liuyulong | #chair liuyulong__ | 14:27 |
| openstack | Current chairs: haleyb liuyulong liuyulong__ | 14:27 |
| liuyulong__ | #link https://bugs.launchpad.net/neutron/+bug/1845150 | 14:27 |
| openstack | Launchpad bug 1845150 in neutron "[FT] "keepalived" needs network interfaces configured as in its own config" [High,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 14:27 |
| ralonsoh | the patch is here | 14:28 |
| ralonsoh | #link https://review.opendev.org/#/c/684249/ | 14:28 |
| ralonsoh | as commented in the patch, if keepalived starts and doesn't see the tracked interfaces with the needed ips | 14:28 |
| liuyulong | ralonsoh, thanks, it looks good to me. | 14:28 |
| ralonsoh | the process stops | 14:29 |
| ralonsoh | ok, thanks! thats all | 14:29 |
| liuyulong | haleyb have +wed it :) | 14:29 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1844688 | 14:30 |
| openstack | Launchpad bug 1844688 in neutron ""radvd" daemon does not work by default in some containers" [Medium,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 14:30 |
| ralonsoh | yes, thanks for the comment | 14:30 |
| liuyulong | The fix: https://review.opendev.org/683207 | 14:30 |
| ralonsoh | I need to add a reno | 14:30 |
| ralonsoh | and test cases | 14:30 |
| *** ociuhandu has quit IRC | 14:31 | |
| liuyulong | A new config is added for this bug, so IMO, a release note is needed. Users should know such config option in many ways. | 14:31 |
| *** ociuhandu has joined #openstack-meeting | 14:31 | |
| ralonsoh | for sure, today I'll push a new patch | 14:31 |
| liuyulong | ralonsoh, np | 14:32 |
| liuyulong | Next | 14:33 |
| liuyulong | #link https://review.opendev.org/683207 | 14:33 |
| liuyulong | #undo | 14:33 |
| openstack | Removing item from minutes: #link https://review.opendev.org/683207 | 14:33 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1845227 | 14:33 |
| openstack | Launchpad bug 1845227 in neutron "new l3 agent router factory breaks neutron-fwaas functional tests" [Critical,Fix committed] | 14:33 |
| ralonsoh | fixed here #link https://review.opendev.org/#/c/652812/ | 14:33 |
| *** rpittau is now known as rpittau|afk | 14:34 | |
| liuyulong | A Critical bug, and it is fixed. | 14:34 |
| *** diablo_rojo has joined #openstack-meeting | 14:34 | |
| liuyulong | Last one from me | 14:35 |
| liuyulong | #link https://bugs.launchpad.net/neutron/+bug/1845145 | 14:35 |
| openstack | Launchpad bug 1845145 in neutron "[L3] add abilitiy for iptables_manager to ensure rule was added only once" [High,In progress] - Assigned to LIU Yulong (dragon889) | 14:35 |
| liuyulong | Basically I did not see that WARNING log from our local queens deployments. But when create new agent extension with parallel processing, I noticed many warning log. | 14:38 |
| liuyulong | In order to ensure the reentrancy of rule addition, I raised such bug. | 14:39 |
| slaweq | liuyulong: but doesn't it means that Your agent extension is doing something wrong? | 14:39 |
| liuyulong | slaweq, we can add more code to prevent that, but it will need more cache and event more lock to achieve that. A simple reentrancy or idempotent can be a nice approach. | 14:42 |
| liuyulong | IMO, iptables_manager should have that ability. | 14:42 |
| slaweq | I wonder why it wasn't dont like that in the past | 14:43 |
| liuyulong | I did not find any reason that we want the duplicated rules in that managers rule cache. | 14:44 |
| slaweq | maybe haleyb or mlavalle will remember something | 14:44 |
| liuyulong | #link https://review.opendev.org/#/c/684237/ | 14:44 |
| * haleyb sticks his head up | 14:44 | |
| liuyulong | The fix is quite simple. | 14:45 |
| haleyb | slaweq: we've never needed that before, i don't know why it's happening now | 14:45 |
| haleyb | that's why i was wondering if something outside neutron was adding a rule, or if we were calling something twice | 14:47 |
| slaweq | haleyb: IIUC Neutron U/S don't need it still but liuyulong is doing some d/s agent extension which needs it to make his life easier | 14:48 |
| liuyulong | haleyb, yes, that upstream log search can approve that. | 14:48 |
| liuyulong | LOG a warning does not make sense to me, we are neutron developers, so we can make it better. | 14:48 |
| haleyb | right, i'm just wondering how that ironic devstack setup is causing issues, since that seems to be where that port rule is added, not via the neutron api | 14:49 |
| liuyulong | This fix should still use list() instead of set(), because the rule needs the proper order. | 14:50 |
| haleyb | basically i'm saying it would be good to better understand how it's happening before adding a duplicate check | 14:50 |
| liuyulong | And that's why the patch set 2 has functional test failures. | 14:50 |
| liuyulong | I will run test locally and update it soon. | 14:51 |
| *** enriquetaso has quit IRC | 14:51 | |
| liuyulong | Sorry, I have to run now. Let's stop here. | 14:51 |
| liuyulong | Thank you for your participation. | 14:52 |
| ralonsoh | bye | 14:52 |
| liuyulong | and sorry for my bad network connection... | 14:52 |
| liuyulong | Bye | 14:52 |
| slaweq | thanks | 14:52 |
| slaweq | o/ | 14:52 |
| liuyulong | #endmeeting | 14:52 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 14:52 | |
| openstack | Meeting ended Wed Sep 25 14:52:41 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:52 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-09-25-14.00.html | 14:52 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-09-25-14.00.txt | 14:52 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-09-25-14.00.log.html | 14:52 |
| *** liuyulong__ has quit IRC | 14:53 | |
| *** liuyulong has quit IRC | 14:53 | |
| *** brault has quit IRC | 14:57 | |
| *** e0ne has quit IRC | 14:59 | |
| *** brault has joined #openstack-meeting | 14:59 | |
| *** e0ne has joined #openstack-meeting | 14:59 | |
| *** brault has quit IRC | 15:00 | |
| *** belmoreira has quit IRC | 15:05 | |
| *** brault has joined #openstack-meeting | 15:10 | |
| *** zhubx has quit IRC | 15:13 | |
| *** ttsiouts has quit IRC | 15:13 | |
| *** boxiang has joined #openstack-meeting | 15:14 | |
| *** eharney has quit IRC | 15:14 | |
| *** artom has quit IRC | 15:15 | |
| *** artom has joined #openstack-meeting | 15:15 | |
| *** ttsiouts has joined #openstack-meeting | 15:15 | |
| *** ttsiouts has quit IRC | 15:15 | |
| *** ttsiouts has joined #openstack-meeting | 15:20 | |
| *** sridharg has joined #openstack-meeting | 15:23 | |
| *** ttsiouts has quit IRC | 15:24 | |
| *** ekcs has joined #openstack-meeting | 15:28 | |
| *** ekcs has quit IRC | 15:29 | |
| *** pcaruana has quit IRC | 15:30 | |
| *** gyee has joined #openstack-meeting | 15:35 | |
| *** whoami-rajat has joined #openstack-meeting | 15:37 | |
| *** macz has joined #openstack-meeting | 15:40 | |
| *** markvoelker has quit IRC | 15:45 | |
| *** macz has quit IRC | 15:47 | |
| *** rosmaita has joined #openstack-meeting | 15:51 | |
| *** davidsha has joined #openstack-meeting | 15:52 | |
| *** _pewp_ has joined #openstack-meeting | 15:53 | |
| *** woojay has joined #openstack-meeting | 15:53 | |
| *** lbragstad_ has joined #openstack-meeting | 15:54 | |
| *** lbragstad has quit IRC | 15:55 | |
| *** lbragstad has joined #openstack-meeting | 15:57 | |
| *** lbragstad_ has quit IRC | 15:58 | |
| jungleboyj | #startmeeting Cinder | 16:00 |
| openstack | Meeting started Wed Sep 25 16:00:23 2019 UTC and is due to finish in 60 minutes. The chair is jungleboyj. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: Cinder)" | 16:00 | |
| openstack | The meeting name has been set to 'cinder' | 16:00 |
| davidsha | o/ | 16:00 |
| lseki | o/ | 16:00 |
| rosmaita | o/ | 16:00 |
| jungleboyj | courtesy ping: jungleboyj whoami-rajat rajinir lseki carloss pots woojay erlon geguileo eharney rosmaita enriquetaso e0ne smcginnis davidsha walshh_ xyang hemna _hemna tosky sfernand | 16:00 |
| jungleboyj | @! | 16:01 |
| _pewp_ | jungleboyj ヾ(-_-;) | 16:01 |
| *** carloss has joined #openstack-meeting | 16:01 | |
| whoami-rajat | Hi | 16:01 |
| walshh_ | Hi | 16:01 |
| jungleboyj | Oh pewp bot. I guess you know how my week is going. | 16:01 |
| woojay | o/ | 16:01 |
| e0ne | hi | 16:01 |
| rosmaita | e0ne: you have been a reviewing maniac, thank you | 16:02 |
| jungleboyj | e0ne: ++ | 16:02 |
| jungleboyj | Though the mental image of e0ne being a maniac is a little scary. | 16:03 |
| jungleboyj | Ok. It looks like we have the usual crowd here. | 16:04 |
| jungleboyj | #topic announcements | 16:04 |
| *** openstack changes topic to "announcements (Meeting topic: Cinder)" | 16:04 | |
| jungleboyj | Not a lot to say other than the fact that we are at RC week for Train. | 16:04 |
| jungleboyj | #link https://etherpad.openstack.org/p/cinder-train-RC-time | 16:04 |
| rosmaita | there's nothing unreviewed on there, just things that need another review | 16:05 |
| jungleboyj | Oooh, look at the PTL to be being organized! | 16:05 |
| * jungleboyj is happy | 16:05 | |
| rosmaita | :D | 16:05 |
| rosmaita | hopefully the stuff in the gate will merge OK | 16:05 |
| rosmaita | there's a small problem with the default types patch, i think | 16:05 |
| jungleboyj | I will take a look through all of these after the meeting. | 16:05 |
| rosmaita | cool | 16:06 |
| rosmaita | the release note needs a revision, but that patch is so big, i don't want to request a change | 16:06 |
| jungleboyj | rosmaita: I see you comments. | 16:06 |
| rosmaita | so i think it would be good to merge it and then do a followup | 16:06 |
| jungleboyj | @whoami-rajat Any concerns with the plan there? | 16:06 |
| *** rfolco is now known as rfolco|bbl | 16:06 | |
| rosmaita | but it will have to be quick because once the stable branch is created, the release notes behave weirdly | 16:06 |
| rosmaita | my proposal would be to propose the RC-1 release tomorrow afternoon? | 16:07 |
| davee__ | o/ | 16:07 |
| whoami-rajat | rosmaita: jungleboyj that sounds good, thanks, i've been trying to get zuul+1 since past few days and if the update is releasenotes related, i would be glad to do it in a followup | 16:07 |
| rosmaita | which is a day later than usual for cinder, but i think it is still within legal time | 16:07 |
| e0ne | rosmaita, jungleboyj: thanks. I'm doing my best to make cinder better | 16:07 |
| jungleboyj | whoami-rajat: Ok. Cool. Lets do that instead of complicating things. | 16:08 |
| rosmaita | e0ne: ++ | 16:08 |
| rosmaita | whoami-rajat: that would be great | 16:09 |
| rosmaita | i will put more notes on your current patch about what i think is needed | 16:09 |
| jungleboyj | Are there any bug fixes that are not on Brian's list? | 16:09 |
| whoami-rajat | jungleboyj: rosmaita thanks, also i would like to ask when is cinderlib release planned? | 16:09 |
| e0ne | jungleboyj, rosmaita, smcginnis: fyi, https://review.opendev.org/#/c/684050/ | 16:10 |
| rosmaita | i didn't look at the unmerged changes, i just pulled the FFEs and bugs i knew about | 16:10 |
| *** yonglihe has quit IRC | 16:10 | |
| rosmaita | whoami-rajat: cinderlib is a cycle-trailing release, so probably within a month after actual Train release | 16:10 |
| rosmaita | it can wait longer, but i think gorka will want to release as soon as it makes sense | 16:11 |
| jungleboyj | e0ne: Thank you. I just put a hold on that. | 16:11 |
| rosmaita | yes, thanks! | 16:11 |
| whoami-rajat | rosmaita: great, i've a dependent patch of the same feature in cinderlib so i think it will be in on time. | 16:11 |
| walshh_ | I have a few driver rev | 16:11 |
| walshh_ | I have a few driver reviews not on the list above | 16:12 |
| jungleboyj | walshh_: I looked at the one this morning you sent. Can you add yours to the list in the etherpad? | 16:12 |
| jungleboyj | That way everyone can take a look. | 16:12 |
| jungleboyj | Not just me. :-) | 16:12 |
| walshh_ | thank you. Just a few more, all have clean CI now, minor fixes | 16:12 |
| jungleboyj | Okie Dokie. | 16:12 |
| jungleboyj | Bug fixes are good. | 16:12 |
| jungleboyj | Must mean people are actually using your driver? :-) | 16:13 |
| e0ne | :) | 16:14 |
| walshh_ | one or two :-) | 16:14 |
| jungleboyj | That is a good thing. | 16:14 |
| jungleboyj | Ok, if you can add your reviews in the etherpad that would be great and we will review. | 16:14 |
| walshh_ | I will, thank you | 16:15 |
| jungleboyj | Otherwise testing of the RC would be good if anyone is able to do that. | 16:15 |
| rosmaita | ok, i am in prague time zone this week, so let's say to be in RC-1, stuff must be merged by 22:00 UTC tomorrow (26 Sept) | 16:15 |
| rosmaita | i will also commit to looking at walshh_'s patches | 16:15 |
| * jungleboyj would like to go to Prague some time. | 16:15 | |
| hemna | any pressing reviews? | 16:16 |
| jungleboyj | hemna: https://etherpad.openstack.org/p/cinder-train-RC-time | 16:16 |
| e0ne | jungleboyj: +1 | 16:16 |
| jungleboyj | Where you been man? ;-) | 16:16 |
| hemna | this job thing.... | 16:16 |
| jungleboyj | hemna: ^^ | 16:16 |
| jungleboyj | hemna: Yeah, I feel your pain. | 16:16 |
| hemna | kinda gets in the way of cinder sometimes | 16:16 |
| jungleboyj | rosmaita: Anything else on RC discussion? | 16:17 |
| rosmaita | i don't mean to be a jerk, but i think we need someone to commit to reviewing the default types patch | 16:17 |
| rosmaita | (it's a bit of a monster) | 16:17 |
| rosmaita | code looks pretty clean, just a lot of files are touched | 16:17 |
| hemna | heh | 16:17 |
| hemna | yah that's a big one the last time I looked | 16:17 |
| jungleboyj | rosmaita: I have been through it before. | 16:17 |
| jungleboyj | So, not sure that I am going to have additional input. | 16:18 |
| rosmaita | i don't think it has changed too much since your last comments | 16:18 |
| hemna | so this is a new feature | 16:18 |
| hemna | are we ok with merging this at this stage of the game? | 16:18 |
| rosmaita | yes, it got a FFE last week | 16:18 |
| hemna | what could go wrong? | 16:18 |
| hemna | :P | 16:19 |
| davee__ | It could provide an excuse to drink lots of caffeine | 16:19 |
| rosmaita | well, if people have set CONF.default_type already, i don' think there will be any problems | 16:19 |
| whoami-rajat | hemna: it's adding a volume type with db migration, migrating all existing untyped volumes/snapshots to the default type, that's it :) | 16:19 |
| jungleboyj | :-) What could possibly go wrong. :-) | 16:20 |
| hemna | well if we are going to merge it, it's best to merge early | 16:20 |
| hemna | and test it | 16:20 |
| hemna | we can always back it out... | 16:20 |
| whoami-rajat | rosmaita: that's the aim :) | 16:20 |
| hemna | theoretically | 16:20 |
| jungleboyj | hemna: Agreed. | 16:20 |
| *** brault has quit IRC | 16:20 | |
| rosmaita | or, more positively, we can backport a minor bugfix :) | 16:21 |
| jungleboyj | rosmaita: Agreed. | 16:21 |
| hemna | ok I'll take a look today | 16:21 |
| jungleboyj | hemna: Thank you. | 16:22 |
| rosmaita | hemna: ty! | 16:22 |
| jungleboyj | Is there anyone planning to test it? | 16:22 |
| whoami-rajat | hemna: thank you! | 16:22 |
| rosmaita | i won't have time until after RC to really test it out | 16:22 |
| *** ociuhandu has quit IRC | 16:22 | |
| hemna | so we need to create a bunch of vols/snaps w/o a type | 16:22 |
| *** priteau has quit IRC | 16:23 | |
| hemna | I presume there is a migration script that has to be run to migrate those ? | 16:23 |
| *** ociuhandu has joined #openstack-meeting | 16:23 | |
| whoami-rajat | hemna: the type migration of existing vols/snaps is done with online migration | 16:24 |
| whoami-rajat | https://review.opendev.org/#/c/639180/41/cinder/db/sqlalchemy/api.py | 16:24 |
| *** mattw4 has joined #openstack-meeting | 16:25 | |
| jungleboyj | whoami-rajat: Is there a way to do it without having to install Stein and try to upgrade? | 16:25 |
| jungleboyj | To test it? | 16:26 |
| whoami-rajat | jungleboyj: we can run the db migrations (cinder-manage db sync) and online migrations (cinder-manage db online_data_migrations) | 16:27 |
| jungleboyj | Ok. Cool and then it should deal with the untyped volumes? | 16:27 |
| *** electrofelix has quit IRC | 16:27 | |
| *** ociuhandu has quit IRC | 16:28 | |
| whoami-rajat | jungleboyj: untyped volumes and snapshots and if any new volume is created without any provided type will be assigned `__DEFAULT__` | 16:28 |
| jungleboyj | whoami-rajat: Cool. | 16:28 |
| jungleboyj | Will try to test but not promising I will get to it right away. | 16:29 |
| rosmaita | whoami-rajat: you aren't going on vacation during the next 2 weeks, are you? | 16:30 |
| jungleboyj | rosmaita: :-) Good question. | 16:30 |
| hemna | ok so most of the filed changed in that patchset are unit tests | 16:30 |
| hemna | te rest looks ok I guess | 16:30 |
| whoami-rajat | jungleboyj: thanks for the help :) | 16:30 |
| jungleboyj | Yep. | 16:30 |
| whoami-rajat | rosmaita: no | 16:31 |
| rosmaita | ok, great | 16:31 |
| jungleboyj | Ok. That is good. | 16:31 |
| rosmaita | hemna: before you arrived, we agreed to do the release note changes in a follow up patch | 16:31 |
| rosmaita | the gate has been really slow, don't want to hold up the main patch | 16:31 |
| hemna | ugh ok | 16:31 |
| hemna | ok, it's on it's way | 16:32 |
| whoami-rajat | jungleboyj: rosmaita hemna thanks for the reviews! | 16:33 |
| jungleboyj | hemna: Thanks. | 16:33 |
| jungleboyj | whoami-rajat: Let me know when you have the updates for the release note done. | 16:33 |
| rosmaita | whoami-rajat: what jungleboyj said | 16:34 |
| whoami-rajat | jungleboyj: rosmaita sure, will do it quickly after the meeting | 16:34 |
| rosmaita | thanks, please add it to the etherpad when you post it | 16:35 |
| jungleboyj | Sounds good! | 16:35 |
| whoami-rajat | rosmaita: ok | 16:35 |
| whoami-rajat | will do | 16:35 |
| jungleboyj | Added notes to the patch on what we have discussed today for documentation purposes. | 16:35 |
| jungleboyj | So, anything else we need to discuss? | 16:36 |
| rosmaita | nothing from me | 16:36 |
| jungleboyj | A friendly reminder to add topics to the etherpad for the Stein PTG: https://etherpad.openstack.org/p/cinder-shanghai-ptg-planning | 16:36 |
| whoami-rajat | jungleboyj: i think there is one thing to discuss | 16:36 |
| jungleboyj | whoami-rajat: Ok. Go ahead. | 16:37 |
| whoami-rajat | it's regarding the tempest schema validation | 16:37 |
| jungleboyj | Oh yeah. | 16:37 |
| whoami-rajat | it seems to be breaking cinder in stable branches | 16:37 |
| rosmaita | well, that is because they aren't testing first | 16:37 |
| whoami-rajat | https://145b0d5be77f0cbe05ff-d67a6226f6eb6d34ab4efd951744a16f.ssl.cf5.rackcdn.com/684658/1/check/tempest-full/9e1de42/testr_results.html.gz | 16:37 |
| rosmaita | it happened in O and now it's happening in P. | 16:37 |
| *** markvoelker has joined #openstack-meeting | 16:38 | |
| rosmaita | guess it will happen in Q next week | 16:38 |
| whoami-rajat | i checked the cinder stable/pike code and seems fine | 16:38 |
| whoami-rajat | rosmaita: hah :) | 16:38 |
| rosmaita | yes, my vote is we ignore it, matt has a patch up to pin tempest | 16:38 |
| rosmaita | hopefully they will pin Q also before anything breaks | 16:38 |
| rosmaita | #link http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009674.html | 16:40 |
| rosmaita | just realized that people may not know what i am talking about | 16:40 |
| whoami-rajat | the question is, is the schema validation really necessary for stable branches ? a lot of code is added removed until the introduction of MV's so IMO it will be valid to do the testing after v3 | 16:41 |
| whoami-rajat | rosmaita: i should've started with the thread link, thanks | 16:42 |
| jungleboyj | rosmaita: We had asked why they were doing this right? | 16:42 |
| jungleboyj | I don't remember the answer though. | 16:42 |
| rosmaita | well, i thought eharney had a well-reasoned discussion going on this last time | 16:42 |
| rosmaita | but it wasn't really followed up on | 16:42 |
| rosmaita | so, in my capacity of being a general PITA, i am inclined to not worry about this and let the qa team keep breaking the stable branches and then having to pin tempest to fix it | 16:43 |
| rosmaita | but it is distressing that they are merging stuff without checking to see what they will break | 16:44 |
| whoami-rajat | there are different reasons for failures, last one was because of not doing an MV bump/ testing a dynamic response in tempest | 16:44 |
| rosmaita | and with that, i will shut up | 16:44 |
| whoami-rajat | rosmaita: seems valid | 16:44 |
| jungleboyj | rosmaita: ++ | 16:45 |
| *** igordc has joined #openstack-meeting | 16:46 | |
| jungleboyj | So, any further discussion required there? | 16:47 |
| whoami-rajat | not from my side | 16:47 |
| jungleboyj | Ok cool. | 16:48 |
| *** markvoelker has quit IRC | 16:48 | |
| jungleboyj | For those who didn't see, they have announced that the next event after Shanghai will be a PTG like event in Vancouver. | 16:48 |
| * jungleboyj is happy! | 16:48 | |
| *** rosmaita has left #openstack-meeting | 16:49 | |
| jungleboyj | Then a summit like event later next year. Location still not determined. | 16:49 |
| e0ne | jungleboyj: sounds good | 16:49 |
| *** zhangchi has joined #openstack-meeting | 16:49 | |
| jungleboyj | :-) | 16:50 |
| jungleboyj | Also, if you know anyone that might be new to OpenStack and in Shanghai please let them know about OpenStack Upstream Institute. | 16:50 |
| jungleboyj | Would like to see more interest there. | 16:50 |
| whoami-rajat | jungleboyj: ++ | 16:51 |
| jungleboyj | Ok. That is all I had for this week then. Please go review the outstanding train patches. | 16:52 |
| *** markvoelker has joined #openstack-meeting | 16:53 | |
| whoami-rajat | jungleboyj: Thanks! | 16:54 |
| jungleboyj | Yep. Thanks everyone. | 16:54 |
| jungleboyj | #endmeeting | 16:55 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 16:55 | |
| openstack | Meeting ended Wed Sep 25 16:55:03 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:55 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-09-25-16.00.html | 16:55 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-09-25-16.00.txt | 16:55 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-09-25-16.00.log.html | 16:55 |
| *** ekcs has joined #openstack-meeting | 16:55 | |
| davidsha | Thanks! | 16:55 |
| *** davidsha has quit IRC | 16:55 | |
| *** zhangchi has quit IRC | 16:55 | |
| jungleboyj | Sorry for being distracted this week. :-) | 16:55 |
| *** woojay has left #openstack-meeting | 16:56 | |
| *** brault has joined #openstack-meeting | 16:56 | |
| *** e0ne has quit IRC | 16:59 | |
| *** mattw4 has quit IRC | 17:04 | |
| *** ociuhandu has joined #openstack-meeting | 17:09 | |
| *** sridharg has quit IRC | 17:14 | |
| *** ociuhandu has quit IRC | 17:16 | |
| *** panda|bbl is now known as panda | 17:19 | |
| *** mattw4 has joined #openstack-meeting | 17:44 | |
| *** njohnston has quit IRC | 17:51 | |
| *** igordc has quit IRC | 17:52 | |
| *** njohnston has joined #openstack-meeting | 17:56 | |
| *** njohnston has quit IRC | 17:59 | |
| *** tesseract has quit IRC | 18:08 | |
| *** armax has quit IRC | 18:18 | |
| *** e0ne has joined #openstack-meeting | 18:25 | |
| *** ekcs has quit IRC | 18:43 | |
| *** armax has joined #openstack-meeting | 18:50 | |
| *** ricolin has quit IRC | 18:55 | |
| *** belmoreira has joined #openstack-meeting | 18:55 | |
| *** gary_perkins has quit IRC | 18:57 | |
| *** gary_perkins has joined #openstack-meeting | 18:57 | |
| *** belmoreira has quit IRC | 18:58 | |
| *** ociuhandu has joined #openstack-meeting | 19:17 | |
| *** e0ne has quit IRC | 19:23 | |
| *** ociuhandu has quit IRC | 19:25 | |
| *** zhangchi has joined #openstack-meeting | 19:52 | |
| *** zhangchi has quit IRC | 19:57 | |
| *** radeks has quit IRC | 20:12 | |
| *** diablo_rojo has quit IRC | 20:24 | |
| *** ykatabam has joined #openstack-meeting | 20:29 | |
| *** ralonsoh has quit IRC | 20:30 | |
| *** slaweq has quit IRC | 20:55 | |
| *** ekcs has joined #openstack-meeting | 20:59 | |
| *** alecuyer has joined #openstack-meeting | 21:01 | |
| timburke | #startmeeting swift | 21:02 |
| openstack | Meeting started Wed Sep 25 21:02:15 2019 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:02 |
| *** openstack changes topic to " (Meeting topic: swift)" | 21:02 | |
| openstack | The meeting name has been set to 'swift' | 21:02 |
| timburke | who's here for the swift meeting? | 21:02 |
| kota_ | o/ | 21:02 |
| *** zaitcev has joined #openstack-meeting | 21:03 | |
| mattoliverau | o/ | 21:03 |
| rledisez | o/ | 21:03 |
| alecuyer | o/ | 21:03 |
| *** igordc has joined #openstack-meeting | 21:03 | |
| timburke | agenda's at https://wiki.openstack.org/wiki/Meetings/Swift | 21:03 |
| tdasilva | hello | 21:03 |
| timburke | i've got more stuff ahead of updates than usual, so i'll try to hustle :-) | 21:04 |
| timburke | #topic User survey results | 21:04 |
| *** openstack changes topic to "User survey results (Meeting topic: swift)" | 21:04 | |
| *** raildo has quit IRC | 21:04 | |
| timburke | #link http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009501.html | 21:04 |
| timburke | this was actually available last week, but i finally got around to looking at them with some amount of depth | 21:04 |
| timburke | we only had a couple swift-specific questions: what features are you using, and a free-form "Other feedback" field | 21:05 |
| timburke | we didn't get any free-form comments, but i still saw some interesting things | 21:06 |
| timburke | 62 of 342 respondents included feedback about which Swift features they use | 21:06 |
| *** diablo_rojo has joined #openstack-meeting | 21:06 | |
| mattoliverau | and... what did they use? | 21:07 |
| timburke | but there were an additional 6 who "contribute maintenance resources, such as patches for bug fixes and code reviews on master or stable branches" to Swift without mentioning which Swift features they use | 21:07 |
| timburke | top 5 features: | 21:07 |
| timburke | S3 compatibility (54) | 21:07 |
| timburke | Quotas (35) | 21:07 |
| timburke | Object versioning (26) | 21:07 |
| timburke | Multi-region clusters (26) | 21:07 |
| timburke | Temporary URLs (25) | 21:08 |
| kota_ | interesting | 21:08 |
| timburke | so i feel rather vindicated in the recent focus on S3 versioning ;-) | 21:08 |
| mattoliverau | yeah, nice one | 21:08 |
| tdasilva | multi-region == 26!! | 21:09 |
| timburke | tdasilva, yeah, definitely interesting | 21:09 |
| timburke | there were other questions posed (by the TC, i think?) asking about stable branch usage | 21:09 |
| timburke | 54 of 62 Swift respondents use stable branches to some degree, with 28 "using only official point releases" | 21:09 |
| timburke | ...so i should maybe tag more stable releases | 21:10 |
| rledisez | tdasilva: 26 means 26 clusters in 26 regions or 1 cluster/1 ring with 26 "zones/regions" ? | 21:10 |
| timburke | 26 of the 62 respondents said they "use" multi-region clusters | 21:11 |
| tdasilva | rledisez: i have no idea, i'm hoping it's 26 different people, meaning 26 different clusters | 21:11 |
| rledisez | ah, ok :D | 21:11 |
| rledisez | i was wondering who would do a 26-regions ring ;) | 21:11 |
| timburke | O.O | 21:11 |
| timburke | anyway, thought i would share. i'll make sure i get the full tally of feature usage up somewhere for the curious | 21:12 |
| timburke | (or you can grab the dataset yourself and take a look ;-) | 21:12 |
| tdasilva | timburke: i'd argue we should almost do a stable branch tag almost every time we do a stable backport (or a string of backports) | 21:12 |
| timburke | yeah... i kinda agree | 21:12 |
| tdasilva | like if there's nothing else left in the pipeline, tag | 21:12 |
| mattoliverau | or maybe at least every time we do a release we should see if there has been any backports and if so.. tag | 21:13 |
| timburke | i think the biggest hurdle has been a sense that "oh, if i'm proposing a tag, i should probably see what other bug fixes maybe ought to get backported..." | 21:13 |
| timburke | mattoliverau, definitely. one of the nice things about hand-curating the release notes is that i get a whole bunch of patches and their impacts loaded into my head anyway, which can then be used on my previous concern | 21:14 |
| timburke | #topic Vancouver | 21:15 |
| *** openstack changes topic to "Vancouver (Meeting topic: swift)" | 21:15 | |
| mattoliverau | \o/ back to YVR | 21:15 |
| timburke | i know we haven't even gotten to Shanghai yet, but apparently we're going back to Vancouver for the next OpenStack event in June | 21:15 |
| timburke | #link http://lists.openstack.org/pipermail/foundation/2019-September/002794.html | 21:15 |
| zaitcev | What's in Vancouver? Also, which Vancouver? | 21:15 |
| kota_ | wow | 21:16 |
| zaitcev | reading now | 21:16 |
| * rledisez loves Vancouver | 21:16 | |
| timburke | rledisez, and your flight got so much shorter! | 21:16 |
| mattoliverau | lol | 21:16 |
| rledisez | yeah, makes it even better :) | 21:16 |
| timburke | you all now know as much as me about all this :-) i'm sure we'll find out more in the coming months | 21:17 |
| timburke | #topic Shanghai | 21:17 |
| *** openstack changes topic to "Shanghai (Meeting topic: swift)" | 21:17 | |
| timburke | just a quick reminder again to put topic on the etherpad | 21:18 |
| timburke | #link https://etherpad.openstack.org/p/swift-ptg-shanghai | 21:18 |
| timburke | (i need to do it, too) | 21:18 |
| timburke | #topic train release | 21:18 |
| *** openstack changes topic to "train release (Meeting topic: swift)" | 21:18 | |
| timburke | so within the next week or so, i'd like to tag a 2.23.0 | 21:18 |
| timburke | i updated the priority reviews wiki to highlight a few patches i'd like to get in if we can | 21:19 |
| timburke | #link https://wiki.openstack.org/wiki/Swift/PriorityReviews | 21:19 |
| *** ekcs has quit IRC | 21:19 | |
| timburke | the top two in particular would be good -- and one of them is actually a bug that still needs a patch | 21:20 |
| timburke | specifically, those have to to with our new py3 support -- and one of them is a (now public) security bug | 21:21 |
| timburke | if you know of any other patches that ought to get in, feel free to update the wiki. i'll try to get a patch up for https://bugs.launchpad.net/swift/+bug/1844368 in the next couple days | 21:22 |
| openstack | Launchpad bug 1844368 in OpenStack Object Storage (swift) "fallocate_reserve cannot be specified as a percentage running under python3" [High,Confirmed] | 21:22 |
| zaitcev | If anyone knows anything about eventlet monkey-patching, there | 21:22 |
| timburke | mattoliverau or zaitcev, you've done a good bit with py3 -- would you be able to review https://review.opendev.org/#/c/684041/ ? | 21:23 |
| zaitcev | 's also this: https://review.opendev.org/684380 | 21:23 |
| zaitcev | timburke: will do | 21:23 |
| timburke | thanks. and i can try to take a look at the triple-o patch -- sounds kinda like the issue i hit in https://review.opendev.org/#/c/675227/ where the module that i wanted patched was imported before i realized that it would be | 21:25 |
| timburke | ok, updates! | 21:26 |
| timburke | #topic versioning | 21:26 |
| *** openstack changes topic to "versioning (Meeting topic: swift)" | 21:26 | |
| zaitcev | As you can see, it is not a review in Swift. The story is, Swift requires eventlet 0.25 for py3. So, I made RDO update the required eventlet. Result is, TripleO fell apart, because the module git has issues with eventlet. It's a mad word, but if anyone can understand what happend, they could add +1. | 21:26 |
| *** ekcs has joined #openstack-meeting | 21:26 | |
| timburke | tdasilva, what's the state of the world? | 21:26 |
| *** whoami-rajat has quit IRC | 21:27 | |
| tdasilva | with the new object versioning patch per se, we've been making progress adding some unit and functional tests | 21:27 |
| tdasilva | there are still some corner cases to discuss the design about. e.g., how to handle error scnearios of creating versions container, how to handle versions and overwrites when versioning is suspended | 21:28 |
| tdasilva | but it's coming along. For now we are also punting on the naming scheme, as we have started discussion on the null character piece | 21:29 |
| clayg | did i miss the whole thing? | 21:30 |
| tdasilva | clayg: nope | 21:30 |
| timburke | clayg, i think you're right on time :-) | 21:30 |
| tdasilva | timburke: so we've made progress, but there's still quite a bit of work to do :) | 21:31 |
| timburke | i know we've got some etherpads for both versioning and the null-byte ideas | 21:31 |
| timburke | #link https://etherpad.openstack.org/p/swift-object-versioning | 21:32 |
| tdasilva | if folks can keep an eye and contribute to both the discussion on the etherpad and the patches listed here, it would be super helpful: https://etherpad.openstack.org/p/swift-object-versioning | 21:32 |
| timburke | #link https://etherpad.openstack.org/p/swift-null-namespace | 21:32 |
| mattoliverau | oh i didn't realise there was a null namespace one. I'll read it today | 21:32 |
| timburke | mattoliverau, i think clayg added it today ;-) | 21:33 |
| clayg | yeah that's new fresh off the presses | 21:33 |
| mattoliverau | oh, well then I was alseep anyway :P | 21:33 |
| kota_ | interesting, the null namespace idea | 21:33 |
| tdasilva | kota_: it's revolutionary! :D | 21:34 |
| clayg | 🤣 | 21:34 |
| kota_ | it seems like similar idea s3api (swift3) uses `+` for versioning namespace that character can not be used by users. | 21:35 |
| timburke | clayg, tdasilva: anything in particular you'd like input on this week? | 21:35 |
| timburke | kota_, i think that may be where we got the idea :D | 21:35 |
| kota_ | +1 | 21:35 |
| clayg | kota_: yes, lots of s3 features and s3api implementation rely on resources the user can't access except through approved apis! | 21:35 |
| clayg | tdasilva: do we have the versioning api draft anywhere? | 21:36 |
| clayg | something that might eventually become the docs? | 21:37 |
| tdasilva | timburke: yeah, i think in terms of order, the null character would come first?? it would probably be good to get a good design on that | 21:37 |
| tdasilva | clayg: no yet, /me needs to get started on that :( | 21:37 |
| clayg | tdasilva: right, everyone please read and comment on the null-namespace etherpad - I'm going to be implementing what's in there so expect to be doing some early review stuff soon | 21:37 |
| timburke | 👍 | 21:38 |
| clayg | tdasilva: i was thinking more fast/dirty wiki style for folks to get an idea - but... I guess we have the s3 api for that 🙄 | 21:38 |
| tdasilva | clayg: yeah, I was thinking about adding to the etherpad to begin with, it makes for editing very easy | 21:39 |
| tdasilva | by anyone | 21:39 |
| clayg | tdasilva: I had an idea about versioned_writes - that maybe we call the new mode "object versioning" and the old one was *truly* "just" versioned_writes (i.e. no versioned reads, or restore, or listing) | 21:39 |
| clayg | might help us avoid calling the "object versioning" feature "new versioned writes" | 21:39 |
| clayg | or maybe.. there's the "object versions api" and "legacy versioned writes" or something - idk - i get hung up on names | 21:40 |
| clayg | tdasilva: hell yeah, etherpad it up! | 21:40 |
| timburke | FWIW, i feel like the S3 docs aren't always sufficient -- exploratory things like tdasilva's http://paste.openstack.org/show/779176/ are certainly useful, too | 21:40 |
| tdasilva | clayg: i've been really scratching my head on naming, open to suggestions | 21:41 |
| tdasilva | timburke: yeah, also found out from testing that s3 doesn't allow POSTing to older versions, I haven't seen that documented | 21:42 |
| timburke | clayg, would it be worth breaking out https://review.opendev.org/#/c/682953/1/test/s3api/test_versioning.py to its own patch as a lace for us to document what AWS does? | 21:42 |
| timburke | tdasilva, GTK | 21:42 |
| clayg | tdasilva: yeah i breifly let myself think that we might back off on allowing POST to static links via versioning and instead stick with our re-direct behavior (i.e. allow POST key?version=) | 21:43 |
| timburke | all right, i still want to check in on other endeavors -- maybe we can continue this in -swift | 21:43 |
| timburke | #topic lots of small files | 21:44 |
| *** openstack changes topic to "lots of small files (Meeting topic: swift)" | 21:44 | |
| clayg | timburke: I guess that'd be ok just for organization - I don't think we need/want s3api tests that fail against swift merged to master? | 21:44 |
| timburke | clayg, fair enough | 21:44 |
| timburke | alecuyer, i saw another patch this morning! | 21:44 |
| timburke | an updated version of https://review.opendev.org/#/c/666378/ | 21:44 |
| alecuyer | yes ! I'm getting to the patches that were pending, | 21:45 |
| timburke | have you had a chance to sort out the bugs found when trying to deploy to prod? | 21:45 |
| alecuyer | I've updated it to work with HTTP | 21:45 |
| alecuyer | so about the bug, the good news, it's unrelated to the "rpc over http" patch | 21:45 |
| timburke | 👍 (on both messages) | 21:45 |
| alecuyer | we quickly tacked on a patch to bring back the hashes.pkl - and hum the "hang" was actually that the algorithm cost would go up with the square of the suffixes below the partition | 21:46 |
| alecuyer | so, hm, that was quickly reverted | 21:46 |
| timburke | eep! | 21:46 |
| alecuyer | and I'll look at it later. But, no issues found with the HTTP patch itself | 21:46 |
| timburke | is there anything we can be helping with? i'll look at the py3 failures | 21:47 |
| alecuyer | thanks ! haven't looked at it yet but I will. So yes that would be great if you could take a look at the patch | 21:48 |
| alecuyer | then I'll go back to the other pending patches (they are smaller than this one) | 21:49 |
| timburke | sounds good, thanks! | 21:49 |
| timburke | #topic sharding | 21:49 |
| *** openstack changes topic to "sharding (Meeting topic: swift)" | 21:49 | |
| timburke | lots of cleave-context cleanup! | 21:50 |
| mattoliverau | lol, yeah, I felt guilty that we weren't using an existing last-modified timestamp and adding a new one which would be annoyting for NM and needless storage.. So fixed that. | 21:50 |
| mattoliverau | Then clayg went and cleaned it up even further! | 21:51 |
| timburke | thanks for that, i think it's a bunch better :-) | 21:51 |
| timburke | mattoliverau, is there anything else there that you want to bring up, or do i just need to get to reviewing more patches? ;-) | 21:51 |
| clayg | minor stuff, paper cuts | 21:51 |
| clayg | is there more to review!? 😬 | 21:51 |
| timburke | always :D | 21:51 |
| timburke | https://review.opendev.org/#/c/675820/ for the empty handoff problem | 21:52 |
| timburke | https://review.opendev.org/#/c/675014/ to latch stats reporting | 21:52 |
| mattoliverau | nope, nothing to bring up. but reviews welcome. I'll review timburke's sharding patches | 21:52 |
| timburke | and of course, the autosharder chain: https://review.opendev.org/#/c/667030/ https://review.opendev.org/#/c/667579/ | 21:52 |
| timburke | #topic open discussion | 21:53 |
| *** openstack changes topic to "open discussion (Meeting topic: swift)" | 21:53 | |
| timburke | anything else people would like to bring up? | 21:53 |
| zaitcev | Not me | 21:53 |
| kota_ | not | 21:54 |
| timburke | one thing i thought of -- the last time we thought much about our current meeting time, there were more of us in SF -- does this time slot still work well for people? | 21:55 |
| mattoliverau | daylight savings changes in a few weeks for me then it'lll be 8am.. a much more civilised time :P | 21:56 |
| timburke | something to think about, anyway. it's hard with the spacing of US, Europe, and Japan/Australia | 21:56 |
| alecuyer | I'm fine with the time - similar to mattoliverau, in winter time it'll be 10pm vs 11 :) | 21:57 |
| timburke | on that note, let's let alecuyer go to bed ;-) | 21:57 |
| timburke | thank you all for coming, and thank you for working on Swift! | 21:57 |
| timburke | #endmeeting | 21:57 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 21:57 | |
| openstack | Meeting ended Wed Sep 25 21:57:42 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:57 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-09-25-21.02.html | 21:57 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-09-25-21.02.txt | 21:57 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/swift/2019/swift.2019-09-25-21.02.log.html | 21:57 |
| mattoliverau | thanks timburke | 21:57 |
| *** zaitcev has left #openstack-meeting | 21:57 | |
| *** rfolco|bbl is now known as rfolco | 22:09 | |
| *** rcernin has joined #openstack-meeting | 22:15 | |
| *** diablo_rojo has quit IRC | 22:16 | |
| *** diablo_rojo has joined #openstack-meeting | 22:16 | |
| *** thgcorrea has quit IRC | 22:27 | |
| *** ekcs has quit IRC | 22:31 | |
| *** ekcs has joined #openstack-meeting | 22:49 | |
| *** mriedem has quit IRC | 22:53 | |
| *** zhangchi has joined #openstack-meeting | 22:55 | |
| *** zhangchi has quit IRC | 23:09 | |
| *** igordc has quit IRC | 23:10 | |
| *** gyee has quit IRC | 23:44 | |
| *** carloss has quit IRC | 23:47 | |
| *** mattw4 has quit IRC | 23:50 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!