21:00:17 <timburke> #startmeeting swift
21:00:18 <openstack> Meeting started Wed Jun 26 21:00:17 2019 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:21 <openstack> The meeting name has been set to 'swift'
21:00:31 <timburke> who's here for the swift meeting?
21:00:32 <clayg> o/
21:00:33 <mattoliverau> o/
21:00:39 <alecuyer> hello
21:01:56 <timburke> not too much on the agenda
21:01:59 <timburke> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:22 <timburke> #topic Shanghai CFP
21:02:30 <timburke> first, just a heads-up
21:02:45 <timburke> the call for proposals for summit talks ends in about a week
21:03:08 <timburke> that and other details were on the mailing list
21:03:10 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007373.html
21:03:25 <timburke> there isn't a dedicated "Storage" track or anything, but i feel like we could tell some interesting stories under "Private & Hybrid Cloud" or "Public Cloud" (which are awfully broad topics)
21:03:40 <timburke> or maybe even "Open Development", "AI, Machine Learning & HPC", or "Container Infrastructure"
21:04:31 <timburke> so, if you've got something you might like to present on, you've got until July 2 to submit it!
21:05:00 <timburke> any questions on the summit?
21:05:07 <mattoliverau> Yeah, I assume it'll be asia heavy, specicially china (obviously), usually that community is quiet, so I wonder if we'll see any interesting swift talks from them
21:05:39 <mattoliverau> well quiet though the usual channels we use
21:06:23 <timburke> me too. and apparently "Sessions will be presented in both Mandarin and English, so you may submit your presentation in either language." which is also interesting
21:06:30 <tdasilva> heelo, sorry i'm late
21:06:40 <mattoliverau> tdasilva: o/
21:07:02 <timburke> next up, updates!
21:07:06 <timburke> #topic py3
21:07:16 <timburke> we finished porting the unit tests!
21:07:37 <mattoliverau> \o/
21:07:40 <timburke> there are still a handful of patches to get py2 func tests passing against py3 services though
21:07:41 <alecuyer> bravo :)
21:08:14 <timburke> i updated the set of patches on the priority reviews wiki
21:08:17 <timburke> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews
21:08:59 <timburke> there *is* a patch to actually have py2 tests against py3 running in the gate though!
21:10:10 <mattoliverau> Sorry it's hack week this week at SUSE, we're not suppose to do normal work and instead on  what ever we want instead. So haven't really gone and reviewed much py3.  Had a few distractions with the girls and flu shots but otherwise have been focusing on auto sharding because HACKWEEK
21:10:36 <timburke> so -- how are people feeling regarding a release? should i continue waiting for py2 func tests passing, or should i known-issues the last few things?
21:10:52 <timburke> yay autosharding! i saw those patches go by, i'm excited :-)
21:11:40 <timburke> (fwiw i'm already planning on known-issuing the ECAppIter bug)
21:12:04 <mattoliverau> Well it would be nice to have a way of func testing.
21:13:07 <mattoliverau> is the first py3 release going to be marked as experimental?
21:13:12 <timburke> yeah. and i kinda want to push people to try it out in devstack around the same time as i announce the experimental support...
21:13:20 <timburke> yeah, that's my plan
21:14:22 <mattoliverau> ok if it's experimental that func tests are only good to have.
21:15:56 <mattoliverau> is it just the 4 patches to get py2 - py3 func working?
21:16:02 <timburke> idk -- like, there's a workaround for https://bugs.launchpad.net/swift/+bug/1691075 but it'd be nicer if it just *worked*...
21:16:03 <openstack> Launchpad bug 1691075 in OpenStack Object Storage (swift) "swift-object-server fails to start in python 3.5 environment with devstack" [Undecided,Confirmed]
21:16:18 <timburke> yeah. https://review.opendev.org/#/c/665494/ to address ^^^
21:17:22 <timburke> https://review.opendev.org/#/c/653548/ to clean up some lingering memcache issues (that i think relate to how keystonemiddleware/keystoneauth use it?)
21:17:53 <timburke> https://review.opendev.org/#/c/662546/ to work around a cpython bug related to non-ascii headers
21:18:25 <timburke> and https://review.opendev.org/#/c/666942/ to actually have the full suite running in the gate
21:19:14 <timburke> i think those are roughly in order of importance -- if i could get the first two, i'm more willing to known-issues the third (which is hairier anyway)
21:19:44 <timburke> can anyone commit to reviewing one or both of those top two?
21:19:50 <mattoliverau> well I t hink having some kinda of func tests happening in py3 would be awesome, especially as we add new Swift code. So how about we focus on the py2 func tests testing py3 swift, at least give it until the end this week/early next week. And then cut a release?
21:20:14 <mattoliverau> I'll take a look at them, I do have more time (HACKWEEK!)
21:20:30 <timburke> thanks mattoliverau!
21:20:50 <timburke> sounds like a plan: release early next week
21:21:18 <timburke> then we can move on to getting func tests ported ;-)
21:21:27 <mattoliverau> +1
21:21:34 <timburke> #topic lots of small files
21:21:46 <timburke> alecuyer, how's it going?
21:22:00 <alecuyer> I wrote more tests,
21:22:38 <alecuyer> found some issues (heh!), things like, if fsck removes the "wrong" file, you're not able to write to a partition until the operator fixes it
21:23:28 <timburke> eep! good thing we're finding them :-)
21:23:46 <alecuyer> So, some fixes, and had to refactor some things for testing. I'd like to stop adding new tests for that patch because it's getting large, so 'ill try to clean it and "finish" it before I write tests for more things
21:24:51 <alecuyer> timburke:  Yes :) it's not something that I expect would happen often but we do know if it can happen, it will ;) so, that's it
21:25:12 <timburke> that sounds like a plan
21:25:25 <timburke> anything the rest of us can do to help you?
21:25:56 <mattoliverau> nice find, capturing all edge cases is hard/impossible. great to find and know about these kind of issues :)
21:26:09 <alecuyer> I think for this review it's best to wait a little because I know it's not ready
21:26:23 <alecuyer> maye if anyone wants to think about grpc / eventlet and find something i couldn't
21:27:17 <timburke> i can try to do that. i've been digging around in eventlet a bit lately anyway
21:27:28 <timburke> i'll also plan on merging from master sometime this week (hopefully today)
21:27:38 <alecuyer> great, thanks!
21:27:57 <timburke> be warned, now you'll *have* to make things pass on py3 ;-)
21:28:07 <mattoliverau> lol
21:28:13 <alecuyer> heh yes I know I'll have some work there… :-)
21:28:22 <timburke> #topic open discussion
21:28:36 <timburke> that's it for the agenda; does anyone have anything else to bring up?
21:28:39 <mattoliverau> As mentioned earlier I've working on autosharding.
21:28:56 <mattoliverau> #link https://review.opendev.org/#/c/667030
21:28:58 <mattoliverau> and
21:29:09 <mattoliverau> #link https://review.opendev.org/#/c/667579
21:29:15 <mattoliverau> so far.
21:29:24 <mattoliverau> it's still very rough and a WIP
21:29:42 <mattoliverau> working on a bit of patch chain to get something up
21:30:00 <timburke> oh! and speaking of sharding, i started thinking about caching shard ranges
21:30:01 <mattoliverau> the latter one is just something I pushed up to get it off my machine late last night.
21:30:04 <timburke> #link https://review.opendev.org/#/c/667203/
21:30:23 <mattoliverau> timburke: yeah I saw that and plan to look more closely at it, great work
21:31:00 <mattoliverau> 667579 has a large commit message ramble that might be interesting
21:31:21 <timburke> i started with just the updating set -- listings were left for future work -- i mainly wanted to get the container GET out of the object write path
21:31:57 <mattoliverau> ie. currently we use the OP tool to scan all ranges and the push all of them in to start sharding, whereas the existing auto sharding (we only use in tests) has been designed to scan X at a time per round
21:33:25 <timburke> cool
21:33:46 <mattoliverau> I wonder if to simplify and to decrease the potential known edge cases, is if we do something similar. Ie the leader scans for ALL ranges. That was we can bail on inserting them (on the second election, ie am I still the leader) if any of the other primaries are in the SHARDING or SHARDED state
21:34:28 <mattoliverau> ie, the OP has adding the ranges or there was a weird election issue etc.
21:35:14 <mattoliverau> Anyway. We have other larger priorities, so just wanted to bring it to peoples attention
21:36:13 <mattoliverau> I'll push along and get something to play with up
21:36:14 <timburke> i know that we (SwiftStack) always do all ranges at once -- i'd be ok going either way, i think. the one nice thing about just doing a few at a time is that you're less likely to do a bunch of work and then discard the result
21:36:30 <mattoliverau> yeah
21:36:55 <timburke> oh -- swiftclient's gate is currently busted! https://review.opendev.org/#/c/667477/ is a step in the right direction, but needs more work :-/
21:38:07 <mattoliverau> anyway, that's all I got atm.
21:38:16 <timburke> anyone have anything else to bring up?
21:38:17 <alecuyer> Unrelated to swift dev but I'll have to help our ML team at OVH with working with swift (with the "JOSS" java library). If there are nice use cases there I'll let you know :)
21:38:25 <zaitcev_> 2019-06-26 10:21:50.802482 | ubuntu-bionic |    Requirement(package='sphinx', location='', specifiers='!=1.6.6,!=1.6.7,!=2.1.0', markers="python_version>='3.4'", comment='# BSD', extras=frozenset())
21:38:25 <zaitcev_> 2019-06-26 10:21:50.802588 | ubuntu-bionic | WARNING: possible mismatch found for package "sphinx"
21:38:37 <zaitcev_> step in the right direction indeed
21:39:08 <timburke> alecuyer, cool! sounds like the sort of thing that could turn into a summit talk ;-)
21:39:16 <timburke> thanks zaitcev_
21:40:06 <timburke> i think there's also something that needs to happen related to https://review.opendev.org/#/c/588103/ :-/
21:40:33 <timburke> (wherein we switched to using auth_uri in the test config)
21:42:02 <timburke> any other topics?
21:43:16 <timburke> then i think we'll break a bit early
21:43:25 <clayg> noice
21:43:32 <timburke> thank you all for coming, and thank you for working on swift!
21:43:37 <timburke> #endmeeting