21:00:26 <notmyname> #startmeeting swift
21:00:27 <openstack> Meeting started Wed Dec 14 21:00:26 2016 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:30 <openstack> The meeting name has been set to 'swift'
21:00:33 <acoles> here :)
21:00:34 <notmyname> who's here for the swift team meeting?
21:00:44 <nadeem> o/
21:00:47 <bkeller`> o/
21:00:48 <tdasilva> eu
21:00:49 <timburke> o/
21:00:50 <jrichli> hey
21:00:51 <clayg> o/
21:00:52 <joeljwright> o/
21:00:56 <mathiasb> o/
21:01:00 <cschwede> o/
21:01:08 <mattoliverau> o/
21:01:36 <notmyname> welcome everyone
21:02:07 <kota_> hello
21:02:20 <notmyname> kota_: I was worried for a second (only a second)
21:02:21 <notmyname> :-)
21:02:35 <notmyname> thought you wouldn't make it
21:02:37 <kota_> :-)
21:02:59 <notmyname> a few things to highlight this week, but first I want to talk about next meetings
21:03:03 <notmyname> #topic next meetings
21:03:18 <notmyname> it's the end of the year, and people will be in and out
21:03:45 <notmyname> we've got 2 weeks left in the year. so the question is, "do we have a meeting during the next two weeks, or do we skip until january 4?"
21:04:26 <notmyname> would anyone like to advocate for either having meetings or not?
21:04:40 <tdasilva> i'm fine with skip
21:04:56 <clayg> see you next year suckers!
21:05:16 <mattoliverau> I'm around for the next 2 weeks, so happy to still do them, but not having them means sleep in.. so I'm really good either way ;)
21:05:23 <notmyname> heh
21:05:44 <kota_> either works to me
21:05:45 <joeljwright> I can be here next week, after that I'm travelling
21:05:55 <acoles> I won't be around for next two weds
21:06:16 <notmyname> well, the other question is if there's something that will be happening in the next 2 weeks that will need a formal meeting time to discuss
21:06:37 <notmyname> ok, based on the responses, let's skip until January 4, 2017
21:06:42 <mattoliverau> kk
21:06:50 <joeljwright> kk
21:06:50 <notmyname> #agreed skip meetings until January 4
21:07:04 <notmyname> #info skip meetings until January 4
21:07:10 * notmyname doesn't know meetbot
21:07:24 <notmyname> #topic FYI things happening in broader openstack
21:07:57 <notmyname> there's been a lot going on, even just this week, that I want to highlight (more than usual)
21:08:04 <notmyname> oh...
21:08:06 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:08:26 <notmyname> FWIW, there's a new #openstack-meeting-5. so lurk there if you like to lurk in meeting rooms
21:08:55 <notmyname> FYI, the openstack individual board of directors election is coming up
21:09:01 <notmyname> you can see the candidates at https://www.openstack.org/election/2016-individual-director-election/
21:09:04 <notmyname> linked from there
21:09:21 <notmyname> and the call for presentations for Boston opened today http://lists.openstack.org/pipermail/community/2016-December/001690.html
21:09:42 <notmyname> so if you want to speak at that openstack forum event, start working on your talk submission
21:09:50 <notmyname> ok, those were the fast things
21:09:58 <notmyname> next up... storyboard
21:10:07 <notmyname> I came across this blog post today https://storyboard-blog.sotk.co.uk/
21:10:35 <notmyname> the summary is that for a few years some people in openstack have been working on a replacement for launchpad that's called storyboard
21:10:57 <mattoliverau> I didn't realise storyboard was still a thing.. I thought we stopped working on it a while ago.. so there you go. cool its still a thing
21:10:58 <notmyname> except it's not completely a replacement in that it doesn't map 1:1 with what LP does
21:11:05 <notmyname> oh, yeah. it's more than a thing
21:11:46 <notmyname> there's been a private email thread that's been really active recently that's summarized as "ok, so this is about to be a Thing in openstack and everyone will be using it, so what do you think"
21:12:14 <cschwede> storyboard is quite nice, once you get used to it. i used it a bit in the past
21:12:15 <notmyname> point is, well... I'm not sure yet of the point, other than it's a big change that affects everyone's workflow
21:12:32 <timburke> oh goody
21:12:49 <notmyname> cschwede: awesome! I could use your help then. i tried to figure it out, and found it ... not obvious as to what i was supposed to do
21:13:42 <notmyname> so again, this is an FYI, and I think you can see it in use in some infra projects. if you've got a few minutes, it might be worth a look
21:14:29 <notmyname> and finally, the golang discussions have started again (or continued?)
21:14:42 <acoles> the story/task thing sounds a bit like Jira
21:14:55 <notmyname> there's a thread on the ML about technical requirements for using golang at http://lists.openstack.org/pipermail/openstack-dev/2016-December/108875.html
21:15:02 <tdasilva> notmyname: what's the link to storyboard?
21:15:16 <acoles> tdasilva: https://storyboard.openstack.org/#!/page/about
21:15:36 <notmyname> and there's a governance proposal about a new TC process to approve not-python being discussed at https://review.openstack.org/#/c/398875/
21:15:54 <notmyname> acoles: thanks
21:16:50 <notmyname> both of the golang discussions are about the same sort of things. the ML thread is about what's a "release", how to manage dependencies, and what about any common libraries that might be needed
21:17:29 <notmyname> the TC resolution basically asks a lot of those same questions saying "if you're going to use not-python, have an answer for all of these things"
21:17:58 <clayg> neat
21:18:40 <notmyname> the most controversial part of the TC resolution is towards the bottom where it talks about common libraries. it strongly implies that the first project to use not-python is responsible for writing oslo equivalents in the new language before using the language in the project
21:19:31 <acoles> but surely the language would need to be approved before anyone would invest time in writing libraries?
21:19:34 <clayg> notmyname: if you don't start using the common thing from the get-go it's really hard to migrate to it later when it's more fleshed out without your legacy of use-case built in
21:19:38 <notmyname> however, an in-person conversation I had last week said that was more to mean "projects writing stuff in not-python shouldn't expect the oslo team to write stuff for you"
21:20:18 <clayg> i don't think the idea with oslo was ever to get someone else to write code you needed for you
21:20:54 <notmyname> the fear is that across the many openstack projects, it's taken a very long time to get everyone using the same versions of common libraries. one would have something in-tree, another would have an old copied version of that, another would be using a common library from oslo, etc
21:21:28 <notmyname> and the hope is that when starting with a new language, it isn't set up to repeat that diversification of libraries doeing sortof the same thing all over again
21:22:14 <notmyname> and furthermore, it's "known" than configs and logging are something that needs to be done, regardless of language, so that's why those are called out
21:22:44 <clayg> sweet
21:22:48 <notmyname> so that's the background to that section
21:23:31 <acoles> fwiw I did make a start on an oslo config like thing in golang, just as a vehicle for learning the language
21:23:36 <notmyname> I suspect that this proposal will be approved by the TC, but I suspect it will be slightly modified (in what ways, I'm not entirely sure)
21:24:20 <notmyname> I think the point is that new-lang needs to read existing configs and log the same way
21:24:32 <notmyname> (note, that's not actually what's written in the proposal)
21:25:16 <cschwede> well, that makes sense, right? we don't want 2.5 different config styles for one service
21:25:35 <notmyname> cschwede: yes, absolutely
21:26:15 <notmyname> hm...rather than being careful with what i say in a logged channel, I'll keep going ;-)
21:26:22 <acoles> but isn't oslo.config more about declaring config schema etc? the actual config is just an ini file?
21:26:28 <cschwede> also, it sounds like acoles is already solving that issue with the config;)
21:26:41 <acoles> unless I am missing something
21:27:17 <notmyname> so if you're frustrated by this being a seemingly good idea when there were two similar proposals earlier this year, you arent' alone
21:27:20 <acoles> so with oslo.config I imagine the desire will be to have a common developer pattern - the op experience is just an ini file
21:27:27 <cschwede> acoles: i think it's also about auto-generating config samples, using only the settings that your project is using
21:27:52 <clayg> +1 acoles and design the grand unified theory of swift/keystone/hummingbird/pastedeploy configuration
21:27:55 <acoles> cschwede: yes, but again that's not s much an op experience as a dev experience
21:28:01 <clayg> +2 if he solves configuration once and for all for all of opnestack - huzzah
21:28:16 <notmyname> I find this very frustrating that this proposal is seemingly making progress, while one proposed by the swift team was rejected because it was proposed by the swift team
21:28:37 <acoles> clayg: actually I got as far as being frustrated by lack of generics and gave up!
21:28:42 <acoles> clayg: JK
21:28:43 <notmyname> but this is the current state of the golang discussion in openstack. so I'm happy to hear that acoles is working on figuring out some basic config stuff in golang!
21:28:44 <clayg> progress is progress
21:28:49 <cschwede> notmyname: well, i think it has more to do with timing and discussions inbetween.
21:29:10 <clayg> cschwede: or it could be what notmyname said
21:29:28 <notmyname> porque no los dos?
21:29:43 <clayg> progress is progress
21:30:29 <notmyname> so all that being said, we've got a large body of work ahead of us related to golang (with little recent progress), and these are the places where we can be involved in that in the larger openstack community
21:31:15 <clayg> huzzah
21:31:21 <notmyname> and I'd like to transition to talking about that. the stuff going on in swift (starting with golang)
21:31:28 <notmyname> #topic things happening in swift
21:31:43 <clayg> i hear 2.12 is the best swift ever
21:31:55 <notmyname> oh yeah. releases :-)
21:32:18 <notmyname> stable releases happened. 2.12.0 should happen today or tomorrow, depending on the openstack-releases team
21:32:28 <kota_> oh,  release?
21:32:31 <notmyname> ie i'm plannign on submitting that patch this afternoon
21:32:51 <notmyname> kota_: yeah, needed to get a few things out. last chance this year
21:33:07 <kota_> notmyname: good call
21:33:11 <clayg> doo eeeet!
21:33:13 <notmyname> I know this one is a little more "sudden" instead of talking about it for a few weeks first. sorry if it's taking anyone by surprise
21:33:28 <notmyname> but it's a good release. best swift ever, in fact ;-)
21:33:46 <notmyname> so I spent some time this week looking at what's starred, what's on the ideas page, what's being reviewed, etc
21:33:56 <notmyname> spoiler: it's a lot
21:34:00 <notmyname> surprise!
21:34:30 <notmyname> the short list is still pretty intimidating
21:35:00 <notmyname> here's the "big stuff" I wrote down (it's on the meeting agenda)
21:35:12 <acoles> looks like the container sharding bullet point got sharded :)
21:35:24 <tdasilva> or replicated
21:35:35 <mattoliverau> as it should :P
21:35:47 <joeljwright> :)
21:35:49 <notmyname> golang, global ec, policy migration and tiering (and symlinks), part power increase, container sharding, and metadata searching
21:35:53 <notmyname> adn container sharding too
21:36:05 <notmyname> :-)
21:36:11 <mattoliverau> the list got so long, it needed sharding
21:36:16 <notmyname> no kidding
21:36:21 <clayg> that's no *so* bad
21:36:34 <clayg> i feel like most of them are stuff that we've had ongoing
21:36:35 <notmyname> clayg: it's only, like, 1.5 years of work ;-)
21:36:47 <acoles> container sync, keymaster+barbican
21:36:48 <clayg> well but maybe some of them are close?
21:37:00 <notmyname> nah, you're right. most (all?) are well on their way
21:37:00 <clayg> i keep thinking when I go look at symlinks that'll be mostly done
21:37:07 <clayg> we'll make some sort of progress on golang
21:37:12 <clayg> composite rings has *been* done
21:37:40 <notmyname> acoles: I specifically left those off. not because they aren't important, per se, but because somethign had to be dropped. I'd still like to see those land
21:37:41 <clayg> policy migration and auto tiering are the future - the elasticsearch sync stuff could be a door opener too
21:37:46 <clayg> increase part power is basically done
21:38:00 <clayg> global ec can make progress before it's "done"
21:38:00 <notmyname> yeah, composite rings, part power, ec replication are all pretty close
21:38:03 <clayg> this is a great list
21:38:11 <acoles> notmyname: yep, and they are more incremental than others
21:38:32 <clayg> this is great
21:38:34 <cschwede> clayg: absolutely! i just need to work on the comments by Pete and you can merge it :)
21:38:42 <clayg> cschwede: ack
21:38:50 <tdasilva> notmyname: would it make sense to prioritize that list somehow?? not sure if it's feasible or not
21:38:53 <notmyname> we've all realized that one of the problems is that we've got a huge list. this is my attempt at focusing in down
21:38:58 <kota_> i'm still waiting to get merged :-)
21:39:11 <notmyname> yeah! first priority is to merge kota_'s stuff!
21:39:13 <notmyname> ;-)
21:39:31 <mattoliverau> speaking on sharding, I've moved a bunch (not the sharding itself) but interacting with shards into the proxy which has cleaned up some not so pretty code, and will make clayg happier. but it does add a little overhead. but so far its better. Although I've been debugging some of that in my benchmarks. But I think its worth it.
21:39:35 <notmyname> tdasilva: it could be prioritized to some extent, but also we can do several in parallel
21:41:09 <notmyname> what I'd like to do next, myself, is organize that list a little more to show what's going on in each
21:41:35 <notmyname> and I think for everyone, I'd ask that you prioritize reviewing things on this list
21:41:50 <clayg> it's good to know what's close and what still might need to have scope trimmed
21:41:52 <notmyname> (obviously, stuff comes up, like the fun with ec 2 weeks ago)
21:42:09 <clayg> it's also good to know when something is close who is going to rally to push it over the finish line
21:42:20 <notmyname> yep, exactly
21:43:17 <notmyname> I'll associate patches with each of these so we can see that
21:43:59 <notmyname> as a first pass, i'll update the priority reviews page. ie before I invent a totally new (and awesomer) dashboard
21:44:23 <notmyname> so....
21:44:33 <notmyname> #topic open discussion
21:44:43 <notmyname> anythign else to bring up or discuss this week?
21:45:02 <acoles> so is the kind of thing that story board should help with - tracking patches and progress under topics?
21:45:04 <clayg> isa-l-rs-cauchy is the best ec thing evar
21:45:07 <joeljwright> would love people to look at https://review.openstack.org/#/c/365371 if they have time
21:45:09 <notmyname> acoles: I have no idea
21:45:13 <timburke> i could use some eyes on https://review.openstack.org/#/c/310075/ (joeljwright, maybe?)
21:45:27 <notmyname> oh! there's a couple of things I'd like to see land in swiftclient (like the new tempurl support) this week and then do a release of that next week
21:45:31 <clayg> we should merge pyeclib into liberasurecode (and future golang bindings should *also* ship inside of liberasurecode)
21:45:44 <notmyname> clayg: I like that idea
21:45:45 <joeljwright> timburke: kk
21:45:59 <clayg> joeljwright: I keep seeing that get updated
21:46:43 <timburke> oh yeah, kota_: are you on board with what i pushed over at https://review.openstack.org/#/c/405929/ ?
21:46:46 <joeljwright> clayg: the updates recently have been rebase/fixes, I believe it's 'done' now
21:46:50 <notmyname> FWIW the one timburke mentioned (https://review.openstack.org/#/c/310075/) is getting some priority inside of swiftstack, so we'll likely try to land that soon
21:46:50 <cschwede> clayg: we should do that probably around the same time the next big release is coming (Ocata)
21:46:53 <clayg> joeljwright: timburke if normally my goto "crazy new client/api feature awesome sause!?!!" - but it looks like he's already looking at it?
21:47:27 <cschwede> clayg: otherwise it might get messy for distributors if we switch to a single pkg
21:47:35 <kota_> timburke: yeah, sorry no response but that last rev looks super great to me.
21:48:01 <notmyname> cschwede: interesting. so when, exactly woudl that be? ie why later as opposed to now?
21:48:03 <mattoliverau> clayg: I had a play with your concurrent gets testing patch and pushed up a new verson, hope that's ok.
21:48:10 <acoles> we should get this in shape and merged https://review.openstack.org/#/c/402043 Optimize hash calculation when suffix hash invalidated
21:48:12 <timburke> and we should probably do https://review.openstack.org/#/c/407829/, maybe add a gating job for it...
21:48:13 <notmyname> from their perspective, aren't we already working on ocata
21:48:48 <clayg> commit message on patch 310075 seems reasonable - i'll star it at least - didn't realize it was a priority
21:48:55 <tdasilva> cschwede: would we have a single package?
21:49:05 <cschwede> notmyname: well, i'd like Pete's view on this; but if we ship fixes for stable versions and switch from two to one package it might get more difficult for packagers
21:49:05 <notmyname> lol @ "here's some priorities" followed by a flurry of patches not about those ;-)
21:49:12 <cschwede> tdasilva: that was my understanding
21:49:14 <tdasilva> cschwede: i think we could still hae multiple packages, no?
21:49:16 <timburke> kota_: i'm not sure, but maybe we can drop the 'WIP'? not sure what to do about tests though...
21:49:22 <clayg> mattoliverau: of course - that's fine - always push over me
21:49:26 <notmyname> cschwede: yeah, definitely should get input from pete and onovy. good call
21:49:56 <tdasilva> cschwede: one repo, one release, but i think we would want multiple packages (libec, python-binding, golang binding)
21:49:59 <cschwede> tdasilva: hmm, good question! i don't know, but might be a possible interim solution?
21:50:01 <tdasilva> not everything in one big package
21:50:24 <clayg> i never checkout my own branches, once it's in gerrit i only get it from `git review -d XXX` so I don't even really "notice" - it's not like there's a tracking branch non-ff merge sorta worry
21:50:28 <joeljwright> timburke: 310075 looks dead useful, I'll get eyes on it
21:50:36 <kota_> timburke: er, the reason I added WIP to it is because I was thinking about how to get merged that smoothly but in your version, we can do it really easy so +1 to remove the WIP
21:51:01 <clayg> acoles: +1 on the rehash patch 402043 - top priority - that is so frustrating!
21:51:53 <clayg> timburke: I'd rather work on merging pyeclib to liberasurecode and dodge the issue of trying to test pyeclib with a matrix of liberasurecode versions
21:52:05 <acoles> clayg: uhuh
21:52:05 <timburke> clayg: fair enough
21:53:05 <tdasilva> If no one is interested, I could start looking at merging pyeclib and libec
21:53:27 <clayg> tdasilva: do it do it do it
21:53:35 <cschwede> fix it!
21:53:40 <notmyname> tdasilva: I think you just volunteered ;-0)
21:53:42 <kota_> clayg: i like the idea to merge pyeclib to libec as a binding but still am thinking of how to package the binding to pypi repo or with c binary
21:54:02 <kota_> perhaps someone has great idea :P
21:54:11 <notmyname> kota_: I'll bet tdasilva has a great idea
21:54:13 <tdasilva> one of my concerns are the current proposed patches on gerrit
21:54:30 <clayg> kota_: what is normally _pyeclib.so just has all the liberasure up in it IMHO
21:54:53 <notmyname> tdasilva: worst case, we'd abandon and repropose. but something to consider
21:55:08 <tdasilva> kota_: i'm going to start by checkign with zaitcev and onovy on the packaging question
21:55:15 <cschwede> kota_, you mean shipping the pypi pkg with the C source? that's totally possible IIRC
21:55:23 <kota_> clayg: may be. so removing liberasurecode.so. sounds working.
21:55:25 <tdasilva> but i'm pretty certain it is at least possible to have multiple packages
21:55:28 <clayg> cschwede: pyeclib already has c code in it
21:55:43 * onovy is here
21:55:54 <notmyname> onovy: go to bed ;-)
21:55:58 <tdasilva> lol
21:55:59 <onovy> notmyname: i'm in bed :)
21:56:01 <clayg> I think pyeclib just goes away "this was a bad idea, but technically liberasurecode can comiple to .so - and this could sorta work; but it's not maintained"
21:56:02 <notmyname> lol
21:56:09 <notmyname> onovy: we've got a packaging question, but it's not urgent
21:56:16 <onovy> np, just ask
21:56:19 <clayg> liberasure code just grows some CPython code and a setup.py
21:56:20 <timburke> tdasilva: on patches, libec looks clean. most of the pyeclib ones are mine, and a lot of them have to do with the fact that they're two repos
21:56:24 <notmyname> onovy: wondering about combining pyeclib and liberasurecode
21:56:29 <clayg> we publish python-liberasurecode and call it a day
21:56:33 <onovy> "combining"?
21:56:38 <cschwede> clayg: yes, but i was thinking about including libec, and that should work fine too (?)
21:56:43 <notmyname> onovy: like, how much pain does that cause for distros.
21:56:49 <timburke> i can apply them again wherever you want them. or just merge them before it matters :-)
21:56:56 <notmyname> onovy: yep. combining. one code repo
21:57:08 <onovy> notmyname: two projects is better imho
21:57:09 <notmyname> onovy: not sure if one deliverable or multiple yet. tdasilva will figure that out for us
21:57:14 <onovy> liberasurecode is used by other projects
21:57:18 <onovy> without python wrapper
21:57:38 <notmyname> the idea is to have the language bindings in the same project. python, future golang, etc
21:57:50 <onovy> ah
21:57:54 <onovy> yep, it's possible, but not needed
21:58:07 <notmyname> onovy: well, it would really help on the dev side
21:58:36 <onovy> it will make packaging only little bit harder
21:58:47 <onovy> because you join it and i will split it :)
21:58:55 <notmyname> heh
21:59:04 <onovy> because some projects should want C binding only
21:59:06 <notmyname> onovy: but like I said, it's not time-critical to figure out in the next 1 minute. but you and tdasilva should talk about it at a more convenient time
21:59:17 <clayg> lots of source repos produce seperate packages
21:59:31 <onovy> notmyname: ok
21:59:32 <notmyname> we're at full time for the meeting
21:59:35 <onovy> clayg: yep, np
21:59:43 <notmyname> thank you everyone for coming
21:59:48 <notmyname> thank you for working on swift
21:59:48 <clayg> i think this workflow makes more sense than two independent repos that must always and forever be updated/repackaged together at the same time and track their dependencies very closely
21:59:49 <tdasilva> clayg: exactly
22:00:00 <onovy> clayg: sounds good
22:00:04 <clayg> pull new source; build; you get n+ packages
22:00:08 <notmyname> clayg: +1 from me
22:00:11 <notmyname> #endmeeting