21:00:26 #startmeeting swift 21:00:27 Meeting started Wed Dec 14 21:00:26 2016 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:28 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:30 The meeting name has been set to 'swift' 21:00:33 here :) 21:00:34 who's here for the swift team meeting? 21:00:44 o/ 21:00:47 o/ 21:00:48 eu 21:00:49 o/ 21:00:50 hey 21:00:51 o/ 21:00:52 o/ 21:00:56 o/ 21:01:00 o/ 21:01:08 o/ 21:01:36 welcome everyone 21:02:07 hello 21:02:20 kota_: I was worried for a second (only a second) 21:02:21 :-) 21:02:35 thought you wouldn't make it 21:02:37 :-) 21:02:59 a few things to highlight this week, but first I want to talk about next meetings 21:03:03 #topic next meetings 21:03:18 it's the end of the year, and people will be in and out 21:03:45 we've got 2 weeks left in the year. so the question is, "do we have a meeting during the next two weeks, or do we skip until january 4?" 21:04:26 would anyone like to advocate for either having meetings or not? 21:04:40 i'm fine with skip 21:04:56 see you next year suckers! 21:05:16 I'm around for the next 2 weeks, so happy to still do them, but not having them means sleep in.. so I'm really good either way ;) 21:05:23 heh 21:05:44 either works to me 21:05:45 I can be here next week, after that I'm travelling 21:05:55 I won't be around for next two weds 21:06:16 well, the other question is if there's something that will be happening in the next 2 weeks that will need a formal meeting time to discuss 21:06:37 ok, based on the responses, let's skip until January 4, 2017 21:06:42 kk 21:06:50 kk 21:06:50 #agreed skip meetings until January 4 21:07:04 #info skip meetings until January 4 21:07:10 * notmyname doesn't know meetbot 21:07:24 #topic FYI things happening in broader openstack 21:07:57 there's been a lot going on, even just this week, that I want to highlight (more than usual) 21:08:04 oh... 21:08:06 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:08:26 FWIW, there's a new #openstack-meeting-5. so lurk there if you like to lurk in meeting rooms 21:08:55 FYI, the openstack individual board of directors election is coming up 21:09:01 you can see the candidates at https://www.openstack.org/election/2016-individual-director-election/ 21:09:04 linked from there 21:09:21 and the call for presentations for Boston opened today http://lists.openstack.org/pipermail/community/2016-December/001690.html 21:09:42 so if you want to speak at that openstack forum event, start working on your talk submission 21:09:50 ok, those were the fast things 21:09:58 next up... storyboard 21:10:07 I came across this blog post today https://storyboard-blog.sotk.co.uk/ 21:10:35 the summary is that for a few years some people in openstack have been working on a replacement for launchpad that's called storyboard 21:10:57 I didn't realise storyboard was still a thing.. I thought we stopped working on it a while ago.. so there you go. cool its still a thing 21:10:58 except it's not completely a replacement in that it doesn't map 1:1 with what LP does 21:11:05 oh, yeah. it's more than a thing 21:11:46 there's been a private email thread that's been really active recently that's summarized as "ok, so this is about to be a Thing in openstack and everyone will be using it, so what do you think" 21:12:14 storyboard is quite nice, once you get used to it. i used it a bit in the past 21:12:15 point is, well... I'm not sure yet of the point, other than it's a big change that affects everyone's workflow 21:12:32 oh goody 21:12:49 cschwede: awesome! I could use your help then. i tried to figure it out, and found it ... not obvious as to what i was supposed to do 21:13:42 so again, this is an FYI, and I think you can see it in use in some infra projects. if you've got a few minutes, it might be worth a look 21:14:29 and finally, the golang discussions have started again (or continued?) 21:14:42 the story/task thing sounds a bit like Jira 21:14:55 there's a thread on the ML about technical requirements for using golang at http://lists.openstack.org/pipermail/openstack-dev/2016-December/108875.html 21:15:02 notmyname: what's the link to storyboard? 21:15:16 tdasilva: https://storyboard.openstack.org/#!/page/about 21:15:36 and there's a governance proposal about a new TC process to approve not-python being discussed at https://review.openstack.org/#/c/398875/ 21:15:54 acoles: thanks 21:16:50 both of the golang discussions are about the same sort of things. the ML thread is about what's a "release", how to manage dependencies, and what about any common libraries that might be needed 21:17:29 the TC resolution basically asks a lot of those same questions saying "if you're going to use not-python, have an answer for all of these things" 21:17:58 neat 21:18:40 the most controversial part of the TC resolution is towards the bottom where it talks about common libraries. it strongly implies that the first project to use not-python is responsible for writing oslo equivalents in the new language before using the language in the project 21:19:31 but surely the language would need to be approved before anyone would invest time in writing libraries? 21:19:34 notmyname: if you don't start using the common thing from the get-go it's really hard to migrate to it later when it's more fleshed out without your legacy of use-case built in 21:19:38 however, an in-person conversation I had last week said that was more to mean "projects writing stuff in not-python shouldn't expect the oslo team to write stuff for you" 21:20:18 i don't think the idea with oslo was ever to get someone else to write code you needed for you 21:20:54 the fear is that across the many openstack projects, it's taken a very long time to get everyone using the same versions of common libraries. one would have something in-tree, another would have an old copied version of that, another would be using a common library from oslo, etc 21:21:28 and the hope is that when starting with a new language, it isn't set up to repeat that diversification of libraries doeing sortof the same thing all over again 21:22:14 and furthermore, it's "known" than configs and logging are something that needs to be done, regardless of language, so that's why those are called out 21:22:44 sweet 21:22:48 so that's the background to that section 21:23:31 fwiw I did make a start on an oslo config like thing in golang, just as a vehicle for learning the language 21:23:36 I suspect that this proposal will be approved by the TC, but I suspect it will be slightly modified (in what ways, I'm not entirely sure) 21:24:20 I think the point is that new-lang needs to read existing configs and log the same way 21:24:32 (note, that's not actually what's written in the proposal) 21:25:16 well, that makes sense, right? we don't want 2.5 different config styles for one service 21:25:35 cschwede: yes, absolutely 21:26:15 hm...rather than being careful with what i say in a logged channel, I'll keep going ;-) 21:26:22 but isn't oslo.config more about declaring config schema etc? the actual config is just an ini file? 21:26:28 also, it sounds like acoles is already solving that issue with the config;) 21:26:41 unless I am missing something 21:27:17 so if you're frustrated by this being a seemingly good idea when there were two similar proposals earlier this year, you arent' alone 21:27:20 so with oslo.config I imagine the desire will be to have a common developer pattern - the op experience is just an ini file 21:27:27 acoles: i think it's also about auto-generating config samples, using only the settings that your project is using 21:27:52 +1 acoles and design the grand unified theory of swift/keystone/hummingbird/pastedeploy configuration 21:27:55 cschwede: yes, but again that's not s much an op experience as a dev experience 21:28:01 +2 if he solves configuration once and for all for all of opnestack - huzzah 21:28:16 I find this very frustrating that this proposal is seemingly making progress, while one proposed by the swift team was rejected because it was proposed by the swift team 21:28:37 clayg: actually I got as far as being frustrated by lack of generics and gave up! 21:28:42 clayg: JK 21:28:43 but this is the current state of the golang discussion in openstack. so I'm happy to hear that acoles is working on figuring out some basic config stuff in golang! 21:28:44 progress is progress 21:28:49 notmyname: well, i think it has more to do with timing and discussions inbetween. 21:29:10 cschwede: or it could be what notmyname said 21:29:28 porque no los dos? 21:29:43 progress is progress 21:30:29 so all that being said, we've got a large body of work ahead of us related to golang (with little recent progress), and these are the places where we can be involved in that in the larger openstack community 21:31:15 huzzah 21:31:21 and I'd like to transition to talking about that. the stuff going on in swift (starting with golang) 21:31:28 #topic things happening in swift 21:31:43 i hear 2.12 is the best swift ever 21:31:55 oh yeah. releases :-) 21:32:18 stable releases happened. 2.12.0 should happen today or tomorrow, depending on the openstack-releases team 21:32:28 oh, release? 21:32:31 ie i'm plannign on submitting that patch this afternoon 21:32:51 kota_: yeah, needed to get a few things out. last chance this year 21:33:07 notmyname: good call 21:33:11 doo eeeet! 21:33:13 I know this one is a little more "sudden" instead of talking about it for a few weeks first. sorry if it's taking anyone by surprise 21:33:28 but it's a good release. best swift ever, in fact ;-) 21:33:46 so I spent some time this week looking at what's starred, what's on the ideas page, what's being reviewed, etc 21:33:56 spoiler: it's a lot 21:34:00 surprise! 21:34:30 the short list is still pretty intimidating 21:35:00 here's the "big stuff" I wrote down (it's on the meeting agenda) 21:35:12 looks like the container sharding bullet point got sharded :) 21:35:24 or replicated 21:35:35 as it should :P 21:35:47 :) 21:35:49 golang, global ec, policy migration and tiering (and symlinks), part power increase, container sharding, and metadata searching 21:35:53 adn container sharding too 21:36:05 :-) 21:36:11 the list got so long, it needed sharding 21:36:16 no kidding 21:36:21 that's no *so* bad 21:36:34 i feel like most of them are stuff that we've had ongoing 21:36:35 clayg: it's only, like, 1.5 years of work ;-) 21:36:47 container sync, keymaster+barbican 21:36:48 well but maybe some of them are close? 21:37:00 nah, you're right. most (all?) are well on their way 21:37:00 i keep thinking when I go look at symlinks that'll be mostly done 21:37:07 we'll make some sort of progress on golang 21:37:12 composite rings has *been* done 21:37:40 acoles: I specifically left those off. not because they aren't important, per se, but because somethign had to be dropped. I'd still like to see those land 21:37:41 policy migration and auto tiering are the future - the elasticsearch sync stuff could be a door opener too 21:37:46 increase part power is basically done 21:38:00 global ec can make progress before it's "done" 21:38:00 yeah, composite rings, part power, ec replication are all pretty close 21:38:03 this is a great list 21:38:11 notmyname: yep, and they are more incremental than others 21:38:32 this is great 21:38:34 clayg: absolutely! i just need to work on the comments by Pete and you can merge it :) 21:38:42 cschwede: ack 21:38:50 notmyname: would it make sense to prioritize that list somehow?? not sure if it's feasible or not 21:38:53 we've all realized that one of the problems is that we've got a huge list. this is my attempt at focusing in down 21:38:58 i'm still waiting to get merged :-) 21:39:11 yeah! first priority is to merge kota_'s stuff! 21:39:13 ;-) 21:39:31 speaking on sharding, I've moved a bunch (not the sharding itself) but interacting with shards into the proxy which has cleaned up some not so pretty code, and will make clayg happier. but it does add a little overhead. but so far its better. Although I've been debugging some of that in my benchmarks. But I think its worth it. 21:39:35 tdasilva: it could be prioritized to some extent, but also we can do several in parallel 21:41:09 what I'd like to do next, myself, is organize that list a little more to show what's going on in each 21:41:35 and I think for everyone, I'd ask that you prioritize reviewing things on this list 21:41:50 it's good to know what's close and what still might need to have scope trimmed 21:41:52 (obviously, stuff comes up, like the fun with ec 2 weeks ago) 21:42:09 it's also good to know when something is close who is going to rally to push it over the finish line 21:42:20 yep, exactly 21:43:17 I'll associate patches with each of these so we can see that 21:43:59 as a first pass, i'll update the priority reviews page. ie before I invent a totally new (and awesomer) dashboard 21:44:23 so.... 21:44:33 #topic open discussion 21:44:43 anythign else to bring up or discuss this week? 21:45:02 so is the kind of thing that story board should help with - tracking patches and progress under topics? 21:45:04 isa-l-rs-cauchy is the best ec thing evar 21:45:07 would love people to look at https://review.openstack.org/#/c/365371 if they have time 21:45:09 acoles: I have no idea 21:45:13 i could use some eyes on https://review.openstack.org/#/c/310075/ (joeljwright, maybe?) 21:45:27 oh! there's a couple of things I'd like to see land in swiftclient (like the new tempurl support) this week and then do a release of that next week 21:45:31 we should merge pyeclib into liberasurecode (and future golang bindings should *also* ship inside of liberasurecode) 21:45:44 clayg: I like that idea 21:45:45 timburke: kk 21:45:59 joeljwright: I keep seeing that get updated 21:46:43 oh yeah, kota_: are you on board with what i pushed over at https://review.openstack.org/#/c/405929/ ? 21:46:46 clayg: the updates recently have been rebase/fixes, I believe it's 'done' now 21:46:50 FWIW the one timburke mentioned (https://review.openstack.org/#/c/310075/) is getting some priority inside of swiftstack, so we'll likely try to land that soon 21:46:50 clayg: we should do that probably around the same time the next big release is coming (Ocata) 21:46:53 joeljwright: timburke if normally my goto "crazy new client/api feature awesome sause!?!!" - but it looks like he's already looking at it? 21:47:27 clayg: otherwise it might get messy for distributors if we switch to a single pkg 21:47:35 timburke: yeah, sorry no response but that last rev looks super great to me. 21:48:01 cschwede: interesting. so when, exactly woudl that be? ie why later as opposed to now? 21:48:03 clayg: I had a play with your concurrent gets testing patch and pushed up a new verson, hope that's ok. 21:48:10 we should get this in shape and merged https://review.openstack.org/#/c/402043 Optimize hash calculation when suffix hash invalidated 21:48:12 and we should probably do https://review.openstack.org/#/c/407829/, maybe add a gating job for it... 21:48:13 from their perspective, aren't we already working on ocata 21:48:48 commit message on patch 310075 seems reasonable - i'll star it at least - didn't realize it was a priority 21:48:55 cschwede: would we have a single package? 21:49:05 notmyname: well, i'd like Pete's view on this; but if we ship fixes for stable versions and switch from two to one package it might get more difficult for packagers 21:49:05 lol @ "here's some priorities" followed by a flurry of patches not about those ;-) 21:49:12 tdasilva: that was my understanding 21:49:14 cschwede: i think we could still hae multiple packages, no? 21:49:16 kota_: i'm not sure, but maybe we can drop the 'WIP'? not sure what to do about tests though... 21:49:22 mattoliverau: of course - that's fine - always push over me 21:49:26 cschwede: yeah, definitely should get input from pete and onovy. good call 21:49:56 cschwede: one repo, one release, but i think we would want multiple packages (libec, python-binding, golang binding) 21:49:59 tdasilva: hmm, good question! i don't know, but might be a possible interim solution? 21:50:01 not everything in one big package 21:50:24 i never checkout my own branches, once it's in gerrit i only get it from `git review -d XXX` so I don't even really "notice" - it's not like there's a tracking branch non-ff merge sorta worry 21:50:28 timburke: 310075 looks dead useful, I'll get eyes on it 21:50:36 timburke: er, the reason I added WIP to it is because I was thinking about how to get merged that smoothly but in your version, we can do it really easy so +1 to remove the WIP 21:51:01 acoles: +1 on the rehash patch 402043 - top priority - that is so frustrating! 21:51:53 timburke: I'd rather work on merging pyeclib to liberasurecode and dodge the issue of trying to test pyeclib with a matrix of liberasurecode versions 21:52:05 clayg: uhuh 21:52:05 clayg: fair enough 21:53:05 If no one is interested, I could start looking at merging pyeclib and libec 21:53:27 tdasilva: do it do it do it 21:53:35 fix it! 21:53:40 tdasilva: I think you just volunteered ;-0) 21:53:42 clayg: i like the idea to merge pyeclib to libec as a binding but still am thinking of how to package the binding to pypi repo or with c binary 21:54:02 perhaps someone has great idea :P 21:54:11 kota_: I'll bet tdasilva has a great idea 21:54:13 one of my concerns are the current proposed patches on gerrit 21:54:30 kota_: what is normally _pyeclib.so just has all the liberasure up in it IMHO 21:54:53 tdasilva: worst case, we'd abandon and repropose. but something to consider 21:55:08 kota_: i'm going to start by checkign with zaitcev and onovy on the packaging question 21:55:15 kota_, you mean shipping the pypi pkg with the C source? that's totally possible IIRC 21:55:23 clayg: may be. so removing liberasurecode.so. sounds working. 21:55:25 but i'm pretty certain it is at least possible to have multiple packages 21:55:28 cschwede: pyeclib already has c code in it 21:55:43 * onovy is here 21:55:54 onovy: go to bed ;-) 21:55:58 lol 21:55:59 notmyname: i'm in bed :) 21:56:01 I think pyeclib just goes away "this was a bad idea, but technically liberasurecode can comiple to .so - and this could sorta work; but it's not maintained" 21:56:02 lol 21:56:09 onovy: we've got a packaging question, but it's not urgent 21:56:16 np, just ask 21:56:19 liberasure code just grows some CPython code and a setup.py 21:56:20 tdasilva: on patches, libec looks clean. most of the pyeclib ones are mine, and a lot of them have to do with the fact that they're two repos 21:56:24 onovy: wondering about combining pyeclib and liberasurecode 21:56:29 we publish python-liberasurecode and call it a day 21:56:33 "combining"? 21:56:38 clayg: yes, but i was thinking about including libec, and that should work fine too (?) 21:56:43 onovy: like, how much pain does that cause for distros. 21:56:49 i can apply them again wherever you want them. or just merge them before it matters :-) 21:56:56 onovy: yep. combining. one code repo 21:57:08 notmyname: two projects is better imho 21:57:09 onovy: not sure if one deliverable or multiple yet. tdasilva will figure that out for us 21:57:14 liberasurecode is used by other projects 21:57:18 without python wrapper 21:57:38 the idea is to have the language bindings in the same project. python, future golang, etc 21:57:50 ah 21:57:54 yep, it's possible, but not needed 21:58:07 onovy: well, it would really help on the dev side 21:58:36 it will make packaging only little bit harder 21:58:47 because you join it and i will split it :) 21:58:55 heh 21:59:04 because some projects should want C binding only 21:59:06 onovy: but like I said, it's not time-critical to figure out in the next 1 minute. but you and tdasilva should talk about it at a more convenient time 21:59:17 lots of source repos produce seperate packages 21:59:31 notmyname: ok 21:59:32 we're at full time for the meeting 21:59:35 clayg: yep, np 21:59:43 thank you everyone for coming 21:59:48 thank you for working on swift 21:59:48 i think this workflow makes more sense than two independent repos that must always and forever be updated/repackaged together at the same time and track their dependencies very closely 21:59:49 clayg: exactly 22:00:00 clayg: sounds good 22:00:04 pull new source; build; you get n+ packages 22:00:08 clayg: +1 from me 22:00:11 #endmeeting