21:01:00 #startmeeting swift 21:01:01 Meeting started Wed May 4 21:01:00 2016 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:01:02 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:01:06 The meeting name has been set to 'swift' 21:01:07 who's here for the swift team meeting? 21:01:16 o/ 21:01:17 o/ 21:01:18 o/ 21:01:18 * torgomatic is just here for the snacks 21:01:19 hello! 21:01:20 me 21:01:24 o/ 21:01:28 o/ 21:01:28 oh wait are there snacks? 21:01:29 snacks, where? 21:01:30 hello 21:01:37 hi 21:01:42 here 21:01:48 o/ 21:01:49 ...did you guys not bring snacks? i brought snacks 21:02:01 timburke: did you bring enough to share with everyone? 21:02:02 ah, found some chips 21:02:08 o/ 21:02:14 acoles probably has some topics 21:02:14 o/ 21:02:18 notmyname: if you can get here, probably 21:02:27 mmm, topics ... 21:02:37 not on pay day 21:02:43 ha! 21:02:43 *groan* 21:02:59 acoles: so yuri had some snacks.... 21:03:07 GROAN 21:03:24 ok, let's get started :-) 21:03:29 agenda is at 21:03:34 wait, is yuri on board? 21:03:34 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:04:04 hi 21:04:20 #topic post summit 21:04:27 so we had a summit last week 21:04:34 I though it was pretty great. what'd you think? 21:04:53 was great :) 21:04:55 (a few topics to cover, but I'd love to hear any feedback first) 21:05:24 it was great! :) 21:05:43 so...much...meat... 21:05:51 good stuff. yep 21:05:53 timburke: +1 21:06:01 maybe more chairs 21:06:14 was good meeting newcomers in person 21:06:26 bkeller`: +1 recurring problem 21:06:34 acoles: +1 21:06:59 how was the balance of conference talks to fishbowl sessions (ops/community feedback) and the working sessions? 21:07:11 anything we need to focus on or do differently next time? 21:07:19 was a shame to lose people during the working sessions 21:07:20 did we have to much time or to little time? 21:07:34 no, I think it was a great mix 21:07:41 joeljwright: +1 - too many conflicts this time 21:07:41 op sessions were good, and fishbowl format seemed to be ok for them 21:07:44 joeljwright: because of the conference stuff? 21:07:47 meeting to fish bowls i mean 21:07:58 meetings to fish bowls was a good mix 21:08:11 hello - sorry i'm late 21:08:12 i think some of the stuff in the meetings could have been moved to friday before everyone split up 21:08:15 notmyname: having *two* fishbowls on ops (because of ops summit) was nice - there's always stuff to discuss there 21:08:25 joeljwright: same for me, to many schedule conflicts 21:08:26 but the talks overlapping so much with the working sessions was a shame 21:08:35 who would have thought 2 ops sessions where completely taken up.. that's awesome 21:08:50 the overlaps are always an issue, to some extent 21:08:54 +1 fishbowls for ops good 21:08:58 notmyname: won't be anymore! 21:09:07 we'll see. TBD I think 21:09:09 the working sessions were the most informal/unstructured we've had at any summit 21:09:12 notmyname: imho we did not have too little time. did we even get to composite tokens? sharding? 21:09:17 It's a tough week, I propose that we change the summits to one day a week for a month. 21:09:20 acoles: nope 21:09:25 notmyname: yeah the working sessions felt pretty hack-a-thon-y 21:09:32 clayg: in a good way? 21:09:58 redbo: or "meet from 12-4, then again from 9pm to 1am?" 21:10:24 If the summits could be from 8pm to 4am, that would help me too. 21:10:25 notmyname: it's fine for us? some people seem to have maybe struggled with lots going on, lots of people talking about different things, no way to "drop-in" to swift working session for a specific topic and still make it out to other teams more "scheduled" stuff 21:10:26 I though thursday breaking up was good. IMHO, friday was too loose. for example, there was nothing captured on etherpad for that day at all. 21:10:56 ^ 21:11:05 i guess because friday, i didnt know where people went. i only know about the groups i was in 21:11:22 jrichli: we should probably just organizie our friday's like we do the other days - it's supposed to be different - but we sorta know how we like to work when we get together - just a little structure 21:11:27 was that different that the firday at previous summits? 21:11:28 anyway - it was great 21:11:38 clayg: that's a great idea 21:11:50 notmyname: no, friday summit *used* to be the closest we get to hack-a-thons 21:11:59 friday at tokyo was better - agian, cause i could at least see the different groups and find out what was being discussed if i wanted to 21:12:05 ok 21:12:09 notmyname: but what we really want is somewhere in the middle - and I think wednesday afternoon and thursday we hit it 21:12:27 clayg: +1 21:12:29 i'm trying to remember friday tokyo 21:12:38 clayg: +1 21:12:38 clayg: my thought exactly :) 21:12:47 I think I went to robot restaurant 21:12:48 it was a tiny room, so some left - but sort of lingered around the room 21:12:57 while I really like the "multiple conversations at once" think (despite bad acoustics), we had some whole-group discussions with slightly more group discussion and structure 21:13:07 clayg: i don't think you can remember friday tokyo ;) 21:13:16 my last comment was about tokyo 21:13:24 tdasilva: yeah, we kept wondering where he and paul were ;-) 21:13:28 hehe 21:13:29 jrichli: oh right - i remmeber a lot of hall way at toyko - was the friday in the same throne room? we talked about keymasters in the hall? 21:13:31 friday tokyo was like speed dating meets swift topics 21:13:49 yep, keymaster in the hall ... wiht the knife 21:14:08 jrichli: has a thing about knives ;) 21:14:15 lol 21:14:17 lol. i was thinking of Clue 21:14:32 jrichli: acoles' people probably have a different name for it 21:14:51 jrichli: and he probably invited the game (but didn't get credit) 21:14:52 we do. whatever it might be. 21:15:00 yeah the game is cluedo 21:15:03 clayg: lol 21:15:04 I can tell we all jsut spent a week together. so many inside jokes 21:15:06 mattoliverau: yeah that's it! 21:15:22 good times 21:15:24 ok, so austin formats... 21:15:26 I got several knives in tokyo, I like them. 21:15:39 let's try to do more like wed afternoon and thursday 21:15:50 what's up with texas and knives 21:15:58 jrichli: clayg OIC I get it now, in the hall... 21:16:17 redbo: ill have to check it out *if* i happen to go to San Antonio soon. acoles :-) 21:16:25 (or at least the parts of thursday that weren't the multi-hour goland discussion? ;-) 21:16:42 notmyname: we always seem to mis-fire on our first session, then get going, maybe next time we should try to remind ourselves how to kick start the sessions 21:16:58 remove a table! 21:17:00 yeah 21:17:03 jrichli: yes! 21:17:42 shall we move on to specific post-summit follow up stuff now? 21:17:45 so whatever context we are in, first remove a table. sounds like a good axiom and I like those. 21:18:10 acoles: might be awkward at dinner 21:18:18 rofl 21:18:22 rofl too 21:18:27 lol. could be abstracted to removing barriers 21:18:29 I would appreciate it if all table jokes going forward were updated to the appropriate SQL syntax... 21:18:32 (╯°□°)╯︵ ┻━┻ 21:18:43 #topic post-summit: no more specs 21:18:45 it's too much 21:19:06 one thing we discussed at the summit is how specs are (not) working for us 21:19:15 redbo: so great 21:19:16 here's the email I'm going to send 21:19:17 #link https://etherpad.openstack.org/p/ZHWszY0GuN 21:19:33 but first I need to copy over the existing open specs to a wiki page 21:19:57 once that's done (this afternoon?) then I'll send that email to the -dev mailing list 21:20:19 great 21:21:07 I like it 21:21:15 :-) 21:21:24 notmyname: i like the postive message - i think the problems you've identified with specs-as-code-review are sort of universially accepted 21:21:48 also, it's pretty close to your original wording? ;-) 21:22:20 it's definitely the result of a group effort. thank you for your help on it 21:22:21 notmyname: but this message has a hint of hopeful ignorence - there's a lot of stuff we're going to have to "figure out" that's not listed there - esp. the note about "project management planning" that review-landed-specs were sort of granfathered into 21:22:31 notmyname: love the 'where hope goes to die" part! 21:22:39 mattoliverau: that was fun to write :-) 21:23:05 clayg: yeah, definitely. and jrichli has brought up some very valid concerns related to discoverability of previous conversations 21:23:05 don't forget 'facilitate communication without encouraging despair' 21:23:10 notmyname: it's probably fine to broadcase the new reality - but the nuanced situation on the ground is we're moving a lot closer back to square one - despite a good attempt at specs - we're still trying to figure it out - and for now we want less rules not more 21:23:11 another choice phrase :D 21:23:40 clayg: yep. my feelings exactly 21:24:00 yeah, i was wondering if that should be made mroe clear, that we are still just trying to figure out... 21:24:23 if anyone has further comments, feel free to reach out to me publicly or privately 21:24:44 also, do you think it would be helpful to point out to folks that when writing specs it is usefult to use a tool that makes providing comments back easy 21:25:00 the mail LGTM 21:25:23 tdasilva: for now, less structure the better IMO. let's address that if we actually have people who are using tools that don't encourage feedback 21:25:34 tdasilva: i'm not sure we know that for sure - if the easiest way to discuss is a spec is to pick up a phone/skype/hangout - maybe that's an imporvement over comments on a etherpad/wiki/gerrit ? 21:25:54 notmyname: +1 21:26:17 notmyname, clayg: good points 21:26:23 #topic post-summit: crypto merge plan 21:26:29 oh snap 21:26:29 acoles: jrichli: wheeee! 21:26:37 good luck 21:26:43 is https://trello.com/b/63l5zQhq/swift-encryption up to date? 21:27:01 notmyname: reasonably up to date and jrichli has added more this week 21:27:05 greap 21:27:09 *great 21:27:14 should be, for the most part. there were some items disucssed last week - i think we tried to capture those 21:27:32 will look at that in the next couple days 21:27:37 and I wrote an overview doc draft which will help as an on-ramp I hope for reviews. it sould be a spec , but... 21:27:45 ok. looks like a lot in the todo column 21:28:06 https://review.openstack.org/312096 21:28:26 https://review.openstack.org/#/q/status:open+project:openstack/swift+branch:feature/crypto 21:28:30 notmyname: make sure you are looking at "reduced scope" column 21:28:48 jrichli: yeah, that's the longest one! 21:29:00 :-( oh. 21:29:19 i guess it grew from last week. believe it or not, that is progress in a way 21:29:26 with the current crypto patches open, where's the place to start? 21:29:58 https://review.openstack.org/#/c/305794/ is next to merge 21:29:58 jrichli: patch 305794 - swift (feature/crypto) - crypto - only set body crypto meta if body is read... 21:30:17 ok 21:30:22 notmyname: swift priority review page 21:30:37 ah. perfect 21:31:19 timburke: thanks for the review 21:31:27 notmyname: there are a few "cleanup" patches in a chain from patch 305794 which I'd like to merge soon 21:31:27 acoles: https://review.openstack.org/#/c/305794/ - swift (feature/crypto) - crypto - only set body crypto meta if body is read... 21:31:44 and so patch 156923 is what we need outside fo the crypto branch 21:31:44 notmyname: https://review.openstack.org/#/c/156923/ - swift - Refactor server side copy as middleware 21:32:10 notmyname: oh yeah, of course if we have review cycle please review copy middleware ^^ 21:32:22 will help so much to get that on master 21:32:35 ok 21:32:52 yes. tdasilva said he was going to upload a patch soon to address the recent comments 21:33:07 tdasilva has the ball 21:33:19 acoles: and did you already pull out the BYOK stuff? I don't think I realized that it was actually done (or alternatively that it was so relatively small) 21:33:22 almost there 21:33:26 tdasilva: thanks 21:34:18 notmyname: not yet, its not huge 21:34:27 ok 21:34:32 did we decide to pull it out now for sure? i thought we were maybe going to see if we could find solutions to potential issues now 21:34:45 the etherpad says we did :) 21:34:49 :-) 21:34:52 boom! 21:34:54 etherpad don't lie 21:34:57 but I saw a video of a demo where it worked! 21:34:57 ah, ok. lol 21:35:04 i know, right? 21:35:05 I thought no BYOK for initial merge, it'll be a follow up to master 21:35:15 right. that's what we said last week 21:35:24 ok 21:35:30 to keep scope smaller to focus on getting something done rather than everything done all at once 21:35:49 and having to potentially define all the areas that wont work with it 21:36:16 and IMO also because jsut getting cluster-key encryption is a lot easier that trying to describe to users "if you do it this other way, some stuff probably breaks" 21:36:31 mattoliverau: yep 21:36:32 mattoliverau: that's the biggest issue I have with it 21:36:32 yep, much as I'd like to have it there, I agree with John that we shoot for minimum useful feature set to get on master and then improve from there. 21:36:36 i acctually had some thoughts on how a byok scheme might work with an integrated keyserver & auth & service accounts - and it could potentially allow some neat features for users (secure erase) - but I didn't want to bring it up because I didn't want to distract from storage-service-managed keys 21:36:41 acoles: +1 21:36:53 clayg: cool 21:36:58 notmyname: exactly, BYOK could drag us into pondering too many unknowns 21:37:07 clayg: write it down somewhere for later :-) 21:37:10 ok, looking at the time, let's move on to the next topic 21:37:14 clayg: topic -- link 21:37:19 it's a little more than MVP tho - there's a bunch of caveats with a pure BYOK (cluster only has access to decrypted data when user hands them a key) 21:37:46 #topic feature/hummingbird plan 21:37:54 zomgreplication! 21:37:55 the other biggest topic of the week... 21:38:07 and all the buzz on the ML! 21:38:13 summary of what we talked about last week and the plan for hummingbird within swift: 21:38:16 #link http://lists.openstack.org/pipermail/openstack-dev/2016-May/093680.html 21:38:40 which prompted the TC to ask for a governance resolution, thus resulting in this email 21:38:42 #link http://lists.openstack.org/pipermail/openstack-dev/2016-May/093795.html 21:38:48 (and subsequent replies) 21:38:54 and this patch https://review.openstack.org/#/c/312267/ 21:38:55 notmyname: patch 312267 - governance - update approved programming languages to include go 21:39:17 notmyname: you've been busy 21:39:26 so lots of stuff going on there right now. 21:40:04 the biggest questions are around the CI/gate support. eg how to reference dependencies and stuff like that. there's been a little bit around sharing code and all that 21:40:23 but the good news is that there's at least one other project that's already also looking at having a go component too 21:40:42 * clayg suspects redbo was involved ^ 21:40:43 (designate -- the dns service thing) 21:40:57 huh- i didn't know that 21:40:59 redbo loves dns 21:41:02 lol 21:41:15 point is, there's lots of questions to answer, and not just about what we do with our own codebase 21:41:27 iirc one of the keynotes, Intel also ported (part of?) Nova internally, so there is some interest in other projects as well 21:41:28 the good news is that we've got a little while to figure some of this out 21:41:52 cschwede: yeah. they did. but they kinda did it completely separately from the rest of the community, at that hasn't gone over well 21:42:08 notmyname: yep, indeed 21:42:10 redbo loves community 21:42:16 I'm actually hoping that doesn't "poison the well" about this stuff 21:42:18 :D 21:42:35 I haven't really seen it, but I did watch one episode where they play D&D and it was pretty good. 21:43:08 lol, great comeback! 21:43:09 notmyname: you should just add the header "adapt or die" to help grease the wheels 21:43:19 notmyname: so... in the meantime 21:43:27 dfg_: yeah, ultimatimes are awesoem 21:43:29 dfg_: good one! 21:43:29 ok, so I'm hoping that while acoles and jrichli and team are getting the last crypto patches landed, we can answer the questions specifically about the minimal feature set that we want to land 21:44:06 then we spend the rest of the cycle actually getting that ready to land and helping the rest of the openstack community figure out the other questions 21:44:30 and that's the plan 21:44:34 #topic post-summit: other stuff 21:45:02 client updates? tiering? symlinks? sharding? other stuff was discussed or should have been discussed and there's always a ton going on 21:45:02 notmyname: did you get anywhere with docs versioning? 21:45:08 any quick updates from those? 21:45:14 joeljwright: nope, not yet 21:45:16 i will say i've been surprised about how receptive things have been so far. when i think back to all the flame wars on openstack dev over the years... 21:45:25 but anyway... 21:45:49 I got a brain dump from hrou on symlinks and plan to get together a wiki or something that summarizes history and where we are at now 21:46:09 jrichli: sounds great. 21:46:11 jrichli: thank you for taking over on that one! :-) 21:46:26 jrichli: just encrypt that and throw away the key until after your *other* huge feature lands!? 21:46:36 :D 21:46:38 jrichli, thank you 21:47:02 I'm not super happy that if people try things out on their own, like if it's something they don't even know if it will work, everyone screams "4 opens!" and shuns them forever. 21:47:28 I pulled Sam and John aside thurning Thursday sessions and talked about sharding problems I had.. Sam gave me enough to go on. I really also need to do some benchmarking (IBM guys were doing that but have disapeeared). 21:47:54 redbo: you have a point there 21:48:02 i've been talking to searchlight about notifications. hopefully i'll have a plan put together by next week for feedback 21:48:07 mattoliverau: remember mmotiani is willing to help! 21:48:10 redbo: well idk about the shuns forever part - but there's something to be said about failing openly 21:48:26 jrichli: i wrote some stuff down on the plane about what clayg and I and maybe others discussed about symlinks 21:48:36 it is hard to make the transition from private to public though 21:48:38 mattoliverau: I can help 21:48:39 hard to let go 21:48:42 pdardeau: mmotiani_ great! 21:48:49 jrichli: he's only saying that so he can document how wrong I was later when he has a better idea 21:48:51 acoles: great! where? 21:49:11 jrichli: acoles: seat-back pockent of 33D 21:49:16 :D 21:49:32 notmyname: sounds about right! 21:49:33 on paper :( battery was dead. and I'd also like us to write down once and for all why a POST cannot update two resources - what we went through on a whitebard in Austin 21:49:33 lol 21:49:36 33D? at least he had an aisle seat! 21:49:44 notmyname: not far off 21:49:47 acoles: yes. so we don't have to do it yet again! 21:49:59 can jrichli go look for that whiteboard 21:50:00 acoles: I had a good discussion here on Monday of lessons learned with tiering in gluster and ceph, need to find a way to share and continue discussion... 21:50:06 acoles: you mean toyko - or maybe you mean austin hackathon '15 21:50:18 clayg: those too ;) 21:50:28 ok...gotta move on 21:50:37 tdasilva: oh that's interesting... 21:50:45 clayg: actually I'm on board with your thinking 21:50:48 if you're thinking about something, write it down and link to it on https://wiki.openstack.org/wiki/Swift/ideas 21:50:53 I talked about high-level overview of composite ring with pdardeau in Thu morning, and probably we need more continuous discussin for implementation secific side in this cycle. 21:51:12 #topic rolling upgrade test status 21:51:13 notmyname: exactly. 21:51:22 cschwede: you're still waiting on patch 297311 to land 21:51:23 notmyname: https://review.openstack.org/#/c/297311/ - openstack-infra/devstack-gate - Run swift services on subnode 21:51:27 cschwede: that's the blocker right? 21:51:31 clayg: maybe we need a failed ideas wiki page too 21:51:35 notmyname: no, the blocker is https://review.openstack.org/#/c/304465/ 21:51:35 cschwede: patch 304465 - openstack-infra/devstack-gate - Use subnodes for Swift storage nodes in a multinod... 21:51:45 redbo: it's a large set :\ 21:51:52 cschwede: ok, thanks 21:51:54 jrichli, acoles, tdasilva I'm interested in being a part of further understanding and discussing symlinks.. Was a part of most of the discussions at the summit.. 21:52:15 notmyname: i might abandon the other one, need to re-check. but patch 304465 is the important one 21:52:15 cschwede: https://review.openstack.org/#/c/304465/ - openstack-infra/devstack-gate - Use subnodes for Swift storage nodes in a multinod... 21:52:49 notmyname: i will ping infra beginning of next week, asking for comments on it 21:52:50 cschwede: ok. etherpad needs updating 21:52:51 ntata: great! we will keep you informed 21:52:59 cschwede: great. let me know where I can help 21:53:04 jrichli, thanks! 21:53:06 ok, last topic 21:53:06 ntata: sure, hopefully jrichli can keep us all in the loop somehow 21:53:10 #topic notmyname needs help 21:53:17 acoles, +1 21:53:34 while we could spend a *long* time on this topic... 21:53:39 lol 21:53:44 :) 21:53:48 i have to pee 21:53:56 there are 2 things specifically I'd like to get help with 21:54:00 cherish that, clayg 21:54:12 first, does anyone remember gholt's ring docs? 21:54:17 oh. that is one for my quotes compilation 21:54:38 notmyname: I do! 21:54:39 turns out they disappeared from the internet, but I talked to him and he found an old copy on the wayback machine 21:54:55 notmyname: that was back from before I hated rings with a burning passion of a thousand suns - good times 21:55:01 notmyname: yay! 21:55:09 and greg gave me a pdf version of it and his blessing to include it upstream in swift's doc tree 21:55:21 so I'ld like help from someone to convert it to .rst and propose it to swift 21:55:30 notmyname: i'll take it 21:55:33 whoot! we need to add a section on stable rebalancing tho - it was always decidely lacking from the blog series 21:55:37 pdardeau: thanks. I'll email it to you 21:55:44 next thing... 21:56:16 I've been promising to migrate the pyeclib/liberasure code repos into the openstack namespace (yay big tent), but I obviously haven't done this yet 21:56:24 ohyeah 21:56:31 yup 21:56:34 I've looked at some of the docs on how to do it, but I havne't had time to sit down for a few hours and get it all figured out 21:56:56 I'd love for someone to take point on this. I'll be happy to help, but I'm admitting defeat on getting it done myself 21:57:36 I can do it 21:57:40 mostl it will be walking through the -infra guide and updating yaml files and doing git commands (I think) 21:57:43 * tellesnobrega is away: I'm busy 21:57:43 tdasilva: thank you! 21:58:06 tdasilva: I can email you the summary and links to what I've found 21:58:14 notmyname: sounds good 21:58:20 pdardeau and tdasilva thank you very much 21:58:31 #topic open discussion 21:58:38 anythign else in the last 2 minutes to discuss? 21:58:52 tdasilva: do you have any experience with the stackforge/openstack-affiliated project stuff? 21:59:08 In my jeglagged state, I've been looking at Andy's Openstack-Ansible swift repo, that also is used to inst 21:59:08 all swift into containers and run functional tests. And attempting to turn it into another SAIO that simul 21:59:08 ates the world better.. watch this space.. i hope. 21:59:18 clayg: a while while back we had to move swiftonfile to stackforge and then to openstack 21:59:23 kota_: wouldn't this be basically the same thing you did with swift3? 21:59:28 mattoliverau: cool 21:59:35 wow, nice splitting of a line irc client 21:59:36 clarkb: iirc, stackforge is in archive mode 21:59:43 eh, sorry, clayg ^^ 21:59:53 clayg: ah, i was missing the recent last comment in meeting 21:59:59 cschwede: the stackforge namespace is archived but there are non openstack project hosted under openstack/ 22:00:05 sounds like tdasilva is an expert 22:00:15 kota_: notmyname was trying to trick someone into doing the pyeclib/libearsure repo migration to openstack 22:00:17 really openstack/ is an implementat detail that has to do with git replication to github. "stackforge" the idea continues to live on 22:00:26 ok, we're at full time 22:00:49 thank you everyone for coming. it was great to see you last week, and than you for your work on swift 22:00:50 clarkb: thx for the details! 22:00:50 #endmeeting