21:00:29 #startmeeting swift 21:00:30 Meeting started Wed Nov 4 21:00:29 2015 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:32 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:34 The meeting name has been set to 'swift' 21:00:43 who's here for the swift team meeting? 21:00:45 o/ 21:00:47 o/ 21:00:48 o/ 21:00:49 hello 21:00:50 o/ 21:00:51 o/ 21:00:52 hi 21:00:55 . 21:00:56 o/ 21:01:00 o/ 21:01:13 hi 21:01:48 welcome 21:01:59 o/ 21:02:18 ok, first thing. somethign that just came up about 8 minutes ago :-)_ 21:02:35 hi 21:03:04 some of you may have heard of globo.com before. they're a pretty Big Deal in Latin America. based in Brazil, they're kinda like a yahoo type site for latic america 21:03:22 they've used swift for a while, and some of their ops have been in the -swift channel asking questions 21:03:50 first off, I was told to tell everyone thanks for that. 21:03:57 they're grateful for the help 21:04:35 and just yesterday they launched a new service (globo-play) which is like netflix for brazil. some of the static assets are stored in swift, and their video team is looking at putting the videos in swift too 21:04:46 I thought that was pretty cool :-) 21:04:49 +1 21:05:56 oh, and globo-play is already the #1 downloaded app in the brazilian app store 21:06:10 yay! everyone using swift every day, even if they don't know it :-) 21:06:21 yay swift :-) 21:06:22 cool! 21:06:31 nice! did they share any numbers (size)? always curious about that :) 21:06:47 cschwede: I don't know. ask NM if you see him in channel. not sure what he can share 21:06:53 k 21:07:29 ok, so moving on to the actual topics for the meeting... :-) 21:07:46 Awesome user story 21:07:50 I want to start with the py3 info, just because of the time zones haypo and cschwede are in. it's late for them 21:07:55 http://globoplay.globo.com/ nice, now you can all watch a whole lot of brazilian soap operas 21:07:59 #topic py3 status info 21:08:01 tdasilva: lol 21:08:24 last week several people met together to talk about py3, and haypo wrote it up on the ML 21:08:25 #link http://lists.openstack.org/pipermail/openstack-dev/2015-October/078058.html 21:08:43 haypo: cschwede: anything you'd like to add or comment on about this py3 plan? 21:08:50 ohia 21:09:12 notmyname: i think there is an upcoming new pyeclib release that no longer requires the workaround in https://review.openstack.org/#/c/199034/? 21:09:21 i heard rumors about that ;) 21:09:28 cschwede: correct! 21:09:41 cschwede: there is a review to bump PyEClib to 1.1 21:09:42 should have landed before the summit, but it didn't 21:10:00 but i don't understand how libjerasure & gf-complete will be installed 21:10:03 yeah, we'll get 1.1.x and won't have to deal with the sed-ness in tox.ini 21:10:19 haypo: they won't, by default 21:10:38 notmyname: is it an issue to run the check jobs on gates? 21:11:09 haypo: no. liberasurecode has it's own simple EC scheme so it can do stuff with no dependencies. that's one of the reasons to bump the version 21:11:26 hello, sorry i'm late 21:11:26 oh ok 21:12:02 on the py3 topic, in general, I think the plan is workable, but I think the reality is that it looks like a long road to go down. we'll end up with a constant set of "port to py3" patches 21:12:13 but the most important part is a voting gate job 21:12:22 ie nothing else really matters without that 21:12:32 but it should be more easy once we have voting patches, avoiding regressions 21:12:43 right. "easy" 21:12:46 notmyname: do you mean a voting job that tests everything - or the green py3 dot that means "doesn't work on py3" 21:13:06 notmyname: i don't care if it takes a long time to port swift, it's up to reviewers ;) 21:13:08 clayg: the green dot that means "some part does work on py3, but not everything" 21:13:26 clayg: a voting job that tests the parts that have been migrated to be py3-usable 21:13:30 There's no alternative to a constant stream of "fix in py3" patches. The important part is to ratched new tests up constantly somehow. 21:13:50 haypo: that's not entirely fair. you've told us a big port won't work and haven't given us an idea of the scope. so it's really hard to assume anything other than "it will take a long time" 21:14:05 zaitcev: right 21:14:33 zaitcev: it's not true to say there "is no alternative" - if we had a patch that fixes all the py3 - even if it conflicted frequently it'd be easy to show up on a wednesday and say "that's it i'm done - go +A" 21:15:22 clayg: really? how did you fix rfc822 in it? 21:15:33 https://review.openstack.org/#/c/238771/ Bump PyECLib version to 1.1.0 21:15:36 clayg: except that the py3 patch author (haypo) has said that he won't be able to do that 21:15:47 notmyname: i know 21:16:05 zaitcev: so you're saying all of this is for naught? You can't acctually have a py3 swift because of pyeclib and other depends? 21:16:21 notmyname: sorry, i have no idea of how much code requires to be modified to support py3 21:16:26 rfc822 referring to the MIME message doc backport thingy? 21:16:26 clayg: oh, wait, n/m. I think you said that you actually had that patch already, not "if we had a patch", sorry. 21:17:02 notmyname: for some projects, it took 3 or 4 months, for some projects it will take 1 year or more (like nova) 21:17:23 IMO I'm fine with the plan as it stands, but I do with there were a way to have a faster port. ie something other than a new patch every week for a year with no real known end date 21:17:27 zaitcev: oh right - totally hypothetical - I'm just saying we're on a path - and the that's fine - but it's hardly an enevitibility that this is the way 21:17:48 notmyname: what i see is that i already proposed py3 patches for swift, and it always take 1 month at least to get reviewed. that's why i expect that the port will take 6 months or much more 21:17:50 ok, so that was fun 21:17:52 "I'm fine with..." == it's somethign I can live with and it sets expectations 21:18:11 notmyname: the problem is that we need first to have a voting py3 job 21:18:17 I agree with that 21:18:34 (sorry, i'm in the middle of another meeting, it's hard to follow) 21:18:36 and getting the pyeclib version bumped is probably the next step 21:18:56 right - how do you break off fast-POST, encrytpion, container-sync, etc to give py3 reveiew bandwidth?! 21:19:03 torgomatic: says he can do py3 reviews on the train 21:19:22 maybe I can make a filter to find +2'd py3 changes and +A them 21:19:33 Since I'm not doing much of any technical work on Swift anymore, I tried to focus on haypo's patches recently... by being a stick in the mud most of the time. Like, I hate if six.PY3: thing and always try to challenge it. 21:19:43 (then spend the next hour rebasing all of my changes) 21:19:44 notmyname: for pyeclib, it took 3 months to get my py3 merged into pyeclib. pyeclib 1.0.8 was released with my fix but it's not used yet in swift 21:19:44 zaitcev: thanks! 21:20:01 notmyname: and it looks like it will still take a few weeks to get pyeclib >= 1.0.8 21:20:07 it's annoying to have to wait so long :-/ 21:20:12 haypo: leave pyeclib out of it for now. that's going to be resolved within a week, I hope 21:20:21 zaitcev: yawn - there's tons of ugly code in swift that is probably masking bugs - a few if PY3 branches won't kill us 21:20:30 well tons is probably a strong word 21:20:46 notmyname: https://review.openstack.org/#/c/238771/ the change on requirements for pyeclib is not merged yet :-/ 21:20:50 even if it was only 1/2% by line count I'd call it "way more than I would like" 21:21:00 I think it's safe to say that nobody is particularly happy with the speed of the py3 work 21:21:04 haypo: is the new pyeclib released?! 21:21:12 notmyname: hehe 21:21:38 clayg: tuskar wants to use pyeclib 1.1 which was released at least 2 weeks ago 21:21:43 patch 238771 has a -1 from kota 21:21:43 acoles: https://review.openstack.org/#/c/238771/ - Bump PyECLib version to 1.1.0 21:21:44 but the point is that we seem to have as close as possible agreement on something which might actually end with py3 support. even if nobody is particularly happy with that plan either 21:21:47 clayg: the first step is to upgrade it in global requirements 21:22:09 clayg: never say "new pyeclib". say "pyeclib-1.1.x" or something. This log will be archived you know :-) 21:22:26 well then what do you want me to do? it's not useful to say "this patch is a month old" if it's not ready to merge 21:22:35 notmyname: i don't really understand why we are not making progress. IMHO the sed hack is an acceptable compromise to unblock the situation 21:22:54 notmyname: always having to wait for other changes (like pyeclib) doesn't help 21:22:58 haypo: I think the sed hack looks stupid and don't see the value of hacking around it instead of ust waiting 21:23:08 haypo: but I'm *trying* to stay out of it 21:23:13 nope. nobody likes the sed hack and there is a plan in progress (delayed by the summit) for updating pyeclib 21:23:48 but I want to move on instead of spending 30 minutes in this meeting going over all this again 21:23:56 sounds good! 21:24:01 acoles: pyeclib 1.1.0 seems to have an issue for upgrading, so my -1 for safety 21:24:16 we have a py3 plan. it's in the link above. the first part is getting a passing test job (and yes I'm working on pyeclib updates to unblock that) 21:24:17 we already have a plan, everybody knows it, let's move on 21:24:22 kota_: right! just pointing out why it has not merged 21:24:24 tdasilva: :-) 21:24:31 kota_: if the gate is happy with it it's probably fine - anyone running master can probably figure it out 21:24:33 acoles: kota_: and I'm in the same boat on that 21:24:38 (or hack the requirements and be done with it) 21:24:54 ok, moving on 21:25:09 #topic summit recap 21:25:13 thanks everyone for looking at those patches! 21:25:31 I want to ask a few questions about the summit. please let me know what you thought 21:25:42 1) what did you like? 21:25:43 I'll make sure Fedora has a pyeclib-1.1.x 21:26:00 I liked the number of fishbowls to working sessions 21:26:13 +1 21:26:14 jrichli: ie lots of working sessions, few fishbowls? 21:26:18 yes 21:26:39 friday's free for all talks were great but it was hard to hear everyone in that room 21:26:39 more swift talks, as peluse counted. that’s great! 21:26:56 if I did it again, I'd probably put the crypto stuff in a fishbowl because of how crowded the room was 21:26:56 we had lots of extra space in the fishbowls - I think a number of people in our working sessions were not around for the fishbowls 21:26:59 yeah, I completely fell out of Friday in the end 21:27:28 having displays in the working sessions was nice (as compared to the previous summit) 21:27:31 Haidi is super loud. Not to knock him, it's just the kind of voice he has. 21:27:38 heh 21:27:45 I think there's a disconnect between the size of the room and format of the topics that isn't really captured in "fishbowl vs. working-session" 21:27:55 hrou: not sure if I should add that as +1 or -1 ;-) 21:28:01 clayg: agreed 21:28:19 i may be getting our working sessions and Friday mixed up 21:28:32 We needed more room to "break off" on Friday 21:28:35 ok, what did you not like? 21:28:44 4 days 21:28:46 rooms were too small for the number of people interested in swift 21:29:00 we were spilling into the hall, and carrying whiteboards around the crowded table 21:29:02 (working sessions) 21:29:05 yeah 21:29:25 I wonder if you could run a Friday in something more like the lunchroom? 21:29:31 my main note from Summit was put fast-post is on my bucket list 21:29:41 FUCK YEAH FAST POST! 21:29:41 zaitcev: yay! 21:30:04 clayg: in paris? vancouver? we had a big room shared with cinder and we all thought the other project was being too loud 21:30:06 the next summit shirt should say swift: now with fast post 21:30:18 blmartin: now it works like you always assumed it did 21:30:21 sorry container-sync, ec follow on, encyrption, etc - fast-POST first 21:30:24 notmyname: no, they thought WE were too loud! 21:30:30 haha 21:30:42 acoles: right. they though we were loud. we thought they were loud 21:30:56 hehe 21:31:12 notmyname: yeah that's what I'm thinking of! I liked that big room - i forgot it was shared - I donn't remember it being a problem - but I do remmeber running into a corner with a crowd to hit specific topics working well 21:31:15 what did you think of the scheduling? we tried the "blocks of time" approach instead of the "40 minutes for a topic" approach 21:31:27 notmyname: SO MUCH BETTER 21:31:32 should we try that again next time? 21:31:33 ok :-) 21:31:34 notmyname: I heard some other projects were doing the same thing 21:31:35 worked pretty well for me 21:31:38 +! 21:31:40 +1 for topic blocks 21:31:43 +1 even 21:31:47 no, clay, we need to you to balance redbo on python side; I have no clue what that "posrep" thing does better than ssync 21:31:49 blocks seemed great to me 21:31:55 I thought we were being so clever 21:32:04 great, I'm glad that worked well. I was happy with it too 21:32:07 +1 21:32:19 if we're clever, it probably means we could fit more stuff in too. 21:32:37 anyone else notice that we seem to be gradually turing the summit into one of our hackathons? 21:32:40 did we work through all our breaks? :) 21:32:51 I think bunching client topics together was good, at least we can have e.g. Tim and Zack schedule easier next time etc. 21:32:57 notmyname: yeah, great, isn't it? 21:33:02 notmyname: yes and I'm sure there's a downside hiding in there 21:33:08 zaitcev: right. next time is to actually get tim and joel in the room! ;-) 21:33:31 clayg: yeah. too much inward focus and we miss out on some cross-project stuff and hallway stuff 21:33:32 notmyname, I was giving a summary about the summit and that was my one bigger comment; felt a lot like the hackathon ; ) 21:33:41 hrou: was that good or bad? 21:34:13 ok, so was it a productive week overall? if not, what was missing? 21:34:15 it was good 21:34:20 * notmyname is writing all this down 21:34:22 zaitcev, I'll take that as a +1 ; ) (but yea, you're right, and I get that a lot, and need to keep that in mind) 21:34:34 bunching of "swift immense topics" together was kinda challenging for a stupid guy like me; I basically tuned out Matthew and sharding because they came last 21:34:39 notmyname, oh I didn't mind at all, I thought it was neat actually and effective. 21:35:05 zaitcev: yeah, I think everyone feels that after 8 hours of discussions 21:35:24 notmyname: matt_____: acoles: I wonder if we haven't yet found the final form of dense topics at hack-a-thons 21:35:28 someone proposed that we should have a 11am-1pm + 8pm-1am summit schedule ;-) 21:35:39 clayg: oh, I'm pretty sure we haven't yet 21:35:53 notmyname: Overall, it was a productive summit. Especially for container sharding, I feel that we made some good architectural progress. 21:36:02 great 21:36:04 I felt like I was able to dissmeninate good info about the ring stuff - but my progress came on Saturday coding - not during the "working" session 21:36:12 notmyname: i think we did more "intro to a topic" than at a hackathon e.g. clayg ring talk. but maybe others feel they needed more of an on-ramp to some topics. 21:36:13 yeah 21:36:27 what do you wish for the next summit? 21:36:41 not to talk about fast-post ;) 21:36:46 lol 21:37:03 lol 21:37:11 lol 21:37:20 #agreed merge fast-POST before the next summit 21:37:21 ;-) 21:37:21 seriously, more chairs in the room. 21:37:27 what room? 21:37:32 less chairs in the room 21:37:36 more whiteboards 21:37:44 are we talking about Friday or Thursday 21:37:50 wed/thurs 21:37:50 I dunno, if I didn't hear you explain it, I would never get why 1 timestamp causes irreconcileable difference between nodes. The spec is impregnable sadly. So maybe it needs talkig about until it's in. 21:38:03 Weds/Thursday - I don't like seeing people having to sit on floor 21:38:15 acoles: yeah, I agree there 21:38:23 larger tables would be nice too - to use laptop 21:38:29 Friday I would say get rid of all furniture except whiteboards 21:38:39 acoles: yeah - i think that's an artifact of the "working session" - we had a bunch of people show up for encyrption that don't show up for swift generally - it should have been a fishbowl 21:38:50 friday was a little chaotic, but I dont really have any good suggestions. it shouldn't be formal, either. 21:38:51 acoles: heh - maybe 21:38:52 clayg: yep 21:39:31 thanks for all the feedback. anything else to add about the summit before we move on? 21:39:46 schedule the meetings so we can get food before its all gone D: 21:39:54 lol 21:40:08 overall I was just glad to see so many people interested in swift...don't think people were expecting to have so many folks, so that's why it felt so cramped in there??? 21:40:14 I think this was about as good as they come and tweaking it may ruin next summit. However, I remember that at hackathon at SwiftStack offices it was quite useful to do breakouts into separate rooms where participants could really focus, in case topic was challenging. 21:40:40 This may be not feasible in Summit venue, but if we can get it, it would be super, IMHO. 21:40:52 and thanks to kota_ for being a great host!!! 21:40:53 zaitcev: no, swiftstack will not host the summit at our office 21:41:03 yes! kota_ did a great job! thanks! 21:41:08 :-) 21:41:19 kota_: !!++!! 21:41:35 Yea kota_ it was really awesome thanks so much ! 21:42:02 +1 21:42:23 #topic swift-init return codes 21:42:26 i can write a report "Swifters were sufficient" 21:42:41 patch 230352 21:42:41 notmyname: https://review.openstack.org/#/c/230352/ - swift-init return codes 21:42:51 clayg: you want to run this discussion? 21:43:06 yeah so there's things legacy beahvior that's stupid 21:43:15 i tried to change it once and redbo made me put it back 21:43:24 and now distro's are complaining 21:43:46 we could fix it - and maybe redbo less happy - maybe you too if you have a script that's effected (I checked I don't) 21:43:54 or we could work around it - and live with the cruft 21:44:05 normally we just live with the cruft - but meh 21:44:09 any opinions? 21:44:13 swift-init all start will start everything that has a config 21:44:30 the proposal is to make that fail with an exit code if a config for a referenced server isn't found 21:44:50 well "referenced" is maybe a strong word 21:44:50 RDO and Fedora prohibit swift-init anyway, so from my perspective you can tweak it as much as you want 21:44:52 the question is if we need the first one or if we can move to the 2nd (which is what distros want, it seems) 21:45:11 just as long as you don't make it completely brain dead 21:45:28 I think rax does "swift-init all start" on object nodes and may still expect non-zero exit even if they only have /etc/hummingbird 21:45:50 er.. still expect successful exit 21:45:51 who knows 21:45:53 ...and no RAX dev is here 21:46:16 I think the interesting doc reference in the reviews is about the LSB standards 21:46:23 (we tell users to avoid swift-init because launching with it inherits wrong SElinux context) 21:46:32 well, maybe added the RAX devs simply to the review? 21:46:39 I think some daemons (like container-sync) are optional - it's possible swift-init all start would exit non-zero if you don't have a [container-sync] config section 21:46:42 asking them on their opinion? 21:46:55 s/added/add/ 21:47:00 cschwede: the idea was we could get that opinion in this meeting 21:47:23 where the fuck is dfg redbo hurricanerick or nadeem or alan or btorch ;a;sdlkfj;laksdjf 21:47:32 er... alex sorry 21:47:55 oh well - i tried - I'll put my opinion on it (whenever I decide what that is) and we'll go from there 21:48:01 I'll probably just say live with the cruft 21:48:08 to easy to punt the pain on future me 21:48:25 FWIW, I think making it more inline with other init scripts to error on config not found is reasonable 21:48:49 acoles: does helion use swift-init? 21:48:55 notmyname: clayg i need to speak to people here to know for sure if we care either way 21:49:00 tdasilva: what about swift-on-file doesn't it like not run the auditor or something? 21:49:03 acoles: thanks 21:49:10 added myself to review 21:49:18 that's main vs. rest isn't it 21:49:34 zaitcev: all main and rest are the aliases that are there 21:49:49 notmyname: off top of my head i don't recall if we use it 21:50:11 you know maybe we could fix it so that if you *don't* use an alias then --strict is implied 21:50:23 so swift-init all start will make all configured services start 21:50:24 clayg: I need to double check our product docs, but I think we tell users to use 'service start' 21:50:31 but swift-init object start will error if you don't have a config 21:50:47 ^ which I think would achive the goal of the author 21:50:49 clayg: that also seems reasonable 21:51:09 clayg: do you want to move forward with that? 21:51:15 ok, so "do both" - should have thought of that before hand 21:51:18 clayg, yep that's the goal 21:51:24 peterlisak: ! 21:51:33 great. go forth and "do both" 21:51:38 peterlisak: yeah so let's do that maybe - seems like a good way to punt on concensus 21:51:41 last topic 21:51:46 #topic keymaster name 21:51:55 jrichli: you ran into the hard problem of naming 21:51:57 I'm *pretty* sure no one was epecting swift-init proxy start to exit zero when it didn't start the proxy 21:52:06 jrichli: keymaster 21:52:09 stupid naming things - this is why we have replicanths 21:52:16 jrichli: keymaster 21:52:19 yes. we are planning on having a "less trivial" keymaster now 21:52:23 jrichli: keymaster 21:52:27 ^ why are we doing that? 21:52:28 hello .. 21:52:31 :-) 21:52:34 oh naming the keymaster 21:52:56 so, what should this new "simple" keymaster be called? 21:52:58 keymaster vs simple_keymaster vs keymaster_v1 vs ?? 21:53:04 Gozer 21:53:05 just "keymaster"? 21:53:11 :-p 21:53:23 jrichli: the other names imply simplicity or future versions neither of which may turn out to be true 21:53:25 mabye 21:53:29 the point of the keymaster we talked about last week is to provide enough functionality for the operator-level feature (cluster has access to the key, probably in a config) 21:53:34 jrichli: and "keymaster" is less key strokes :) 21:53:37 acoles: ah. gotcha 21:53:58 basic_keymaster decent_keymaster acceptable_keymaster 21:53:59 so trivial_keymaster that isn't supposed to be used in prod (a la tempauth) isn't right 21:54:09 less_trivial_keymaster 21:54:14 Gozer 21:54:19 *the* keymaster 21:54:42 i would go with Gozer ... why not 21:54:46 well, gozer 21:55:03 http://ghostbusters.wikia.com/wiki/Vinz_Clortho 21:55:14 then we can use the word "keymaster" to represent whichever one actually gets loaded 21:55:16 ^ this is me *not* being helpful 21:55:28 just "keymaster" 21:55:38 ^ this is notmyname being helpful 21:55:59 ok. but then there wont be a generic form 21:56:13 it's full name is swift.common.middleware.keymaster.KeyMaster - but we just call him the "key-man" for short 21:56:30 keyapprentice 21:56:42 jrichli: I'm not too worried about building extensibility for something not yet implemented into a name of something else not yet implemented fully 21:56:48 clayg: if you like vinz then go +A this https://review.openstack.org/183014 21:56:59 gotcha 21:57:17 acoles: nice 21:57:18 keymistress ? 21:57:22 point is, use keymaster and make future-me solve any future-problems that come up. for now, we need a keymaster, and "keymaster" makes more sense than "steve" or "bob" 21:57:29 + 1 to just keymaster ; ) 21:57:41 swift.common.middleware.keymaster.KeyMaster is a great name 21:57:44 ok. agreed. 21:57:47 honestly keymaster sounds great 21:57:56 briancline: do you use swift-init in prod? 21:58:09 great. agreed on "keymaster" then 21:58:22 #agreed the keymaster should be called "keymaster" 21:58:30 lol 21:58:30 notmyname: yeah 21:58:36 ^ so much better than replicanth 21:58:48 lol 21:58:55 briancline: clayg is goign to ask you a question about swift-init in the #openstack-swift channel 21:59:35 and peterlisak ^ 21:59:47 thanks everyone for coming today 21:59:54 it was great to see (most of) you last week! 21:59:58 thanks for working on swift 22:00:01 #endmeeting