21:00:10 <notmyname> #startmeeting swift
21:00:11 <openstack> Meeting started Wed Feb  1 21:00:10 2017 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:14 <openstack> The meeting name has been set to 'swift'
21:00:20 <notmyname> who's here for the swift team meeting?
21:00:21 <joeljwright> o/
21:00:24 <timburke> o/
21:00:26 <m_kazuhiro> o/
21:00:33 <kota_> hi
21:00:33 <dmorita> hi
21:00:34 <zaitcev> o/
21:00:52 <mathiasb> o/
21:01:02 <tdasilva> hi
21:01:44 <notmyname> ...waiting just a bit for a few more people
21:01:59 <joeljwright> c'mon clayg!
21:02:10 <sgundur> hi
21:02:17 <timburke> i can see him...
21:02:21 <notmyname> joeljwright: he was so excited about it today ;-)
21:02:33 <joeljwright> :D
21:02:37 <notmyname> acoles_ won't be here today
21:02:49 <zaitcev> a door jammed in a toilet stall?
21:02:55 <clayg> o/
21:02:56 <notmyname> I'm not sure about jrichli (she might be in an airport)
21:03:03 <notmyname> same with bkeller
21:03:04 <jungleboyj> o/
21:03:18 <jungleboyj> Going to try and sit in on meetings more often.
21:03:30 <notmyname> joeljwright: it's the best part of the week
21:03:30 <clayg> jungleboyj: great!
21:03:35 <notmyname> ok, let's get started
21:03:42 <notmyname> welcome, everyone to the swift team meeting
21:03:46 <jungleboyj> :-)  A new friend from Lenovo!
21:03:58 <jungleboyj> Or an old IBM friend ... depending on how you look at it.
21:04:24 <notmyname> only two big topics to cover this week. so let's do the faster one first
21:04:30 <notmyname> #topic ptg prep
21:04:36 <notmyname> #link https://etherpad.openstack.org/p/swift-ptg-pike
21:04:54 <notmyname> that etherpad is for the three days we have at the ptg in a few weeks
21:05:01 <notmyname> please add stuff and fill out info
21:05:10 <notmyname> each of those topics need more info on them
21:05:25 <notmyname> (yes I know I added most, so it's my fault more than others that it isn't filled out...)
21:05:50 <notmyname> however, if you've got something to talk about, or if you know about somethng that's already written down, add info about it to the etherpad
21:06:11 <clayg> notmyname: I feel like we used to have stuff like "wtf is this; who care about this; relevent docs/code/ideas" or some stuff?
21:06:42 <notmyname> starting next week (about a week from now) we'll organize who's interested in what so we do schaduling of the big topics
21:07:06 <notmyname> clayg: in the past I've done a small template of name/description/interested for each
21:07:25 <clayg> notmyname: something like that would probably be good - you don't have to do it now
21:07:53 <clayg> or do it now
21:07:55 <clayg> :P
21:08:10 <notmyname> heh
21:08:23 <notmyname> ok, I added a template. all those existing things need to be filled out
21:08:32 <notmyname> and new stuff, too, as it's added
21:08:56 <notmyname> the PTG shouldn't have any surprises for us, with the way the swift room will work
21:09:07 <notmyname> we'll do this very much like we've done past summits and hackathons
21:09:16 <tdasilva> notmyname: sounds like we won't have a projector there thou...
21:09:22 <notmyname> we'll rearrange tables/chairs and break apart
21:09:25 <tdasilva> or TV
21:09:30 <notmyname> tdasilva: oh, interesting. I missed that. whiteboards?
21:09:39 <tdasilva> idk
21:09:51 <tdasilva> I thought I saw an email from ttx on that
21:10:22 <notmyname> the etherpads for all the other projects are at
21:10:24 <notmyname> #link https://wiki.openstack.org/wiki/PTG/Pike/Etherpads
21:10:53 <notmyname> so take a look at those if there's a topic that we need to participate in (especially for monday-tuesday)
21:11:13 <zaitcev> ok
21:11:32 <notmyname> eg the py35 support will probably be something we should be a part of to some extent
21:12:38 <notmyname> FYI there will be several people who are only at the PTG for wed-fri
21:12:58 <notmyname> any questions on the PTG or prep?
21:13:59 <notmyname> ok, then let's move on to the release
21:14:03 <notmyname> #topic upcoming release
21:14:14 <notmyname> before the ptg the openstack ocata release will be finalized
21:14:21 <notmyname> that's happening the week of Feb 13
21:14:36 <notmyname> we are responsible for getting a tag on swift
21:14:42 <notmyname> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews
21:14:59 <notmyname> the priority reviews wiki page has the stuff that we're working through to get done before the release
21:15:12 <zaitcev> looks like good progress
21:15:18 <notmyname> yeah, it is
21:15:30 <notmyname> thanks everyone for reviewing patches and responding to the reviews
21:15:56 <notmyname> https://review.openstack.org/#/c/425493/ is the last of the "fix stuff to be better" patches that needs to land
21:16:18 <notmyname> timburke already has a +2 on it, so just needing one more core to take a look
21:16:25 <notmyname> is anyone currently reviewing that?
21:16:28 <notmyname> clayg: do you know?
21:16:41 <timburke> torgomatic around? no?
21:16:44 <notmyname> looks like mattoliverau and sam have looked
21:17:23 <clayg> know what?
21:17:26 <notmyname> IIRC mattoliverau just went on a trip or something (left today)
21:17:32 <notmyname> clayg: who elese might be looking
21:17:43 <clayg> notmyname: yeah that's ready to go
21:18:00 <clayg> I think torgomatic and be tricked into a +A
21:18:16 <clayg> mattoliverau: knows the problem space and signed off on the design
21:18:39 <notmyname> kk
21:19:00 <notmyname> let's bug torgomatic and see if we can get his +A
21:19:11 <notmyname> clayg: what was the patch you wanted to make people aware of?
21:19:13 <clayg> it'd be nice for acknowledgement from addl. core about my hi-jacking of handoffs_*first* - but everyone seems to understand the use-case for EC to prefer revert is *super* important
21:19:16 <notmyname> with the upgrade issue
21:19:55 <clayg> so this seems like the most straight forward path - although going into a major release it's possible there's some important meta-work even if we want the code to do exactly what's written
21:20:17 <zaitcev> I'm very weak in the reconstructor area, unfortunately, but I can give it a good try...
21:20:30 <clayg> https://review.openstack.org/#/c/419787 needs to land
21:20:37 <clayg> it's the final form of the suffix hashing fix
21:20:50 <clayg> it's at least as important as the handoffs_first mode for EC fix
21:20:57 <clayg> was hoping to get PavelK to sign off
21:20:58 <notmyname> ah right. and I'd missed it on the wiki
21:21:04 <timburke> clayg: did you ever give those docs a look through to see if any of your patches change things?
21:21:05 <notmyname> onovy: ^
21:21:21 <clayg> in the 11th hour we'd discovered the data format migration of hashes.pkl isn't as backwards compatible as we thought
21:21:38 <clayg> so if you rollback you'll be unhappy until you delete all your hashes.pkl
21:21:44 <clayg> but... you know - don't rollback
21:21:55 <clayg> there was a bunch of %^&*ing bugs in the old code!
21:21:57 <notmyname> so clean upgrade path, but with a rollback issue
21:22:05 <clayg> notmyname: correct!
21:22:23 <notmyname> when it comes time for the changelog, please help me remember that :-)
21:22:25 <clayg> but srly, don't rollback - that junk on master is broken
21:22:45 <notmyname> I'm really curious how many times ops have done a partial swift upgrade and rolled back
21:22:51 <clayg> i stuck an UpgradeImpact in the commit - dunno if that helps
21:23:06 <clayg> I've only done it once #neveragain
21:23:39 <zaitcev> I never had to do it. Not even when policy indexes were added.
21:24:12 <clayg> not that anyone cares - but I'm going to merge cschwede's patch to fix this thing that really get's under my skin in swift-ring-builder https://review.openstack.org/#/c/326967/
21:24:27 <notmyname> I'm not aware of it being a thing people do. good to note int eh changelog though
21:24:41 <notmyname> clayg: ack
21:24:50 <notmyname> ok, kota_'s global ec patch
21:24:53 <notmyname> kota_: what's the status there?
21:25:04 <notmyname> I really would like to see that in this release :-)
21:25:11 <kota_> notmyname: I hope so
21:25:27 <kota_> notmyname: i think clayg and timburke is continuing the reviews
21:25:42 <kota_> notmyname: only one rebase happend in this week
21:25:44 <clayg> also on the hashes fix (https://review.openstack.org/#/c/419787) - it's a trixy little bit of code that took a bunch of work to get right - cschwede and PavelK (onovy) know what's going on
21:25:59 <clayg> I'm not sure who/when is going to +A
21:26:23 <notmyname> clayg: do you want to get cschwede to give his blessing?
21:26:28 <notmyname> kota_: that's good :-)
21:26:42 <notmyname> kota_: looks like the current patch set is ready for next round of reviews
21:26:49 <clayg> notmyname: ok well it's *really* important to note that ec_duplication is not a complete solution to global EC
21:26:50 <kota_> notmyname: yes
21:27:25 <notmyname> kota_: clayg: timburke: what's your sense of landing this patch this week? likely? possible? no way it's happening?
21:27:57 <clayg> notmyname: I'm all for getting it in - but we won't be "delivering" anything to anyone that runs swift - it's only for dev/testing - first step on a journey - i'm acctually not entirely clear the right way to docuemnt or "release" ec_duplication - but I'm definately interested in the global ec story (I think the reconstructor is finally ready for it!)
21:29:05 <clayg> notmyname: I'd just be leary focusing on ec_duplicates during the final RC weeks if there's other operationally impactful stuff that needs to go in the release.
21:29:06 <timburke> kota_: can we ensure good dispersion of the duplicates? if we can't, should we still land it (as experimental) and iterate?
21:29:51 <clayg> but for my part - I'm done with operational stuff as soon as the handoffs_first and better suffix hashing land - then I'm full on 'development on future"
21:30:00 <kota_> timburke: not yet. to ensure the dispersion, we need to have one more big patch (i.e. composite ring) imo
21:30:01 <notmyname> clayg: AFAIK, the other operationally impactful stuff has been landed (or just about to)
21:30:14 <timburke> kota_: make sense
21:30:21 <clayg> so ec_duplciates is the right place for me to focus AFAIK - but it's not relevant in/out of the release for me
21:31:06 <notmyname> my understanding is that "in the release" is actually more important for kota_. so I'd defer to him on that question
21:31:15 <clayg> I think the important thing is that we have ec_duplicates under our belt by PTG so we can make progress on what needs to happen to finish global EC in the next cycle
21:31:39 <notmyname> (bringing in the "fun" reality that we work for employers who want to see certain things happening)
21:32:06 <clayg> I'll commit to "will land before Atlanta" - I might advocate for *not* in the release because of the experimental nature - but i'm open to being pushed around because of the long history of the patch
21:32:39 <clayg> I still stand by the reconstrcutor wasn't ready for work to begin until now - kota_ literally lives in the future - we've finally caught up
21:32:44 <kota_> clayg: the commit sounds ok, "will land before Atlanta"
21:32:47 <notmyname> kota_: what do you need or want? does this need to land in this OpenStack ocata release?
21:32:54 <notmyname> ah ok
21:32:57 <kota_> hopefully it's in Ocata
21:33:44 <notmyname> the truth is, the window between the ocata release and the PTG is Friday February 17
21:33:48 <notmyname> clayg: ^
21:34:08 <clayg> notmyname: when does the RC get pulled off master?
21:34:13 <notmyname> I'll likely have to tag the ocata release on the 16th
21:34:27 <notmyname> it's up to us, but the 16th is likely the last day possible
21:34:45 <notmyname> and the ptg starts the following monday
21:35:34 <notmyname> so that being said, let's say we'll land this patch both "before atlanta" and also "in the ocata release"
21:36:08 <notmyname> sound ok?
21:36:15 <kota_> great
21:36:30 <notmyname> clayg: timburke?
21:36:39 <timburke> i think we can do that
21:37:21 <clayg> notmyname: I think "in ocata release" puts a bigger burdon to ensure the experimental nature is properly documented - but maybe only marginally so
21:37:22 <timburke> and it leaves us in a place where we can justify backporting the composite rings work
21:37:31 <clayg> oh dear goodness
21:37:36 <notmyname> lol
21:37:43 <notmyname> or not
21:38:04 <clayg> look - if you don't backport this feature ec_duplciation_factor doesn't give you global ec!?
21:38:53 <timburke> lumpy duplicates seems like a pretty bad place to be -- look, i can get m + k fragments! wait, i still can't reconstruct...
21:38:58 <notmyname> no need to figure out backport policies for future work. we've got enough to think about for now
21:39:07 <timburke> true enough
21:39:24 <notmyname> for now, land the patch we've got and we'll try hard to have good words in the release notes and docs about it
21:39:53 <notmyname> ok, now we need to have the same conversation about the part power increase patch https://review.openstack.org/#/c/337297/
21:40:07 <notmyname> again, an old patch with a ton of work
21:40:21 <notmyname> I didn't see if cschwede was actually here today or not
21:40:38 <notmyname> timburke and zaitcev have been looking at it, along with some others
21:40:53 <notmyname> and mattoliverau
21:41:04 <notmyname> but again, IIR mattoliverau's out for the rest of the week
21:41:20 <notmyname> so where on we on this patch? will it land? what needs to happen?
21:41:48 <timburke> i'm fairly confident we can get it landed. seems really really close
21:41:51 <tdasilva> it would be great to see it land before ocata
21:42:12 <zaitcev> I convinced myself that it does the right thing, but I'd like to have someone else look, I may be getting dull to something obvious (like %f format hehe).
21:42:48 <notmyname> tdasilva: have you been looking at this one? I see you were added, but I don't see any comments
21:42:54 <notmyname> kota_: same question ^
21:43:09 <tdasilva> notmyname: did not
21:43:13 <notmyname> tdasilva: ok
21:43:17 <kota_> notmyname: not yet
21:43:21 <notmyname> kota_: ok
21:43:33 <notmyname> kota_: you've got the EC patch to spend time on
21:43:56 <kota_> notmyname: true
21:43:58 <timburke> i just want to say, i'm very sad that we do `... >> (32 - part_power)` instead of `... & ((1 << (part_power + 1)) - 1)`
21:43:58 <zaitcev> Yeah. I didn't bug Kota
21:44:10 <notmyname> is there someone else who can take a look at this part power increas patch?
21:44:51 <clayg> timburke: i don't get it
21:45:21 <notmyname> clayg: makes more sense when he draws it on a whiteboard
21:45:31 <timburke> clayg: the way things are, increasing part_power moves *everything*. if we did it the other way, only half the world moves
21:45:34 <clayg> notmyname: doubt it?
21:46:08 <clayg> notmyname: the part-power-increase is a good example of something that has the potential to acctually impact swift operational deployments in octocat
21:46:23 <timburke> it's the difference between mapping X to either 2X or 2X + 1, and mapping X to either X or 2^n + X
21:46:24 <notmyname> tdasilva: would you be able to look at this patch this week?
21:47:00 <clayg> notmyname: if we're resourced constrained and need to make hard calls you know where my heart lies
21:47:38 <tdasilva> notmyname: yeah, i can. One of the reason I haven't was more because we (rht) didn't feel comfortable we pushing it all (christian, pete and I)
21:48:08 <tdasilva> so I could review and not +A ??
21:48:52 <notmyname> tdasilva: ack, but (1) I've not ever seen that as a problem in all the years of swift and (2) it's also got a huge about of input from rax and swiftstack
21:48:53 <timburke> tdasilva: if it makes you feel better, i'm happy to be the +A
21:49:12 <timburke> ...when the time comes, of course
21:49:20 <tdasilva> notmyname, timburke sounds good
21:49:24 <notmyname> ok, thanks
21:49:36 <notmyname> anything else to cover about the upcoming release?
21:49:54 <clayg> timburke: does it acctually effect the consumption of inodes?  do the parts that "don't move" still need hardlinks under the epoch-tree or whatever it's called?
21:50:01 <cschwede> notmyname: sorry, just arrived home :/
21:50:12 <clayg> timburke: because if you can really decrase the inode useage by 2x - that should totally happen
21:50:23 <notmyname> cschwede: no worries
21:50:51 * cschwede goes reading the backlog
21:50:53 <zaitcev> clayg: yes, temporarily 2x inodes, then back when relinker finishes
21:50:55 <clayg> notmyname: is symlinks really further out than ec_duplicates and part-power-increase?
21:51:08 <clayg> notmyname: what about tdasilva's patch that fixes 200 after DELETE
21:51:34 <notmyname> tdasilva: symlink question should go to you
21:51:54 <clayg> zaitcev: not just 2x - but +2x - a 200% increase (not just 100%)
21:52:27 <clayg> zaitcev: I wasn't clear if timburke's "I wish" was an oppertunity to get it down to 100%?
21:52:28 <tdasilva> re: symlinks, timburke has left some great comments (but I think they are all minor fixups) so I've been waiting for more comments to try to limit the number of changes.
21:53:03 <tdasilva> the big ticket item that is left is the symlink target path on the container db that we talked about
21:53:49 <clayg> oh right forgot about that one
21:53:53 <notmyname> clayg: no. he want's a time machine to change the way the ring was written in 2009
21:53:57 <timburke> clayg: my "i wish" would get us from a 2x during the migration down to a 1.5x, only we can't do it, because that's not how part_power works
21:54:51 <clayg> timburke: notmyname: ok great - I've wiped the conversation from my mind - thank you!
21:55:08 <clayg> tdasilva: do you want to merge w/o putting the target in the container db or do you want someone to write a POC
21:55:36 <tdasilva> clayg: I was planning on working on that as a dependent patch that would evolve on its own
21:55:43 <tdasilva> again to limit the number of patchsets
21:56:00 <tdasilva> and if we think it is good, maybe we can squash ??
21:56:02 <tdasilva> wdyt?
21:56:05 <timburke> tdasilva: looks like there were still some func test failures -- is that just because of the post-as-copy stuff? would a rebase fix most of those?
21:56:13 <tdasilva> timburke: yes
21:56:46 <clayg> yeah doing the POC in the follow on patch would be a great idea - my question is about merge.
21:56:51 <notmyname> oh, and we need to get the community qa cluster redeployed with fast post I think too
21:56:52 <tdasilva> timburke: actually, not a rebase, we might need to skip symlink func tests if slow-post is on
21:57:19 <tdasilva> since we don't plan on supporting that at all
21:57:30 <clayg> either we think we should merge w/o names in the listing (and effectively agree it just NBD if it ever happens, same as other manifest objects) or we want to see the POC and make the go-no-go call *before* we merge
21:57:44 <tdasilva> notmyname: right
21:57:46 <clayg> ... specifically becase we recognize it's hard to go back if you don't start with the data at point A
21:58:07 <tdasilva> clayg: agree, might also be an argument for not landing it for ocata
21:58:10 <notmyname> we're running out of time in the meeting room
21:58:11 <timburke> yup, 'cause we've still got the post-as-copy coverage. i still think we *could* support it if we really wanted, but maybe this'll be a nice carrot to incent people to get off the old, bad, slow code?
21:58:40 <tdasilva> timburke: don't really want ;)
21:58:40 <clayg> ok so no symlinks in octocat - wfm - we'll POC the names in the listings as soon as we can find a box of round tuits
21:58:50 <jrichli> sorry im late o/
21:59:16 <tdasilva> clayg: yeah, my plan is to start playing with that POC before the end of this week
21:59:39 <tdasilva> clayg: jrichli is also looking at container sync for symlinks, so yeah, there's still some WIP
21:59:44 <clayg> tdasilva: well given the release crunch - it might be better to have you on some other outstanding issues
21:59:47 <notmyname> ok, so the plan is the landing of the smaller patches (most of which already have the reviews)
22:00:05 <clayg> I'd like to pick up advocating for the GET 200 after DELETE fix (the 404 x-backend-timestamp bug thing)
22:00:05 <tdasilva> notmyname: fair enough
22:00:24 <tdasilva> clayg: that I do need help with
22:00:26 <notmyname> clayg: ok. add it to the wiki page
22:00:44 <clayg> tdasilva: ^ +1 get it on the wiki page
22:00:59 <clayg> that's time yeah?
22:01:03 <tdasilva> clayg: i think we are done here
22:01:04 <notmyname> ....and the plan for the others is to continue to iterate and review the ec repl patch and the part power increase patch
22:01:16 <notmyname> yeah, we're over time
22:01:25 <notmyname> thanks everyone for coming. thanks for working on swift
22:01:28 <notmyname> #endmeeting