19:01:15 <notmyname> #startmeeting swift
19:01:16 <openstack> Meeting started Wed Jun 18 19:01:15 2014 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:21 <openstack> The meeting name has been set to 'swift'
19:01:23 <notmyname> who's here?
19:01:30 <elambert> o/
19:01:31 <mattoliverau> o/
19:01:50 <gvernik> hello
19:01:54 <peluse_> yo
19:02:07 <acoles> here
19:02:14 <torgomatic> .
19:02:25 <tdasilva> hi
19:02:27 <notmyname> portante said he'd be late. and creiht has to leave early
19:03:03 <notmyname> this week, as normal for recent meetings, we gotta talk abou tstorage policies. but I hope this is the last meeting to ask when it will be merged
19:03:15 <notmyname> #topic storage policy status
19:03:39 <notmyname> we've had the patches proposed to master for quite a while now
19:03:45 <notmyname> and a freeze on other stuff landing
19:03:59 <notmyname> but it's getting to be "too long" to keep at it much longer
19:04:24 <cutforth> here
19:04:38 <creiht> ..
19:04:54 <notmyname> so there is a thought of "just merge it" unless there is a serious obvious flaw and the iterate in QA and later
19:05:33 <notmyname> I'm not sure I have a specific question there. what are everyone's thoughts on the state of things?
19:05:55 <torgomatic> seems to me like it's close enough; I'm inclined to merge it and fix things as they come up
19:06:03 <creiht> I agree
19:06:18 <torgomatic> I'm down to nitpicking stuff in tests at this point
19:06:30 <zaitcev> Merging it allows for releases for wider distribution.
19:06:34 <notmyname> with a Swift 2.0 release (after a 2 week RC period) once it lands in master? that's been the plan so far
19:06:43 <zaitcev> That's what kernel's -rc releases were for
19:06:45 <notmyname> and what I'd still like to do
19:06:47 <peluse_> and allows for easier tweaking
19:07:08 <notmyname> peluse_: ya, I think that's important
19:07:36 <clayg> weeeee - i was doing comments
19:07:40 <acoles> agree, except would like to have the timestamp format decided first. i think.
19:07:41 <clayg> lost track of time
19:07:46 <mattoliverau> Well the problem is, until its merged, there is a virtual dam so no other fixes can get through now, they've been scrutenized.. so I think its merge time and everybody get ready for fire fighting.
19:07:52 <clayg> acoles: everyone has there pet-peeves ;)
19:07:52 <notmyname> clayg: no worrie :-)
19:08:11 <acoles> clayg: just saying :)
19:08:49 * peluse_ thinks he might hear a drum roll
19:09:09 <notmyname> clayg: summary, and outstanding question, is: do we call it "good enough to merge" and then iterate in RC and later?
19:09:57 <clayg> notmyname: torgomatic's been putting his +2's out there where he thinks they're "good enough to merge"
19:10:05 <clayg> notmyname: I think others have been as well
19:10:21 <clayg> notmyname: I think everyone wants them to be good enough - but keeps finding stuff I mucked up?
19:10:29 <peluse_> bah
19:10:35 <clayg> ... and typos and leftover prints ;)
19:11:12 <notmyname> right. of the things that you are tracking that are outstanding I think the most impactful one is about the timestamp format, right?
19:11:50 <clayg> notmyname: yeah... and the list endpoints thing... and the header/literal thing - but that one really would be easier to do as a single patch on the chain rather than trying to interleave it
19:12:17 <acoles> +1 for leaving the header/literal change for later
19:12:23 <notmyname> ok. the header/literal change I agree would be better after the fact in one patch
19:12:33 <clayg> and the timestamp change has been sort of a burden to manage since it touches so much - when I go to merge down a fixup into an earlier change I have to go and fix all the calls to the class that doesn't exist yet
19:12:35 <peluse_> +1 for the same on list endpoints, what's the impact there if that takes another week?
19:13:17 <clayg> ok, well so on the timestamp format - has anyone besides acoles commented if they think 12 digits is the right number?
19:13:47 <notmyname> acoles: why 12 over 8?
19:13:51 <clayg> or even if the value of not doing the deca-mirosecond bump is worth all the churn/risk of chaging how we resolve time!?
19:14:03 <clayg> notmyname: it's BIGGER (obviously)
19:14:07 <acoles> notmyname: well, i shot for 16 decimal and clayg met me half way
19:14:10 <clayg> notmyname: since when is bigger not better!?
19:14:11 <peluse_> clayg:  my only comment is that if we think it needs to be bigger do it now but I think its OK
19:14:12 <notmyname> lol
19:14:20 <torgomatic> it's easier to have extra, but widening it later is hard, so I say use more
19:14:32 <clayg> torgomatic: that sounds like a vote for 16!
19:14:33 <torgomatic> screw it, we're using 256 digits
19:14:44 <clayg> whoa, torgomatic went there
19:14:52 <clayg> it's only bytes
19:15:05 <acoles> torgomatic: clearly i lack ambition
19:15:10 <torgomatic> :)
19:15:16 <mattoliverau> lol
19:15:37 <clayg> I think the 64 digit number system was the way to go
19:15:37 <notmyname> ok, timestamp bump to 16
19:15:43 <clayg> zomg
19:15:55 <peluse_> rock it
19:16:06 <notmyname> clayg: and the list_endpoints change is to do the auto v1/v2 detection based on the underscore?
19:16:09 <clayg> lets see... 10 billion objects times 17 bytes...
19:16:30 <clayg> notmyname: it's to make it so the endpoitns response for the non-zero policies are useable
19:16:46 <clayg> notmyname: for backwards compat; yeah I think you do something to v2 the response
19:16:56 <tdasilva> clayg: there's also an outstanding comment from gholt on the two vector timestamp patch, it that a concern?
19:17:07 <peluse_> clayg:  I can always take that for yo post merge if you want (LE)
19:17:34 <clayg> tdasilva: I *think* he just wanted me to not be such a jackass in the commit talking about stuff that doesn't exist
19:17:50 <clayg> tdasilva: but you know... he wouldn't ever *say* that
19:18:32 <notmyname> list_endpoints can wait until after a 2.0 release I think. or perhaps be finished in the RC period. we'll have another release after this one before the juno integrated release. it should be done by then
19:18:42 <peluse_> agree
19:19:00 <notmyname> also, pconstantine is on vacation this week, and I think we can ask him/zerovm to contribute for that
19:19:04 <peluse_> so just vector width pre-merge?
19:19:21 <notmyname> peluse_: yes, that's my opinion
19:19:21 <clayg> umm... so did we decide on 16 digits for the offset?  seems like a lot for the only existing use case which only needs to represent like idk - maybe an integer as high as.... 12!
19:19:40 <creiht> gotta run sorry
19:19:45 <notmyname> creiht: thanks
19:19:49 <clayg> notmyname: oh it's not that much code, and pconstantine's patches take forever to merge :P
19:19:59 <clayg> notmyname: but i'm fine with going in post merge
19:20:04 <torgomatic> I think acoles wanted real timestamps to fit in there, so 16 decimal would work, I think?
19:20:13 <torgomatic> I mean, normalize_timestamp() only uses 15 decimal digits
19:20:21 <clayg> notmyname: but it's sorta weird how the list endpoints trys to support storage policies but doesn't really provide a useable response
19:20:39 <peluse_> you're welcome
19:20:44 <acoles> clayg: 16 hex takes us way past year 2286, no? maybe overkill
19:20:56 <notmyname> clayg: describe not usable. as in badly formatted or as in just plain wrong
19:20:58 <clayg> torgomatic: oh oh oh - so are we going with 15 digits!?
19:21:12 <acoles> torgomatic: i'm cool with hex. or base 64.
19:21:16 <clayg> notmyname: like it tells you an object is here and then you go there and the server says it's not there
19:21:25 <notmyname> clayg: ack
19:21:34 <torgomatic> clayg: we'll go with max(all the serious suggestions)
19:21:39 <clayg> notmyname: (becauase you didn't send the right storage policy index, because you don't KNOW the storage policy index)
19:21:41 <torgomatic> so probably not 256 hex digits :)
19:21:51 <notmyname> acoles: but the second part is for finer resolution of the first part. not for representation of the timestamp. it's not a timestamp itself, right? its basically a counter
19:21:59 <notmyname> clayg: right
19:22:13 <clayg> notmyname: in acoles world you can't really do a fixed counter
19:22:27 <acoles> notmyname: i think clayg and converged on it being delta time from the normal timestamp part
19:22:37 <notmyname> ok
19:22:47 <clayg> torgomatic: yeah if we keep with hex (which seems a totally reasonable usage of available ascii bytes) then we really don't need 16 whole digits
19:23:18 <clayg> acoles: well part of that was just to deal with limited space, if we have more room it may be simpler to troubleshoot with extra digts
19:23:37 <notmyname> clayg: what
19:23:48 <clayg> acoles: my main concern was adding all these bytes that I don't really use - even the fast post stuff is just sort of an idea - it may not acctually turn out to be the best way do it
19:23:58 <torgomatic> clayg: true; 9999999999999999 fits in 14 decimal digits
19:24:00 <clayg> notmyname: wat?
19:24:05 <notmyname> clayg: what's the cost of updating it to 16 hex digits? another 2 days of review? or is it something that can be done today?
19:24:30 <clayg> notmyname: idk, have to see how many tests break - but why did we decide on 16 hex digits?
19:24:43 <acoles> clayg: fair point.
19:25:05 <notmyname> clayg: maybe I misread above?
19:25:11 <notmyname> if not 16 then what?
19:25:24 <notmyname> I guess that's the question :-)
19:25:42 <notmyname> oh, acoles said 12
19:26:09 <clayg> meh, it's not that many tests - i mean there's lots of asserts/expects in a loop but it seems like there's only a handful of tests that get all anal about the literal value of the timestamp (maybe that's ok)
19:26:30 <clayg> mathbot: 16 ** 12
19:26:36 <clayg> stupid mathbot
19:27:26 <torgomatic> 16 hex seems like a reasonable upper bound on the things people have asked for
19:27:53 <clayg> yeah it's a way higher resolution than the existing timestamps
19:27:55 <torgomatic> also, it's 64 bits, which is a nice round number, so there should be fewer questions about it
19:28:08 <torgomatic> instead of like "why 14 hex digits? what's going on there?"
19:28:13 <notmyname> torgomatic: makes the port to C easier?
19:28:38 <torgomatic> I'll get right on that
19:29:16 <acoles> so if we go large, its easier to shrink afterwards, if we decide there's no need. but not other way round. right?
19:29:21 <torgomatic> correct
19:29:36 <torgomatic> or just leave big and padded with zeros; that's easy too
19:29:45 <notmyname> ok, so what are we doing? clayg updates the patch to do 16 bytes?
19:30:00 <clayg> ok, well 12 hex is *almost* as big as 15 9's which is sorta the max time representable in swift with full resolution
19:30:16 <acoles> torgomatic: yeah, just thinking about clayg's space usage concern.
19:30:22 * peluse_ has to pay attention to another meeting for 15 min or so...
19:30:45 <torgomatic> 16 bytes doesn't seem like a lot in the grand scheme of things, but it doesn't matter all that much to me
19:31:03 <torgomatic> as long as it's useful for what all folks want it for
19:31:09 <clayg> either way i'd like to point out again that the reconciler doesn't need any of this (decamicro second bump worked fine) and even if it does want an offset to make reconciliation transparent it definately does need to do it 16 ** 16 times
19:31:10 <notmyname> it's not 16 bytes. its 8 more that today (and onlu 4 more than the proposed 12)
19:31:57 <notmyname> is anyone arguing against 16 bytes?
19:32:06 <torgomatic> not I
19:32:19 <clayg> o/ it's overkill at best
19:32:41 <notmyname> clayg: what do you want instead? the current 8? or 12? or?
19:32:50 <clayg> but i guess we could eventually use the leading digits to do something to fix the 2286 bug
19:32:56 <clayg> idk
19:33:04 <clayg> torgomatic's probably right harder to add more
19:33:13 * portante wanders in ...
19:33:21 <clayg> i guess that's what harddisks or for
19:33:32 <notmyname> portante: talking about the resolution for the second part of the timestamp
19:33:38 <portante> k thanks
19:33:38 <clayg> yay more bytes!  why aren't we doing 256 again?
19:34:25 <portante> clayg: because inode cluster allocation at higher inode sizes becomes problematic
19:34:43 <portante> if by 256 you meant inode size on xfs
19:34:55 <notmyname> portante: no. not at all :-)
19:35:23 <portante> :)
19:37:07 <acoles> suggestion: move to 12 or 16 for RC, i will try to progress something on fast-copy to validate usefulness or not, review number digits next week?
19:37:28 <acoles> (realise that means work right now making change)
19:37:45 <notmyname> acoles: that's a good proposal I think
19:38:09 <notmyname> do 16 now
19:38:33 <clayg> acoles: no I think 8 was the wrong number, so it's either 12 or 16 and i think the tempature is leaning twoards 16 - which is TON of room for the eixting use case (more resolution than we have in normalize'd timestamps) - but it's a good round number and no one can imagine any one wanting more than that
19:38:57 <acoles> i meant fast-post of course!
19:39:14 <clayg> acoles: it'd be nice if you could make copy faster too
19:39:28 <clayg> acoles: would three dementional time help?
19:39:46 <acoles> sure would, and i can trim every 16 bytes down to 8 :)
19:40:05 <notmyname> ok. yay resolution, then
19:40:15 <notmyname> change it to 16 bytes
19:40:56 <clayg> weeeeeee
19:40:57 <notmyname> then, if there is nothing else big (and nothing is known now), clayg and I will work on getting in landed as a merge commit from a feature branch
19:41:20 <notmyname> so that means one more patch chain push for the 16 bytes
19:41:22 <clayg> s/clay and I/notmyname/
19:41:31 <notmyname> clayg: ack
19:41:35 <clayg> notmyname: :P
19:42:08 <torgomatic> gate pass rate seems decent right now; if the merge-commit dance takes too long, we might be able to simply land all the patches
19:42:18 * torgomatic has no idea how long a new feature branch would take to set up
19:42:53 <notmyname> torgomatic: not long, I think
19:43:01 <torgomatic> okay then; never mind :)
19:43:44 <notmyname> everyone ok with the state of things then?
19:43:59 <torgomatic> sure
19:44:01 <clayg> umm... did anyone look at the merge_timestamps changes - that was the big diff from last night
19:44:39 <clayg> acoles: had a thing the replicator where a corrupt db could blow up until the auditor cycles
19:45:55 <acoles> yeah, i wasn't sure on that one whether it was so different to what might already happen
19:46:57 <clayg> acoles: pretty sure I could get a test that would make replicate 500
19:48:41 <clayg> torgomatic: some of those print statements might be useful to the next guy!
19:48:48 <torgomatic> hehe
19:50:22 <notmyname> I think that's it for the meeting then
19:51:19 <notmyname> thanks everyone for coming
19:51:22 <notmyname> #endmeeting