21:02:58 #startmeeting swift 21:02:58 Meeting started Wed Dec 2 21:02:58 2015 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:02:59 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:03:01 The meeting name has been set to 'swift' 21:03:01 meetbot? 21:03:12 ah, lag on my end or somewhere 21:03:18 who's here for the swift team meeting? 21:03:22 o/ 21:03:22 o/ 21:03:23 yo 21:03:24 yo 21:03:24 o/ 21:03:26 o/ Happy Humpday! 21:03:31 o/ 21:03:32 hi 21:03:33 o/ 21:03:34 o/ 21:03:36 o/ 21:03:49 o/ 21:03:53 cutforth: but its thursday :P 21:04:03 hello, everyone 21:04:18 mattoliverau: sorry for you 21:04:25 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:04:27 o/ 21:04:27 hello 21:04:27 agenda ^ 21:04:44 several things to go over this week 21:04:48 mattoliverau: you guys are so wrong down there 21:04:52 going a little out of order... 21:04:59 peluse: just stand on your head. makes it easier ;-) 21:05:16 #topic tracking TODO things 21:05:17 * peluse totally gets it now 21:05:31 peluse, notmyname: lol 21:05:48 I'm just from the future 21:06:03 there's always been stuff to do that's not exactly been bugs, but also not really been "oh you should make a spec about that" 21:06:16 and we've struggled to track it and give people a place to find it 21:06:29 hi 21:06:32 I've used https://wiki.openstack.org/wiki/Swift/ideas in the past. and it's been ok. but not great 21:06:39 so new plan! 21:06:47 (as you'll see on that wiki page) 21:07:13 track todo items as bugs in launchpad, but mark them with the "wishlist" status 21:07:27 that way there's one place to find them 21:07:37 there's an opportunity to have a discussion on them 21:07:48 and you (should be) looking at the LP bugs anyway ;-) 21:08:23 nice 21:08:27 so I think it could help unify a workflow (ie remove a wiki page) and give some benefits around discoverability of "what should I work on" 21:08:44 I can't claim credit for this (IMO very good) idea. it was clayg's idea :-) 21:08:57 notmyname: take the credit 21:09:11 +1 I was actually worried you are going to suggest yet another wiki page 21:09:23 MOAR WIKI! 21:09:24 I am still collecting them... 21:09:29 nom nom nom 21:09:29 the last point I'd make about wishlist items: they don't have to be complete or even be a good idea 21:09:43 notmyname: well what *do* we do with the bad ideas tho? 21:09:50 I have some! 21:09:57 ie just because something has a wishlist bug, doesn't mean (a) you must work on it (b) it's awesome as it's written down 21:10:00 like "swift should deploy it's own rings" - no it shouldn't - that was on purpose 21:10:08 notmyname: clayg: thanks for the idea.. really like making more use of LP 21:10:17 * torgomatic is here but late 21:10:19 if there's a bad idea, then that should be mentioned in the comments on that specific item 21:10:31 notmyname: that part makes sense 21:10:42 notmyname: can we *confirm* wishlist items? 21:10:45 then we can drop it as a "won't fix" or something like that 21:10:48 notmyname: like yup - good one 21:11:03 notmyname: oh ok - yeah won't fix would be the hammer - i like it 21:11:07 nice thinking 21:11:07 no. bugs aren't confirmed, really. at least there's not "accepted" status like blueprints 21:11:14 sweet - answer is "always file it as a wishlist" - it's simple 21:11:23 no? 21:11:23 yeah 21:11:39 i mean it has a status... 21:11:49 so maybe as it get's prioritized as "low, mid, high" if someone is working on it? I'm not sure 21:12:02 priority is different from status 21:12:06 oh right 21:12:21 I think we can totally use wishlist/new and wishlist/confirmed - and then there's wishlist/won'tfix 21:12:31 notmyname: this is a good plan you had 21:12:35 dfg: !!! 21:12:39 hey 21:12:40 wishlist is a priority too 21:13:02 yeah, wishlist/new and wishlist/confirmed is a good idea 21:13:12 notmyname: wishlist is *not* a status - it's *only* a priority 21:13:16 right 21:13:24 notmyname: ok - let's give it a shot 21:13:32 notmyname: thanks for pushing up the first batch 21:13:45 sound reasonable to everyone? any questions or concerns to raise right now on it? 21:13:51 notmyname: is there some useful lauchpad filter links on *the* wiki page? 21:13:54 what does wishlist/confirmed mean? wouldn't we just change from wishlist to low or medium? 21:14:10 acoles: wishlist is lower than lower 21:14:11 clayg: not that I could find. no short links that would actually not error in LP 21:14:13 lower than low 21:14:18 do the ideas auto expire after about 5 months of inactivity or something? 21:14:22 notmyname: stupid launchpad 21:14:41 blmartin: what's an "idea" - wishlist bugs won't expire 21:14:42 acoles: small real bugs would be low. idea to do something would be wishlist. IMO 21:14:57 blmartin: I'd hope not 21:15:11 ah sorry, misunderstanding on my part 21:15:25 notmyname: yes, but when a wish becomes agreed as a good idea is it confirmed or moved to hi/lo/medium 21:15:39 acoles: for now, I'd say just move to confirmed and keep as wishlist 21:15:46 acoles: I think wishlist dicussions might eventually be like a precursor to a spec - like one way to close a wishlist is to write a spec? Another way would be to write a patch. 21:16:06 notmyname: +1 wishlist/confirmed 21:16:06 blmartin: good question, though. might need to address it if there's some LP janitor that expires stuff. however, seeing as there are bugs listed from 2013, I'm not too worried ;-) 21:16:33 acoles: I mean unless the wishlist item is acctually a low priority bug? 21:16:55 I'll write up a short email to the ML about this 21:17:00 then the pritorty gets changed - but's that just because it was misfiled to begin with or we learned something 21:17:03 ignore me, i'm just confused 21:17:28 next topic! 21:17:36 notmyname: may be helpful to have examples of what would be items for wishlist? 21:17:40 acoles: for me this started when intel started having people write unittests for swift and I remember how janky some of our old test code is 21:17:44 pdardeau: check to see what's there now :-) 21:17:57 acoles: I wanted to write down "module abc should work like xyz" and then never do anything about it 21:18:07 it's not a bug 21:18:23 #opic patches -- ring change 21:18:24 notmyname: did I already kudos the inital population?! 21:18:29 kudos 21:18:29 patch 241571 21:18:29 notmyname: https://review.openstack.org/#/c/241571/ - Put part-replicas where they go 21:18:37 cschwede: thanks for the update 21:18:45 clayg: right. 21:18:46 hurricanerix: dfg: what's the word? 21:19:15 notmyname: you typo-ed topic 21:19:16 clayg still playing with it, was off most of last week, and, you know, ring code... =) 21:19:19 clayg: yw! hope to add my +2 soon 21:19:21 #topic patches -- ring change 21:19:25 mattoliverau: thanks 21:19:31 notmyname: acoles has a bloke doing some testing that *may* have an issue with zero-weight devices not being agressive enough at sheading parts - it's not entirely clear tho - and it may not be so critical - it depends on how many parts are being left behind and why - removing the device always reassigns 21:19:47 yeah, I saw that. lorcan 21:19:54 cschwede: even if you +2 I might say leave off the +A 21:20:00 so I'm hoping you'll get his ring and find what it is 21:20:17 cschwede: it'd be nice to have feedback from hurricanerix and maybe this lorcan fellow - or kota_ mattoliverau whoever 21:20:23 more eyes the better on something like this I'd say! 21:20:29 cschwede: yeah. if you add a +2, ;et's hear from rax and hp before +A 21:20:32 clayg: that was my idea as well, to get a broader view on this 21:20:32 but... it's a terrible bug - we should totally close it - we've got working code 21:20:35 clayg: +1 21:20:45 well... "working" - at least some people think it's not terrible - it WOMM 21:20:52 clayg: I think we should have a goal of having it landed by the next meeting 21:21:02 glange: !!! 21:21:26 RAX has been looking (or said they're looking! ;-) for a few weeks now. looks like we'll see lorcan's (HP) status soon 21:21:28 ok great that was fun 21:22:07 love to have it shutdown by next week - maybe put the merge hammer down if we make it to next Wednesday? Anyone need more time than that? I'm sure we'd love to close it ASAP 21:22:12 and it's demonstrably better than master today. so let's get it landed soon unless something unexpected shows up 21:22:19 clayg: we've run some normal operations on our rings through it and they've looked fine. i'm going to try to get one of our ops guys to check it out. 21:22:27 dfg: nice! 21:22:29 notmyname sorry, i am slow, and i have never messed with the rings before. so i am kinda taking my time to make sure i do it right. =) 21:22:33 ... but i'm not sure exactly what the hard requirement is - other than I really need to get started on packaging and get it shipped 21:22:38 but ya- you should prob just set a deadline. 21:22:41 dfg: could you add a short comment in gerrit saying that? 21:22:53 ok. deadline is next tuesday! 21:22:54 notmyname: it's in irc logs now - we can copy paste it 21:22:55 you know know what waiting on us is like :p 21:23:19 #agreed (by fiat) merge this by next tuesday 21:23:23 hurricanerix: dfg: np, thanks for the help 21:23:51 ok, next up 21:24:01 #topic testr change 21:24:06 patch 214206 21:24:06 notmyname: https://review.openstack.org/#/c/214206/ - Modify functional tests to use testr 21:24:07 yay... 21:24:14 hurricanerix: yay! 21:24:25 another "why hasn't this landed yet?" ;-) 21:24:45 last Iheard, the concurrency stuff has causing an issue 21:24:53 gmmaha: uploaded a new patch set 21:25:18 hmmm, it WFM now 21:25:18 oh, mattoliverau did the last one :-) 21:25:38 currently failing in the gate, though. mattoliverau or gmmaha any info on that? 21:25:47 so the concurrency fix is letting it pass func tests on an SAIO 21:26:03 +1 on mattoliverau comments 21:26:04 but when in-prcoess, it hangs at the end of the account func tests 21:26:16 that's whats failing in the gate. 21:26:21 havent checked the Ci failure.. can do that now 21:26:42 I started debuggung it yes arvo, but still haven't figured out why. 21:26:50 mattoliverau: thanks for looking 21:27:12 I even had a quick look at what the ironic guy said about os-testr, same issue. so debugging tests now. 21:27:25 mattoliverau: anything you need from anyone else? 21:27:58 I'll poke people if I totally fail today :) 21:27:59 are we still in doubt why concurrency needs to be 1 - there's a TBD in a comment on that? 21:28:17 seemed to me that it was to do with state being setup by tests 21:28:37 acoles: seems like merging the infra changes and making it run faster should be orthogonal 21:28:44 acoles: I don't know how testr splits things, we might be reusing account names or something 21:29:02 step 1, lets get it to work 21:29:05 yes! 21:29:18 mattoliverau: we do reuse accounts, thats the point 21:29:21 ok, mattoliverau will save us on this one. and gmmaha is looking too? 21:29:59 lol, we'll I'll continue where I left off yest.. anyone else is welcome to look too :) 21:30:05 notmyname: yes, i will assist mattoliverau in anyway i can 21:30:13 gmmaha: thanks man 21:30:16 mattoliverau: which is why i think concurrency must be 1 21:30:18 ok, good 21:30:27 thanks for looking at it 21:30:36 mattoliverau: my pleasure 21:30:53 ok, last small patch topic (I think) before the 2 bigger ones 21:30:58 #topic pyeclib updates 21:31:04 tsg not here 21:31:09 but i think I jsut saw somethign land 21:31:12 in -infra 21:31:36 so progress is being made there 21:31:47 mostly I wanted to check if there was any blocker from our end 21:31:50 o/ 21:31:53 tsg: yo 21:31:54 tsg: !!! 21:32:04 current status? any blockers on our end for pyeclib updates? 21:32:07 hey guys 21:33:21 so after a long huddle with -infra folks, we were able to get liberasurecode-dev installed by default on the CI slaves 21:33:29 the changes merged just an hour ago 21:33:29 yay 21:33:37 tsg: whoo! 21:33:42 rock n roll! 21:33:47 this means the >=1.0.7 patch can land? 21:33:50 yep 21:33:52 nice 21:34:04 then we've got to land a >=1.1.1 one, right? 21:34:09 it unblocks us to get pyeclib 1.2 upstream with no liberasurecode integrated 21:34:16 oh. 1.2 now :-) 21:34:31 so we'll jump from 1.0.7 to 1.2? 21:34:40 its a mad mad mad world 21:34:48 yes, next step is to land >=1.0.7, then update GR with >=1.2, and update swift req to that rev 21:34:53 good 21:34:56 we can make it 1.1.2 :-) 21:35:05 will this fix the ImportError in the py34 gate? http://logs.openstack.org/12/232712/3/check/gate-swift-python34/d401741/console.html#_2015-11-30_09_55_39_874 21:35:08 but removing liberasurecode integration is a change! 21:35:10 IMo make it whatever the lates is 21:35:22 timburke: yes, it unblocks some py3 stuff too 21:35:29 yep 21:35:33 sweet 21:35:40 this will move things for the work haypo has been pushing for 21:36:02 ok, then when we get the version updated in swift, we can do the thigns necessary to move pyeclib to the openstack namespace, which willlet more people help tsg out with getting releases done 21:36:10 so progress! 21:36:18 yep, progress for sure! 21:36:27 every release is a little better than the next 21:36:31 ok, that's great 21:36:44 and thanks to lifeless, clarkb etc on -infra for putting up with my persistent pestering ;) 21:36:46 clayg: it's because the numbers keep going up. you know that 1.2 is better than 1.1 because it's one bigger 21:36:57 notmyname: that's not what I said - but ok 21:37:01 heh 21:37:31 tsg: thanks for beign dogged about it. and thanks to -infra for helping getting it done :-) 21:37:32 notmyname, clayg: 1.1.x will continue to work if that matters at all 21:38:06 ok, I think that's good on status update patches. now a couple that are kinda significant IMO 21:38:10 notmyname: I will get Kevin on the next meeting so we can discuss the openstack namespace migration 21:38:18 great 21:38:22 #topic quoted etag patch 21:38:29 patch 250145 21:38:29 notmyname: https://review.openstack.org/#/c/250145/ - Unquote etag value in getting SLO/DLO 21:38:38 normal object don't quote etag values 21:38:44 the spec says to quote them 21:38:53 we don't because at one point long ago it broke clients 21:39:00 but *LOs quote etags 21:39:06 stupid etags 21:39:07 so we're inconsistent with ourself 21:39:13 therefore the broken clients don't use large objects 21:39:29 torgomatic: or maybe learned to handle it 21:39:35 notmyname: I wouldn't count on it 21:39:44 this patch proposes a change to resolve the internal inconsistency 21:39:53 notmyname: nope nope nope 21:40:24 if you have one thing that follows spec you don't break backwards compat to be ... was this just a "call in the -2's" 21:40:35 ... or am I misunderstanding 21:41:00 well, actually, I'm curious if we can resolve it by adding quotes to normal objects at this point 21:41:15 I'd generally be against it because we've seen this sort of thing break clients. but I have no recent data 21:41:28 but does that break our api? well existing clients out there anyway? 21:41:55 how long have we been quoting *LOs? 21:42:00 mattoliverau: always 21:42:06 right 21:42:09 notmyname: well then there you go! 21:42:09 ok 21:42:16 notmyname: I don't think we can quote etags 21:42:20 we've changed one or two things in the past to make them more conformant with the http spec 21:42:22 notmyname: not in v1 api anyway 21:42:40 notmyname: ... and had to revert them? 21:42:45 we can't add the quote- it would break a bunch of our customers 21:42:46 but there's also been a few places we haven't because it broke stuff 21:43:01 dfg: yeah, IIRC it was cloud files users/clients that broke 21:43:09 ya 21:43:09 notmyname: I only remember the ones where we didn't because it broke stuff 21:43:10 maybe we can make a couple public cloud providers turn on etag quoting and see who squawks /s 21:43:20 eg we also havne't "fixed" timestamp timezones, ETag vs Etag 21:43:30 notmyname: tbf cloud files *noticed* it (and also were the ones that changed it) 21:43:35 we all learned our lesson together 21:43:37 yeah 21:43:42 notmyname wasn't there a draft for a /v2 that had all this stuff on it? 21:43:46 to be pedantic, timestamp timezones are something we do wrong; ETag/Etag is something clients do wrong 21:43:58 hurricanerix: fuck yeah! api /v1.1 is the only hope! 21:44:51 hey now, isn't this a "family project"? :) 21:44:57 I just wish I had better info on who or what actually breaks when we start following the http spec closer 21:45:14 side note: if we really want to adhere to the spec (and probably break some clients), i suppose the etag for DLOs that have more segments than fit in a container listing really ought to be W/"d41d8cd98f00b204e9800998ecf8427e" 21:45:22 notmyname people who wrote custom code that expects things to look exactly like they do. =) 21:45:32 notmyname: we have the saem big client we used to that are still runnign the same they were 5 years ago :) 21:45:37 big clients 21:46:01 notmyname: unfortunately we will know about all users only afterwards :/ 21:46:13 dfg: I held out some small hope that the introduction of *LOs might make them learn how to etag properly ;-) 21:46:26 cschwede: yeah. fun. 21:46:37 ok. done 21:46:41 notmyname: thats a pretty small hope :p 21:46:50 dfg: I'm idealistic ;-) 21:46:56 I can go -2 that patch if one of you doesn't do it first 21:46:57 more of a hop 21:47:20 ok, next topic 21:47:28 #topic entrypoints patch 21:47:33 patch 206105 21:47:34 notmyname: https://review.openstack.org/#/c/206105/ - Use entrypoints for storage policy implementation ... 21:47:34 dfg: hop? 21:48:08 basically this loads object controller, diskfile, and policy classes from entry points 21:48:20 it's interesting, I think 21:48:21 notmyname: i like this one in theory - i don't understand people that think entrypoints will bring doom and destruction - it's just a better way to load shit than monkey patching! 21:48:36 yeah, I like it in theory too 21:48:47 * clayg hasn't looked at the implementation at all - i may have tons of tactical grief - but I like it in theory 21:48:58 in part because it makes known use cases in the existing ecosystem easier 21:49:25 however, torgomatic has a *really* good point that this gets us a lot closer to actually having to more formally support this internal API 21:49:43 (and that the current abstractions, especially diskfile, aren't the right things to make plugable) 21:50:26 so i think this one needs some review from a varied audience and more than jsut 2 +2s 21:50:54 so this is a call for reviews on it 21:51:10 unless there is a specific thing to bring up here and now on it 21:51:34 tdasilva: ? 21:52:09 notmyname: that's fair - but torgomatic and I disagree - a) we can change whatever we want - let 'em belly ache b) the only problem in computer science that *can't* be solved by another level of abstraction is too-many-abstractions and three is not too many - when we have 18 and more docs than base-classes we can bitch about how it's hard to get started with the plugins and have a whiteboard session to make it simpler, bum 21:53:20 notmyname: ok, so if we have time we should look at entrypoints and it should take more +2's than normal - so it's not going anywhere anytime soon 21:53:24 yeah, I think there will be valid disagreements and arguments on both sides. this is more of a "strategy" change in the patch than the actual technical changes, and I want to make sure we the community are ok with it 21:53:28 right 21:53:51 notmyname: I just want to make sure the actual technical changes are good - anything is better than monkey patching 21:54:01 ok, last topic (looking at the time) 21:54:09 #topic next swiftclient release 21:54:11 notmyname: I've had to write code that interfaces with swift - we all want entrypoints - for sure 21:54:22 just this morning patch 252308 landed 21:54:22 notmyname: https://review.openstack.org/#/c/252308/ - Remove py26 support from swiftclient (MERGED) 21:54:29 so there is no more py26 testing 21:54:34 whoa 21:54:49 (meta comment, it sure is nice than -infra patches take about 8 minutes to land instead of hours/days for ours!) 21:55:10 {'cool': True for v in python_versions if v >= 2.7} 21:55:12 so I propose that we tag what's HEAD of master now and release it as swiftclient 2.6.1 21:55:54 if there is any other patch that has not landed, it must be very very special and it must have a "I've personally run this on py26 and it works" from a core reviewer 21:56:00 notmyname: so we *just* dropped 2.6 support - so push that out to everyone? when was the last release that supported 2.6 then? 21:56:16 oh oh wait 21:56:19 clayg: currently, all releases of swiftclient support py26 21:56:26 I'd say this would be the last one 21:56:30 clayg: so that will be the last to support it 21:56:42 so makes sense to me 21:56:45 I'd say draw the line in the sand right where we are and call py26 done 21:57:01 and going forward we are py27,py3 only 21:57:13 +1 21:57:40 notmyname: ok perfect 21:57:40 works for me 21:58:08 +1 21:58:16 no objections here 21:58:23 kk. thanks 21:58:30 did timburke or joel have anything pressing to merge first? other wise I am +1 21:58:32 I'll work on that over the next 24 hours 22:00:10 I think we're out of time 22:00:38 only other this was to say I'll look for swift patches for a release there, but we've got a few big ones still outstanding we all want, i think (like rings) 22:00:43 thanks everyone for coming today 22:00:53 thanks for your work on swift. you make the code and the community awesome 22:00:56 #endmeeting