21:00:42 <notmyname> #startmeeting swift
21:00:43 <openstack> Meeting started Wed Jul  5 21:00:42 2017 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:46 <openstack> The meeting name has been set to 'swift'
21:00:48 <notmyname> who's here for the swift team meeting?
21:00:52 <joeljwright> hey
21:00:52 <mattoliverau> o/
21:00:52 <jungleboyj> o/
21:00:53 <m_kazuhiro> o/
21:01:17 <kota_> o/
21:01:20 <clayg> weee
21:01:22 <acoles> hello
21:01:48 <torgomatic> hi
21:01:50 <notmyname> cschwede won't be able to make it this week
21:01:51 <rledisez> o/
21:02:19 <notmyname> tdasilva: around?
21:02:30 <tdasilva> hello
21:02:55 <notmyname> welcome everyone
21:03:15 <notmyname> I think this will be a relatively quick meeting this week
21:03:24 <notmyname> Link https://wiki.openstack.org/wiki/Meetings/Swift
21:03:44 <notmyname> #topic upcoming events
21:03:51 <notmyname> the PTG and the summit are coming up
21:03:59 <notmyname> #link https://etherpad.openstack.org/p/swift-ptg-queens
21:04:09 <notmyname> that's the etherpad to start collecting topics for the PTG
21:04:37 <notmyname> please register for the PTG and get your hotel soon so that you have a close place before they are all booked
21:05:01 <tdasilva> is the conference hotel already full?
21:05:08 <notmyname> I hope not!
21:05:23 <notmyname> I just booked mine last week, so I'd guess there is still room
21:05:36 <notmyname> https://www.openstack.org/ptg has all the PTG info
21:06:12 <acoles> I booked ok last week, except the Saturday before was full
21:06:26 <notmyname> the schedule for the PTG will be similar to the last one in atlanta: monday-tuesday will have a lot of "cross project" things to invest in. wednesday-friday will have rooms dedicated to the swift team
21:06:47 <notmyname> please, *please* encourage all ops people to attend the PTG
21:06:52 <jungleboyj> Just checking.  There is room.
21:07:00 <notmyname> if you have to choose, choose the PTG over the sydney summit
21:07:00 <mattoliverau> the last one was good, I recommend it to others
21:07:33 <notmyname> yeah, I expect it to be a very productive week
21:07:34 <joeljwright> we need a mattoliverau lifesize cutout
21:08:02 <mattoliverau> well maybe not life sized.. you could at least make it slimming ;)
21:08:09 <joeljwright> :D
21:08:13 <notmyname> the other upcoming event, of course, is the summit in sydney. info at https://www.openstack.org/summit/
21:08:36 <notmyname> the call for presentations for sydney is only open for about another week, so if you're interested in giving a talk, now's the time to submit it
21:08:50 <notmyname> note that the sydney summit is only 3 days long
21:09:06 <mattoliverau> :(
21:09:18 <mattoliverau> 3 days is teh dumb
21:09:31 <notmyname> while I expect to be going, I do not expect there to be much in the way of deep, productive swift team time (ops or otherwise)
21:09:44 <notmyname> therefore, that is why I am encouraging all devs and ops to be at the PTG instead
21:09:53 <mattoliverau> except for visiting the beach house right?
21:09:58 <mattoliverau> notmyname: ^
21:10:23 <notmyname> for most of the world, sydney is a *very* long and expensive trip for a 3 day conference that doesn't have any dedicated time for our team
21:10:43 <clayg> OMG the sessions is still empty :\
21:10:48 <clayg> it's like way out in september tho?
21:10:50 <jungleboyj> mattoliverau:  That is my plan.
21:11:02 <clayg> july, august - it seems like we have awhile yet
21:11:05 <notmyname> clayg: for the PTG?
21:11:07 <mattoliverau> jungleboyj: ;)
21:11:08 <notmyname> yeah
21:11:18 <jungleboyj> mattoliverau:  Bucket list check point, here I come.
21:11:28 <notmyname> any questions about the ptg or the summit?
21:12:19 <notmyname> all right. moving on, then
21:13:14 <notmyname> I wanted cschwede to share his follow-up for bug triage, but he's not able to make it today. he said he'd follow up via email
21:13:27 <notmyname> #topic docs migration work
21:13:43 <notmyname> have you been following the mailing list threads about the docs migrations?
21:14:14 <notmyname> the summary is that for various reasons, the stuff that is currently managed by the docs team will be moved into respective project repos
21:14:48 <notmyname> and the work to do so is on each team (in large part because nearly the whole docs team got laid off at the same time mattoliverau did :-( )
21:15:04 <mattoliverau> :(
21:15:12 <notmyname> I was traveling last week and holiday early this week, but i'm starting to look at that migration work now
21:15:13 <jungleboyj> How about that 1.2 Gig Docs repo?
21:15:21 <notmyname> heh, only 760MB ;-)
21:15:43 <notmyname> so there's a couple of steps here
21:15:44 <jungleboyj> Yeah, after I did a build it ballooned and almost killed my VM.  :-(
21:15:56 <kota_> sounds so huge docs
21:16:02 <notmyname> first is that we'll have to import a bunch of stuff to the swift and swiftclient repos
21:16:15 <notmyname> *after* that, we can rearrange, de-dup, etc
21:16:17 <jungleboyj> <little kids voice>  So big! </little kids voice>
21:16:24 <timburke> but that's across all projects, right? not 3/4 gig just for swift?
21:16:33 <jungleboyj> timburke:  Right.
21:16:34 <notmyname> timburke: correct. it's just the source repo
21:16:41 <notmyname> jungleboyj: have you already started looking at the docs migration work for swift?
21:16:47 <tdasilva> notmyname: isn't the first step to start using the new openstackdocs theme? https://review.openstack.org/#/c/478402
21:16:47 <notmyname> cause if so... yay!
21:16:48 <patchbot> patch 478402 - swift - Switch from oslosphinx to openstackdocstheme
21:16:58 <kota_> oic
21:17:01 <jungleboyj> notmyname:  No, I was supposed to do it for Cinder.
21:17:06 <notmyname> jungleboyj: ah ok
21:17:23 <notmyname> tdasilva: perhaps. I'm getting caught back up to it. seems like that would be a simple thing to do
21:17:23 <jungleboyj> Panic'ed today when I started noticing that our links out on the web are broken.
21:17:37 <notmyname> yeah, I heard that happened (all the 404s)
21:17:44 <jungleboyj> notmyname:  Depending on how long this takes for Cinder I could help out with Swift.
21:18:04 <notmyname> jungleboyj: cool. ping me when you're done with cinder, and we'll see where the swift migration is
21:18:11 <tdasilva> notmyname: i wish, I think it really breaks the look and feel of the docs, so I think we will have to do meaningless tweaks over time just for things to look nice again :/
21:18:15 <notmyname> jungleboyj: try to go "swiftly" on cinder
21:18:33 <notmyname> tdasilva: :-(
21:18:40 <jungleboyj> notmyname: Sure.  Is anyone starting on it?
21:18:43 <notmyname> I am
21:19:15 <jungleboyj> notmyname:  Ok, let me know if you have questions.  I have found, already, that the build fails when just copying things over.
21:19:22 <notmyname> :-(
21:19:30 <jungleboyj> They have also removed everything from the openstack-manuals tree.
21:19:32 * notmyname was hoping for a few days of work and be done with it
21:19:40 <clayg> tdasilva: what breaks?  patch 478402 looks fine?
21:19:41 <patchbot> https://review.openstack.org/#/c/478402/ - swift - Switch from oslosphinx to openstackdocstheme
21:19:43 <jungleboyj> So you need to go back to the 'before-migration' tag.
21:20:22 <tdasilva> clayg: builds don't break, it's very subjective, I'm just used to the old looks, so I don't like how things look in the new theme
21:20:36 <tdasilva> clayg: compare: http://docs-draft.openstack.org/02/478402/5/check/gate-swift-docs-ubuntu-xenial/7563106//doc/build/html/associated_projects.html#associated-projects
21:20:38 <clayg> oh... sorry I don't have opinions on that sort of thing
21:20:43 <tdasilva> to https://docs.openstack.org/swift/latest/associated_projects.html#associated-projects
21:20:47 <tdasilva> lol
21:21:11 <clayg> yeah I mean both pages seems to have the same information :\
21:21:34 <clayg> I assumed since someone spent time on it they must think one is better than the other?
21:21:46 <notmyname> we can work on the prettiness later
21:22:05 <clayg> maybe one has the denting wrong?
21:22:05 <notmyname> yes, this work is not insignificant, and it is being forced upon us as a team by external forces
21:22:09 <jungleboyj> notmyname:  ++
21:22:24 <jungleboyj> I was planning to just get Cinder's stuff moved and then work on cleanup afterwards.
21:22:24 <clayg> it sorta looks wrong on both tho - un-even
21:22:32 <notmyname> but docs are important, so we need to do it
21:22:39 <clayg> let's do it!
21:23:04 <notmyname> I'll start, and hopefully finish, without needing to bring in too many other people. but I may need to ask for help, depending on how it goes
21:23:40 <notmyname> but also since this is a "do it or lose your docs" sort of situation, I expect the eventual reviews to be cursory "yup, content moved, +2/+A
21:24:01 <notmyname> so don't freak out when a lot of new stuff gets added :-)
21:24:09 <jungleboyj> notmyname:  Let me know if I can help.  I am hoping to get a couple of patches up yet today for Cinder.  I am traveling tomorrow but will probably spend Friday and the weekend working on the rest.
21:24:10 <acoles> tdasilva: did someone figure we're all getting older and increase the font size accordingly? ;)
21:24:21 <notmyname> jungleboyj: thanks
21:24:22 <tdasilva> acoles: hehe
21:24:23 <notmyname> acoles: yes!
21:24:27 <clayg> WHO MOVED MY CHEESE
21:24:42 <notmyname> ok, any questions about the docs migrations?
21:24:42 * jungleboyj shakes my head
21:24:46 <jungleboyj> I refuse to get older!
21:25:22 <notmyname> ok, moving on
21:25:31 <notmyname> #topic open discussion
21:25:32 <clayg> toward the cheese
21:25:48 <notmyname> I expect there to be swift and swiftclient releases in the next few weeks
21:25:50 <notmyname> jFYI
21:26:05 <tdasilva> we had a libec/pyeclib release last week
21:26:09 <notmyname> I'll keep the priority reviews page updated
21:26:11 <clayg> why is patch 470158 crossed off on priority reviews?
21:26:12 <notmyname> tdasilva: nice!
21:26:12 <patchbot> https://review.openstack.org/#/c/470158/ - swift - Write-affinity aware object deletion
21:26:17 <tdasilva> working now to get rpm out for it
21:26:50 <notmyname> clayg: good question. I think christian did that. looks like a typo
21:26:51 <mattoliverau> I don't know. I need to go loop back around and review that
21:26:58 <tdasilva> is onovy building .deb packages?
21:27:00 <clayg> mattoliverau: do you know how len(local_primary_nodes) is calculated?
21:27:24 <notmyname> now uncrossed
21:27:27 <mattoliverau> yeah, using the is_local method
21:27:40 <mattoliverau> and looping thorugh the primaries..
21:27:52 <notmyname> torgomatic: anything we need to know about your patch 340526 vs zaitcev's PUT+POST?
21:27:53 <patchbot> https://review.openstack.org/#/c/340526/ - swift - Unify Putter and MIMEPutter into a single class.
21:28:00 <clayg> ahh... so it *is* per part - that's EPIC
21:28:00 <mattoliverau> it kinda does it when it needs it.. I wonder if that can be cleaned up to be in one place.
21:28:06 <kong> morning, guys
21:28:24 <torgomatic> nothing to know yet; we'll see how things shake out
21:28:42 <torgomatic> I had some thoughts around using different verbs for versioning instead of putting it in the path, but no code yet
21:28:50 <clayg> kong: I think mattoliverau is telling us we can probably merge patch 470158 - so that's exciting!
21:28:50 <patchbot> https://review.openstack.org/#/c/470158/ - swift - Write-affinity aware object deletion
21:29:18 <torgomatic> that way, we get a 405 when the object server is old, and if we need v3 or v4 or whatever, we just have to make up and implement some new verbs
21:29:25 <kong> clayg: hooray!
21:29:33 <mattoliverau> I'll go give it another whirl today, and if all good I'll give it the second +2 ;)
21:29:34 <torgomatic> but like I said, no code yet
21:29:37 <mattoliverau> kong: ^
21:30:05 <kong> mattoliverau: thanks, feel free to ping me at irc if you have any question
21:30:33 <mattoliverau> kong: cool, we share a similar timezone, so that works well ;)
21:30:39 <kong> ataraday_: i see ;)
21:30:41 <notmyname> torgomatic: seems a little weird to invent new verbs all the time instead of actually, you know, a protocol version number
21:30:45 <kong> mattoliverau: i see :)
21:31:29 * kong stops disturbing swift meeting
21:31:33 <clayg> kota_: have you been following kevin & tushar on the isa-l bug?
21:31:34 <torgomatic> notmyname: true, but this lets you do it on an operation-by-operation basis... you can come up with a new bit of stuff for object PUT, then later on do something special for GET, but you don't have to tie them together with a single version number
21:32:03 <notmyname> kong: you are very welcome here. you aren't disturbing anything. this meeting is for you :-)
21:32:04 <clayg> acoles: kota_: do we have more exposure on this isa-l rs_vand >= 5 parity thing or not?
21:32:07 <kota_> clayg: is that update recently in github issue traacker?
21:32:17 <clayg> kota_: yeah pretty sure
21:32:32 <notmyname> clayg: ah, right. thanks for bringing that up
21:32:44 <kota_> i looked at it briefly but i don't think kevin and tuchar following that
21:33:08 * kong is reading what you are saying carefully to pretend he understands everything
21:33:26 <clayg> kota_: I also want to get some details on urgency of lp bug #1701472
21:33:27 <openstack> Launchpad bug 1701472 in OpenStack Object Storage (swift) "ring builder can converge dead state in rebalance" [Undecided,New] https://launchpad.net/bugs/1701472
21:33:48 <kota_> clayg: yeah, i'm still working on how it works
21:34:03 <kota_> sorry i have been sick in the primary half of this week.
21:34:05 <notmyname> clayg: kota_: how should we proceed on the isa-l bug? kota_ can you ping them? do we need to ignore it and let intel deal with it (that seems bad)?
21:34:15 <clayg> this is the EC bad frag decode bug https://github.com/01org/isa-l/issues/10
21:34:42 <clayg> the heart of the issue that caused us to refuse to start swift if some isa-l policies were not deprecated
21:34:54 <notmyname> kong: please don't hesitate to ask for clarity. IRC meetings are the hardest time for non-english speakers because everything goes by so quickly
21:35:57 <kong> notmyname: sure, i will. btw, for some people who don't know, we (catalyst cloud in New Zealand) have been using Swift at production since last year.
21:36:14 <kong> we are keen to help fix bug :-)
21:36:15 <clayg> wooo!  wtg kong !!!!
21:36:27 <mattoliverau> kong: oh your with catalyst!
21:36:42 <kong> mattoliverau: yeah
21:36:44 <notmyname> kong: :-)
21:36:52 <mattoliverau> kong: say hi to bruno ;)
21:37:03 <kong> mattoliverau: hah, i promise i will
21:37:13 <tdasilva> kong: https://bugs.launchpad.net/swift there's only 500 of them :D
21:37:24 <notmyname> heh
21:37:35 <mattoliverau> kong: also tell him to hire me ;P
21:37:37 <kong> i will try my best to figure out what i can help there
21:37:54 <kong> mattoliverau: hah, actually, we were talking about that in yesterday's meeting
21:38:14 <tdasilva> kong: just kidding, all help is welcome and thanks for working on patch 470158, I remember Bruno saying that's very important to your customers
21:38:14 <patchbot> https://review.openstack.org/#/c/470158/ - swift - Write-affinity aware object deletion
21:38:45 <kong> tdasilva: yes it is. after merging, i will backport it to our internal repo
21:38:52 <kong> we are using a old version of swift
21:39:14 <clayg> kota_: so but the ring rebalanace bug - just to understand the priority ... it is annoying ... but do you have a workaround?
21:39:19 <tdasilva> kong: would be great to see someone from catalyst at the PTG
21:39:48 <clayg> kota_: like just increase overload or add a node or something?  i guess you can just use the any/all trick get the rings you want and ship it out - out-of-band change only!
21:39:57 * kong will ask Bruno to arrange some money for PTG :)
21:40:03 <acoles> clayg:  https://github.com/01org/isa-l/issues/10 seems to be saying that there may be an issue with 4 parity frags if num data frags > 21
21:40:06 <notmyname> kong: great! thanks
21:40:15 <notmyname> acoles: that's what scares me
21:40:21 <kota_> for rebalance, it's not super high priority but working soon is better because it looks like making dead lock state to rebalance on some cases
21:40:49 <clayg> kota_: I only ask because the all/any change isn't obvious to me - it seems the code did intend to do what it was saying - like it wasn't just a typo - i'm freaked out that more tests don't rub up against that condititonal?
21:40:50 <kota_> i notice some ways to mitergate the dead lock by hand operation though
21:41:23 <kota_> clayg: yup, changing overload or weight could help to rebalance work
21:42:16 <clayg> kota_: ok, tagged, torgomatic is working around lp bug #1697543
21:42:18 <openstack> Launchpad bug 1697543 in OpenStack Object Storage (swift) "Ring refuses to save even when 100% parts move" [Medium,Confirmed] https://launchpad.net/bugs/1697543
21:42:35 <kota_> clayg: ok, current code is in your intention, I need to rethink the algorism because I WAS not sure if it was intended or not
21:42:46 <tdasilva> acoles: not sure this is related, but found this: https://github.com/ceph/ceph/blob/master/src/erasure-code/isa/README#L23 and this https://github.com/ceph/ceph/blob/master/src/erasure-code/isa/ErasureCodeIsa.cc#L381
21:43:22 <clayg> kota_: which is definitely a different issue - but christian pointed out that the associated change doesn't fix the ring/script in the bug - so maybe rubbing some braincells together will help
21:44:24 <kota_> clayg: kk, thx. I will look at.
21:44:31 <clayg> kota_: yeah... it was there to prevent picking up parts from one disk and putting them back on the same server
21:44:47 <clayg> kota_: but... if the disks on the server are already not balanced!!!  :\
21:45:05 <notmyname> tdasilva: should we add those same constraints to libec?
21:45:38 <kota_> clayg: yeah, and my simulation with current code, in some case, the gatherd parts seems to go back to same servers as reassignment.
21:45:55 <kota_> which I reported in the lp bug.
21:46:11 <tdasilva> notmyname: i'm not sure, but I definetely found interesting that they will just change the k for the user, don't allow at all
21:46:14 <clayg> tdasilva: acoles: notmyname: yeah I think the rules outlined in the bug report are acctually correct
21:46:15 <acoles> tdasilva: thanks. so that also shows cap of 32 data frags
21:46:36 <notmyname> tdasilva: yes. "interesting"
21:47:22 <notmyname> clayg: the rules listed on https://github.com/01org/isa-l/issues/10#issuecomment-310944022 ?
21:47:49 <clayg> yeah
21:48:07 <notmyname> maybe our answer then is to user isa-l v2.19 then
21:48:43 <tdasilva> last comment is funny...there's no source,  i just know it!
21:49:06 <clayg> i mean... that's probably a good idea - but also probably orthogonal - unless they expose a "check if these params are cool" sorta method?
21:49:21 <clayg> brute force
21:49:52 <clayg> but that last one is interesting - it makes it sound like a 7+6 is ok!
21:49:52 <notmyname> are there any other topics to bring up this week in the meeting? does anyone else have something to discuss?
21:51:13 * kota_ need to look at the k, m conditions on https://github.com/01org/isa-l/issues/10#issuecomment-310944022 to clarify
21:51:30 <notmyname> I think we're done, then
21:51:48 <jungleboyj> Thanks.
21:51:51 <notmyname> thanks for coming, everyone. thanks for working on swift
21:51:54 <notmyname> #endmeeting