21:00:33 <notmyname> #startmeeting swift
21:00:34 <openstack> Meeting started Wed Sep 20 21:00:33 2017 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:35 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:38 <openstack> The meeting name has been set to 'swift'
21:00:39 <notmyname> who's here for the swift team?
21:00:46 <mattoliverau> o/
21:00:46 <notmyname> ...team meeting
21:01:01 <m_kazuhiro> o/
21:01:04 <tdasilva> hi
21:01:08 <rledisez> hi o/
21:01:23 <acoles> hello
21:01:30 <timburke> o/
21:01:49 <timburke> there's a chance i'll need to drop out for some other meeting though :-(
21:02:05 <notmyname> yeah, I'm sitting in another one myself
21:02:46 <notmyname> I hope everyone got home from the PTG safely
21:03:06 * kota_ is almost read only
21:04:29 <notmyname> I just got back home late last night, so I'll admit I haven't fully caught up yet
21:05:09 <notmyname> but, mostly this week I want to review the PTG, now that we've had a chance to sleep on it, and see what's obvious that we need to keep doing
21:05:18 <notmyname> ther PTG etherpad is at
21:05:20 <notmyname> #link https://etherpad.openstack.org/p/swift-ptg-queens
21:05:24 <notmyname> (for review)
21:05:44 <notmyname> and the retrospective we did (with the few action items recorded) is at
21:05:47 <notmyname> #link https://postfacto.io/retros/swift
21:05:54 <notmyname> #topic ptg review
21:05:59 <notmyname> so...
21:06:12 <notmyname> what did you think?
21:06:56 <notmyname> here's what we said in denver: https://postfacto.io/retros/swift/archives/8680
21:06:58 <acoles> notmyname: I liked the retrospective tool we used
21:07:07 <timburke> i got lots of patches merged -- i'm happy
21:07:09 <notmyname> acoles: cool. I did too
21:07:19 <tdasilva> Enter the password to access Swift Denver PTG.
21:07:20 <acoles> is that 'meta-retrospective'?
21:07:26 <notmyname> tdasilva: hmm
21:07:31 <notmyname> sorry
21:07:34 <mattoliverau> I thought it went great. Especially thanks for acoles, timburke and torgomatic  for spending so much time on Sharding. It's coming along nicely now
21:07:53 <acoles> mattoliverau: no speed bumps on a feature branch :)
21:07:58 <tdasilva> notmyname: https://postfacto.io/retros/swift is fine, shows action items
21:08:02 <mattoliverau> :)
21:08:28 <notmyname> tdasilva: yeah, the archive one shows the stuff we said was good/bad/questions
21:08:41 <notmyname> here's a way to rephrase the question. now that you're back home, you've thought more about the PTG and you've probably talked to coworkers about it and whatever you missed at work while in denver.
21:09:08 <notmyname> so has your perspective on what was done in denver or what you need to to next any different than what happened last week
21:10:08 <notmyname> eg since the PTG I've had several conversations (in the office and otherwise) that reenforce the s3api work we talked about
21:10:31 <tdasilva> cool, i've started some of that conversation on my side too
21:10:45 <notmyname> also I almost immediately ran into a support question that made me want to have policy migration landed a long time ago ;-)
21:12:39 <notmyname> hmm... /me is doing a stellar job of prompting discussion ;-)
21:12:49 <tdasilva> heh
21:12:50 <joeljwright> :)
21:12:52 <acoles> ok, I'll pitch in
21:12:53 <notmyname> ok, how about the bug triage work?
21:12:55 <mattoliverau> I think the ptgbot kinda worked, except I kept forgetting to check it
21:13:16 <tdasilva> so one of the things I've been thinking is how do we continue with the action items...things we said we woud do in the future
21:13:25 <tdasilva> like bug triage work and virtual meetings
21:13:30 <acoles> ...for me, I'm still happy with the outcomes of the week. feature branch is definitely working well.
21:13:52 <mattoliverau> I think we getting better at hacking in a room, we managed to get things done without a projector :)
21:14:04 <notmyname> acoles: glad to hear it
21:14:07 <mattoliverau> Which I didn't miss
21:14:18 <notmyname> mattoliverau: yeah, the room layout went pretty well. I agree
21:14:20 <acoles> ...I found it helpful to understand what is important to different people and I think we need to make sure we continue to communicate those priorities amongst ourselves
21:14:45 <acoles> ...and I appreciate people travelling to be there because it is always good to meet face to face
21:14:50 <notmyname> tdasilva: what have your thoughts been?
21:15:10 <notmyname> acoles: I too like that people shared what was important to them and their employers
21:15:53 <tdasilva> notmyname: just how can we make concrete plans...for example: is it too early to try to schedule a virtual mid-cycle?
21:15:58 <kota_> +1
21:16:30 <tdasilva> notmyname: how will we continue making progress with bug triage? is that a monthly meeting? every-other? how do we drive down the bug count?
21:16:35 <torgomatic> I was glad to make progress on container sharding. That was really my one goal for the PTG; anything else that happened was just gravy.
21:16:48 <notmyname> torgomatic: a gravy train?
21:16:50 <notmyname> tdasilva: I don't think it's too early. we're not in a particular rush (eg we don't need it planned by next week), but starting planning now isn't a bad idea
21:17:20 <tdasilva> notmyname: yeah, I wasn't thinking planning as in topic, but maybe start thinking about dates cause then people can plan for that
21:17:25 <notmyname> tdasilva: for bug triage, we agreed to have one meeting a month focused on it, right?
21:17:39 <notmyname> tdasilva: yeah. that's good
21:19:07 <tdasilva> notmyname: the action items says 'dedicate weekly meeting to bug triage', but i'm not sure what that would look like...
21:19:28 <tdasilva> if we try to triage bugs in this meeting we will get through very few of them
21:19:51 <notmyname> do you want to try to do it next week and see what happens? and try to plan it before then?
21:20:35 <notmyname> you're right that going through in a meeting won't work great (similar to how we can't do code review in an hour long IRC meeting), but what else could we do?
21:20:45 <notmyname> report on stuff triaged?
21:20:52 <notmyname> pass out things to triage for people?
21:21:13 <acoles> how about if we advertise that at next meeting we will review bug status and *maybe*. that will act as an incentive for us to nail a few more *before* that meeting?
21:22:02 <notmyname> ok
21:22:22 <mattoliverau> How would a virtual midcycle work? All try and meet at the same timezone but remotely, cause it's one thing to do when you in the same city but harder from home.. especially for those of us who are opposite timezones? Or run it for a week and try and find overlaps to different discussions. I'm not against either, just need to think about how to make it work.
21:22:24 <tdasilva> yeah...I was thinking about what acoles said back in Denver. how maybe just the idea of designating a day is helpful. Like "On Sept. 27th, will be swift bug triage day" and hopefully people can dedicate a good chunk of their time on that day for that
21:22:35 <clayg> but I'm *here*
21:23:18 <notmyname> anyone know someone who's run virtual midcycles before?
21:23:20 <clayg> lol @ meta-retrospective!
21:23:25 <mattoliverau> I know ironic did some virtual midcycles
21:23:34 <tdasilva> manila is running this week
21:23:35 <mattoliverau> I think it's was the former.
21:24:01 <tdasilva> i was kind of waiting end of the week to ping a few manila folks and ask about it
21:24:09 <mattoliverau> Cool
21:24:18 <notmyname> yeah, that'd be great
21:24:54 <notmyname> ok, thinking more about a bug triage day
21:25:10 <notmyname> it's interesting to think of it on a meeting day (eg sept 27, next week)
21:25:42 <notmyname> because for EU, you'd end talking about what happend, for US we'd do it in the middle of the day, and for APAC it's the start of the day
21:26:31 <clayg> @timburke where are those fancy charts and graphs for bugs published?
21:26:33 <notmyname> wednesdays seem to be "meeting days" for me, so that might lend itself to picking off some bugs to triage here and there in between meetings
21:26:42 <notmyname> /bugday?
21:26:49 <mattoliverau> True, tho APAC would be the day after
21:27:08 <mattoliverau> Unless we make our triage day the day after yours ;)
21:27:16 <acoles> APAC can do the day before and after :)
21:27:17 <timburke> yeah, http://status.openstack.org/bugday/ -- but it only keeps like three days worth of history i think?
21:27:22 <mattoliverau> Lol
21:27:24 <clayg> I really like the idea of trying to prioritize it with gamification
21:27:41 <timburke> ...and that's when it's actually working...
21:27:56 <notmyname> ok, not just me getting a blank page for that
21:28:00 <clayg> @timburke conducive to short focused group effort!
21:28:09 <acoles> I know it be a pipe dream but I think that somehow knowing there is a collective day of action will make me pay attention
21:28:12 <tdasilva> also checkout https://github.com/thiagodasilva/launchpad-reporting for swift launchpad graphs
21:28:17 <notmyname> they just upgraded gerrit, so probably all busted
21:28:43 <timburke> what, no gh-pages? :P
21:28:58 <clayg> tdasilva: yes!  I think i've seen that before - maybe we could get it hosted & published somewhere?
21:29:08 <acoles> gerrit mail seems to be stalled too, I've seen none today
21:29:29 <notmyname> acoles: I find that a relief. my unread count is going up, not down ;-)
21:29:33 <mattoliverau> tdasilva: awesome!
21:29:51 <mattoliverau> If only I still had cloud I'd host one there
21:30:17 <clayg> tdasilva: what's least effort for you?  Mostly you just need everyone to do X bug triage per week?
21:30:54 <clayg> tdasilva: how do we know we're doing that - how can I hold myself accountable - how can someone else hold the team as a whole accountable?
21:31:01 <tdasilva> https://snag.gy/Wa1qRl.jpg
21:31:39 <clayg> can you zoom in on that drop?
21:31:43 <acoles> notmyname: hmmm, actually, may be that my mail filter needs tweaking to include feature/deep in what I see. IDK!
21:32:05 <clayg> I think just having that IN OUR FACE all the time would make a huge difference in driving behavior
21:32:19 <mattoliverau> +1
21:32:34 <clayg> I see absoultely ZERO reason that tdasilva shouldn't just feel totally comfotrable throwing that at us every week - YOU GUYS SUCK - LOOK AT THESE BUGS - DO BETTER!
21:32:40 <clayg> totally WFM
21:32:56 <notmyname> tdasilva: if you don't have a place, I can find one to host it
21:33:05 <tdasilva> clayg: TBH, I'm not sure. Ideally, we need everyone to commit to do X bug triage per week, but that's hard to hold people accountable. and we also need people to commit to start fixing bugs, me included
21:33:06 <clayg> notmyname: yeah you host it!
21:33:21 <acoles> clayg: +1 - hold us accountable
21:33:22 <clayg> right fixing them... one step at a time ;)
21:33:36 <notmyname> ok, I see two things
21:33:42 <tdasilva> but yeah, graph probably helps
21:33:44 <notmyname> one, host tdasilva's tool somewhere
21:33:56 <notmyname> two, designate a day for global bug traige
21:34:00 <notmyname> *triage
21:34:03 <clayg> yeah - i think that could be fun
21:34:20 <clayg> and with the graph at the end where we can *see* the drop of our efforts (and brag to our bosses about effort *well spent*)
21:34:23 <notmyname> shall we say next wednesday for bug triage? would that work?
21:34:30 <clayg> ... I think that'd be a huge improvment
21:34:34 <clayg> ... ad tdasilva is a hero
21:34:57 <mattoliverau> Next Wednesday or Thursday :p
21:35:09 <mattoliverau> Ill assume the day of the meeting
21:35:25 <notmyname> heh, ok. next week, "day of the meeting" then :-)
21:35:36 <kota_> mattoliverau has good question
21:35:41 <tdasilva> would it be helpful to hold a 'virtual video room' open for high bandwidth conversations? or is IRC enough?
21:35:43 <clayg> idk, I have other meetings in the morning... but at least it's not a release week... I haven't really "checked" with anyone - but I don't think my plate is over full ATM :\
21:36:03 <mattoliverau> Unless we want to look at the graph at the meeting to see how we're doing then Wednesday is good ;)
21:36:04 <clayg> I'll can put it on my calendar as not busy and do my best?
21:36:19 <clayg> mattoliverau: *I* want a graph!?
21:36:22 <notmyname> clayg: yeah, since there are other meetings, I wonder if that might make it easier to pick off some bugs in between instead of trying to find a giant block of uninterrupted time
21:36:52 <notmyname> yeah, the graph is good, regardless of bug day :-)
21:36:56 <notmyname> just needs to get hosted somewhere
21:37:03 <clayg> tdasilva: IRC is plenty for me - I don't recall needing anything particularlly high bandwidth for triage - it's somewhat decomposable and parallelizable
21:37:26 <tdasilva> notmyname: if you can help with the graph hosting part, that would be great
21:37:32 <clayg> notmyname: maybe you and tdasilva could schedule a time to high-bandwidth/screen share to get that good business hosted!?
21:37:32 <notmyname> tdasilva: on it
21:37:54 <clayg> tdasilva: be sure to write down all the sekret passw0rds
21:38:02 <notmyname> tdasilva: I've added it to my todo list
21:38:17 <notmyname> tdasilva: i'll ping you about it later (probably tomorrow at the earliest)
21:38:19 * clayg dusts hands (it's on notmyname's todo list)
21:38:43 <notmyname> I think to start with IRC should be enough for a bug triage day
21:38:56 <tdasilva> agree
21:38:57 <clayg> +1
21:39:10 <clayg> but video adhocary would be awesome
21:39:26 <notmyname> #agreed next week, "day of the meeting" (in your timezone) is a bug triage day!
21:39:30 <clayg> it might be enough just knowing everyone is on the same page - I like to chat folks - I'M NICE
21:40:24 <notmyname> yeah, I think that collective sense of common task will be very helpful
21:40:27 <clayg> notmyname: tdasilva: it's on my calendar - it will be
21:40:41 <notmyname> #topic open discussion
21:40:44 <notmyname> what else to talk about?
21:40:51 <notmyname> joeljwright: what's up with ambles?
21:41:09 <joeljwright> I was hoping you could tell me that
21:41:09 <clayg> kota_: did you look at the ring rebalance bug?  does it fix the rings for that one team that discovered the issue?
21:41:10 <joeljwright> :)
21:41:17 <tdasilva> oh lol, one more thing for next week...forgot about Pete's meeting
21:41:32 <m_kazuhiro> tdasilva: I think all related patches for symlink are merged. Can we start review cycle on Symlink?
21:41:57 <joeljwright> I've not seen any further comments on https://review.openstack.org/#/c/365371/
21:41:58 <patchbot> patch 365371 - swift - Add Preamble and Postamble to SLO and SegmentedIte...
21:41:59 <tdasilva> m_kazuhiro: yeah, i pushed a new patchset of symlinks today
21:41:59 <clayg> kota_: I was thinking about dusting off https://github.com/swiftstack/ring-check and throwing some CPU at it
21:42:20 <clayg> tdasilva: on the PUT+POST
21:42:49 <tdasilva> clayg: yep, we had talked about scheduling one hour with Pete. is that still desirable for next week?
21:43:51 <clayg> tdasilva: torgomatic: what's the status - zaitcev says he wants to talk about it - I want to be able to test backed object nodes with non-eventlet wsgi servers (e.g. gunicorn multi-process + rate-limit thread per request)
21:43:57 <kota_> clayg: is that the tool for ring graph?
21:44:17 <torgomatic> clayg: gunicorn in the what now?
21:44:35 <clayg> kota_: no, but I did get permission to open source that - in the process I also realized that project doesn't have an adequate license
21:44:40 <m_kazuhiro> tdasilva: Thank you. I will start review on symlink patch #232162.
21:44:41 <patchbot> https://review.openstack.org/#/c/232162/ - swift - Symlink implementation.
21:44:53 <tdasilva> kota_: timburke ^^^ need your help too
21:44:59 <clayg> kota_: that tool just runs rebalance on a bunch of rings and outputs stats via CSV so you can import them into a data analysis tool (like google sheets or w/e)
21:45:34 <notmyname> joeljwright: can you take care of the merge conflict on https://review.openstack.org/#/c/365371/ ?
21:45:35 <patchbot> patch 365371 - swift - Add Preamble and Postamble to SLO and SegmentedIte...
21:45:44 <joeljwright> sure
21:45:47 <clayg> kota_: but I'm thinking about packaging swift next week for some other bugs and I'd love to have the ring fix in there (we have some customers effected)
21:45:54 <kota_> clayg : i beleave your fix is nice to have. Sorry I've been vacation after ptg hence not so long time to see
21:45:55 <joeljwright> I'll sort that out first thing tomorrow
21:46:09 <notmyname> joeljwright: thanks
21:46:17 <clayg> kota_: so I was mostly curious if you could try that patch against the rings that specifically discovered the issue for you sometime in the next... 5 days?
21:46:25 <kota_> mobile is hard to type :/
21:46:27 <clayg> kota_: "i'm no sure" or "not likely" is fine - i can plan
21:46:41 <clayg> kota_: YOU'RE ON VACATION!?  THAT'S GREAT!!!
21:47:11 <kota_> oic, try the csv tool
21:47:20 <clayg> kota_: do you mind if I ask when you think you'll be "back to work" (again I'm just trying to plan)
21:47:23 <clayg> kota_: will do!
21:47:29 <notmyname> hmm... kota_ is on vacation? I don't think I still have /kick access in this channel ;-)
21:47:44 <clayg> torgomatic: I just want to not use eventlet when I hit the disks!  I hate the eventlet with the blocking.  SO much.
21:48:04 <clayg> but we're tied to eventlet in terrible terrible superficial way in the WORST place
21:48:11 <clayg> it's the same problem with golang and EC
21:48:17 <clayg> or *anything* not eventlet and the object server
21:48:33 <kota_> The next week I'm back so..
21:48:34 <clayg> the OVH/LoSF work *also* is getting bit by eventlet
21:48:51 <clayg> kota_: great!  NP!  I'll resync with you at the next meeting :D
21:49:07 <kota_> Sorry
21:49:13 <clayg> kota_: it's great!
21:49:33 <clayg> tdasilva: yeah so I'm *interested* but not actively *investing*
21:50:10 <clayg> but depending on how likely things are stuck vs. moving forward - I'm curious what options we have
21:50:28 <clayg> torgomatic: I think you're 1) busy with other stuff and 2) after that stuff thinking about container sharding
21:50:45 <tdasilva> clayg: I was thinking if we could just touch base with Pete so that he is brought up to sync with everyone, You have some ideas, torgomatic I think also has some other ideas, and pete has that patch
21:50:48 <clayg> so that leaves zaitcev to save golang/LoSF/anything-not-eventlet-in-the-worst-place-to-be-stuck-with-eventlet
21:50:50 <torgomatic> clayg: that sounds correct; I'd like to work more on the object PUT protocol, but I can't really find the time
21:51:11 <clayg> ok, i'm cool to touch base
21:51:41 <notmyname> tdasilva: can you set that up for next week, depending on availability?
21:52:07 <tdasilva> notmyname: sure. so I followed your advice and set this up: https://doodle.com/poll/mfcd2ypt28fmce8y
21:52:23 <tdasilva> if you want to attend, can you please add your name there and availability...
21:52:46 <notmyname> that is a lot of timeslots ;-)
21:52:50 <notmyname> thanks
21:53:14 <notmyname> #link https://doodle.com/poll/mfcd2ypt28fmce8y
21:53:19 <notmyname> (for easy finding later)
21:53:38 <clayg> torgomatic: could you please also make time to fill out doodle and try to catch up with pete (cc timburke & acoles) on this topic?  I'd love to level set if we have to have follow-up conversations about prioritization
21:53:45 <notmyname> ah, calendar view is simpler
21:53:51 <torgomatic> clayg: sure, I'll try to get to that today
21:54:02 <acoles> clayg: on it
21:54:08 <tdasilva> also make sure to see it in your timezone
21:54:13 <tdasilva> you can change that
21:54:25 <notmyname> thank
21:54:29 <acoles> notmyname: good tip...calendar thanks
21:54:34 <clayg> I think lp bug #1496636 has really got us in a sticky whicket
21:54:35 <openstack> Launchpad bug 1496636 in OpenStack Object Storage (swift) "EC: Chunked transfer/commit protocol is *not* HTTP" [Medium,In progress] https://launchpad.net/bugs/1496636 - Assigned to Pete Zaitcev (zaitcev)
21:54:37 <notmyname> anything else to bring up today?
21:54:44 <notmyname> clayg: is it a right royal bug?
21:55:00 <acoles> I should jolly well think so
21:55:08 <mattoliverau> Lol
21:55:34 <timburke> got some backports i proposed this morning, but i can bug notmyname offline
21:55:42 <notmyname> yeah
21:55:49 <tdasilva> notmyname: maybe tomorrow we can talk more about s3api plans?
21:56:03 <notmyname> tdasilva: sure
21:56:07 <tdasilva> thanks
21:56:39 <notmyname> tdasilva: tomorrow morning will work better for me (my time)
21:56:49 <kota_> Tomorrow?
21:57:10 <mattoliverau> kota_: you just need to holiday ;)
21:57:23 <notmyname> kota_: you're on vacation! stay on vacation :-)
21:57:24 <kota_> Ok
21:57:34 <kota_> Thanks
21:58:02 <notmyname> I think that about covers it for this week
21:58:08 <notmyname> bug triage next week
21:58:15 <notmyname> talk with pete next week (see doodle poll)
21:58:29 <notmyname> tdasilva and I will host the LP bugs/graphs
21:58:52 <acoles> doodle is great
21:58:55 <notmyname> and m_kazuhiro is ready for reviews on migrations and joeljwright will resolve the merge conflict
21:58:59 <jungleboyj> argh
21:59:03 <notmyname> good meeting
21:59:04 * jungleboyj forgot to join the meeting
21:59:22 <notmyname> thanks everyone for coming. thanks for your work on swift
21:59:26 <notmyname> #endmeeting