20:59:56 <notmyname> #startmeeting swift
20:59:57 <openstack> Meeting started Wed Oct 31 20:59:56 2018 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:59:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:00 <openstack> The meeting name has been set to 'swift'
21:00:04 <notmyname> who's here for the swift team meeting?
21:00:08 <mattoliverau> o/
21:00:11 <timburke> o/
21:00:13 <kota_> hi
21:00:22 <rledisez> o/
21:00:30 <notmyname> welcome zaneb
21:00:33 <zaneb> o/
21:00:58 <notmyname> clayg: tdasilva: ping
21:01:09 <clayg> ohai!
21:01:10 <tdasilva> hello!
21:01:29 <notmyname> zaneb: glad you could make it. do you have any particular schedule restrictions? (leaving early, etc)
21:01:48 <notmyname> agenda this week is at...
21:01:49 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:01:54 <zaneb> notmyname: nope, I can be here for the duration
21:02:24 <notmyname> "normal" stuff, but the main topic is the tc vision proposal that zane is championing
21:02:29 <notmyname> zaneb: great!
21:02:53 <notmyname> #topic tc vision document
21:02:55 <notmyname> #link http://lists.openstack.org/pipermail/openstack-dev/2018-October/136031.html
21:03:03 <notmyname> #link https://review.openstack.org/#/c/592205/
21:03:04 <patchbot> patch 592205 - governance - [DRAFT] Add a Technical Vision statement - 4 patch sets
21:03:24 <notmyname> ok, so the first question is "who actually read it?" :-)
21:03:35 <zaneb> :D
21:03:42 <clayg> I skimmed it - but then I wanted to start making comments - so I stopped
21:03:47 <timburke> i read... some of it, anyway
21:03:53 <kota_> :P
21:04:04 <clayg> the patch seemed fine, but then people start talking - it makes me want to drink
21:04:09 <notmyname> heh
21:04:20 <notmyname> I think the frustrating part of this kind of thing is that it is very meta. it's not about solving problems; it's for solving problems about how we solve problems.
21:04:36 <clayg> some people are better at that sort of thing
21:04:51 <timburke> i remember thinking "event notifications where appropriate" could use some fleshing out -- that's definitely something that swift's been interested in, but it's hard
21:05:08 <notmyname> rledisez: tdasilva: kota_: did you get a chance to read it?
21:05:19 <timburke> Bidirectional Compatibility seemed to be stepping outside of the service-only purview
21:05:27 <timburke> it's on the client to figure out how to maintain compatibility with older services, just as it's on the services to figure out how to maintain compat with older clients
21:05:28 <kota_> briefly, not get in detail.
21:05:36 <rledisez> notmyname: not really, I'm going through it right now briefly
21:05:39 <notmyname> ok
21:05:51 <zaneb> timburke: it does apply to some stuff like Heat and Mistral on the service side
21:06:00 <notmyname> here's my summary ( zaneb, please correct me if I get things grossly wrong)
21:06:03 <notmyname> tc want's to write down some guidelines used for defining the "in" and "out" for openstack. it's doing this by describing and defining basic things about cloud infrastructure and then declaring them to be in scope for openstack. so that if project x shows how they match one or more of these basic principles, they are then in scope for being in openstack
21:06:12 <timburke> felt like fault-tolerance/high-availability might be another good pillar (and just so happens to be something we're good at ;-)
21:06:14 <zaneb> timburke: and arguably to e.g. microversions and stuff in APIs
21:06:15 <notmyname> swift often has a different perspective on openstack because we pioneered "independently deployable" and also have a data-centric API (instead of provisioning). this document describes several cloud principles that swift does, so zane says this should finally put that debate to bed
21:07:28 <notmyname> zaneb: is that about right? maybe my understanding of the overall purpose is more limited than you have in mind
21:07:36 <zaneb> notmyname: that's a fair summary, although there are other reasons for doing it also
21:07:46 <notmyname> yes. please share :-)
21:08:11 <zaneb> one is, as you said, that the TC currently has no standard for judging which projects should be allowed to join, and it's a pain point for us
21:08:37 <zaneb> but another is that we want to give projects something they can refer to when making design decisions
21:08:39 <mattoliverau> I've skimmed it, seems notmyname you've had some interesting discussions in gerrit about some definitions
21:09:05 <notmyname> mattoliverau: I tried to avoid the biggest traps, I think :-)
21:09:08 <zaneb> and a lot of that stuff have previously only been at the level of folklore that people picked up by osmosis, if at all
21:09:41 <notmyname> zaneb: meaning that if eg we have a new feature we are designing, we could compare it against this list to see if it's good or bad?
21:09:53 <notmyname> well, appropriate or not
21:10:12 <zaneb> and it'd be nice to have a kind of framework within which the services sit so that the people independently designing them eventually produce something coherent
21:10:54 <notmyname> that last thing sounds terrific. design a system so that independent parties converge on a cohesive whole. (also sounds really hard)
21:11:35 <zaneb> yes, really hard to get it happen on purpose, but absolutely not going to happen by accident :)
21:11:47 <clayg> ^ that's rich
21:11:48 <timburke> designing converging systems in general is hard -- see our troubles with replication, dark data, and ghost listings :P
21:12:13 <notmyname> zaneb: so are you thinking this document will guide (or advise) feature development?
21:12:15 <timburke> it's even worse when the system's made up of people instead of programs
21:12:40 <zaneb> notmyname: to some extent, yes, but to different degrees for different projects
21:13:08 <zaneb> and a lot of it is stuff that we old-timers already know, but hadn't written down in one place
21:13:47 <notmyname> TBH I'm struggling with ... well, I'm struggling with how to express my thoughts via text chat :-)
21:14:06 <mattoliverau> meme? interpritive dance?
21:14:12 <notmyname> lol
21:14:41 <clayg> lol
21:15:00 <zaneb> one thing I personally would love to get is events out of every service. imagine if we'd had Storlets but for everything 5 years ago - we'd have invented Lambda before AWS did
21:16:00 <notmyname> I'm very concerned when someone starts defining boundries in the openstack ecosystem, because swift has oftentimes been put in the "other" group. that's why I pay attention to these sorts for discussions. but on the other hand, it's hard to see how it matters on an ongoing day-to-day basis
21:16:59 <clayg> I think that might be baggage - I feel more in-crowd cool kids these days
21:17:13 <notmyname> clayg: it totally could be
21:17:51 <zaneb> fwiw I think everyone else has a lot to learn from Swift, and allowing it to be defined as 'other' has prevented the rest of OpenStack from doing that as much as we should
21:18:45 <zaneb> so as I said in the email, I hope drawing the boundary where it is in this doc will help people to *stop* seeing Swift as 'other'
21:18:53 <clayg> :hugs:
21:19:12 <notmyname> clayg: that's the emoji thing I'm looking for! :-)
21:19:15 <clayg> zaneb: what would be most helpful from "us" (and by us I mostly mean other people besides me)
21:19:32 <clayg> ... but especially from notmyname :P
21:19:40 <clayg> :hugs:
21:19:50 <notmyname> heh
21:19:52 <zaneb> lol
21:20:02 <clayg> mattoliverau!  mattoliverau is helpful
21:20:06 <mattoliverau> lol
21:20:37 <kota_> lol
21:20:52 * notmyname waits for zaneb before asking more questions
21:21:49 <zaneb> the most helpful thing for me is people engaging with the review, because in the end this will be successful if people feel like we all came up with it together, and it's not something handed down from the TC
21:22:17 <notmyname> zaneb: the outreach you're doing to specific projects in great. thank you
21:22:19 <zaneb> if I start typing gibberish, it's because a miniature human is attacking the keyboard
21:22:40 <zaneb> I mean, more gibberish than usual
21:22:48 <notmyname> clayg: timburke: you both said you had some comments but help back from posting them
21:23:04 <notmyname> were they about the doc in general, any specifics in it, or just the commentary about the doc?
21:23:05 <mattoliverau> lol
21:23:46 <clayg> notmyname: honestly 9/10 of my "comments" were "hell, yeah!" the others were like "no YOU'RE WRONG, this is good."
21:24:17 <clayg> I'm paraphrasing
21:24:33 <zaneb> lol
21:25:10 <timburke> i think mostly just the comments i made here. seemed like actually posting would mostly add more noise (to what will definitely be a noisy review) rather than help, but i could be wrong
21:25:57 <zaneb> timburke: you mentioned fleshing out the 'event notifications where appropriate' - would be very interested in what sort of stuff you'd like to see there
21:26:17 <timburke> mostly just what "where appropriate" means
21:26:38 <timburke> because i honestly don't know where the appropriate line would be in swift
21:26:44 <notmyname> timburke: in my experience with openstack, it's very good to have noisy reviews early in order to get a right version of the doc from the start (but like clayg, that may be my own historic baggage)
21:26:52 <zaneb> that section is kind of toned down because that is TOTALLY MY HOBBY HORSE and I would write the whole thing about just that if I could ;)
21:27:42 <timburke> "on every swift operation" feels *incredibly* noisy, like we'd almost certainly cause whatever message queue we're putting those notifications in to fall over
21:27:57 <clayg> TIL https://en.wikipedia.org/wiki/Hobby_horse#Other_meanings
21:28:36 <zaneb> clayg: it's fortunate that didn't take a much darker turn :D
21:29:01 <zaneb> timburke: maybe we should let users filter at the source?
21:29:21 <notmyname> zaneb: do you think this document will take a turn towards more "provisioning-only" language? one commenter seemed to want to move it in that direction
21:29:48 <timburke> notmyname: i was noticing that too...
21:30:27 <zaneb> notmyname: by that you mean just-Nova-providing-a-VPS?
21:30:49 <zaneb> aka the "free VMWare" model?
21:30:54 <notmyname> more generally, like "you ask openstack to give you resources that you can then use directly"
21:31:17 <zaneb> oh, ok
21:31:58 <notmyname> I have two general concerns with things that try to define openstack. the first is provisioning vs "data-plane" api disctinctions. the second is a focus on compute-only or compute-first language
21:33:04 <clayg> ... so many half typed sentences ...
21:33:12 <zaneb> imo there's such a mixture of different things that it's never going to make sense to say openstack is control-plane-only
21:33:12 <notmyname> lol
21:33:31 <clayg> I actually need to cut out (happy halloween everybody!)
21:33:42 <kota_> lol
21:33:45 <timburke> notmyname: yeah, i kinda want to point kevin at https://governance.openstack.org/tc/reference/principles.html#openstack-primarily-produces-software and say that while others may reimplement the swift protocol (we have no control over that, they can make their own decisions), that's not Swift
21:33:52 <clayg> +1 to everything zaneb says
21:34:14 <notmyname> clayg: thanks for joining while you could. I definitely appreciate your comments and perspective
21:34:25 <clayg> notmyname: was there anything else on the agenda?
21:34:30 <clayg> rledisez: ohai!?
21:34:37 <clayg> I'll read the logs
21:34:43 <zaneb> that comment from Kevin was interesting food for thought, but I don't think it's going to change the direction of the document in any major way
21:34:45 <notmyname> clayg: nah, jsut "normal" stuff like "go land the s3api patches" and review losf :-)
21:34:54 <clayg> k
21:35:00 <clayg> l8r
21:35:04 <zaneb> clayg: thanks o/
21:35:19 <rledisez> clayg: ohai???
21:35:50 <notmyname> kota_: rledisez: what do you think about this doc (or this discussion)? we haven't heard from you yet
21:36:26 <timburke> rledisez: don't worry, i speak clayg. "oh, hi there!" and in this particular context, "sorry we couldn't chat"
21:37:04 <rledisez> so, i just read it. it is interesting, i'm not sure i see how it could drive everydays decissions about feature/implementation. if we need something, will we really restict ourself to what this document allow?
21:37:13 <rledisez> timburke: thx for the translation :D
21:37:21 <kota_> notmyname: i have (probably) same concerns with you on the definition of the openstack because swift provides data-centric apis.
21:37:33 <notmyname> rledisez: yeah, I completely agree about the feature development influence
21:37:37 <kota_> and also, Storlets does...
21:38:24 <notmyname> kota_: yeah. did you notice anything that gives you concern?
21:38:33 <rledisez> notmyname: but I like the turn it takes about the place of Swift inside Openstack
21:38:42 <kota_> just a guideline could be fine but if it'll be restriction, I could not sure I could follow it, maybe?
21:39:26 <notmyname> kota_: from what I understand of it, and from what zaneb has said, I do not expect there to be any definitions that turn in to "data-centric is bad"
21:39:47 <kota_> notmyname: ok,
21:40:00 <zaneb> as far as features in Swift go, I'd be most interested in hearing if the things documented are consistent with the decisions about features you've made in the *past*, so we can hand the knowledge on to some of the newer teams
21:40:05 <kota_> thanks for clarification.
21:40:23 <notmyname> zaneb: ah, yes, that cuts to the chase
21:41:06 <kota_> zaneb: that sounds nice
21:41:17 <notmyname> zaneb: I question some of the "hardware virtualization" section. it mentions storage, but what we do with object storage is sortof hard to fit into a "storage virtualization" world. at least much harder than cinder
21:41:19 <zaneb> notmyname: that reminds me, we should discuss you comment about regions, but not sure if this is the time
21:41:47 <notmyname> zaneb: sure. we can discuss the regions in the review, I think. it doesn't seem like a major issue to me
21:41:55 <zaneb> ok
21:42:57 <zaneb> notmyname: IMHO hardware virtualisation is necessary for a cloud, but it also tends to scale in coarse-grained ways (from the application perspective)
21:43:18 <zaneb> so we need both that and things like Swift that provide more fine-grained scaling
21:44:21 <mattoliverau> yeah, just the term regions means different things in Swift and keystone, they can fit together depending on your deployment topology but doesn't have to. We need a venn diagram :P Just like Swift containers and containers (though they don't overlap at all) :P
21:44:30 <notmyname> yeah. the HW virtualization works well in the "free vmware" mindset. but swift (object storage) is really more about not thinking about hardware at all (from the app perspective). and that normally means the service is runnign on bare metal
21:44:32 <kota_> i think it depends on the definition. IMO Swift API virtualized actual hardware and data placement.
21:44:41 <kota_> we could assume.
21:45:24 <notmyname> kota_: yeah :-). I think the definition is vague enough that we could make what swift does fit in to it. swift abstracts hard drives. you can swap them out and everything just keeps working.
21:45:34 <timburke> notmyname: i think that whole section doesn't really apply, because we don't
21:45:50 <timburke> have a service "provided by a specialised piece of hardware"
21:45:54 <zaneb> kota_: you could say that about almost anything ;)
21:45:57 <notmyname> "a hard drive"
21:46:47 <clayg> Lol @ tburke is my opposite of Obama’s “Luther”
21:46:55 <zaneb> my 2c: they're separate things; we need both in OpenStack
21:47:19 <notmyname> zaneb: I'm not really sure what to do next. my summary of what we've said in here is that most people are mostly ok with it (or at least not too concerned about the current version). we have some smaller wording questions, but nothing that's very troubling
21:47:57 <notmyname> zaneb: so to repeat what clayg said earlier, what do you need from us now? have we given you anything that will help you? do we need to do something else?
21:48:36 <notmyname> based on the comments from the rest of the team, I'm ok with putting my own +1 on it as a swift representative.
21:48:45 <zaneb> comments on the review if you have changes to be addressed would be great
21:49:05 <zaneb> there will be another patchset coming soon, but +1s on that would be great
21:49:19 <notmyname> ok. I'll wait until the next patchset
21:49:58 <zaneb> we have a Forum session at the summit, so hoping to get folks along to that to discuss where we're at and what we need to do to finalise it
21:50:23 * kota_ will be there.
21:50:26 <notmyname> ok. I'll look for that on the schedule
21:50:33 <notmyname> (my time in berlin is a bit limited)
21:50:57 <zaneb> #link https://www.openstack.org/summit/berlin-2018/summit-schedule/events/22818/vision-for-openstack-clouds-discussion
21:51:26 <notmyname> ok, last few minutes (thanks for being patient with a longer meeting this week)
21:51:40 <zaneb> really appreciate everyone's input, thank you!
21:51:41 <notmyname> zaneb: thank you for taking the time to specifically ask us about this doc
21:51:50 <notmyname> #topic other topics
21:52:03 <notmyname> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews
21:52:16 <notmyname> priority reviews page is looking pretty similar to where it's been
21:52:24 <mattoliverau> thanks zaneb for coming, listening and taking the time, great work on the vision work
21:53:01 <notmyname> timburke: I'm wondering about the s3api patches. I'd really like to land them soon. maybe if we threaten to jsut land them then maybe mattoliverau and kota_ will look at them? ;-)
21:53:11 <zaneb> mattoliverau: my pleasure, thanks!
21:53:50 <notmyname> rledisez: I saw new patches (or patch sets) from alex about losf?
21:53:59 <zaitcev> timburke was doing such a good job adapting proxy that at one point got my manager to write down a working py3 proxy as a checkpoint on 10/10. But now he's away doing god knows what that Swiftstack needs more... Like completely random things.
21:54:00 <kota_> it looks 3 patches are remaining
21:54:08 <kota_> for s3api in prioriry
21:54:13 <notmyname> kota_: yeah
21:54:13 <mattoliverau> Yeah, sorry, I've joined a new virtual team (happens every 6 months or so) so been busy ramping up. So haven't been reviewing as much as I should. I'll make sure I find work time to devote to more reivews :)
21:54:20 <timburke> notmyname: merge 'em all!
21:54:23 <notmyname> :-)
21:54:53 <rledisez> notmyname: maybe, tbh i was not focused on that. but as tim said, "merge 'em all!"
21:55:21 <timburke> zaitcev: i think we *are* really close to a working py3 proxy (at least for some requests?)
21:55:45 <timburke> having tests that verify that will be interesting, though :-/
21:56:19 <kota_> i think I could look at 2 patches w/o patch 592231 but only 592231 is still not sure it goes fine way...
21:56:20 <patchbot> https://review.openstack.org/#/c/592231/ - swift - s3api: Include '-' in S3 ETags of normal SLOs - 3 patch sets
21:56:41 <kota_> i'd like to get another person's opinion for that
21:57:23 <notmyname> kota_: ok. could you leave a comment with your concerns or questions in gerrit? right now only tdasilva is on it
21:57:38 <notmyname> so between the two of you, maybe we can get it landed quickly
21:58:04 <kota_> and sorry of my slow pace for the reviews, various things goes in work and private...
21:58:13 <kota_> notmyname: ok
21:58:41 <notmyname> kota_: do not worry. I completely understand
21:59:35 <notmyname> zaitcev: I also wanted to say thanks for your py3 work. it's often tedious and thankless and hard to see progress, but as you know it's important. so thanks for working on it
21:59:41 <notmyname> zaitcev: also, good luck with IBM :-)
22:00:16 <mattoliverau> ^ +100 to both these things
22:00:17 <zaitcev> notmyname: I'm quite interested in IBM's public cloud, they have some storage running in there. Not sure if it's Swift or not.
22:00:51 <notmyname> zaitcev: the softlayer stuff is still, I think. eveything else is cleversafe (spectrum scale is the name now, i think)
22:00:58 <notmyname> ok, we're at full time
22:00:58 <mattoliverau> and I guess thats time.
22:01:02 <notmyname> thanks for coming today
22:01:14 <notmyname> thanks for working through the tc doc
22:01:18 <notmyname> and thanks for your work on swift!
22:01:22 <notmyname> #endmeeting