19:01:38 <notmyname> #startmeeting swift
19:01:38 <peluse_> rock and roll
19:01:38 <notmyname> no meet bot?
19:01:38 <notmyname> who's here?
19:01:38 <zaitcev> o/
19:01:38 <chmouel> o/
19:01:38 <tdasilva> hello
19:01:38 <portante> o/
19:01:38 <acoles> hi
19:01:39 <notmyname> not sure if we'll get a meetbot or not
19:01:39 <openstack> Meeting started Wed Apr 30 19:01:38 2014 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:40 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:43 <openstack> The meeting name has been set to 'swift'
19:01:46 <notmyname> ah ok then
19:01:54 <portante> o/
19:02:08 <peluse_> welcome meetbot
19:02:19 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
19:02:23 <notmyname> agenda for this week
19:02:48 <notmyname> I've been catching up after being out last week (and a little bit of news stuff this morning...)
19:03:00 <notmyname> so let's see where we are
19:03:37 <notmyname> looks like there are been some patches landing (yay)
19:03:48 <notmyname> the correct version is making it's way through zuul now
19:04:12 <notmyname> what happened is that the bot had a bug, so the milestone-proposed branch never got merged
19:04:20 <portante> whoops
19:04:32 <notmyname> then it got fixed, but seemingly fell through the cracks with the gerrit upgrade and heartbleed stuff
19:04:44 <notmyname> so it was only just this morning retriggered
19:05:18 <portante> so once that merges and we rebase we should be good?
19:05:30 <notmyname> so that means the window of inaccurate version numbers was a little longer, but ya--should be fine after that lands
19:05:48 <notmyname> speaking fo patches that have landed recently....
19:05:55 <notmyname> #topic in process func tests
19:06:01 <notmyname> portante: you should be ahppy about that :-)
19:06:02 <portante> yes
19:06:06 <portante> we all should be
19:06:10 <notmyname> :-)
19:06:14 <notmyname> of course
19:06:15 <portante> that was not a fun patch to work on, for sure
19:06:33 <notmyname> and it looks like you're working with -infra to get that added to the auto test suite
19:06:43 <portante> but I am hoping that it will help us keep the tree more stable as the big sp patches go through
19:06:46 <portante> yes
19:06:52 <chmouel> +1
19:07:05 <portante> I am hoping that we can all agree to make sure that new code added is covered by functional tests
19:07:10 <notmyname> portante: are both the devstack func tests and the in-process func tests needed? should we prefer one over the other?
19:07:22 <peluse_> way cool portante, I have yet to do anything policy specific with it but its on my todo list
19:07:25 <notmyname> my understanding is the main difference is the coverage report for in-process
19:07:38 <chmouel> notmyname: the devstack one would test the other openstack components
19:07:45 <chmouel> if we don't break them or they don't break us
19:07:52 <zaitcev> I thought in-process didn't cover something that regular did. I func-tested against Keystone at some point.
19:08:18 <portante> notmyname: too much, lemme sum up, the six fingered man ...
19:08:31 <notmyname> that's right. the in-process won't do keystone/ceilometer/glance stuff. but isn't that covered by the tempest suite?
19:08:52 <portante> notmyname: yes, one main difference is that in-process allows us to track coverage of the main code base under functional test
19:09:17 <portante> one *could* that with a contrived SAIO installation and running all the servers by hand, but that is more difficult
19:10:05 <portante> the second, is that it allows folks to run functional tests quickly without an SAIO, so that there should be little excuse for not running functional tests before a commit is made
19:10:26 <notmyname> portante: `tox -e func` right?
19:10:29 <portante> yes
19:10:31 <chmouel> notmyname: the tempest suite is not as comprehensive as the swift functional tests
19:10:41 <notmyname> chmouel: right
19:11:13 <notmyname> I'm not opposed to running both if they test different things, but I don't want to add multiple runs of the same effective test if it's not adding any value
19:11:18 <portante> I'd like to see folks look for gaps in coverage during reviews and help out by adding tests that increase the coverage
19:11:30 <notmyname> that's the outstanding question I have about adding it to jenkins
19:12:05 <portante> notmyname: I see the in-process tests in the gate as a way to make sure a commit does keep them from working
19:12:29 <portante> Now if we started tracking coverage of the project ...
19:12:57 <zaitcev> I prefer to make sure a commit did not keep from working.
19:13:00 <portante> there are jobs that run coverage specifically after a patch lands, and that would be need to get that information public and easikly viewable
19:13:13 <notmyname> yup
19:13:41 * notmyname imagines a graph on his desktop of historic swift code test coverage
19:13:49 <notmyname> like the gate success rate one I have now
19:13:49 <portante> would be nice
19:13:53 <portante> ;)
19:13:59 <portante> hopefully that will look better
19:14:01 <portante> ;)
19:14:18 <notmyname> heh
19:14:26 <sc68cal> notmyname: for commit in swift - nosetests -c >> log.txt
19:14:29 <portante> so are folks on board with reviewing code changes to catch coverage of new changes by both unit and functional tests?
19:14:39 <notmyname> portante: what do you need from the rest of this?
19:14:45 <portante> that ^
19:14:48 <sc68cal> ;)
19:14:49 <notmyname> *rest of us on this
19:15:56 <notmyname> portante: do you mean checking functional test coverage for new patches in addition to unit test coverage? and adding more coverage to functests?
19:16:02 <portante> I'd like us all to be comfortable with running the functional tests that way, write new functional tests to add coverage, and when reviewing check for coverage, so that we are less likely to cause the kinds of problems we have had over the past six - 8 months
19:16:08 <portante> yes
19:16:17 <peluse_> agreed
19:16:18 <notmyname> ya, I'm totally on board witht hat
19:16:30 <portante> does anybody have any concerns with that?
19:16:59 <peluse_> we just need to put our money where our mouths are (or keyboards are I should say)
19:17:07 <notmyname> I think we'll need to update some docs and simply remind each other for a while until it becomes natural
19:17:19 * peluse_ says not it on docs
19:17:29 <portante> certainly, and I'd be happy to work on docs as folks find places that it would benefit
19:17:56 <notmyname> portante: ya, I was thinking more with the SAIO docs to start with. or wiki pages for new contributors. etc
19:18:07 <notmyname> ie stuff actual contributors would look at, not deployers
19:18:23 <portante> sure, we can talk about that offline then
19:18:28 <notmyname> ya.
19:18:41 <notmyname> two things I can think of that may help it set in with everyone
19:19:00 <peluse_> portante:  I'll test your SAIO docs no problem, just had my share of writting in englush for a while
19:19:06 <peluse_> see what I mean?
19:19:09 <portante> ;)
19:19:15 <tdasilva> portante: are func test coverage results being created now regardless of if you are running func tests in-process or not?
19:19:27 <notmyname> allow ./.functests to run in both modes (I think you might already have this). and track coverage (people optimize to change published numbers, whatever they are)
19:19:40 <zaitcev> it is allowed
19:19:55 <zaitcev> I run ./.functests and then SWIFT_TEST_IN_PROCESS=1 ./.functests
19:20:11 <notmyname> ah, ok
19:20:13 <portante> tdasilva: I think you have to specifically ask for coverage
19:20:14 <zaitcev> I hoped that it would increase confidence in PBE
19:20:45 <peluse_> ..and policies and for sure EC when we get there
19:20:54 <portante> I use "tox -e pep8,py27,func" before a commit from my laptop, and then in my SAIO use .functests to get the SAIO func test runs
19:21:28 <portante> zaitcev: yes, that is the goal, but we might have to raise the coverage on master before a given patch can be shown as "good"
19:21:37 <notmyname> nice
19:21:49 <portante> I am hoping that the core dev team will help point those kinds of changes necessary
19:22:11 <peluse_> portante:  just so I'm clear (and others too I think) what specifically if anything do we have to do to get coverage to be generated running the in process func tests?
19:22:12 <portante> I'd like to have a way to automatically publish the coverage of master today
19:22:18 <notmyname> who's point on making sure this new functionality is written down and findable for people (not that you have to do all the work, just to be the person to coordinate that)
19:22:31 <notmyname> is that me? portante? zaitcev, you interested?
19:22:33 <portante> notmyname: I'll do that
19:22:37 <notmyname> portante: ok, cool. thanks
19:22:41 <portante> please back me up and check me on it
19:22:52 <notmyname> I'd be happy to
19:22:56 <chmouel> portante: i think there was a gate sometime ago allowing to do that on other openstack project you may want to ping infra about it
19:23:15 <portante> chmouel: yes, I'll do that, thanks
19:23:35 <chmouel> one can always run a bot listening to gerrit stream and run the coverage and post back the results
19:23:55 <portante> peluse_: the .unittests file shows you how to get coverage with unit tests
19:23:56 <chmouel> it should not be too hard (TM)
19:24:11 <peluse_> portante:  K, thanks
19:24:11 <portante> chmouel: agreed
19:24:25 <portante> peluse_: it would be a nice option to add for .functests
19:24:56 <peluse_> portante:  yeah, agree.  thought it ws there as part of what you checked in but am clear now that its a separate thing
19:24:58 <notmyname> portante: +1
19:25:08 <notmyname> peluse_: me too
19:25:12 <notmyname> glad for the clarification
19:25:14 <portante> you get it with tox
19:25:19 <notmyname> anything else on this subject?
19:25:21 <portante> so a tox -e func will get you coverage
19:25:26 <notmyname> ah ok
19:25:29 <peluse_> got it
19:25:31 <peluse_> thanks
19:25:41 <peluse_> (well now I get it)
19:26:30 <notmyname> #topic storage policy update
19:26:47 <notmyname> I haven't sat down with clayg yet, but I see that there has been some good progress
19:26:54 <notmyname> peluse_: can you give a summary?
19:27:16 <peluse_> peluse:  I don't have any 'TODO' coding items.  I have a few still in need of more core reviewers and have spent time going through clayg's stuff but most of this past week has either been non-Swift for me or EC
19:27:28 <peluse_> good progress on EC design work though :)
19:27:40 <portante> so is clayg here?
19:27:42 <notmyname> ok
19:27:50 <portante> hmm, does not seem so
19:27:54 <notmyname> no, clayg got puled into a customer issue
19:27:57 <portante> okay
19:27:59 <peluse_> also, docs seemed to have slowed down on comments and had at least one 'test' of the SAIO w/policies from someone out there so they're looking good
19:28:14 <peluse_> yeah, clayg would have to talk to his stuff.  He's been very busy on it though as you mentioned
19:28:27 <notmyname> AFAIK, he's got a really good reconciler and is getting the patches restaged to propose to master
19:28:34 <portante> so he is working on preparing a patch series to master, right?
19:28:35 <notmyname> but I don't know the details on that
19:28:44 <peluse_> thus my recent plea for more reviews on my outstanding patches so we can focus just on the reconclier
19:28:47 <notmyname> yes, that's my understanding
19:29:18 <peluse_> wrt staged patches to master, the plan was to get the reconsilcer set done first and get the rest of mine merged
19:29:19 <portante> and are we going to finalize the reconcilier into feature/ec and then propose everyting to master? Or do that in parallel?
19:29:25 <portante> ;)
19:29:27 <portante> thanks
19:29:30 <peluse_> but he may have mad additional progress that I'm not aware of
19:29:32 <notmyname> peluse_: I think the first
19:29:38 <portante> love answers before questions asked ...
19:29:44 <notmyname> :-)
19:29:48 <zaitcev> So, what does prevent us from starting pulling SP
19:29:52 <peluse_> so key asks now:  review my outstanding stuff (only a few things) and start looking at clay's long series of aptches :)
19:30:05 <zaitcev> if not " any 'TODO' coding items"
19:30:16 <peluse_> zaitcev:  the idea is that all needs to be on the ec branch first before we can stage a meaningful cohesive set of patches to intro to master
19:30:45 <peluse_> zaitcev:  what I meant is that I don't have any more code to write
19:30:50 <portante> so we'd want to finalize the docs on feature/ec first as well, then?
19:30:58 <peluse_> yes sir
19:31:03 <notmyname> portante: yes, and I think peluse_ has good ones there already
19:31:04 <portante> great
19:31:07 <notmyname> or at least a very good start
19:31:17 <notmyname> that should be the first patch to look at, actually
19:31:22 <portante> agreed
19:31:22 <notmyname> for anyone reviewing SP patches
19:31:32 <notmyname> and first in the chain of stuff to land, IMO
19:31:32 <peluse_> they've had several reviews, may need some RST specific formatting but I think content is good (minus reconslicer)
19:31:37 <portante> and to make sure it has good functional test coverage. :)
19:31:44 <notmyname> "reconslicer" I like that
19:31:45 <peluse_> indeed
19:31:51 <peluse_> yeah, I was close
19:32:00 <peluse_> recon-somthing-or-another
19:32:35 <portante> btw, before SP lands, master has 54% total coverage from just functional tests
19:32:43 <notmyname> good to know
19:33:07 <notmyname> portante: I suspect that much of the gaps are in middleware
19:33:24 <portante> how 'bout I prepare a report of sorts on that then
19:33:31 <notmyname> that would be great
19:33:36 <tdasilva> +1
19:33:38 <portante> maybe it will help us with SP get that shaped up
19:33:52 <notmyname> I think so
19:33:59 <peluse_> I'll run on the S code this week for sure and see what % we're at
19:34:08 <peluse_> S=SP
19:34:17 <notmyname> S=SP=storage policies
19:34:21 <notmyname> ;-)
19:34:27 <zaitcev> I would prefer a movement on SP in master in small steps.
19:34:44 <zaitcev> "Reviewing" it in EC branch is a lot of work.
19:34:56 <portante> zaitcev: do you mean by movement, that the individual patches are small?
19:35:13 <portante> applied to master that is?
19:35:14 <zaitcev> Off-loading the logical structuring to Paul makes it easier
19:35:34 <zaitcev> Well, not small by line count. Small by impact.
19:35:54 <peluse_> zaitcev:  I think what we're planning will make sense.  Series of patches that flow logically that don't erquire any diffs with the ec branch to understand them
19:36:00 <notmyname> the plan is to propose digestible pieces to master
19:36:17 <zaitcev> bring it on
19:36:24 <peluse_> digestible so we don't have a movement :)
19:36:30 <zaitcev> But eventually I'll want to trade.
19:36:55 <peluse_> zaitcev:  oh yeah
19:37:47 <notmyname> good on S=SP=sotrage policies for this week?
19:37:53 <notmyname> for the update, that is
19:37:59 <peluse_> yup
19:38:20 <notmyname> I'll be finding time to wrap my head around it later this week
19:38:23 <peluse_> reminder for everyone:  notmyname and I have a talk at the summit on policies
19:38:32 <zaitcev> bah
19:38:34 <notmyname> ya, that's the other kind of thing that I gotta do :-)
19:38:40 <zaitcev> just start posting patches
19:38:44 <notmyname> 4 summit talks and a day of swift sessions :-)
19:39:06 <chmouel> heh, good luck notmyname
19:39:09 <peluse_> and policies is the first day jsut after the keynotes
19:39:14 <notmyname> zaitcev: `git checkout feature/ec && git branch -m master` ;-)
19:39:39 <zaitcev> notmyname: you can't be serious
19:39:45 <notmyname> zaitcev: of course not :-)
19:40:00 <notmyname> zaitcev: you did say to just propose patches...
19:40:19 <peluse_> heh.. hey SP realted, I'd like to start giving a brief update on EC weekly as well
19:40:24 <peluse_> related
19:40:53 <notmyname> peluse_: I think that would be nice to do after the summit. ie next week is last-presummit, then summit, then let's do that
19:41:05 <notmyname> any questions or concerns on the new gerrit version?
19:41:16 <peluse_> cool, I'd just like to see more folks inovled up front on EC than were on policies is all
19:42:56 <notmyname> I think the -infra team is working on putting together some useful dashboards for the new gerrit instance, similar to some of the bookmarks we have now (eg stuff needing one more +2, stuff that hasn't been reviewed at all, etc)
19:43:01 <notmyname> so that should be good
19:43:13 <portante> notmyname: great
19:43:29 <notmyname> ok, summit stuff then
19:43:40 <notmyname> oh, I should have been using the #topic
19:43:45 <notmyname> #topic summit prep
19:43:59 <notmyname> I pushed a schedule for the 8 swift sessions
19:44:39 <notmyname> my idea was to choose things (and group things) that will be good for discussion at the summit. remember its facilitaded discussion, not presentations
19:45:21 <portante> where did you push it to? :)
19:45:29 <notmyname> I'm looking forward to the sessions on some of the community management stuff. ("swift growing pains"). stuff about making sure we do reviews in a timely manner and encourage new devs and improvign docs
19:45:37 <peluse_> notmyname:  but we'll have a project to show pictures if needed though right?  Will be hard to talk EC without a few pics
19:45:39 <notmyname> but we also have the swift project pod
19:45:44 <notmyname> peluse_: ya, I think so
19:45:51 <notmyname> peluse_: ie we always have in the past
19:46:22 <notmyname> the "project pod" is a table for us. we've got the corner of a fairly large room, so plenty of space. and I'm bringing a projector for it
19:46:52 <zaitcev> peluse_: or you can run to nearest Kinkos/FedEx and make a few copies
19:46:59 <notmyname> unscheduled time there (but we could keep a list on the table or something), but it should be a good place to start and continue discussions
19:47:12 <acoles> notmyname: which days do we have the pod?
19:47:25 <notmyname> acoles: all week AFAIK (tuesday through Friday)
19:47:33 <acoles> notmyname: great
19:47:36 <notmyname> that is, the entire "Design Summit" time
19:47:54 <notmyname> I think these pods may be in lieu of a dev lounge. not sure about that
19:48:15 <notmyname> also, there are some interesting cross-project sessions earlier in the week
19:48:28 <notmyname> full schedule is:
19:48:30 <notmyname> #link http://junodesignsummit.sched.org
19:50:16 <notmyname> any questions or things I can do for you about hte summit?
19:51:02 <notmyname> #topic open discussion
19:51:08 <notmyname> anything else?
19:51:17 <notmyname> dfg: here?
19:52:05 <notmyname> gholt: dfg: I'm a little curious about your "turn of fsync" experiment
19:52:27 <portante> off?
19:52:38 <portante> yes, that sounds very interesting
19:52:56 <portante> is there a commit somewhere documenting it?
19:53:22 <notmyname> https://github.com/rackerlabs/swift/tree/rackspace_production
19:53:33 <portante> in lieu of that, physcx has been posting the buffering patch
19:53:59 <portante> very interesting
19:54:03 <notmyname> oh yeah. anything to discuss on that here? or keep in in -swift?
19:54:30 <portante> just to make folks aware of what he observed
19:54:59 <notmyname> I think it's great. I want to see us all keep looking at performance and efficiency in swift
19:55:16 <portante> that the pipeline through proxy server to disk relies on buffers filling up, otherwise PUT writes to disk are small and everything starts going compute bound
19:56:06 <notmyname> anything else this week?
19:56:56 <notmyname> thanks for coming!
19:56:58 <notmyname> #endmeeting