22:01:23 <vipul> #startmeeting Reddwarf
22:01:24 <openstack> Meeting started Tue Jan 22 22:01:23 2013 UTC.  The chair is vipul. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:01:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:01:28 <openstack> The meeting name has been set to 'reddwarf'
22:01:28 <hub_cap> ohai
22:01:36 <vipul> #link http://wiki.openstack.org/Meetings/RedDwarfMeeting
22:01:56 <vipul> shall we get started?
22:02:08 <SlickNik> yeah, let's do it
22:02:08 <hub_cap> works for me
22:02:09 <vipul> #topic Action Item Updates
22:02:26 <vipul> k, first one was mine - Blueprint on Quotas
22:02:53 <vipul> argh trying to find
22:03:03 <vipul> #link https://blueprints.launchpad.net/reddwarf/+spec/quotas
22:03:17 <vipul> completed, for now, probably need to add some more details
22:03:28 <hub_cap> sweet
22:03:34 <vipul> next item.. TestR blueprint
22:03:43 <vipul> anyone get a chance to take a peek?
22:04:06 <hub_cap> nope. ive been knee deep in cinder code...
22:04:17 <SlickNik> nope, still a todo for me
22:04:21 <hub_cap> well cinder conversations and code :D
22:04:22 <vipul> esp: does this still need another pair of eyes? or is it good to go
22:04:36 <esp1> which part?
22:04:37 <hub_cap> can u link it?
22:04:40 <steveleon> link?
22:04:46 <SlickNik> #link https://blueprints.launchpad.net/reddwarf/+spec/testr-unit-tests
22:04:52 <esp1> thx
22:04:57 <SlickNik> np
22:05:03 <esp1> yeah it wouldn't hurt to have someone look at it.
22:05:07 <grapex> I've got a question: last meeting we decided to use sqlite and mocks strategically, but I ntocied in the blue print it says to never use sqlite.
22:05:23 <esp1> hmm.. that would be a mistake.
22:05:26 <esp1> let me look
22:05:26 <vipul> good thing we're taking a look at it :)
22:05:38 <hub_cap> seems valid
22:05:48 <esp1> I think we were moving along the lines for not mixing mocks and sqlite tests
22:06:01 <juice_> +1
22:06:03 <hub_cap> ive set to approved
22:06:15 <vipul> k, good enough
22:06:18 <grapex> Has Nova stopped using sqlite in their unit tests?
22:06:20 <juice_> I would like to see tests segmented by unit and integration (incl sqllite
22:06:46 <vipul> I personally prefer using sqllite, instead of just mocks all over the place
22:06:50 <grapex> Me too
22:06:54 <hub_cap> im not sure if they have, but i do see that python-mysql is now necessary for test-requires
22:06:58 <esp1> grapex:  I can see the confusion in there.  we do want to use sqlite eventually just having got that far yet.
22:07:08 <grapex> Ok, I know how we can add it easily
22:07:10 <hub_cap> which is annoying to me personally :)
22:07:16 <grapex> I just wrote a test in testr that makes use of it
22:07:27 <grapex> Assuming sqlite can be used with multiple procs, we should be OK
22:07:37 <esp1> yep shouldn't be a big deal.  just don't want to mix paradigms with mock vs sqlite
22:07:41 <SlickNik> For some integration tests, sql-lite does make sense.
22:08:00 <vipul> A lot of the stuff in models, i'd rather have it make real sql calls
22:08:01 <grapex> "unit" tests doesn't mean to only use mocks, its one tool in the box
22:08:13 <juice_> SlickNik I agree that sqllite would be ok in integration but not unittests
22:08:14 <hub_cap> nova seems to mock it
22:08:31 <vipul> i think they tend to lean towards Fixtures
22:08:31 <esp1> yeah, yer all correct.  we need to do a better job of naming things.
22:08:33 <vipul> not complete mock
22:08:37 <juice_> grapex: doesn't have to use mock but shouldn't be using a database
22:08:38 <steveleon> i have some unitests for the guestagent... specifically for query-creation classes...should those be tested using sqlite?
22:08:38 <hub_cap> https://github.com/openstack/nova/blob/master/tools/test-requires
22:08:38 <hub_cap> https://github.com/openstack/nova/tree/master/nova/tests/db
22:08:56 <esp1> maybe just break them out: mock tests, sqlite tests.
22:09:11 <vipul> steveleon: I'd think those types of tests would be better against a sqllite
22:09:17 <juice_> how about "int-lite"
22:09:32 <esp1> juice: nice one.
22:09:45 <vipul> ok looks like we need a longer discussion on this
22:09:48 <esp1> steveleon: I was thinking of doing both
22:09:51 <hub_cap> ya table it vipul
22:09:54 <grapex> juice_: We could put tests like that in the existing tests that run before testr
22:09:56 <vipul> #action Discuss use of mock/sqllite/fixtures
22:10:13 <vipul> Ok, next item.. Mutlipel images
22:10:16 <esp1> sounds good
22:10:21 <vipul> i think we settled that
22:10:33 <juice_> working on it now
22:10:37 <SlickNik> Yeah, we had a discussion on that.
22:10:50 <vipul> k, the agreement was that we'd push percona 'element' to RD-int
22:11:05 <juice_> #agreed
22:11:12 <vipul> next item.. dkehn, SlickNik: setting up CI
22:11:21 <SlickNik> #agreed
22:11:42 <dkehn> SlickNik, has the changes in the vm-gate functions and I'm testing as we speack
22:11:53 <dkehn> once completed will move to CI upstream
22:12:00 <SlickNik> So far I've got the post-devstack changes in for redstack.
22:12:11 <SlickNik> And the simple tests are working.
22:12:23 <vipul> cool
22:12:33 <SlickNik> There are a few black-box tests that are failing on me that I'm still debugging.
22:12:50 <vipul> #action dkehn, SlickNik to complete CI with devstack-vm-gate with blackbox tests
22:12:56 <SlickNik> and dkehn is working on integrating upstream with the vmgate.
22:13:04 <hub_cap> have yall gotten it working on cloud servers?
22:13:10 <hub_cap> rax cloud servers :)
22:13:12 <vipul> k, that's next
22:13:15 <SlickNik> We're doing this on HPCloud server.
22:13:21 <dkehn> hpcloud only
22:13:22 <vipul> anyone have a rax account :)
22:13:24 <SlickNik> I don't have access to RAX cloud servers :)
22:13:30 <hub_cap> ill gladly spawn u one
22:13:36 <dkehn> dkehn, me either
22:13:36 <hub_cap> tomorrow just remind me
22:13:45 <SlickNik> okay, hub_cap, I'll ping you.
22:13:50 <hub_cap> np
22:13:53 <hub_cap> good luck w/ networking
22:13:56 <SlickNik> Will be good to see how/if they're different.
22:13:59 <hub_cap> it seems completely broken
22:14:13 <hub_cap> thats where i stopped :D
22:14:18 <vipul> somehow the CI guys are able to run there though
22:14:23 <vipul> they may be doing some hacks
22:14:27 <hub_cap> ya but they dont do tests from _in_ teh guest
22:14:33 <hub_cap> they just spawn a instance
22:14:34 <vipul> oh i see
22:14:35 <hub_cap> we can do that
22:14:45 <hub_cap> ya...
22:14:51 <hub_cap> maybe someone on your end tho has some fu
22:15:08 <vipul> #action SlickNik to test on Rax Cloud Servers
22:15:21 <vipul> next item.. update on HP CLoud Server and int-tests
22:15:22 <hub_cap> i torqued it a few times too ;)
22:15:32 <vipul> I still haven't been able to get a good run on hpcloud instance
22:15:37 <hub_cap> wtf...
22:15:43 <hub_cap> thats sucky
22:15:43 <vipul> wiht "cheat codes" i was successful
22:15:55 <juice_> vipul: how many tests fail?
22:15:56 <hub_cap> oh thats at least good... but a full run is failing?
22:16:05 <vipul> but the tests have some funky dependency and hard to pinpoint exactly which one
22:16:13 <vipul> mainly the Stop Test
22:16:19 <vipul> but that causes a couple of skips
22:16:31 <vipul> all others pass on a full run
22:16:34 <hub_cap> vipul: maybe we can shed some light... if u can give someone like tim a login to a machine maybe he can play around w/ it
22:16:36 <hub_cap> or me when i get some time
22:16:46 <vipul> K, that will help.
22:17:16 <vipul> #action Vipul hub_cap and grapex to debug int-tests on hpcloud
22:17:19 <hub_cap> word
22:17:24 <vipul> next item: cheat codes
22:17:43 <vipul> cp16net gave us a brief intro to them last meeting
22:17:58 <vipul> i think we're good unless there are other tricks we should know
22:18:13 <hub_cap> i think there were just 1 or 2 right?
22:18:21 <hub_cap> the instance is already created one
22:18:24 <juice_> is it "up - down - up - down - left - right - left - right - a - b - enter" ?
22:18:30 <vipul> lol
22:18:32 <hub_cap> juice_: select start
22:18:40 <vipul> yes, that's the one i saw, what's the other?
22:18:42 <juice_> no wonder it never worked
22:18:42 <hub_cap> my gamepad has no enter key
22:18:50 <hub_cap> grapex: is there another cheat code?
22:18:52 <juice_> what are "cheat codes"
22:19:00 <juice_> in this context?
22:19:12 <vipul> running integration tests, without creating an instance during the run
22:19:13 <hub_cap> juice_: special ways to alter the executiojn of the tests
22:19:16 <grapex> The only other one is TESTS_DO_NOT_DELETE_INSTANCE
22:19:21 <SlickNik> env variables that you can set to speed up tests
22:19:23 <juice_> ah
22:19:30 <grapex> Set that to True and it skips the delete.
22:19:33 <hub_cap> and of course there is
22:19:35 <grapex> There's really only two.
22:19:37 <juice_> gots it
22:19:38 <hub_cap> --group=blackbox --stop
22:19:43 <SlickNik> grapex: I've found that one especially useful.
22:19:45 <hub_cap> that one will stop tests on first failure
22:19:50 <grapex> --stop is a nose argument
22:19:54 <hub_cap> ya
22:20:03 <vipul> cool, good stuff
22:20:05 <vipul> next: hub_cap to talk to heckj and bcwaldon about how they are doing their next gen specs
22:20:08 <hub_cap> ./redstack int-tests --group=blackbox --stop <--useful if u wantt to diagnose
22:20:18 <hub_cap> have not done so yet, its on my list
22:20:31 <vipul> #action hub_cap to talk to heckj and bcwaldon about how they are doing their next gen specs
22:20:34 <hub_cap> ive talked /w my mgr and we are going to start drafting v2
22:20:38 <vipul> next: hub_cap to get the featureset for V2 started, BP & wiki
22:20:39 <hub_cap> next week i will have something
22:20:51 <hub_cap> thats same. put it on the action items
22:20:56 <hub_cap> they are really one in the same
22:21:07 <hub_cap> the v2 api is my top prio after i finish cinder
22:21:18 <hub_cap> so next week sometime
22:21:23 <vipul> K, at some point we need to sync up regarding feature set for v2
22:21:41 <vipul> hub_cap to talk to mikeA about timelines for v1 wadl
22:21:47 <vipul> I think you mentioned that's coming?
22:21:58 <hub_cap> ya
22:22:01 <vipul> any timeline?
22:22:10 <hub_cap> so let me first talk about v2 "sync"
22:22:18 <hub_cap> i totally agree we need to sync up
22:22:31 <hub_cap> ill give u guys teh wiki links so u can add/edit as u see fit adn we can discuss
22:22:50 <hub_cap> i dont want this to be a autocratic spec
22:22:57 <hub_cap> now that im off that soapbox, v1
22:23:13 <hub_cap> mike A is working on it now. he should have it ready soon... he didnt give an exact timeline
22:23:24 <jcooley> hub_cap: also regarding use of wiki.
22:23:26 <hub_cap> #link https://github.com/stackforge/database-api
22:23:41 <vipul> Ok, works for me.  Just need to make sure we're aligned as far as features, and we can iterate on the wiki if that's the way it's done
22:23:46 <jcooley> is there a place we can start putting some of the knowledge we're uncovering around setting up, running, etc?
22:24:00 <hub_cap> vipul: ill find it out some way or antoher and keep u in the email chain/etc/...
22:24:11 <hub_cap> jcooley: id say create some wiki.openstack pages
22:24:18 <hub_cap> maybe we need a "landing" page for reddwarf
22:24:27 <hub_cap> and to put links on that... i _hate_ the wiki they have
22:24:29 <jcooley> ok, didn't know if you had an area you were currently using for similar
22:25:00 <jcooley> i'm just seeing us create knowledge internally that we should be sharing with the community
22:25:00 <vipul> #action, link Reddwarf wiki area (create one if necessary) to http://wiki.openstack.org/Projects
22:25:00 <hub_cap> for instance, there is a page for incubated project
22:25:00 <hub_cap> http://wiki.openstack.org/Heat
22:25:31 <vipul> #link http://wiki.openstack.org/Reddwarf
22:25:36 <vipul> is still not created, probably the place to start
22:25:45 <hub_cap> http://wiki.openstack.org/Reddwarf
22:25:48 <hub_cap> i jsut did
22:25:48 <SlickNik> Probably a good idea.
22:25:53 <hub_cap> god it so slow
22:26:25 <vipul> ok, last item juice: percona elements
22:26:45 <jcooley> hub_cap: sweet
22:27:09 <vipul> juice ^^?
22:27:28 <hub_cap> juice_: ^ ^
22:27:31 <hub_cap> :D
22:27:38 <vipul> sneaky
22:27:41 <hub_cap> pete
22:27:44 <vipul> lol
22:27:46 <SlickNik> heh
22:27:48 <esp1> let me throw something at juice
22:27:49 <hub_cap> oh this isint word association :)
22:28:06 <juice_> what is the question?
22:28:13 <vipul> #action juice_ to push percona bits to Redddwarf-Int
22:28:15 <SlickNik> oh, oh…Do I see projectiles...
22:28:26 <juice_> nerd frisbee please
22:28:29 <juice_> nerf
22:28:32 <hub_cap> either way
22:28:33 <vipul> ok done with action items..
22:28:45 <vipul> #topic CI Updates
22:28:46 <esp1> juice: all I have over here are keyboards and rocks
22:29:09 <vipul> I htink SlickNik and dkehn covered most of this
22:29:18 <dkehn> I think so
22:29:20 <SlickNik> yeah covered it earlier.
22:29:20 <vipul> wanna recap?
22:29:22 <hub_cap> yup, and all thats left is me to spin up a cloud isntacne for yall
22:29:32 <vipul> ok, we're good then
22:29:42 <juice_> what about the other topic quotas
22:29:45 <vipul> we're targetting this by end of week? to push upstream?
22:30:13 <vipul> juice_: it's coming... http://wiki.openstack.org/Meetings/RedDwarfMeeting
22:30:37 <SlickNik> dkehn was looking into pushing upstream, he probably has a better idea of the timeline for that…dkehn?
22:31:22 <dkehn> I'm targeting as ASAP, but we must get it working and then separate it out into their schema (wrapper, etc..
22:31:44 <dkehn> but yes, as soon as possible, I'm not the only one involved with CI
22:32:03 <vipul> just an FYI for hub_cap, grapex: We're going to push upstream while invoking simple-tests for now
22:32:11 <vipul> until we verify that blackbox runs everywhere
22:32:20 <hub_cap> as in, you are testing locally before u push right?
22:32:26 <dkehn> once in upstream then we can add things to it
22:32:26 <hub_cap> oh nm, simple-tests
22:32:30 <SlickNik> I've got the simple tests running successfully on an HP Cloud instance after a post devstack build. Will talk to grapex about some issues I'm running into with the full suite of black-box tests.
22:32:32 <dkehn> yes
22:32:43 <hub_cap> well int-tests should work on the cloud isntance
22:32:50 <hub_cap> but simple-tests wont even work for u as is
22:32:51 <hub_cap> :)
22:32:58 <vipul> right, _should_
22:33:06 <hub_cap> sure, we run em all the time :D
22:33:19 <hub_cap> but we run on old networked cloud servers b4 the new nova switch
22:33:22 <hub_cap> and using openvz
22:33:25 <hub_cap> :D
22:33:26 <grapex> vipul SlickNik: As long as running all of blackbox remains a goal I'm good.
22:33:27 <esp1> should we rip out simple-tests ?
22:33:32 <vipul> right :)
22:33:42 <vipul> grapex: Yep, blackbox is the end goal
22:33:42 <esp1> or should we wait...
22:33:45 <hub_cap> once we get int-tests solid we should remove simple-tests
22:33:52 <esp1> cool
22:33:52 <SlickNik> grapex, I'm with you 100% we want to run all of blackbox.
22:34:09 <hub_cap> id like to figure out why your int-tests are failing, and im volunteering grapex time :) (jk)
22:34:15 <vipul> do you guys run int-tests in KVM?
22:34:20 <hub_cap> ya
22:34:21 <SlickNik> this is the first step so we can iron out issues with pushing CI upstream and work on the testing issues in parallel.
22:34:26 <hub_cap> im 100% public these days
22:34:34 <hub_cap> and your int-tests work in local env right?
22:34:43 <vipul> right, just not in a cloud instance
22:34:47 <hub_cap> coo
22:34:47 <esp1> yeah, they run fine in vmware fusion
22:34:55 <hub_cap> lets figure out what the deal is w/ that this wk
22:35:02 <hub_cap> ill spare some time vipul tomorrow if u want to chat it up
22:35:10 <hub_cap> and give me a login so i can watch em go
22:35:15 <hub_cap> (or not go...)
22:35:23 <vipul> ok, so in the meantime we dont' want that holding up, which is why we push with simple-tests, and change it up later
22:35:27 <vipul> hub_cap: sounds good
22:35:30 <hub_cap> sure
22:35:41 <vipul> k, next topic
22:35:45 <hub_cap> but simple-tests will fail on a cloud server too ;)
22:35:53 <hub_cap> lets roll
22:35:57 <vipul> #topic Unit Test updates
22:36:25 <steveleon> annashen and I have been adding unittests to the guest agents
22:36:27 <vipul> so we've got lots of tests the python-reddwarfclient + guestagent
22:36:48 <vipul> grapex: wanna discuss what we need to do push the last two reviews through?
22:36:57 <grapex> Sure
22:37:05 <SlickNik> yeah hub_cap I'll try and see if I can make the tests (networking) work on cloud server. Get me a login.. :)
22:37:12 <grapex> I've been writing an email about it for awhile
22:37:20 <hub_cap> SlickNik: ill one up u, ill give u a dedicated server!
22:37:54 <grapex> So first off, I want to say I know the code can be confusing and I'm sorry. I also apologize that there are unused functions and cruft and junk in the code. :(
22:37:54 <vipul> grapex: just got the email
22:38:32 <SlickNik> hub_cap: sweet! thx, will ping you after the meeting.
22:38:39 <grapex> That said, while writing unit tests, especially for the reference guest which has no coverage other than the public tests that just started running, feel free to delete methods if you believe they aren't used.
22:38:43 <steveleon> grapex: It's all good... It's been a very good experience for us :-)
22:38:50 <grapex> The integration tests should catch big issues like that
22:39:18 <vipul> Yep, the issue is we don't have a way to check coverage with Integration Tests, unless you go through every test
22:39:24 <grapex> plus we'll catch it in the gerrit reviews, if not we're not doing a good job of reviewing the pull requests.
22:39:33 <grapex> vipul: Yes, that is a problem.
22:40:07 <grapex> Maybe we need to talk to Monty or someone about this, but is it necessary to cover even methods we really can't test in the unit test environment?
22:40:31 <grapex> I think the code would be clearer if we just added no cover directives
22:40:53 <grapex> Then made sure to hit those with either integration tests or functional tests
22:41:10 <vipul> Yep, I agree there is a probably a good balance...
22:41:26 <vipul> at the same time.. eventually I think we'd like to get to a point where we can gate on a Coverage %
22:41:27 <grapex> Another thing about the reference guest - Sneaky Pete has functional tests which do things like make sure the utility code to hit a MySQL db or call other processes is actually working.
22:41:45 <grapex> vipul: We could add no coverage directives and then make the case during the gerrit review as to why
22:42:07 <vipul> That could work..
22:42:16 <hub_cap> vipul: lets be careful about gating on coverage %s too... does any other project do it?
22:42:23 <grapex> if we also added integration or functional tests as we do that, it would be kosher
22:42:33 <vipul> not yet, but they'd like to.. so not saying we're ready to yet.. but eventually
22:42:50 <hub_cap> ok cool id like to watch a openstack project do it for a while first
22:42:56 <hub_cap> see how it works (or doesnt) for them :D
22:43:18 <grapex> Second thing I would like to propose is that we spend some time deleting code that isn't currently isn't covered by the coverage report of the int-tests as run in Tox
22:43:31 <jcooley> yah, was thinking we could be conservative.  just to make sure the coverage doesn't slide as new code is added.
22:43:32 <grapex> When I got the integration tests working in tox the coverage was something like 78%
22:43:55 <grapex> When I looked at what wasn't covered, some of it seemed like it could be cruft
22:44:08 <vipul> ++
22:44:15 <hub_cap> and we should set the entire openstack/common dir to nocover too :D
22:44:19 <grapex> I'm worried that by leaving this junk in there we could be wasting people's time if they cover it. :(
22:44:24 <grapex> hub_cap: Agreed
22:44:33 <jcooley> ++
22:44:33 <hub_cap> god why isisnt that a pip library :D
22:45:03 <grapex> So maybe the thing to do is aggressively remove junk first- kind of like when you're trying to put on muscle, first you lose all the weight you can and make sure the pounds you're adding aren't fat. :)
22:45:09 <vipul> #agreed grapex, we'll need your assistance in Identifying some of the cruft.. since we're in a 'scared to delete cuz we don't know enough about it yet' mode
22:45:42 <SlickNik> #agreed lol, while I don't quite agree with your analogy grapex, I completely echo your sentiments :)
22:46:03 <hub_cap> lol grapex
22:46:09 <grapex> vipul: I really appreciate that you are being careful with deleting stuff, I think that's coming from the right place. However the goal of the integration tests is to find anything wrong in the "happy path" so if necessary code is missing we should find out quick.
22:46:22 <hub_cap> grapex is quite aggressive
22:46:27 <grapex> Plus if anyone deletes something we need we'll be quick to point it out.
22:46:55 <grapex> I'm aggressive, like an inverted junk yard dog that destroys junk rather than protecting it!
22:47:05 <SlickNik> Having dead code to maintain is no fun.
22:47:24 <hub_cap> having terrible analogies to maintain is no fun either :P
22:47:29 <vipul> Ok grapex: so let's do this... can you guys look over the two reviews.. let's figure out what shouldn't be unit tests.. and get some of that merged
22:47:45 <grapex> vipul: Will do.
22:47:45 <hub_cap> agreed. grapex is your man for those
22:47:58 <esp1> grapex: I'll ping ya in a bit to see if we can figure stuff out
22:48:14 <vipul> #action grapex to look over guestagent unit test reviews, work with steveleon to get them merged
22:48:29 <vipul> alright.. let's move on to the next thing
22:48:34 <steveleon> ill take some of the comments and suggestion from grapex and apply them to my patches
22:48:35 <vipul> #topic Quota / Rate limiting
22:48:57 <vipul> Ok we're starting to look at adding supprot for quotas, rate limits in REddwarf this sprint
22:49:07 <vipul> Just quickly looked over repose
22:49:36 <vipul> i guess the discussion is: what should this look like?  since Repose is another standalone service
22:49:44 <vipul> do we want to do something lightweight instead?
22:49:53 <vipul> anyone looked at 'turnstile'?
22:49:55 <hub_cap> well id say that we should not reinvent the wheel here :)
22:50:08 <hub_cap> so wrt turnstyle i know nova said they would move to it eventually
22:50:08 <jcooley> don't do another tier. either repose (ok, maybe turnstile).
22:50:15 <juice_> turnstile here appeared to have limited functionality
22:50:18 <hub_cap> but they are using repose currently
22:50:23 <jcooley> but we should use an openstack project if one exists.
22:50:26 <hub_cap> so i really dont want to have a 3rd type
22:50:33 <hub_cap> jcooley: thats repose :P
22:50:35 <hub_cap> ish
22:50:47 <jcooley> turnstile != repose :)
22:50:54 <hub_cap> thats what they are trying to do w/ repose
22:51:00 <vipul> hub_cap: nova has their own quota class i thought
22:51:01 <jcooley> hub_cap: in agreement on no 3rd option
22:51:21 <hub_cap> ya its legacy
22:51:24 <jcooley> we looked at the repose project at the OS conference in SD.
22:51:26 <hub_cap> its been around for a VERY long time
22:51:29 <jcooley> looks pretty interesting.
22:51:35 <vipul> another potential Issue.. Repose is java
22:51:50 <jcooley> ugh. yah, so was our global proxy tier :(
22:52:04 <vipul> since it's standalone, probably not a big deal.. but surprised that Openstack is choosing that
22:52:28 <hub_cap> well... repose is trying to get into openstack
22:52:41 <hub_cap> and at rax we are using it as a quota system
22:52:41 <jcooley> we may actually have to run it in our prod environment due to the HP deltas from Keystone, but... don't want this in the reference implementation.  want to keep that clean from corporate operational requirements
22:52:51 <hub_cap> but ure right, its not _the_ defacto
22:53:06 <hub_cap> so maybe its not a bad idea to have something, albeit small
22:53:08 <hub_cap> https://github.com/openstack/nova/commits/master/nova/quota.py
22:53:37 <vipul> any reason why this is not in oslo?
22:53:41 <hub_cap> https://github.com/openstack/nova/commit/1cf475d7a135c1078cf7df11c261618af501dc37 :)
22:53:44 <juice_> an embedded class over a standalone service
22:53:46 <jcooley> repose is solving the right requirements.  want rate limiting, white-listing, black-listing, SSL termination, Keystone middleware
22:53:46 <hub_cap> vipul: thats a good question
22:53:53 <juice_> sounds like a good place to start
22:54:24 <juice_> use something like nova-quota unless there is a clear direction
22:54:31 <hub_cap> but even https://github.com/klmitch/turnstile is old as crap
22:54:42 <hub_cap> juice_: nova quota has no per user quota support fyi
22:54:50 <vipul> jcooley, hub_cap: sounds like we wnat both an embedded quota solution + a repose hook
22:55:02 <jcooley> oooh.  need to have per-user or per-tenant
22:55:10 <hub_cap> its VERY limited
22:55:20 <juice_> can we just write our own for the time being in the likes of nova-quota?
22:55:27 <jcooley> vipul: ++
22:55:40 <hub_cap> well why not try to push something to oslo instead?
22:55:42 <jcooley> repose goes in front, so just need to turn off any embeded solution
22:56:14 <vipul> #link https://github.com/openstack/cinder/blob/master/cinder/quota.py
22:56:31 <vipul> yea, that's an option, push a generic framework to Oslo
22:56:43 <hub_cap> ya i wouldnt necessarily use cinder as the golden child vipul
22:56:51 <hub_cap> #link https://github.com/openstack/nova/commits/master/nova/quota.py
22:57:01 <vipul> looks liek they all basically have their own
22:57:10 <grapex> jcooley: Or use both for double protection and give ops the joy of ensuring the configs are in sync. :)
22:57:17 <hub_cap> right lets do this
22:57:24 <jcooley> grapex: yah, that's it!
22:57:26 <hub_cap> we shouldnt decide this w/o buyin from some core peeps
22:57:34 <hub_cap> lets talk to core nova/cinder/oslo
22:57:53 <hub_cap> and see how they feel about either 1) moving something into oslo, or 2) a external wsgi service
22:58:07 <hub_cap> rather than hack up and have YAQS (yet another quota system)
22:58:18 <jcooley> exactly
22:58:30 <vipul> #action hub_cap and juice_ to talk to core nova/cinder/oslo about generic quota embedded vs external service
22:58:30 <jcooley> (it's not just about quotas)...
22:58:38 <hub_cap> ok works for me
22:58:42 <hub_cap> thx vipul
22:58:43 <vipul> right, need to take into account rate limits
22:59:01 <jcooley> need SSL term, DOS protection (white/black listing), etc
22:59:03 <hub_cap> ya those are another form of quotas from repos's perslective
22:59:22 <hub_cap> rate and absolute limits is what we need to start with
22:59:45 <hub_cap> but we digress... lets just work it offline w/ the core teams
23:00:10 <vipul> alright.. let's get the discussion started at least
23:00:11 <juice_> who will take the reigns on that one
23:00:41 <hub_cap> juice_: see above
23:00:44 <hub_cap> its me and u
23:00:44 <vipul> juice_ can you start by talking to mordred, and then we bring it up to the Oslo members
23:00:58 <hub_cap> ill talk w/ nova/cinder/glance ptls
23:01:07 <juice_> ok
23:01:09 <hub_cap> adn we can reconvene in a day or 2
23:01:13 <vipul> i'm sure this is something everyone's thought about
23:01:20 <hub_cap> and eveyrone has a differing answer :D
23:01:26 <hub_cap> woo openstack!
23:01:42 <jcooley> woot!
23:01:50 <vipul> k, we got a plan...
23:01:53 <hub_cap> word
23:01:55 <vipul> i think that's all for items
23:02:00 <vipul> #topic Open Discussion
23:02:01 <SlickNik> that sounds good.
23:02:03 <hub_cap> and i dont really have any "extra" items to discuss...
23:02:07 <vipul> anything else people want to get off ther chest
23:02:18 <SlickNik> nothing from my end.
23:02:21 <grapex> Functional tests
23:02:29 <vipul> k.
23:02:34 <grapex> In the blue print, there's talk about creating some kind of new framework
23:02:55 <grapex> I don't see the point. We can just add new groups in the RDI tests and run them as part of CI.
23:02:57 <esp1> grapex: this is the testr thing right?
23:03:01 <grapex> Yes.
23:03:11 <hub_cap> brb
23:03:41 <esp1> yeah, there were folks who wanted to get closer to what openstack is using.
23:03:52 <grapex> We do that at Racks for some internal stuff relating to billing, things we need to access certain resources for. There's not that many great examples of using it in the public.
23:04:09 <esp1> I think CI guys want to be able to run tests in parallel someday
23:04:29 <lifeless> esp1: CI does run in parallel
23:04:37 <lifeless> esp1: for some projects. Want it for ~all
23:04:46 <vipul> Is this a question of whether to add non-integration tests to probosis?
23:04:57 <grapex> I see. We'll next hackathon at Rackspace I'll be looking at making Proboscis integrate with testr and run non-dependent test series together.
23:04:59 <esp1> lifeless:  right not at the moment but I think it's a goal of their's in the future
23:05:32 <lifeless> esp1: hmm? nova tests *already* run in parallel in CI
23:05:51 <lifeless> esp1: I may be confused about what you mean.
23:06:02 <hub_cap> ya fo sure
23:06:03 <grapex> vipul: My point is we use proboscis to do that internally and it's proven easy. The benefits is that all of the setup code to communicate to the database and other resources already lives there.
23:06:20 <hub_cap> hi lifeless :)
23:06:23 <esp1> lifeless: sorry misread.  but I mean that folks want to run reddwarf tests in parallel one day.
23:06:24 <SlickNik> lifeless: I think esp1 is talking about the current reddwarf tests running through proboscis.
23:06:44 <lifeless> ah right
23:06:52 <grapex> I see the concern about parallel testing though. I think we'll alleviate it by reducing use of "depends_on" (which is overused now) and getting testr to fire off every non-related series of methods.
23:06:54 <vipul> grapex, esp1: Getting probosis to run testr may be the solution here... if we can do that _and not make them all depend on one another_ then i am ok with it
23:06:57 <lifeless> yes, doing that in parallel +1 ;). Happy to advise on any testr questions.
23:07:08 <jcooley> yes, want the tests to run in parallel.  don't want to write tests in probosis; want to use testr.
23:07:09 <grapex> lifeless: Thanks
23:07:27 <lifeless> [and other stuff, but I suspect the only new thing here for you folk is testr :P]
23:07:42 <grapex> jcooley: Keep in mind testr is the runner...we can still make use of test tools and everything.
23:08:09 <jcooley> just don't want to see a) reinventing the wheel, or b) a bunch of red-dwarf-only tools
23:08:33 <esp1> regarding question of "functional" tests we were thinking of creating a separate package for tests that make will make use of sqlite there.
23:08:47 <grapex> jcooley: I agree, I don't want to rewrite tests we've been depending on.
23:08:54 <hub_cap> testr is the pirate of tests for sure
23:08:54 <esp1> perhaps functional is not a good name.
23:09:06 <jcooley> grapex: yep.
23:09:22 <juice_> I think it is misleading to use functional in that context
23:09:33 <jcooley> keep what we have, migrate eventually if we can -- if it makes sense.
23:09:40 <juice_> it seems like the current int-tests are more functional than int-tests
23:09:41 <grapex> So are we talking about unittests + live resources?
23:09:55 <esp1> juice_: yep.  so how about sqlitetests?
23:10:01 <SlickNik> grapex: sounds good, I'm okay with proboscis as long as we can parallelize with testr.
23:10:05 <vipul> grapex: so how about this... new tests that are not integration tests for now are just testr tests
23:10:08 <esp1> grapex: yeah
23:10:20 <esp1> I want to exercise the actual code and not mocks
23:10:27 <jcooley> ++
23:10:30 <juice_> depends on your background but I think the more common definition would be unit tests, integration tests (sqllite with multiple classes) and functional (almost near production / use case based)
23:10:39 <vipul> and things that are integration-tests, and ones that need the additional 'live resources' need to go in as probosis
23:11:29 <hub_cap> ok so are we in agreement?
23:11:35 <grapex> vipul: Agreed, for now let's just keep it to unit tests we can run in tox and integration tests we run in the VM. This will include things that might for example test reference guest methods like "start_mysql"
23:12:07 <vipul> #agreed
23:12:24 <SlickNik> #agreed
23:12:31 <hub_cap> #agrease
23:12:49 <vipul> Ok, if that's all, i'm going to wrap it up
23:13:00 <hub_cap> put a bow on it vipul
23:13:03 <vipul> #endmeeting