22:00:58 <hub_cap> #startmeeting reddwarf
22:01:02 <openstack> Meeting started Tue Jan 15 22:00:58 2013 UTC.  The chair is hub_cap. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:01:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:01:05 <vipul> i see you're rusty
22:01:05 <openstack> The meeting name has been set to 'reddwarf'
22:01:10 <hub_cap> vipul: yes i am
22:01:31 <hub_cap> so i updated the meeting notes a bit
22:01:34 <hub_cap> #link http://wiki.openstack.org/Meetings/RedDwarfMeeting
22:01:42 <cp16net> hi
22:01:48 <datsun180b> hello
22:02:00 <esp> greetings
22:02:05 <grapex> Yo
22:02:05 <SlickNik> I see.
22:02:12 <hub_cap> i just screamed 'meetings now' to our team so they should be comin in
22:02:13 <vipul> full house
22:02:28 <SlickNik> I like the "MERGED!" part of the notes. :P
22:02:28 <hub_cap> so lets start w/ action items
22:02:32 <hub_cap> SlickNik:  ;)
22:02:40 <hub_cap> #topic action items
22:03:06 <cp16net> ya!
22:03:10 <hub_cap> vipul: had a chance to file a BP for quotas?
22:03:31 <vipul> No, not yet, obviously you heard about our policies changing again
22:03:35 <hub_cap> OHHHH ya
22:03:39 <hub_cap> ill file a dummy for ya
22:03:49 <vipul> k, put it back on my plate though
22:03:53 <hub_cap> will do
22:03:55 <jcooley> best if you guys file-em and we edit for now.  we'll take our existing ones through the process
22:04:02 <hub_cap> #action Vipul to file blueprint on quota support in Reddwarf
22:04:12 <hub_cap> yup jcooley ill do that
22:04:20 <hub_cap> just hit me up if u need one filed
22:04:25 <vipul> k
22:04:28 <jcooley> thanks!
22:04:52 <hub_cap> so next one says dan, assuming esp?
22:04:59 <vipul> esp it is
22:05:08 <esp> yep
22:05:16 <hub_cap> was it esp who filed the BP? Dan file a blue print for planning testr -> reddwarf
22:05:21 <hub_cap> the last bp
22:05:29 <esp> yep, that was me.
22:05:38 <hub_cap> ok so u have done that then nice
22:05:38 <esp> probably could use some edits
22:05:55 <hub_cap> cool. lets look it over, ill action that
22:06:02 <esp> yes sir, I have it marked done (completing a BP) on our end
22:06:10 <hub_cap> #action vipul SlickNik grapex hub_cap to look over the testr BP
22:06:10 <esp> thx
22:06:23 <hub_cap> esp: so u cancelled it out? do we need a new bp?
22:06:35 <SlickNik> Cool, will look it over. Thanks esp…
22:06:37 <esp> Oh yeah?  that was silly
22:06:42 <esp> let me take a look at it.
22:06:45 <vipul> #link https://blueprints.launchpad.net/reddwarf/+spec/testr-unit-tests
22:06:57 <hub_cap> vipul: check out blueprint-test-2
22:07:01 <hub_cap> u can use that one
22:07:17 <vipul> hub_cap: thanks
22:07:20 <hub_cap> ok nice vipul on that link
22:07:30 <hub_cap> next, imma assume we have not talked aobut
22:07:35 <hub_cap> the ongoing debate :)
22:07:39 <hub_cap> #action SlickNik, hub_cap, juice, vipul talk to mordred about housing multiple images (elements) in reddwarf
22:07:52 <vipul> dkehn may have some updates?
22:07:52 <juice> yowsers
22:07:57 <juice> still open
22:08:00 <dkehn> I have been talking to mordred about that
22:08:01 <juice> mordred is back
22:08:05 <juice> try to close this one out today
22:08:06 <esp> that BP looks like my work, I can see a bunch of typos and run-on sentences
22:08:07 <hub_cap> ya? whats the story
22:08:21 <dkehn> and he belives that the build code should go into reddwarf,
22:08:38 <dkehn> this can be conditionally called or not to avoid testing issues
22:09:03 <dkehn> in the future we are most likely going to have more (i.e. mariaDB, etc.)
22:09:08 <dkehn> drizzle
22:09:14 <dkehn> etc.
22:09:23 <vipul> hub_cap, dkehn: let's try to get mordred into #reddwarf and hash this out
22:09:26 <hub_cap> cool. id like to have a chat w/ him about it too... so lets bring it in to the open sometime this wk
22:09:31 <dkehn> if someone is not supporting ti they should exclude it
22:09:31 <hub_cap> not today tho :)
22:09:40 <dkehn> made the request
22:09:44 <hub_cap> thx
22:09:56 <hub_cap> so lets talk volume support cp16net vipul
22:10:02 <hub_cap> thats the next action item
22:10:15 <hub_cap> i dont have much context on that.. can yall fill me in
22:10:19 <vipul> volume-support seems to be work as advertised
22:10:30 <cp16net> i did not get to looking into much
22:10:41 <vipul> I think there were some issues before ,but those may be resolved by cp16net
22:10:50 <hub_cap> so is this a action item? or can we move on
22:11:01 <vipul> afaict, it's good for now
22:11:02 <SlickNik> I was able to use volumes too.
22:11:04 <hub_cap> sweet
22:11:06 <cp16net> cool
22:11:07 <SlickNik> I think it's working now.
22:11:14 <hub_cap> Slicknik, your turn
22:11:17 <SlickNik> So might not be an issue.
22:11:17 <esp> I think we are good on volume support
22:11:23 <hub_cap> configuring gate to run devstack with reddwarf's local.sh
22:11:30 <hub_cap> any update to that sucker
22:12:07 <SlickNik> Had to make some changes to local.sh and redstack
22:12:26 <SlickNik> Running with juice's new imagebuilder and still working on making sure the tests are all green.
22:12:41 <SlickNik> So still work in progress.
22:13:02 <hub_cap> cool should we put the same AI on next wks? care to reword it?
22:13:04 <SlickNik> Dkehn's made some good headway into looking into the vm-gate integration.
22:13:20 <hub_cap> good deal. im also working on it in cloud server land
22:13:45 <dkehn> yes, we have a working model, heaviely modifuled and running into issues
22:13:48 <vipul> yea, let's try to get the int-tests running in cloud servers, since that's where jenkins will run the gates
22:14:09 <SlickNik> #action Slicknik working on making integration tests run post devstack install + local.sh run for CI
22:14:10 <dkehn> WebOb, glance 1.0.8 issues
22:14:11 <hub_cap> im trying to get the int-tests runnin on the cloud servers
22:14:30 <vipul> hub_cap, i'm trying to get them running in hpcloud server
22:14:32 <hub_cap> huh? ive got everything runnin except i have routing issues
22:14:40 <hub_cap> w/ the local ip
22:14:45 <hub_cap> from teh bridge....
22:15:00 <hub_cap> ive gone thru these issues before and fixed them w/ some networking fu
22:15:06 <esp> maybe check /etc/hosts to see if you have an entry for 127.0.1.1 ?
22:15:12 <grapex> Quick note re: volumes - looks like the resize volume test isn't enabled.
22:15:14 <grapex> I'll file a bug on it
22:15:17 <hub_cap> grapex: <3
22:15:33 <juice> wonder if those are the same issues SlickNik was having
22:15:45 <SlickNik> #action dkehn slicknik working on setting up CI integrating reddwarf tests with devstack-vm-gate
22:15:46 <hub_cap> naw esp it has to do w/ routing, and the bridge, and the eth device its on, w a little nat masquerading fu to get out to the real world for apt-get
22:16:02 <hub_cap> i should have more info tomorrow
22:16:13 <esp> gotcha
22:16:16 <hub_cap> and give u guys a script of what we have to run in the vm-gate
22:16:40 <dkehn> hub_cap, please ping me with it
22:16:44 <vipul> hub_cap: is this different thna the scripts that are in openstack-ci?
22:16:47 <hub_cap> dkehn: u will see it for sure
22:16:51 <hub_cap> vipul: it will be different
22:16:52 <dkehn> k
22:17:02 <hub_cap> becuase they dont really care about what goes on _in_ the vm
22:17:10 <hub_cap> they just make sure it spawns and goes to active rigth?
22:17:17 <dkehn> y
22:17:19 <hub_cap> we have to log in, and install a guest, and have that guest talk to the world
22:17:23 <hub_cap> so its tricky
22:17:35 <dkehn> there is an issue though with what I mentioned previously https://review.openstack.org/#/q/status:open+message:webob,n,z
22:17:44 <grapex> https://bugs.launchpad.net/reddwarf-integration/+bug/1100058
22:17:44 <hub_cap> ill give u guys the info, and let u work w/ mordred to decide if its _ours_ or belongs in openstack-ci
22:17:47 <dkehn> concerning the glance pip-requires
22:17:51 <vipul> hmm, ok - although that shoudl be done by our tests..
22:18:01 <SlickNik> hub_cap: FWIW, I was having bridge issues between the host and guest after the diskimage-builder changes.
22:18:18 <vipul> unless there is stuff that needs to happen on the host, which may be the case
22:18:24 <hub_cap> hmm, i havent seen those SlickNik... and i havent updated the cloud server im on to use the new builder changes
22:18:30 <hub_cap> vipul: this is all host stuff
22:18:38 <vipul> k
22:18:40 <SlickNik> hub_cap: Still working through them
22:18:42 <hub_cap> its tweaking the host so that it can talk out and that it can route properly
22:18:51 <SlickNik> hub_cap: Will keep you posted if I find something.
22:18:53 <hub_cap> SlickNik:  lets chat tomrrow then
22:18:57 <hub_cap> maybe i can help u
22:19:05 <hub_cap> ive dealt w/ bridges and routing like 100 times at this point
22:19:10 <SlickNik> cool, sounds good.
22:19:15 <hub_cap> and banged my head bloody > 1 time
22:19:28 <hub_cap> ok so now for esp add unit testing details to testr blueprint sqlite vs fakes
22:19:30 <SlickNik> lol, I'm almost there.
22:19:54 <hub_cap> ok cool SlickNik if u arent lets chat tomorrow
22:20:15 <SlickNik> cool, thanks!
22:20:22 <kaganos> hub_cap: just making sure, you're not using virtual ip's, right?
22:20:38 <vipul> floating ips?
22:20:44 <hub_cap> kaganos: naw
22:20:44 <kaganos> yep, sorry ...
22:20:46 <hub_cap> and naw
22:20:56 <hub_cap> fixed ip from the bridge 10.0.0.X
22:21:12 <SlickNik> yep, same here.
22:21:21 <vipul> esp: i think you're up
22:21:21 <kaganos> ok, cause if you set those up, then you can't access out ...
22:21:22 <esp> hub_cap: I'll try flesh out the details of that BP regarding sqlite/fakes
22:21:35 <hub_cap> ok if u need a fakie created esp hit me up
22:21:48 <hub_cap> just a fyi, i filed a BP to make fakes external to the codebase, using fake rabbit.. there is still an issue w/ the implementations for nova/guest etc... but at least itll address that
22:21:50 <esp> last we discussed I think we decided to use both strategies
22:22:24 <hub_cap> yes its VERY nice to have the fakes run. its worth its weight in gold at present for a developer
22:22:25 <esp> ok, I'll checkout that BP as well
22:22:28 <hub_cap> esp: k
22:22:36 <hub_cap> aight done w/ the action items! woo
22:22:40 <hub_cap> 22min good job team
22:22:45 <SlickNik> sweet
22:22:51 <hub_cap> #topic Testing updates
22:22:58 <datsun180b> A new record?
22:23:04 <hub_cap> first one is CI, sholdve prolly added that to topic
22:23:06 <hub_cap> datsun180b: me thinks
22:23:21 <SlickNik> datsun180b: certainly seems like it.
22:23:23 <hub_cap> so as i previously mentioned im working on getting it to work in a cloud server env
22:23:39 <hub_cap> #action hub_cap to work w/ dkehn on running int-tests in a cloud server
22:23:47 <vipul> yep, i'm running inot issues with the Stop Tests, when run as a whole
22:23:58 <vipul> when running just the stop group, everything is good
22:24:00 <hub_cap> local or on your cloud?
22:24:03 <vipul> cloud
22:24:11 <hub_cap> not bad
22:24:18 <hub_cap> thats further than ive gotten w/ the cloud server
22:24:23 <hub_cap> :D
22:24:28 <esp> yeah much better than before all the fixes
22:24:34 <vipul> i think it's possible that the previous resize or restart tests couuld be causing the issues
22:24:45 <vipul> since those aren't run when running the group separately
22:24:52 <cp16net> I'm working on making the tests adapt between kvm and ovz entering the server
22:24:56 <hub_cap> its possible. can u ssh in to the inception vm on the cloud node?
22:25:04 <vipul> yep
22:25:07 <hub_cap> good
22:25:13 <vipul> i see the guest logs show the instance state as SHUTDOWN
22:25:20 <vipul> but that doesn't seem to be reflected in DB
22:25:26 <vipul> or something like that.. still gotta investigate
22:25:26 <hub_cap> ok we might ahve a bug in the tests
22:25:28 <hub_cap> or in the guest
22:25:31 <hub_cap> cool
22:25:44 <vipul> but this only happens in conjuction with another test :p
22:25:52 <vipul> gotta figure out which test that is
22:25:56 <hub_cap> #action vipul to provide update wrt running int-tests on hp cloud
22:26:05 <hub_cap> vipul: 1/2 the fun is not knowing what the hells going on ;)
22:26:11 <vipul> tell me about it
22:26:16 <hub_cap> other 1/2 is solving it :P
22:26:26 <cp16net> yup
22:26:41 <hub_cap> ok lets talk about local tests... is anyone having issues in vmware?
22:26:45 <hub_cap> or w/ any other local virt
22:26:55 <cp16net> no issues as of late
22:27:02 <SlickNik> Seems unfair that you can have the second half of fun only after eradicating the first half… :P
22:27:16 <hub_cap> nice
22:27:19 <hub_cap> ^ ^ SlickNik
22:27:21 <esp> no issues for me lately
22:27:32 <SlickNik> no issues in Virtualbox.
22:27:35 <grapex> I just set up a VM on a new machine today and had all the tests run fine accept for one.
22:27:37 <hub_cap> ok good so i think the rax guys have been pretty successful since cp16net fixed things
22:27:52 <hub_cap> grapex: do u remember what test? was it the same test i wsa having issues w/?
22:28:03 <grapex> test_make_sure_mysql_is_running_after_resize
22:28:22 <vipul> grapex: that passed for me, 148 secs
22:28:43 <hub_cap> hmm
22:28:46 <cp16net> hehee "works for me!"
22:28:56 <hub_cap> test_make_sure_mysql_is_running_after_resize                OK  0.84
22:29:09 <grapex> It's a fresh VM, so maybe I missed something.
22:29:10 <vipul> nvm, wrong one         test_make_sure_mysql_is_running_after_resize                OK  1.67
22:29:13 <grapex> I was glad it got that far.
22:29:17 <hub_cap> i will say... our tests are pretty damn good, but we might see odd errors here or there
22:29:27 <hub_cap> it could mean bugs or timing issues of sorts
22:29:41 <hub_cap> we might see some false positives on the VMs too, so lets be minddful and fix em as they come along
22:29:44 <vipul> i think the issue is they are very inter-twined, so hard to diagnose issues
22:30:07 <cp16net> yeah seems like the tests are fragile
22:30:14 <hub_cap> ya id hate to see them non intertwined tho based on how long it would take to spin up an instance for each one :D
22:30:19 <esp> they scare me
22:30:21 <hub_cap> there is improvement we can make
22:30:23 <vipul> wonder if it's worth making the groups be smaller, so that we can run a subset
22:30:42 <cp16net> vipul: thats the benifit of running some of the cheat codes
22:30:43 <cp16net> :)
22:30:52 <SlickNik> cheat codes?
22:30:53 <vipul> cp16net: you'll have to educate me on that
22:30:55 <hub_cap> LOL
22:31:01 <grapex> Its undoubtable there is a real bug somewhere that's hard to hit. I bet will find there's something we should be checking or asserting which we haven't thought about yet.
22:31:02 <hub_cap> ya thre is a backdoor trojan
22:31:05 <cp16net> basically dont create a server use this existsing server
22:31:09 <hub_cap> grapex: yup
22:31:13 <cp16net> saves massive time :)
22:31:17 <SlickNik> I tried up, up, down, down, etc… didn't do anything :P
22:31:22 <grapex> lol
22:31:27 <hub_cap> so we are getting a bit OT, lets tlak about the "cheat code" offline
22:31:37 <cp16net> #agreed grapex
22:31:38 <hub_cap> #action cp16net to discuss "cheat codes" w HP
22:31:40 <SlickNik> okay, sounds good. I'd be interested as well.
22:31:44 <vipul> nice
22:31:52 <cp16net> sounds good :)
22:32:02 <cp16net> SlickNik: lol
22:32:06 <grapex> Check instances.py, the environment var "TESTS_USE_INSTANCE_ID" can be used to skip to the point where the instance is created.
22:32:27 <vipul> wow
22:32:43 <hub_cap> we def need to update the tests in some form or fashion. lets keep it on the forfront of our brains while we develop
22:32:51 <esp> nice
22:32:56 <hub_cap> anything else w/ the tests for now?
22:33:07 <hub_cap> if not we be movin on
22:33:25 <vipul> i think we shoudl all just try to get them running, to iron out any issues
22:33:26 <SlickNik> good for now.
22:33:32 <vipul> so we can't say 'works on mine' :)
22:33:41 <SlickNik> #agreed
22:33:52 <hub_cap> uh oh we found #agreed :D
22:34:02 <hub_cap> i wonder what that does on the notes cp16net SlickNik
22:34:03 <hub_cap> :P
22:34:06 <hub_cap> we will find out
22:34:14 <hub_cap> #topic Image builder
22:34:26 <hub_cap> we are officially merged w/ the image builder
22:34:33 <hub_cap> juice: can u action item what u want to update
22:34:34 <SlickNik> w00t!
22:34:37 * cp16net shrugs
22:34:41 <cp16net> #agreed
22:34:41 <vipul> lol
22:34:45 <hub_cap> lol
22:34:46 <juice> thanks hub_cap for your effort in reviewing this
22:34:47 <SlickNik> heh
22:34:51 <juice> not much to update
22:34:54 <hub_cap> juice: np!
22:34:55 <juice> just the one tweak
22:35:03 <juice> for linking rather than copying files
22:35:03 <hub_cap> cool. so maybe we just skip on then
22:35:10 <hub_cap> #info its merged - booya
22:35:12 <juice> other than that ready to move on to something else
22:35:16 <vipul> percona!
22:35:17 <vipul> :)
22:35:24 <hub_cap> LOL
22:35:25 <vipul> next week that is
22:35:37 <vipul> once we know where to put it
22:35:39 <SlickNik> Once we figure out where it can live. :P
22:35:41 <SlickNik> yead
22:35:41 <dkehn> will try to get Monty for that one
22:35:42 <SlickNik> yeah*
22:36:03 <hub_cap> imma change things up a bit
22:36:09 <hub_cap> #topic database-api
22:36:22 <hub_cap> so, we had a meeting wrt doc'ing and our api
22:36:38 <hub_cap> #link https://github.com/stackforge/database-api
22:36:55 <hub_cap> basically, the codebase as it stands will be called the V1 api. we have to make sure it stays solid
22:36:58 <vipul> hub_cap: any reason why it's called database-api instead of reddwarf-api?
22:37:09 <hub_cap> ya... anne and jeblair said no codenames
22:37:28 <hub_cap> cp16net: pointed out a mistype
22:37:35 <hub_cap> i meant the reddwarf api/codebase as it stands
22:37:41 <hub_cap> _not_ the database-api project
22:37:46 <hub_cap> will be the V1 apo
22:37:47 <hub_cap> *api
22:38:07 <hub_cap> vipul: note compute-api, volume-api, no mention of cinder/nova/etc...
22:38:38 <hub_cap> our doc writer mike a (ill get him to get on irc.. hopefully ;) ) will be adding it to the repo
22:38:39 <vipul> k, hub_cap: what do you mean by reddwarf codebase = v1 api
22:38:46 <SlickNik> going with the flow.
22:38:50 <hub_cap> well whats in the code as it stands today vipul
22:39:01 <vipul> right ok... yes so all we have now is v1
22:39:03 <hub_cap> correct
22:39:13 <hub_cap> we need to sit down and start designing the v2 api
22:39:15 <SlickNik> So we will doc the reference implementation's API and christen it v1.
22:39:21 <hub_cap> correct SlickNik
22:39:35 <hub_cap> V2 will have all our features that we want to implement over the next 6mo or more
22:39:36 <cp16net> #agree hub_cap v1 of api is what is currently in stackforge/reddwarf
22:39:43 <hub_cap> u missed a d
22:39:48 <cp16net> #agreed hub_cap v1 of api is what is currently in stackforge/reddwarf
22:40:02 <hub_cap> snaps/replication/backup/restore/updates...etc etc will form the V2 api
22:40:16 <hub_cap> but instead of organically growing it we gotta waterfall it w/ a spec
22:40:32 <hub_cap> example: keystone is doing this now w/ their new spec
22:40:37 <mordred> what's up guys?
22:40:45 <hub_cap> mordred: welcome to the party
22:40:52 <SlickNik> w00t! welcome...
22:40:54 <hub_cap> were discussing your favorite thing, api specs
22:41:13 <vipul> only issue i'd have is ... do we have the capability to rev version now... or is that something we have to wait for
22:41:29 <hub_cap> u mean in the code?
22:41:31 <vipul> meaning, can we start working on those v2 features now
22:41:37 <vipul> right, assuming we have spec'd it out
22:41:43 <vipul> or is that a summit hting
22:41:46 <hub_cap> we can start working on them now sure
22:41:48 <hub_cap> no not summit
22:41:55 <hub_cap> itll be irc / wiki thing
22:42:08 <hub_cap> #action hub_cap to talk to heckj and bcwaldon about how they are doing their next gen specs
22:42:10 <vipul> and by spec, just need a wiki tech spec around it?
22:42:17 <hub_cap> vipul: thats how it starts
22:42:18 <vipul> linked to blueprint
22:42:20 <vipul> k
22:42:20 <hub_cap> it will eventually be a wadl
22:42:30 <SlickNik> wadl?
22:42:34 <hub_cap> and tech writers will make it sound nice and generate very basic docs around it
22:42:43 <hub_cap> SlickNik: http://en.wikipedia.org/wiki/Web_Application_Description_Language
22:42:47 <vipul> wadl = wsdl for rest
22:43:00 <SlickNik> ah, gotcha…thanks hub_cap, vipul.
22:43:24 <hub_cap> we will be doing things inline w/ how the other teams _in_ openstack are doing it, ill make sure of that
22:43:26 <hub_cap> ;)
22:43:48 <hub_cap> #action hub_cap to get the featureset for V2 started, BP & wiki
22:44:02 <vipul> ok, so besides the wadl, the boilerplate in database-api will just be copy/paste?
22:44:08 <hub_cap> so lets chat over the next few days w/ what u want for v2 and what we want for v2
22:44:20 <hub_cap> vipul: im not sure what u mean
22:44:33 <vipul> there is some javascript, doc generation code
22:44:37 <hub_cap> itll look like this https://github.com/openstack/compute-api
22:44:57 <vipul> k
22:45:12 <hub_cap> vipul: our doc team has some stuff, but its not really fleshed out yet. ive asked them to find an open way to say, hey, company X, use this to generate internal docs
22:45:13 <jcooley> hub_cap: right on!
22:45:28 <hub_cap> so HP and other companies can use it... they want to do something liek that and say they are working toward it already
22:45:46 <hub_cap> so itll be a bit of a bumpy road, but its not a road we have teo deal w/. mike A will be our POC for that stuff
22:46:07 <hub_cap> as far as we care, its a wadl
22:46:25 <hub_cap> #link https://github.com/openstack/compute-api/blob/master/openstack-compute-api-2/src/os-compute-2.wadl
22:46:27 <vipul> k, makes sense
22:46:28 <hub_cap> an example
22:46:53 <hub_cap> we should be adding to the wadl tho, we are the owners of it
22:47:13 <hub_cap> any other question on doc-fun
22:47:38 <vipul> you have an eta on when we'll have a baseline to start adding to?
22:47:53 <hub_cap> vipul:  thats a good question
22:48:07 <hub_cap> #action hub_cap to talk to mikeA about timelines for v1 wadl
22:48:14 <hub_cap> i hope soon vipul
22:48:18 <vipul> k cool
22:48:56 <hub_cap> ok so now the fun part
22:49:05 <hub_cap> #topic open discussion
22:49:30 <hub_cap> anyone else have any random reddwarf stuff to talk about? anyone annoyed to no end w/ hub_cap and just need to air it
22:49:34 <dkehn> can we talk about the where abouts of the image building
22:49:45 * cp16net raises hand
22:49:46 <vipul> is now a good time to bring mordred into the discussion?
22:49:48 <dkehn> for non-mysql images
22:49:52 <SlickNik> I guess mordred is hwere.
22:49:52 <dkehn> mordred, ?
22:49:54 <SlickNik> here*
22:49:57 <hub_cap> ive got a strict deadline of 10 min
22:50:00 <hub_cap> so if we can get there lets do it
22:50:16 <juice> I'll walk over to see if mordred is still around
22:50:18 <mordred> ok. so - what's the problem here?
22:50:24 <hub_cap> so heres my opinion
22:50:48 <hub_cap> id like reddwarf to be _an_ implementation of a database as a service, ie, a impl of postgres, of mysql, etc...
22:51:02 <juice> nada - afk
22:51:03 <hub_cap> i feel like if we focus on 4 different impls of mysql, we will end up in edge case hell for a long time
22:51:16 <hub_cap> vs focusing on a supported version of each of the major databases
22:51:45 <mordred> what's the actual split though - package name + apt repo?
22:51:55 <hub_cap> my.cnf differences?
22:52:06 <mordred> like, what if the apt package name was parameterizable with a default value?
22:52:11 <mordred> hub_cap: are there?
22:52:15 * mordred asking
22:52:21 * mordred not trolling
22:52:23 <hub_cap> well id assume there were tweaks that HP wanted to put in
22:52:27 <hub_cap> or i _did_
22:52:49 <hub_cap> and percona has removed some features of mysql (mind u its prolly a good idea...)
22:53:03 <hub_cap> but maybe we want to support X or Y that percona deems unsafe
22:53:03 <mordred> so - there's NO WAY that there is a one-size-fixts-all my.cnf across any of these installs, right?
22:53:18 <mordred> hub_cap: I hear that - I'm just wondering what the actual incomptability is
22:53:24 <mordred> because in _theory_ I hear you
22:53:27 <hub_cap> :P
22:53:31 <hub_cap> i get ya man no worries
22:53:34 <mordred> but in practice, if it's literally jus a package name change, you know?
22:53:44 <hub_cap> i dont have a problem if its a package change
22:53:54 <hub_cap> as long as there is an understanding that the environment is a development env
22:54:01 <dkehn> there are defaults and in the case of percona and a devops situation one does not have to take advantage of expert options
22:54:04 <vipul> i think also need to look at what happens when the next db is introduced.. like mariadb or something
22:54:06 <hub_cap> not _the_ env that each of us will be tweaking our production my.cnfs
22:54:14 <mordred> right.
22:54:27 <mordred> I think we _have_ to grok that whatever my.cnf is put in here is for dev purposes
22:54:38 <hub_cap> agreed
22:54:52 <mordred> great. so with that - let's see what the actual split is - and honestly, if there's an elements tree
22:55:21 <vipul> mordred: how does this work in CI, you only test one path?
22:55:24 <mordred> and  there either a parameter on the command line or 2 1-line files, one for percona and one for mysql, it shouldn't be too terrible
22:55:36 <mordred> but then that what we _don't_ want
22:55:50 <mordred> is 4 different super complex and divergent things, right?
22:55:59 <hub_cap> correct mordred
22:56:23 <mordred> vipul: does that seem workable to you?
22:56:28 <hub_cap> for instance we have a diff my.cnf that we use for prod, and test that in our own ci env
22:56:36 <mordred> vipul: also, I think we could potentially set up multiple jobs, one for each
22:56:41 <mordred> hub_cap: exactly.
22:56:43 <hub_cap> vs whats in the development env... whis is honestly throwaway
22:56:45 <hub_cap> *which
22:56:49 <mordred> openstack itself needs a config or two to test
22:56:53 <hub_cap> in terms of production readyness
22:56:58 <mordred> that's "sensible" for testing functionality
22:57:02 <vipul> yep, just need to make sure that we are testing both paths, so nothing gets stale
22:57:08 <mordred> yah. I think that's doable
22:57:44 <hub_cap> ps, v
22:57:45 <hub_cap> http://www.percona.com/software/percona-server/feature-comparison
22:57:46 <hub_cap> lol
22:57:54 <hub_cap> percona has a bunch of extra features
22:58:06 <mordred> but if we get to the point where the percona path starts to get complex, then we should discuss whether or not that should be in an HP CI
22:58:08 <hub_cap> that we cant rely on...
22:58:16 <mordred> hub_cap: you guys should start using percona :)
22:58:25 <dkehn> agreed
22:58:28 <mordred> hub_cap: stewart, their head of engineering, is pretty fun to drink with :)
22:58:40 <hub_cap> im not opposed to it
22:58:48 <hub_cap> but i dont make those decisions ;) my engineers do!
22:58:58 <hub_cap> and ill have a drink w/ him/u next time i see and corner u both
22:58:58 <cp16net> heh
22:59:08 <hub_cap> the only tink i dont want to see is this
22:59:11 <hub_cap> Durability and Reliability Enhancements
22:59:21 <hub_cap> there is a ton of nicities that percona has that stock mysql does not
22:59:27 <hub_cap> so when we go down the path of durability
22:59:33 <hub_cap> we will have a nice code split between them
23:00:04 <mordred> yeah. I would say that reddwarf itself should not depend on percona
23:00:17 <mordred> until such a point as ubuntu replaces its stock mysql with percona
23:00:28 <hub_cap> correct
23:00:43 <dkehn> or mariaDB for that matter, which note has all the percona patches
23:00:44 <vipul> mordred: regarding HP CI, i think that's something we really should be doing, since our production images are different than what will be in reddwarf anyway
23:00:45 <hub_cap> so as long as the team is a-ok w/ not using percona features (winkie) we are good to go
23:01:02 <vipul> hub_cap, i'm cool with not using them in CI
23:01:12 <hub_cap> :D exactly vipul
23:01:15 <mordred> vipul: yes. remind me to show you some slides that aleksander from the hp nova team put together about following upstream closely but doing internal testing too
23:01:21 <SlickNik> Well, we can make a conscious decision not to make any percona specific changes in the reddwarf codebase.
23:01:32 <hub_cap> if u want to use wizbang feature X in your impl i say DO IT!!
23:01:59 <hub_cap> so here is my beef really...
23:02:02 <hub_cap> do we _need_ to test percona
23:02:13 <cp16net> wouldnt that be nice to be able to easily add it via extensions in reddwarf
23:02:22 <hub_cap> if we are relying on ubuntu, stock mysql, and cant use the features in ci
23:02:28 <hub_cap> i get that we cna make it configurable, sure
23:02:39 <hub_cap> i want wizbang mysql product X w/ a nosql backend
23:02:52 <hub_cap> but thats _my_ internal responsibiilty, not something that we need a codepath for in the public
23:03:05 <hub_cap> and then why do we need to run it in ci
23:03:20 <hub_cap> i _do_ want it to be configurable tho
23:03:26 <hub_cap> we agree there
23:03:50 <vipul> i just want to be able to test any 'elements' that support percona in CI
23:04:04 <hub_cap> what cha mean by elements?
23:04:20 <vipul> diskimage-builder elements (flavor)
23:04:20 <kaganos> elements == flavors of an image
23:04:29 <vipul> if we're not introducing any for percona, then fine
23:04:36 <hub_cap> so that goes back to my initial question
23:04:48 <hub_cap> why, if we only support ubuntu and stock mysql, w/ no support for percona
23:04:50 <vipul> (if we decide to drive those values by variables or whatrever)
23:04:58 <hub_cap> do we add the percona elements to the script and put it in ci in the puiblic
23:05:24 <hub_cap> seems like a lot of overhead for something that we arent going to say we officially support
23:05:26 <juice> right now its one folder with one file
23:05:35 <hub_cap> i get that juice its not the size of the files
23:05:40 <hub_cap> its the notion of supporting it
23:06:04 <hub_cap> if someone wants ot use maria, do we really need ot add it to our imagebuilder, and start running CI tests against it?
23:06:21 <juice> if either company is going to support it, I suppose so
23:06:24 <hub_cap> or do we say, hey, we support mysql, and as long as maria does, feel free to run it in your infra and tst against it... and then use maria in production
23:06:25 <kaganos> someone just went through a stop sign about 5 minutes ago … anyone caught his plate number?  ;)
23:06:32 <hub_cap> LOL
23:06:44 <juice> either being rackspace or hp
23:07:12 <vipul> i think we've come full circle
23:07:15 <hub_cap> but then we are making a company responsible for supporting a specific implementation of mysql in a public product
23:07:23 <hub_cap> we have vipul and mordred hath dissapeared
23:07:25 <lifeless> hub_cap: how much confidence do you have that oracle mysql will be representative of performance etc vs maria ;)
23:07:36 <hub_cap> lifeless: it has nothing to do w/ that :)
23:07:40 <mordred> I'm right here
23:07:49 <hub_cap> percona is likely a nice solution
23:07:57 <hub_cap> but since its a drop in replacement for any other mysql
23:08:03 <hub_cap> why do we have to put it in the public codebase
23:08:13 <hub_cap> if u want to support it, i say go for it
23:08:25 <mordred> lifeless: do we have parameterizable elements?
23:08:27 <hub_cap> but does it belong in our CI infra in our public
23:08:36 <hub_cap> when we get maria, or drizzle
23:08:37 <vipul> if we put it in reddwarf... and it's a new element... we have to test it
23:08:45 <lifeless> mordred: in the sense that they are arbitrary shell code, yes.
23:08:50 <hub_cap> then we have 4 complete CI runs w/ different mysql bakcends
23:09:01 <mordred> lifeless: I mean, if there was a "mysql-server" element - and only one
23:09:21 <mordred> lifeless: would there be a way to write it such that at disk-image-build time one could choose to install percona-server instead?
23:09:29 <mordred> lifeless: or would you need a percona-server element?
23:09:42 <hub_cap> im all for making it configurable, dont get me wrong
23:09:44 <lifeless> mordred: thinking out loud
23:09:49 <lifeless> there are two angles they may differ
23:09:53 <lifeless> one is the software source
23:09:55 <hub_cap> but why do we, the community, need to test it in the puiblic CI
23:10:05 <juice> whether it's configurable or not, don't we want all the same ci support for it?
23:10:05 <hub_cap> if HP uses percona, then HP can use HP resources to test it
23:10:08 <lifeless> different repo, and or different package name, or git tree, or whatever.
23:10:08 <mordred> hub_cap: just looking at what's _possible_
23:10:10 <vipul> if it's parameterized, probably not...
23:10:14 <hub_cap> sure mordred i get ya
23:10:25 <lifeless> You can imagine that minor differences are straight forward, but big ones less so.
23:10:46 <juice> hub_cap: is your concern that you don't want to burn any extra cycles on the percona ci process in the public space?
23:10:48 <hub_cap> if we want to run drizzle internally, i dont see that we shoudl be adding it to the public codebase... we support mysql, if drizzle supports mysql, run it in production
23:10:48 <hub_cap> :)
23:10:53 <hub_cap> juice: mainly yes
23:10:56 <vipul> juice: how much different are the two elements?
23:11:01 <lifeless> The second way that they may differ, is that the needed boot-time orchestration to properly run could be very different - nbd or not, is there a ring /quorum system etc.
23:11:08 <juice> its just the name of the packages
23:11:19 <vipul> and additional of another apt-repo?
23:11:29 <lifeless> if its /only/ the package name, and you guys are confident that it will stay that narrow a difference
23:11:36 <juice> also there could be different cnf files needed if we want to take advantage of any percona features
23:11:36 <lifeless> I don't see any benefit in a new element
23:11:55 <lifeless> configuration is outside the diskimage-builder domain, so orthogonal.
23:11:56 <hub_cap> but we def wont have those in the default impl juice (as discussed above)
23:12:07 <mordred> well, the last thing juice said I think is inappropriate for reddwarf upstream
23:12:09 <juice> oh yes the apt-repo. good call vipul
23:12:37 <lifeless> hub_cap: FWIW, if rackspace or whoever runs drizzle for reddwarf, I'd really like to see the support for that - including the status of how well it works etc - in public. Why would it be private?
23:12:45 <juice> does HP have it's own ci processes for reddwarf?
23:12:58 <vipul> so if we make it parameter-driven, but only ever set the community mysql flags, then what does that buy us
23:13:12 <mordred> vipul: who said we'd ever set the community?
23:13:13 <juice> if so, then we should just have some small tweak (tbd) that builds and tests reddwarf with the small changes for percona
23:13:16 <mordred> we'd only ever
23:13:25 <hub_cap> lifeless: i guess i dont see the need for it to be all public... it seems like a lot of work to keep every different impl of mysql running in reddwarf
23:13:34 <lifeless> I mean, surely the same question applies to nova with postgresql. Someone runs postgresql, why does the community CI need to test it ?
23:13:55 <mordred> hub_cap: I think in general though, the hp team is willing to do that work ... which is kinda how it happens with postgres in nova
23:13:58 <mordred> right?
23:14:07 <hub_cap> lifeless: thats a different db entirely tho. i see reddwarf as a reference impl to multiple types of dbs
23:14:08 <mordred> I mean, we made the CI job config self-service for a reason
23:14:10 <lifeless> AFAICT the answer so far has been 'if someone steps up to maintain it, and the code is clean, it goes in'.
23:14:56 <lifeless> hub_cap: it is a different DB, but I was using nova by analogy, which doesn't expose the backend DB to anyone.
23:15:31 <hub_cap> well i think it more close to xen community vs xenserver
23:15:49 <juice> sounds like we need a way to make the image building extensible in reddwarf (without making it complicated)
23:16:06 <hub_cap> does nova support both impls? and devstack too?
23:16:23 <lifeless> hub_cap: nova does, and devstack, and nova gates on it.
23:16:37 <lifeless> hub_cap: sorry, misread - thats the pg answer.
23:16:37 <hub_cap> lifeless: if thats the case my arguement is moot
23:16:47 <lifeless> hub_cap: I don't know about the xen stack at all
23:17:02 <hub_cap> :P
23:17:18 <hub_cap> when cloud servers moved xen impls, they didnt keep support for the previous one did they?
23:17:33 <hub_cap> i see xen vs vmware like i see mysql vs postgres
23:17:50 <hub_cap> 1 api, 2 different products on the backend w/ the ability for choice
23:17:51 <lifeless> hub_cap: I guess I'm confused here. Is it a resource concern, or a philosophical concern you have. If its resourcing, lets state up front what resources are needed for a variant and seek commitment before patches.
23:18:05 <lifeless> If its a philospohical concern, lets focus on that.
23:18:05 <hub_cap> at this point philosophical
23:18:15 <hub_cap> i understand hp has the resources for htis :)
23:18:18 <hub_cap> *this
23:18:25 <hub_cap> likely more than us hehe
23:18:53 <lifeless> heh, no comment :)
23:19:10 <hub_cap> the vision for me is that reddwarf has a reference implementation of each major database player
23:19:34 <lifeless> that makes a huge amount of sense :)
23:19:36 <hub_cap> pg, mysql, (hopefully) sqlserver, etc...
23:19:49 <lifeless> cassandra and riak :)
23:19:50 <hub_cap> and if a company wants to support a different impl of the reference, they can
23:19:58 <hub_cap> eventually lifeless ;)
23:20:03 <lifeless> cool
23:20:16 <hub_cap> but i dont necessarily see it as being part of the public reddwarf code
23:20:20 <lifeless> so, why /wouldn't/ the upstream code base know how to deploy etc a different implementaiton
23:20:30 <lifeless> how does that compromise the vision ?
23:20:45 <hub_cap> well its not a compromise of the vision
23:20:51 <hub_cap> its a question of does it need to
23:21:05 <lifeless> Hmm, I'm confused.
23:21:16 <hub_cap> i guess i dont see it as being necessary
23:21:30 <lifeless> I got the impression you were pushing back *against* it, which speaks - to me- of a concern or problem vs being additional stuff that doesn't have to happen but could
23:21:54 <hub_cap> correct. i really dont know what it could bring in terms of differences
23:21:57 <hub_cap> or if it evn will be different
23:22:01 <vipul> from a end user perspective, i think there is value in providing the tools/support to others interested in reddwarf, to be able to pull it down, set some flags, and they can run what they want under the covers
23:22:16 <vipul> instead of having to learn the intricacies of image building
23:22:16 <hub_cap> vipul: correct im not aruging against that
23:22:24 <lifeless> vipul: right!
23:22:26 <hub_cap> i want the system to be configurable
23:22:36 <hub_cap> i want it to be able to run maria or percona
23:22:53 <hub_cap> but do we need ot devote resources in the public to that, is my problem
23:23:00 <hub_cap> short term it does not seem like a lot of work
23:23:02 <lifeless> hub_cap: so do we have to decide this now? Could we perhaps run an experiment where we put (say) percona in, conditional on no-bad-things-happening, and if bad things happen, we make a commitment to pull it out later.
23:23:20 <hub_cap> lifeless: i guess i dont have an arguement against that :D
23:23:34 <hub_cap> but do we need to run it in the CI stuff?
23:23:44 <hub_cap> as is the CI tests take upwards of 30+min
23:23:57 <hub_cap> so duping it to run them both is gonna be a time sink
23:23:59 <lifeless> ok, so *that* is a good question.
23:24:00 <vipul> hub_cap: that's a concern..
23:24:04 <lifeless> What if we did it in parallel ?
23:24:12 <hub_cap> lifeless: i could live w/ that
23:24:18 <lifeless> so it was still 30+m, but maybe only a little longer for percona?
23:24:30 <hub_cap> so _if_ there is a error w/ percona
23:24:32 <mordred> we have good parallel testing capabilities
23:24:38 <hub_cap> do we throw our hands up and say HP fix this
23:24:45 <hub_cap> from rax perspective
23:24:57 <hub_cap> and then the gating waits?
23:25:11 <lifeless> hub_cap: I think if there is an error isolated to percona, it would demonstrate the shared benefits that accrue to every org choosing to deploy w/percona of having it in the public space.
23:25:51 <hub_cap> my guess is that there wont be honestly
23:26:06 <lifeless> hub_cap: in that it would prove that percona is different enough that saying 'reddwarf works with mysql' is not enough to say 'random percona customer can run reddward+percona safely'
23:26:39 <hub_cap> sure... and again thats what i see as scary, but i know the liklihood is soooo small
23:27:03 <hub_cap> and of course when we decide to use whiz bang trickery to do things that stock mysql doesnt support
23:28:02 <vipul> hub_cap: i think that's where we have to reject such patches.. possibly even have a single my.cnf for the entire project
23:28:13 <hub_cap> vipul: single my.cnf only makes sense
23:28:17 <hub_cap> a developer my.cnf
23:28:18 <hub_cap> :)
23:28:47 <vipul> hub_cap: let's give this a shot.. run both versions in parallel.. see where it gets us
23:28:52 <lifeless> sorry, wife calling, I need to disappear for a few
23:29:12 <vipul> we may come back and say it was a bad idea.. and pull it out.. or it may not be that much of a headache
23:29:13 <hub_cap> ok so 1) we wont support anything that is _not_ stock mysql, 2) we wont allow patches to percona specific stuff, 3) we have a single my.cnf for development
23:29:14 <hub_cap> ya?
23:29:23 <vipul> #agreed
23:29:38 <SlickNik> #agreed
23:30:08 <hub_cap> given those requirements, i guess i dont see the use of adding it, but i will concede :)
23:30:15 <vipul> lol ;)
23:30:46 <juice> we can always remove it
23:30:52 <juice> if things don't work out
23:31:06 <hub_cap> of course :D we will see how it goes
23:31:08 <vipul> yep, this will at least give it a home, even if it's temporary
23:31:11 <hub_cap> someone action item it
23:31:15 <SlickNik> I actually like the idea since there are also benefits of being able to test the reddwarf eco system being able to pick and use a different db implementation under the covers.
23:31:24 <hub_cap> whoever on yalls end for whos gonna test it
23:31:25 <vipul> #action juice to publish percona elements to Reddwarf-int
23:31:47 <hub_cap> coo
23:31:50 <hub_cap> fin?
23:31:55 <vipul> yes sir!
23:31:58 <hub_cap> #endmeeting