17:58:09 <SlickNik> #startmeeting trove
17:58:09 <openstack> Meeting started Wed Jul  8 17:58:09 2015 UTC and is due to finish in 60 minutes.  The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:58:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:58:12 <openstack> The meeting name has been set to 'trove'
17:58:43 <SlickNik> It's still a couple of minutes to the hour, so giving folks a few more minutes to trickle in
17:59:15 <SlickNik> The agenda for the meeting is at:
17:59:23 <edmondk> o/
17:59:24 <SlickNik> #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting
17:59:29 <cp16net> heyo
17:59:38 <dloi_> o/
18:00:09 <pmalik> o?
18:00:29 <cp16net> lolz thats a head scratcher
18:01:11 <SlickNik> hah, okay — let's get started.
18:01:25 <SlickNik> #topic Trove pulse update
18:01:31 <SlickNik> #link https://etherpad.openstack.org/p/trove-pulse-update
18:01:32 <vgnbkr> ol
18:01:51 <vkmc> o/
18:02:05 <sushilkm> hello everyone
18:02:46 <SlickNik> Fewer reviews this week (sort of expected given the long weekend, and holiday in Canada).
18:03:19 <SlickNik> Thanks for reviews for those who were able to take the time to do that.
18:03:36 <edmondk> Is the trove pulse update not loading for anyone else or is it just me?
18:03:47 <cp16net> worked for me
18:03:52 <sushilkm> i am able to see that
18:03:53 <SlickNik> But I think we need to step them up to keep the momentum going.
18:03:54 <edmondk> just stuck on loading screen for me
18:03:58 <SlickNik> working for me.
18:04:01 <cp16net> edmondk: try in a different browser
18:04:16 <SlickNik> edmondk: Ctrl + Shift + R
18:04:21 <edmondk> safari worked
18:04:26 <cp16net> might need to clear the caches for chrome
18:04:48 <SlickNik> (or command + shift + R on a mac)
18:05:04 <cp16net> i had a similar problem and deleted the user/chrome/cache dir and it started worked
18:05:08 <cp16net> working*
18:05:18 <edmondk> all good we can continue :)
18:05:22 <SlickNik> Okay, getting back on topic :P
18:05:24 <peterstac> works for me
18:05:59 <SlickNik> Let's do more reviews this week to ensure that we're not falling behind with our momentum.
18:06:35 <cp16net> holiday is over
18:06:41 <amrith> ./
18:06:44 <SlickNik> And keep an eye out for that review count to ensure it's closer to the average next week. :)
18:07:38 <SlickNik> Okay, that's all I had on this — any other comments / concerns regarding this?
18:07:43 <sushilkm> vacations might have taken its toll on reviews
18:08:00 <sushilkm> it shud be better next week, i guess
18:08:05 <SlickNik> sushilkm: ++
18:08:45 <SlickNik> okay, let's move on.
18:08:52 <SlickNik> #topic Add root-disable api
18:09:06 <amrith> may I paste-bomb?
18:09:12 <SlickNik> #link https://review.openstack.org/#/c/189837/
18:09:29 <SlickNik> amrith: sure
18:09:33 <amrith> The issue with 'root-enabled' (the flag) is that in reality it means, "a root user was enabled on this instance at some time, or on some instance that was used to generate a backup and then subsequently launch this instance".
18:09:33 <amrith> Today, without the root-disable feature, one can follow these steps.
18:09:33 <amrith> 1. create an instance
18:09:33 <amrith> 2. enable root
18:09:33 <amrith> 3. login as root
18:09:35 <amrith> 4. create a new user and grant that user all privileges
18:09:37 <amrith> 5. delete the root user
18:09:39 <amrith> 6. backup the instance
18:09:41 <amrith> 7. launch a new instance
18:09:43 <amrith> The new instance would indicate root-enabled is false.
18:09:45 <amrith> A peculiarity in how ACL's are controlled on MySQL further means that I can ...
18:09:47 <amrith> 1. create an instance
18:09:49 <amrith> 2. enable root
18:09:51 <amrith> 3. login as root
18:09:53 <amrith> 4. create a new user and grant that user all privileges
18:09:57 <amrith> 5. do some nefarious thing as either new user or root user
18:10:00 <amrith> 6. login as the new user (session A)
18:10:01 <amrith> 7. login as the root user (session B)
18:10:03 <amrith> 8. on session A, drop root user
18:10:05 <amrith> 9. backup the database
18:10:07 <amrith> What we need (I think) is a mechanism that will attach the root-enabled status on an instance to the backup and restore that state when a backup is restored. After posting comments in the review, I was writing a BP to propose this and was going to submit code once the BP (approach) was agreed to. But I got scooped ;(
18:11:04 <amrith> FWIW, I believe that the fix in https://review.openstack.org/#/c/199635/ doesn't solve the issue.
18:11:59 * SlickNik reads paste bomb
18:12:04 <edmondk> lol
18:12:14 <cp16net> so my first thought is that we know what instance the backup came from and should be able to tell if that instance had root enabled in the past and mark the new instance with root enabled.
18:12:37 <amrith> cp16net, can't be sure that the instance still exists
18:12:38 <cp16net> that sounds like it would resolve the issue right?
18:12:54 <cp16net> but we keep track of that in the database right?
18:12:56 <amrith> I could extend this to say take a backup, delete the old instance, ...
18:13:17 <amrith> no guarantee that we will keep track of deleted instances forever.
18:13:24 <amrith> that's an admin cleanup thing
18:13:39 <amrith> we should (I believe) persist the root enabled state into the backup
18:13:46 <amrith> or as you say store it in the database
18:13:48 <SlickNik> amrith / craig: that's a possibility. Another option would be to mark this in the metadata, so that we can set the root-enabled flag when we do the restore based on the backup metadata.
18:13:48 <amrith> with the backup
18:13:48 <peterstac> we'd need to store the root-enabled in the backup table as well
18:13:55 <amrith> so extend the schema for the backup list
18:13:58 <amrith> and store root enabled there
18:14:05 <amrith> peterstac ++
18:14:32 <SlickNik> amrith: backups already have a metadata field in the table, if I'm recalling correctly.
18:14:38 <peterstac> SlickNik, yes, that would work too, as long as the user can't muck with the swift meta-data
18:14:58 <amrith> FWIW, the reason I suggest that https://review.openstack.org/#/c/199635/ does little is that we can extend my second repro to have session A delete root user and session B delete the new user.
18:15:16 <amrith> peterstac, HUH?
18:15:53 <amrith> you are suggesting that the one doing trove create is also going to mess with the metadata?
18:16:03 <amrith> well, we can't prevent against that.
18:16:25 <cp16net> we have a root_enabled_history table for every instance that root was enabled on
18:16:25 <amrith> no matter where you store the data (unless you run with the modified remote.py ;)), the person running trove create can muck with it.
18:16:43 <pmalik> cp16net ++
18:16:43 <cp16net> if we look at that table and find it (even if the instance was deleted) we know
18:17:22 <cp16net> i verified that enabling root and backing up an instance and creating a new one still shows the instance as root enabled
18:17:40 <peterstac> cp16net, yes
18:17:43 <cp16net> i deleted all the root users and ran a backup
18:17:53 <cp16net> and it failed because no root user was available
18:17:57 <pmalik> cp16net we could also keep track of when and who enabled/disabled the root access. It would also make it datastore agnostic.
18:18:09 <cp16net> umm its sorta like that
18:18:43 <cp16net> root_enabled_history has id(instance) user(user id) created(time it was enabled)
18:18:50 <peterstac> I verified that if you create a new user (with super-user privileges) and then delete the root user, the new instance created from the backup will show as root-enabled=false, even though you can still log in as the super-user
18:18:52 <amrith> so SlickNik ... procedurally, do we want to spend the meeting finding the solution, or do we want to propose that someone (I volunteer) write up a BP for this and then we review it that way
18:18:56 <cp16net> it doesnt care about datastore
18:18:58 <sushilkm> https://review.openstack.org/#/c/199635/  helps eradicating the problem as it finds any user with access more than USAGE created for anyone except localhost
18:19:04 <peterstac> (^^^ with Duk)
18:19:09 <sushilkm> which could only be created using a root-access
18:19:16 <SlickNik> So — here's where we are at.
18:19:27 * amrith waits for SlickNik to speak
18:19:29 <peterstac> sushilkm, that's very mysql-specific though
18:19:40 <SlickNik> We've identified a hole that exists with this, and want to plug it.
18:19:59 <SlickNik> And we've also identified customers and users who want a root-disabled API.
18:20:08 <pmalik> 22cp16net: Well, but it should help us to avoid having to mess with datastore (mysql-specific) stuff on the GA as we have to do now...
18:20:54 <SlickNik> We're acknowledging that we have a security issue here (existing, and possibly easier to find with the root-disable API).
18:21:39 <SlickNik> Let's open up a bug to fix the issue and discuss the design of how we're going to fix it as part of that bugfix.
18:22:26 <SlickNik> And we can unblock work on blueprint of adding the "root-disable" API.
18:22:49 <peterstac> SlickNik, sounds good
18:22:51 * amrith waits for EOF from SlickNik
18:22:52 <cp16net> yeah i think there was some great convos that came out of the root-disable API
18:23:11 <SlickNik> It would be good to get both of these to land in Liberty, but I don't see why both need to land as part of the same blueprint.
18:23:32 <SlickNik> <end of transmission>
18:23:37 <amrith> SlickNik, ++
18:23:52 <cp16net> #agreed
18:23:59 <amrith> Happy to volunteer to land the code for this proposal (have some of it going, duk and co have framework to test it in)
18:24:24 <amrith> I appreciate the unblocking of the disable root BP. and agree that it would be advisable for both to land in liberty
18:24:27 <SlickNik> Okay, sounds good to me.
18:24:35 <SlickNik> I think we have a good plan going forward.
18:24:35 <amrith> in a way that the latter can be back ported if required. <EOF>
18:24:55 <SlickNik> I'll update the spec with this info immediately after this meeting.
18:25:04 <amrith> thx
18:25:28 <SlickNik> And open a bug for the issue we've identified so that we can continue discussion on that front..
18:25:59 <SlickNik> #action SlickNik to open a bug for the root-enable security hole on restore.
18:26:33 <SlickNik> #action SlickNik update root-disable BP with plan of action from meeting.
18:27:04 <SlickNik> Moving on.
18:27:17 <SlickNik> #topic Trove-compatible images creation automation
18:28:38 <SlickNik> recalling the conversation from last week.
18:29:37 <SlickNik> vkmc was going to split out the image-elements from trove-integration to a new repo.
18:29:47 <amrith> ./
18:30:09 <SlickNik> And come up with a script to build guest images independently of running redstack.
18:30:43 <vkmc> indeed, something like what Sahara is doing with their images
18:31:10 <vkmc> https://github.com/openstack/sahara-image-elements
18:31:14 <amrith> I don't understand the solution/problem
18:31:17 <amrith> we have elements
18:31:24 <SlickNik> Once this was done,  the next step would be to refactor redstack to simply pull in and use this repo and use this script so that the image building wouldn't have live solely in redstack.
18:31:27 <amrith> we can put (in trove-integration) a script to build images
18:31:34 <amrith> heck, you can run redstack build-image
18:31:50 <amrith> or you can use the command in the dev-doc that we have.
18:32:01 <amrith> so I'm not sure how a new repo improves the situation.
18:32:24 <vkmc> so, IMO the problem with trove-integration right now is that in order to create images for trove, you need to get the whole script for trove-integration
18:32:57 <amrith> sorry, what does "whole script for trove-integration" mean?
18:33:08 <vkmc> plus, trove-integration is usually considered to be useful to deploy development environments, not real deployments
18:33:48 <vkmc> with that I mean, development environment setup script, image creation script and integration tests
18:33:52 <vkmc> all in one
18:34:05 * amrith waits for vkmc to finish
18:34:21 <vkmc> when maybe you only want a way to create images, to use in your production environment
18:35:09 <vkmc> we already have the docs amrith write for that, but certainly is not enough
18:35:42 <vkmc> we always get people in -trove asking things related to the image building
18:36:01 <vkmc> and that's pretty much it
18:36:10 <amrith> ./
18:36:19 <amrith> so, a couple of things
18:36:30 <amrith> do you think the issue is that trove-integration is hard for people to use
18:36:37 <amrith> is it that http://docs.openstack.org/developer/trove/dev/building_guest_images.html is hard to read?
18:36:45 <amrith> is it that trove should offer production scripts?
18:36:52 <amrith> is it that trove should offer production elements?
18:36:58 <amrith> is it that we need better documentation?
18:37:12 <amrith> unfortunately, to none of those do I think the answer is "new project/repository"
18:37:20 <tosky> amrith: why do you need a lot of variables related to redstack but not needed for image creation?
18:37:21 <amrith> so I must be missing the question (I think)
18:37:22 <vkmc> is it that when users ask how to create their images, a vital part on trove, they don't have a place to go
18:37:44 <vkmc> they have the docs, yes... but still they find issues
18:37:54 <tosky> amrith: please note that a separate repository dedicated to image creation is something already successfully used elsewhere (Sahara)
18:38:21 <amrith> ok vkmc, I agree that there's an opportunity to improve the docs; we spoke about doing a presentation about that at summit
18:38:26 <amrith> and should try again for Tokyo.
18:38:34 <vkmc> whereas the docs cover the major part, they don't cover things specific to the different distributions
18:38:38 <amrith> and write better docs, and help people learn how to write elements.
18:38:39 <SlickNik> amrith / vkmc: I can also understand an argument made for separation of concern. Right now our image-creation is tightly coupled with our development environments (e.g. passing of env variables regarding where the source code lives on the devstack box, etc.). If we want the elements to be reusable in a manner that lets users create images that they can use, it makes sense to decouple this further.
18:39:11 <amrith> tosky, what variables? I don't get what you are saying.
18:39:13 <SlickNik> It would be nice to refactor this so that the dev-stuff is done in a separate element that gets mixed in only for redstack.
18:39:28 <vkmc> so... if for a customer to use trove requires this customer to write their own elements
18:39:37 <vkmc> then I think we might face an adoption issue
18:39:43 <amrith> SlickNik, you raise a great question, which is probably masked by this solution; should Trove be in the business of offering production elements.
18:39:53 <amrith> I don't profess to have an answer; the quesiton however is the one at hand
18:40:05 <tosky> amrith:
18:40:07 <tosky> export HOST_USERNAME
18:40:08 <tosky> export HOST_SCP_USERNAME
18:40:10 <tosky> amrith: etc etc etc
18:40:30 <amrith> you don't have to export all of those things if you don't want to. redstack build-image will do it for you.
18:40:45 <amrith> the reason you have to set them is because the elements need them.
18:41:03 <amrith> if you don't want to, you can write a wrapper around dib.
18:41:16 <saurabhs> I think we can seperate it by saying trove-image-elements hosts standard elements for installing datastore specific stuff and trove integration provides elements specific for dev environment
18:41:22 <saurabhs> that way people will be less confused
18:41:25 <tosky> amrith: the element that vkmc prepared don't need them, and I'm sure REDSTACK_SCRIPTS is not needed by dib
18:41:55 <amrith> so great ... are we suggesting that trove should 'support' production ready elements?
18:42:07 <vgnbkr> Ultimately, don't the user's operators need access to the nova instances?  How would trove supplied "production elements" support that?
18:42:17 <saurabhs> I don't think trove should support production ready elements
18:42:19 <amrith> vgnbkr ++
18:42:28 <tosky> we are suggesting to have a way to easily build production ready images, with clean elements
18:42:39 <tosky> again, this is something which is working fine in Sahara
18:42:45 <vkmc> vgnbkr, not sure what you meant
18:42:53 <amrith> how many different 'databases' does Sahara support?
18:43:05 <saurabhs> trove should provide elements that will allow somebody to install datastore sepcific packages but not put dev burden on them
18:43:23 <vgnbkr> vkmc, if something breaks in the nova instance underneath the trove instance, how would the operator fix it?
18:43:25 <vkmc> saurabhs, that's exactly my point
18:43:47 <SlickNik> vgnbkr: I don't think the suggestion is that you supply _all_ the elements for a production image. If I understand the suggestion correctly, you'd be supplying most of them which an operator can extend on top of.
18:43:53 <vkmc> vgnbkr, we are not aiming to solve that
18:43:56 <amrith> to all ... I would like to just point out that a solution was proposed, and today we're still having a somewhat challenging time stating what the problem is.
18:44:07 <saurabhs> by moving datastore specific elements to a different repo we will decouple elements for dev and real use
18:44:24 <saurabhs> for building production grade images people will still need other stuff on their images
18:44:36 <peterstac> saurabhs, by real use, you mean 'production-ready' ?
18:44:39 <SlickNik> vgnbkr: Today, the elements are coupled with the dev-env, so you'd have to rewrite them if you wanted to build on top of some of those pieces for a non-development image.
18:44:44 <vkmc> amrith, not really, I think the problem is clear: we don't have a clear, automatic way to create images
18:44:44 <amrith> saurabhs, suppose I propose a change to trove and it breaks one of those "production elements" then who gets to fix it?
18:45:14 <amrith> vkmc, we have a clear way to create redstack images. we have NO way to generate production images.
18:45:16 <saurabhs> amrith: who ever broke it :)
18:45:24 <tosky> amrith: Sahara regularly tests the elements producing images; the solution is... testing, of course
18:45:24 <amrith> so saurabhs that means Trove just expanded.
18:45:27 <saurabhs> I am not suggesting that we make a copy of current image lements
18:45:49 <amrith> tosky, the answer to my question was "1". There is one database you support. Trove today supports close to a dozen.
18:45:52 <saurabhs> I am saying strategically move datastore specific stuff out to a seperate repo
18:46:04 <saurabhs> and keep dev specific elements in trove integration
18:46:06 <vgnbkr> Seems to me that if the user can figure out all the other things they will need to do on top of production versions of the few images we provide, making the production versions would be trivial.
18:46:21 <saurabhs> and when building elements for dev env we will combine elements from both
18:46:25 <amrith> oh, great, so saurabhs is proposing splitting the trove repository into datastore specific ones ;)
18:46:45 <saurabhs> amrith: not sure where I said that :)
18:46:56 <tosky> amrith: Sahara supports 4 plugins, two different version of Hadoop for each of them, on multiple distributions
18:47:06 <amrith> saurabhs, "strategically move datastore specific stuff out to a seperate repo" ;)
18:47:19 <saurabhs> yeh but not per datastore :)
18:47:22 <vgnbkr> Why doesn't someone just make a SourceForge project for whatever database they want to support.  Then there could be different types of "production images" for different types of deployments.
18:47:28 <SergeyLukjanov> in total we're publishing more than dozen of images generated by sahara-image-elements script
18:47:31 <amrith> saurabhs, you can buy me a beer when we meet ...
18:48:07 <saurabhs> I am saying lets move elements which install/configure datastore (all the datastore) into a seperate repo that way its useful for building images for more than dev envs
18:48:37 <vkmc> saurabhs, and once again, that is what I wanted to mean
18:48:39 <saurabhs> let trove integration have elements that allow things like rsync
18:48:54 <saurabhs> vkmc: then we both agree
18:49:00 <vkmc> saurabhs++
18:49:01 <amrith> I'm fine with this as long as a trove developer isn't on the hook to make sure that changes are made and tested in this 'production ready trove CI'. that will do doubt follow.
18:49:03 <vgnbkr> We already decided not to split the guest agent into a separate repo because of gerritt issues - why would we do so for datastore images?
18:49:55 <saurabhs> amrtith: today if a dev breaks mongodb element who is suppose to fix it? I think responsibility will be same when we move to a different repo
18:50:12 <amrith> no, but the current elements are development only.
18:50:25 <amrith> and therefore the bar is lower.
18:50:25 <SlickNik> vgnbkr: The issue there was python packaging and circular dependencies. In this case dib-elements don't pose an issue (the way sahara does image elements today shows that this is a viable approach).
18:50:33 <dougshelley66> saurabhs, today the only elements that gate are mysql 5.6 on ubuntu
18:50:50 <saurabhs> :) we want people to use trove or not or use it only for development purposes.
18:51:01 <amrith> saurabhs, people do use trove
18:51:07 <amrith> and people do build elements and images
18:51:21 <amrith> I just want to know where we draw the line on the responsibility of the trove developer
18:51:37 <amrith> if you stand up a new repo and you have a bunch of differen tproduction ready elements there
18:51:49 <tosky> amrith: so how the real world testing is done, if only dev images are tested?
18:51:50 <amrith> is it the trove developers responsibility to keep them updated
18:52:07 <SlickNik> amrith / saurabhs: FWIW (anecdotal evidence alert) I've had multiple conversations with folks whose top problem is that building trove images is too hard given the current process.
18:52:19 <amrith> SlickNik, I agree
18:52:40 <amrith> but I don't know how this proposed solution does anything to address it!
18:52:45 <tosky> amrith: and moreover, how different are the dev images from at least a simple model of production images?
18:52:58 <saurabhs> amrtith: how about we take it from a different angle lets not call it prod ready elements. lets call them independant elements which don't require trove integration elements but will produce the image in a such a way that datastore specific package and configuration is there
18:53:02 <SlickNik> and most of them are of the opinion that moving to a sahara like model (where the image-elements are separated out) so that they don't have to understand the intricacies of redstack would improve this quite a bit.
18:53:10 <amrith> anyway, maybe this needs to be an in-person conversation. at mid-cycle?
18:53:26 <cp16net> so in this new repo will there be an "easier" script or will it basically be all the same it is today in redstack?
18:53:29 <tosky> amrith: this proposed solution gives you a simple set of files which you don't need to dig from a bigger set, like (sorry for repeating it) sahara-image-elements
18:53:34 <abramley> the major difference will be in where you get the trove code - is it from apt-get or git - is it a release version or HEAD etc
18:53:40 <saurabhs> somebody can go on the top of that and install stuff specific for their production or even non production use
18:53:54 <tosky> cp16net: please check https://github.com/openstack/sahara-image-elements
18:53:59 <SlickNik> cp16net: The proposal was that there would be an easier script (ala sahara) — although I'll let vkmc confirm.
18:54:00 <vkmc> abramley, good point
18:54:00 <tosky> cp16net: the idea is the same
18:54:08 <abramley> and also what we do about users / ssh keys etc in the guests - since we expect certain things in the dev environment that you would never do in production
18:54:10 <vkmc> cp16net, yes, the idea is to make it easier
18:54:35 <abramley> so whilst the database specific elements may look the same in dev vs production - the trove guest agent elements will be different
18:54:48 <SlickNik> abramley: make those artifacts part element that only gets mixed in when test images are being built.
18:55:06 <SlickNik> part of an element*
18:55:06 <amrith> SlickNik, may I propose that we treat this just as we would any other 'feature' and BP it with a clear description of the usage, the use case, ...
18:55:29 <amrith> if this is really so simple a solution, I don't believe writing it up as a BP should be that hard?
18:55:38 <vkmc> cp16net, a simpler script, easier to scale
18:55:42 <peterstac> SlickNik, so we'd maintain a second elements dir with dev stuff in it?
18:55:44 <sushilkm> amrith, ++
18:55:52 <amrith> and oh by the way, a link to sahara elements repo isn't the answer. trove is just a bit different
18:55:55 <vkmc> amrith++
18:55:57 <vgnbkr> If you understand what you need to do, it only takes a few minutes to deal with the trove-integration images.  It's the months you need to put in to figure out all the other stuff.
18:56:09 <vgnbkr> s/images/elements
18:56:16 <SlickNik> peterstac: no, ultimately we'd refactor redstack to call out to this other dib script once it's all set up so we're not maintaining two versions.
18:56:44 <amrith> time check ... 4 minutes to go.
18:56:45 <tosky> amrith: they are closer than many other projects; provide instances with $something, plugins vs datastores
18:57:07 <tosky> regarding this specific issue, they are really really close, for what I've been able to grasp
18:58:14 <SlickNik> Sure, we can have a BP to explain what we're doing in this case, but I don't want to block progress on this.
18:59:14 <SlickNik> I think that simplifying the image-building process is something that we _really_ need to address (based on user feedback), and that this is a step in the right direction.
18:59:25 <vkmc> I'll propose the spec, we can continue discussing over it, if that is ok for all :)
18:59:36 <SlickNik> vkmc: ++ Thanks!
18:59:43 <amrith> vkmc ++
18:59:49 <saurabhs> vkmc++
18:59:51 <vkmc> thanks :)
18:59:52 <cp16net> sounds good
18:59:57 <SlickNik> We're out of time for today's meeting!
18:59:58 <cp16net> i'm still wrapping my head around it
19:00:03 <cp16net> :)
19:00:12 <SlickNik> #topic Open Discussion
19:00:16 <sushilkm> thnx :)
19:00:16 <amrith> cp16net, don't hurt urself ;)
19:00:25 <SlickNik> Any items?
19:00:31 <SlickNik> (for open discussion)
19:00:33 * cp16net thinks i pulled a brain muscle
19:00:37 <amrith> who's coming to mid-cycle ;)
19:00:45 <cp16net> o/
19:00:46 <SlickNik> We might have to take them to the #openstack-trove room
19:00:47 * amrith planning beer deliveries
19:00:49 <SlickNik> o/
19:00:52 <johnma> :) thats hilarious
19:01:09 * amrith wanders off to #openstack-trove
19:01:17 <sgordon> i heard you like trove so i put some trove peeps in yo meeting room
19:01:18 <SlickNik> amrith: I have registrations from evite, I'll send you numbers.
19:01:26 <SlickNik> lol
19:01:31 <SlickNik> #endmeeting