18:02:14 #startmeeting trove-bp-review 18:02:15 Meeting started Mon Sep 8 18:02:14 2014 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:16 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:17 o/ 18:02:19 The meeting name has been set to 'trove_bp_review' 18:02:23 o/ 18:02:28 o/ 18:02:36 o/ 18:02:41 Agenda at: 18:02:44 o/ 18:02:44 #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting 18:02:55 ./ 18:03:13 o/ 18:03:29 o/ 18:03:31 o/ 18:03:32 #topic SUSE support 18:04:16 #link https://blueprints.launchpad.net/trove/+spec/suse-support 18:04:20 iartarisi, floor is yours 18:04:31 thank you 18:04:33 so I think the action item from last time was for me to create a blueprint which we can discuss 18:04:51 that's the blueprint ^^. If everyone's ok, I can start targeting the existing change requests against that 18:05:34 and over the coming weeks I will also start working on a SUSE(zypper) PackagerMixin 18:05:43 how does that sound to everyone? 18:05:48 iartarisi: Thanks for creating that. I had two questions: 18:06:09 1. Is there a list of work items / reviews that this BP entails? 18:06:52 I can quickly come up with one. There are three outstanding reviews + the not yet started PackagerMixin issue 18:07:00 Dan Nguyen proposed a change to openstack/trove-integration: Add a script to package the guest agent in a venv https://review.openstack.org/108562 18:08:16 https://review.openstack.org/108394 https://review.openstack.org/108703 https://review.openstack.org/108972 18:08:25 those are the outstanding reviews 18:08:54 2. There were some concerns around the CI and maintenance around this, since it's not tested by upstream. You guys mention there's a SUSE CI, and you guys will be on the ball for any breaks in SUSE related areas, right? 18:08:56 these are already part of our packages and we have working trove with them. 18:09:09 SlickNik: right! 18:09:31 SlickNik, I believe the bugs being addressed here are the result of the SUSE CI (see, I got the capitaliztion right this time) 18:09:48 amrith: thanks! and that's right! 18:10:01 amrith proposed a change to openstack/trove: Remove unused entries in cfg.py https://review.openstack.org/112995 18:10:12 iartarisi / amrith: got it, thanks! 18:11:02 Sounds good to me. 18:11:17 ok, I have an aside here related to the PackagerMixin 18:11:42 that last review is about adding a NoOpPackagerMixin which wouldn't touch the system at all, but rather assume that the admin has set everything up 18:11:50 are you fine with adding that to trove? 18:12:21 to me, as a distro packager, this seems like a much more sensible way for trove to act, rather than installing things itself which should be outside of the attributions of an app 18:13:20 that's also related to the SUSE support actually, since I believe we'd be more comfortable with just handling everything in packaging rather than in a SUSEPackagerMixin 18:14:27 iartarisi, is this part of the BP as written? I think not, is that correct? 18:14:37 The approach seems reasonable to me that this would be a selectable option if a deployer chose it — although I haven't had a chance to review the code yet, so you might get some more feedback once I review the code. 18:14:55 amrith: this is about one of the standing reviews and it would remove one item from the blueprint 18:15:18 SlickNik: that sounds good, selectable where? 18:15:41 as a guestagent config value? 18:15:47 i assume the selection is that the deployer chooses the NoOpPackagerMixin? 18:16:42 in the review, the NoOpPackagerMixin is only a fallback, in case a compatible OS was not found 18:16:43 So right now they're choses based on the distro, IIRC. 18:17:14 but I would definitely encourage having it as a configuration option, too 18:18:12 iartarisi: Let's hold that thought and keep the discussion on the review. 18:18:33 FWIW: I like the fallback to No-op in case we can't detect the distro. 18:18:38 iartarisi, agreed with SlickNik 18:18:42 (in the interest of time) 18:18:43 SlickNik: alright, so I'll wait for how that goes before starting work on a SUSE Mixin 18:18:59 iartarisi: Sounds good. Thanks for the work on this! 18:19:07 SlickNik: cool. Thank you! 18:19:19 Any other comments on this? 18:19:30 . 18:19:33 I move that this be approved as written. 18:20:07 Seconded. 18:20:30 sounds good to me. 18:20:33 Let's move on — we have a full agenda. 18:20:37 #topic SSL support 18:20:40 kevinconway: around? 18:20:43 o/ 18:20:49 o\ 18:21:57 #link https://wiki.openstack.org/wiki/Trove/TroveSSL 18:21:57 ssl! 18:22:34 so i outlined in the BP how ssl would work for mysql and postgresql 18:22:45 i did some research on the others and wrote up how they would fit in 18:23:07 kevinconway: Yup, went through that -- but one of the concerns I have around this is the chicken and egg problem. 18:23:35 kevinconway: Since we don't have TLS, we don't have a secure way of getting the certs on to the datastore instance. 18:24:01 ah, i see. i should have specified in the BP. the keys would be delivered via the message q 18:24:11 as a part of the prepare call, for example 18:24:39 Ah, I see. 18:24:45 could they be injected into the instance via nova create? 18:25:00 kevinconway, I'm concerned about the work involved in key management and storage. These are non-trivial activities to get right. I'm also concerned about trove getting into the business of key creation, management, storage, etc., 18:25:00 so the precursor chicken lays a proto chicken egg. science! 18:25:06 cp16net, i like this idea 18:25:18 But you shouldn't send a private key unless the connection you send it over is also secured. 18:25:37 amrith, correct, this is a task for barbican =) 18:25:42 amrith: so part of the BP is that the key provider should be an interface. we could have one or two default impls, but deployers should manage keys 18:25:53 vgnbkr, honest true ! 18:26:09 I submit to you that in the fullness of time, trove should merely indicate to the guest instance that it must enable ssl and the guest instance should request the various things directly from barbican 18:26:16 and trove should stay out of the equation. 18:26:37 amrith: when you say "guest instance", do you mean the agent? 18:26:45 grapex: yes 18:26:49 i agree that bbq is a part of the equation, but it doesn't actually support SSL certs. 18:26:52 Because the guest agent is Trove as well 18:26:55 trove itself should neither know nor see any part of SSL 18:27:09 amrith, +1 18:27:13 grapex: understood 18:27:15 terminology 18:27:30 amrith: Your concern is that the Trove server daemons see the SSL certs 18:27:37 grapex: yes 18:27:48 I think that's a solution for today's pre-Barbican era 18:28:00 other than the guest where the keys and the cert are installed, no other part of trove should see these things 18:28:05 In the post-Barbican future Trove could contact it vicariously on behalf of the user 18:28:16 I don't like the fact that a prepare call provides the things 18:28:30 I'd rather the prepare call provide a URI where the guest can go and get the things it wants 18:28:40 amrith: right now we don't have much of a choice. we would need to wait for bbq to support certs 18:28:51 not necessarily true 18:28:58 if a user provides you the certs 18:29:05 amrith: Also, consider in the future that what Trove injects is a URI to Barbican rather than the certs itself 18:29:09 then you could point the guest agent to get them from some location, no? 18:29:24 grapex, I wouldn't assume that 18:29:30 so the way the BP is written now allows for amrith's solution as well as others 18:29:34 then the guest still has to download it 18:29:38 and see it 18:29:40 in the future, I'd assume that all trove infors the guest is that "go forth and configure SSL" 18:29:45 and let the guest do it directly 18:29:47 amrith: I think you trust the guest agent too much 18:30:10 grapex, in an SSL implementation, you have to trust the end point that has the database. 18:30:10 because in the end its a piece of Trove infrastructure running on the server 18:30:18 and we have to rely on the guest agent to get stuff there. 18:30:43 I don't believe that trove should get into the business of generating, or providing any ability to manage keys 18:30:46 now, or in the future. 18:30:54 amrith, +1 18:30:54 amrith that is not in hte BP 18:30:57 so amrith, in the BP it does not dictate how keys are created or managed 18:31:15 it suggests an impl interface that a deployer can provide 18:31:27 so if you want ssl links to the guest and a guest impl that downloads the keys you can have that 18:31:28 but it does indicate that the interface would convey the PKI+cert 18:31:35 if i as another deployer want something else i can do that as well 18:31:49 i'd rather keep proprietary key management out of codebase (unless it's bbq) 18:32:00 IMHO, the extent of the interace should be: prepare (enable_ssl=True) 18:32:00 When we say interface, are we talking about the guest agent RPC calls? Because if so that word may be too kind. :) 18:32:07 denis_makogon: it would be out of the trove trunk if a deployer did a custom impl 18:32:09 and it is up to the guest agent to go figure out how to get it. 18:32:43 that's may work 18:32:49 amrith: that is still a possibility with the BP 18:33:02 yes, it suggests the keys are tx'ed along the q 18:33:07 but that is a matter of impl 18:33:19 kevinconway, that may be true. 18:33:29 but what I'd like is to have the discussion now, about the implementation 18:33:34 rather than have it with code in flight 18:33:47 you still need something outside of the prepare to fix older instances and in this case the mgmt call adds ssl 18:33:53 we had some of the discussion of what the role and bounds of trove should be (at mid-cycle) 18:33:59 so the trove impl could easily be a noop. i imagine deployers will all have different key management strategies 18:34:17 and to that point, I'd like to at least have the discussion now that frames the outlines of what the API changes would look like 18:34:23 and get agreement on that, in principle. 18:34:46 amrith: when you say API are you talking about the RPC calls? 18:35:24 grapex, I'm talking about the interaction between the guestagent and the rest of trove. 18:35:44 One thing I dislike about this notion that the guest agent has to become smart enough to do all of the key handling itself by default. 18:35:57 currently, for example, ssl_payload contains keys and certs. 18:36:01 We already have it talking to Swift but in that case it was unavoidable 18:36:10 that's an API 18:36:26 and I think that part of the API is not Trove's responsibility 18:36:33 amrith: so nothing in the BP should suggest that you _have_ to send the actual keys as payloads 18:36:41 that's why there is also an interface for the guest portion 18:36:51 which consumes the ssl_payload and handles it 18:37:10 kevinconway, what I'm suggesting is that the BP should explicitly state that the keys will not be in part of the payload of the enable_ssl command 18:37:32 the enable_ssl command should only tell the guest ... go forth and enable SSL 18:37:34 i think that's an unnecessary restriction on the impls 18:37:37 amrith: So basically, you're enforcing that no deployer of Trove may ever send the keys? 18:37:41 kevinconway: +1 18:37:55 grapex, I'm proposing that the open source product not get into that 18:38:02 if a deployer wishes to change Trove, so be it. 18:38:55 amrith: I think it's fair to say kevinconway and I disagree 18:39:05 that would be fair to say. 18:39:08 I think we can make it flexible enough to work both ways 18:39:09 I agree too. 18:39:11 I don't see the downside of that 18:39:45 i've got one question 18:39:48 unfortunately, I do. 18:39:52 so in your model amrith, trove would send a simple binary payload to the guest and the guest would perform all work related to ssl. is that correct? 18:40:17 kevinconway, i think that's correct. 18:40:21 no matter how it is going to be implemented, it is really matter how would we test it? 18:40:30 other than preventing a deployer from sending keys to the guest, what is the value added of that over the suggested BP? 18:41:11 kevinconway, it is a matter of what one thinks is the 'role' of trove. 18:41:11 if its the deployers decision whether to send the keys or not, then they know what they are getting into 18:41:28 kevinconway, amriths way is much safer that sending keys over AMPQ 18:41:30 at the mid-cycle, we had conversations about whether or not we should have thresholds for what constituted a delay in replication 18:41:32 Is the objective to merely support SSL connections to the database, or must the database support connecting with a pre-defined certificate? 18:41:51 and the decision was that trove should not decide what constitutes 'degraded'. let the deployer decide that 18:41:54 i think kevinconway has made it clear that trove is not generating the keys 18:42:08 in the same way, the question is whether trove should generate and transmit keys. 18:42:13 I believe it should not 18:42:19 are keys required, yes they are. 18:42:28 then use the no-op ssl provider impl? 18:42:28 but let them be obtained without trove getting into the mix. 18:42:38 I think we're all agreed in that trove shouldn't generate and manage these keys. 18:42:42 kevinconway, you are missing the point 18:42:57 SlickNik +1 18:42:59 I'm suggesting that there shouldn't be an implementation in the open source product that has trove shipping keys 18:43:05 The question is whether the trove API should be the medium through which these keys are transmitted. 18:43:14 what deployers want to do in their versions is their issue. 18:43:25 amrith, +1 18:43:35 relate this to the conversation of triggering what constitutes a degraded situation wrt replication lag. 18:43:45 no one suggests that NO deployer should have such a notion 18:43:46 amrith: so if the initial of the BP did not tx keys, would that satisfy your concern? 18:43:48 amrith: if there weren't dozens of things in Trove we at Rax had no use for I'd agree with this sentiment that Trove shouldn't have extra options you yourself might never use. 18:43:52 just that the open source project should not. 18:44:08 amrith: the initial impl that is 18:44:20 kevinconway, the implication is that a later implementation will? 18:44:30 amrith: no, but a deployers impl could 18:44:35 sure 18:44:45 deployers can have trove sing the national anthem, I don't mind. 18:44:48 that's their call. 18:45:21 there are singing robots, amrith 18:45:25 some quite popular 18:45:42 amrith: Do you agree that having SSL for datastore implementations that are deployed is a valid trove scenario? 18:45:46 kevinconway, and some deployers may have API calls to those too. (out of the scope of this BP) 18:45:47 amrith: What you haven't proven in my mind is how including this would be a detriment to Trove itself. 18:46:08 SlickNik, yes. I believe it is. 18:46:20 grapex, that is likely true. 18:46:29 Deployers already have the right to do whatever they want. This is an apache license piece of open source software so I don't think we need to reiterate that. 18:46:55 i'm simply suggesting an interface that allows for more than one possibility 18:47:16 so the trove reference impl can _not_ send keys, but i don't believe we should prevent other impls that do 18:47:28 so, what does the rest of core think? what about others who were at mid-cycle? should we (in trove) handle keys? pass them around and so on? 18:47:33 i agree SSL is good 18:47:43 the question I have is whether we should be in the business of key transport. 18:47:43 Same here. 18:47:52 I agree that we should support this. 18:48:36 I'm not sure what the reference impl for getting the keys / cert out to the guest should be (quite yet) 18:48:56 i'd suggest to delegate key management to guestagent, and do not involve AMPQ 18:49:32 There should be no need to transmit PRIVATE keys unless we wished to support pre-defined certs. Transmitting public keys is fine. 18:49:41 this is a potential security issue, since we can't know if AMPQ service works with SSL/TLS 18:49:54 vgnbkr, +1 18:50:02 I don't think we're going to get closure on that during this meeting, so we'll probably need to discuss this OOB and perhaps again next week. 18:50:26 SlickNik, +1 18:50:49 amrith / grapex / kevinconway: can we continue the discuss out of band? 18:51:05 SlickNik, +1 18:51:14 And let's move on to the next BP in the interest of time. 18:51:52 #topic ceilometer integration 18:51:56 it's mine 18:52:10 #link https://blueprints.launchpad.net/trove/+spec/ceilometer-integration 18:52:32 for now we have only several notifications - when instance gets created, resized, and deleted 18:52:41 all this notifications are emitted at the start 18:53:33 for billing purposes and resource monitoring (involving ceilomenter) we need to provide fine grained notification through resource life cycle 18:54:14 there are three types of resources: instance, backup and cluster(new one) 18:55:10 there are three levels of notifications: start, end, error 18:55:35 some of those notifications are emitted by taskmanager, some of them by conductor 18:56:12 denis_makogon: So my concern is that we already have instance notifications — does it make sense to first work on a ceilometer plugin using these notifications (https://blueprints.launchpad.net/ceilometer/+spec/trove-plugin) before we go add notifications for all sorts of events in trove? 18:56:54 SlickNik, i guess no, since it would be nice to have all notification once and for all 18:57:09 SlickNik, it means that we need to build our own notifications first 18:57:41 just take a look at nova, it had its notifications even before ceilometer was proposed 18:58:57 notifications in Trove is the fine grained base for going forward with including ceilometer into monitoring workflow 18:59:42 I'm not sure that we'd even use these notifications for billing. Other folks (RAX, ebay) can chime in here, but at HP we bill backups based on Swift data, so something like backup create/delete would not be super useful to us. And clusters just made it in, so probably good to not get ahead of ourselves with this. 19:00:18 Why can't we get a POC with the ceilometer plugin and just instances as a first step? 19:00:37 SlickNik: +1 19:00:40 we might try 19:00:41 SlickNik +1 19:00:43 is there someone who is looking for ceilometer integration in trove? 19:00:44 we use resize/create/delete for billing 19:00:48 that would could make part of this POC? 19:00:52 amrith, i'm going to do that 19:01:25 denis_makogon, not you. I mean someone operating trove in production who wants this feature 19:01:40 Ok, as i can see, initial implementation would take only instance events and ceilo plugin 19:01:49 could we find out what events they want. 19:02:39 If no one is going to deploy trove with this enabled, it seems like a futile effort. 19:03:08 how is this different then the current notifications? 19:03:18 does it extend it? 19:03:23 or replace it? 19:03:27 amrith, Trove only emits notification - it doesn't know who's going to consume them 19:03:37 robertmyers, it'll extend it 19:03:50 robertmyers, for example, we have "instance.create" 19:03:53 denis_makogon: this does not involve any changes to existing messages, correct? 19:04:01 denis_makogon ok 19:04:12 kevinconway, it would modify topics 19:04:36 kevinconway, for example "resize_volume" to "resize.volume.start|end|error" 19:04:50 kevinconway, i mean resize_volume.start|end|error 19:04:58 denis_makogon: could you create new messages with those titles rather than modify existing? 19:05:24 yeah we dont want breaking existing systems which depend on these notifications 19:05:38 kevinconway, to keep working Rax billing system? 19:05:55 denis_makogon: -1 to changing existing messages without a good reason. 19:05:57 denis_makogon: so for example trove.instance.create.end == trove.instance.create 19:05:57 iccha, kevinconway, i can do it 19:06:29 robertmyers, SlickNik, kevinconway, iccha, what about sending several events at one stage ? 19:06:41 that is fine 19:06:59 as long as the currnet is unchanged 19:07:02 so it would keep everything working fine for all 19:07:05 +1 robertmyers 19:07:07 robertmyers, sure 19:07:12 yeah, i wouldn't mind multiple so long as the originals are preserved 19:07:38 cores, what do you think about multiple notifications ? 19:07:57 denis_makogon: I'd really like to see more "integration with ceilometer" pieces, and fewer "get as many events as we can emitted by Trove" pieces as part of this BP. 19:08:10 denis_makogon: I'm ok if you can also demonstrate some use 19:08:27 which I guess SlickNik just said. :) 19:08:28 grapex, some use where ? 19:09:08 i could see some potential use if we use it for metrics like how long does it take to start and end a task 19:09:21 but no point unless someone actually uses it 19:09:41 iccha, +1 19:09:42 so, first i need to implement ceilo part, right ? 19:09:59 if that works for all - fine, i'm cool with that 19:10:30 denis_makogon: Yes, that would be good. 19:10:35 ok 19:10:37 Okay we're already running over. 19:10:44 ya way over :) 19:10:48 can we review the last one ? 19:10:49 Dan Nguyen proposed a change to openstack/python-troveclient: Add mgmt upgrade call to CLI https://review.openstack.org/108796 19:10:58 cassandra clusters 19:10:59 denis_makogon: will have to move to next week. 19:11:07 #endmeeting