18:00:46 <SergeyLukjanov> #startmeeting sahara
18:00:47 <openstack> Meeting started Thu Feb  5 18:00:46 2015 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:50 <openstack> The meeting name has been set to 'sahara'
18:00:52 <SergeyLukjanov> sahara folks, hey!
18:00:56 <aignatov> o/
18:00:58 <elmiko> yo/
18:00:58 <crobertsrh> hello/
18:01:00 <egafford> Hi!
18:01:03 <tellesnobrega> hi
18:01:22 <tmckay> hello
18:01:25 <sreshetnyak> hi
18:01:36 <SergeyLukjanov> okay, let's start
18:01:37 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda
18:01:39 <ylobankov> hi
18:01:46 <huichun> hi
18:01:47 <SergeyLukjanov> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
18:01:54 <SergeyLukjanov> #link https://etherpad.openstack.org/p/sahara-reviews-in-horizon
18:02:05 <SergeyLukjanov> I've updated the etherpad a few days ago
18:02:20 <crobertsrh> I had a couple bug fixes merge, but mostly the same status
18:02:33 <SergeyLukjanov> so, looks like we have a bunch of patches merged and that's awesome
18:02:49 <crobertsrh> Yes, we will take any sort of progress :)
18:02:55 <SergeyLukjanov> so, do we have something urgent?
18:03:20 <crobertsrh> Nothing that I would call "urgent" at the moment that I know of
18:03:26 <SergeyLukjanov> okay
18:03:35 <SergeyLukjanov> crobertsrh, anything else on this topic?
18:03:52 <crobertsrh> Just be sure to take a look at:  https://review.openstack.org/#/c/147677/
18:04:00 <crobertsrh> That is the "guided cluster creation"
18:04:17 <SergeyLukjanov> ack, it's on my todo list
18:04:45 <SergeyLukjanov> #topic News / updates
18:04:49 <SergeyLukjanov> folks, please
18:05:07 <SergeyLukjanov> #info kilo-2 is mostly here, /me will release rest of the components today
18:05:36 <tmckay> spark-swift integration is merged, but there is still a question of the jackson version compat issue.  We can talk about it in open discussion.  Essentially, wait for spark community to fix it (ongoing) or carry our own spark assembly for a while
18:06:16 <elmiko> security doc chapter is coming along, i'm hoping to have a review up soon. i also have some ideas about how we can integrate barbican for some of our credentials storage.
18:06:24 <tmckay> also, the group working on cdh plugin has identified maybe a shortfall in Java EDP support for hbase, where an extra classpath value needs to be set.  I am trying to repro on a CDH cluster so I can verify and spec a change if we need one
18:06:29 <elmiko> #link https://bugs.launchpad.net/openstack-manuals/+bug/1415218
18:06:41 <elmiko> for anyone who wants to keep track
18:06:53 <sreshetnyak> patch for quota checks, new integration tests
18:07:09 <egafford> Working to integrate Sahara with TripleO. There's a puppet review ongoing here: https://review.openstack.org/#/c/145509. Thanks to those who have already reviewed; any more reviews are useful. A tripleo-image-elements commit is likely soon, and a tripleo-heat-templates commit afterward.
18:08:12 <weiting> Working on Sahara integration test for service - key store value
18:08:17 <tmckay> in https://review.openstack.org/146659 we noticed that launch_command.py  (for spark jobs) was not in the MANIFEST.in.  Do we need to backport that to Juno?
18:08:41 <Nikolay_St> egafford: I'll take a look tomorrow, I suppose
18:08:56 <SergeyLukjanov> egafford, yay!
18:09:18 <egafford> Nikolay_St: Thanks. SergeyLukjanov: :)
18:09:41 <SergeyLukjanov> any other updates?
18:09:51 <SergeyLukjanov> let's move on
18:09:54 <SergeyLukjanov> #topic How to improve Horizon changes
18:10:06 <egafford> Interested in answer to tmckay's question re: launch_command.py.
18:10:19 <SergeyLukjanov> so, we've discussed a bit this topic on the last cross-project meeting
18:10:19 <tmckay> we can take it up in open discussion
18:10:27 <egafford> tmckay: Sane and just.
18:10:38 <crobertsrh> +1:  we should improve horizon changes
18:10:44 <tmckay> :)
18:10:49 <SergeyLukjanov> and agreed that we need to at least be sure that all our changes we need to merged into horizon
18:11:03 <SergeyLukjanov> has own blueprints and bugs in horizon targeted to the milestone
18:11:17 <SergeyLukjanov> it should increase review rate a lot
18:11:39 <SergeyLukjanov> to do it I think we should make some process of inter-command prioritization
18:12:02 <SergeyLukjanov> and ping horizon team to accept high-prio things to the milestones
18:12:03 <crobertsrh> What do you mean by "process of inter-command prioritization"
18:12:19 <SergeyLukjanov> I mean to discuss the urgency of things inside the sahara team
18:12:34 <SergeyLukjanov> and then push the list to the horizon team
18:12:44 <SergeyLukjanov> and always ensure that milestones assigned
18:12:54 <crobertsrh> Ok, so maybe take our etherpad of outstanding changes, order them by priority and  ping #horizon-people
18:13:58 <crobertsrh> Is there a go-to person on the horizon side?  Is an in-channel request all we need, or should we send an email?
18:14:00 <SergeyLukjanov> and ensure that all of them has own issues and blueprints
18:14:14 <david-lyle> crobertsrh: ping me
18:14:23 <crobertsrh> I know they were talking about this issue, but I missed this week's meeting.
18:14:25 <SergeyLukjanov> I think that we could initially ask david-lyle to go through the etherpad
18:14:27 <crobertsrh> thanks david-lyle :)
18:14:32 <SergeyLukjanov> david-lyle, oh, hey :)
18:14:34 <SergeyLukjanov> david-lyle, thanks
18:14:45 <david-lyle> saw horizon, came running
18:14:52 <SergeyLukjanov> :)
18:15:20 <david-lyle> I just need to be made aware of items you are needing, then I can prioritize them in horizon
18:15:40 <SergeyLukjanov> crobertsrh, could you please update the etherpad with bugs/blueprints and links to reviews?
18:15:41 <crobertsrh> Sounds fine, david-lyle.
18:15:49 <crobertsrh> I can do that
18:16:01 <SergeyLukjanov> crobertsrh, cool, thx
18:16:07 <crobertsrh> links to reviews should contain the bp/bugs links in them already
18:16:23 <crobertsrh> I can add them to the etherpad though if that somehow speeds things up
18:16:27 <SergeyLukjanov> david-lyle, you're not using specs process in horizon?
18:16:42 <david-lyle> not yet, will be moving to that in lemming
18:16:59 <tmckay> lemming, that's the next release?
18:17:00 <SergeyLukjanov> david-lyle, nice, so, it'll be easier for us before Lemming
18:17:03 <SergeyLukjanov> tmckay, nope
18:17:20 <SergeyLukjanov> http://surveymonkey.com/r/openstack-l-naming voting for the next release name
18:17:24 <tmckay> I hope not
18:17:34 <SergeyLukjanov> closing next week, hurry up to vote
18:17:38 * tmckay wipes brow
18:17:43 <elmiko> there was some talk about lemming being a poor choice on the ml
18:18:01 <david-lyle> could be Love, so weigh your options carefully
18:18:10 <tmckay> +1 London
18:18:36 <SergeyLukjanov> -4 to Love, I don't want to say this word while talking with customers ;)
18:18:48 <SergeyLukjanov> okay, let's move on
18:18:50 <elmiko> lol
18:18:52 <tmckay> there is a town here called Lizard Lick
18:19:01 * david-lyle goes back to Horizon land
18:19:02 <SergeyLukjanov> #topic Open discussion
18:19:10 <SergeyLukjanov> david-lyle, thx for participating
18:19:13 <elmiko> i'd like to talk about deployments
18:19:28 <elmiko> as they pertain to advice we will be giving in the security doc
18:19:34 <tmckay> okay, spark assembly issue for me.  and backport manifest tweak necessary for Juno?
18:20:36 <elmiko> are there any objections to us recommending that sahara controllers be deployed into cloud instances?
18:21:14 <tmckay> elmiko, as opposed to running on the openstack controller?
18:21:27 <tmckay> is that bad for some reason? ^^
18:21:37 <tosky> maybe after that the daemon splitting is in place...
18:21:43 <elmiko> yea sorta, as opposed to being deployed directly on a host connected to the cloud
18:22:07 <elmiko> i'm being asked to create a recommended deployment strategy
18:22:52 <elmiko> and security-wise it sounds like it's easier to recommend deploying to cloud instances as the security issues will be elevated past the host related issues.
18:22:56 <elmiko> does that make sense?
18:23:31 <elmiko> does anyone have opinions about sahara deployment in production?
18:23:31 <alazarev> elmiko, what do you mean by 'cloud instances'?
18:23:52 <elmiko> alazarev: a server spawned by openstack as opposed to a host connected directly to the cloud
18:24:28 <elmiko> for example
18:24:29 <elmiko> https://mimccune.fedorapeople.org/data_processing.png
18:24:43 <elmiko> i shared that diagram as a basic example of a deployment
18:25:20 <elmiko> the pushback i got was if i was implying that sahara needs to be a separate host connected to the cloud, or could it be an instance within the cloud.
18:25:49 <alazarev> elmiko, do you want to recommend dedicated node for sahara controller?
18:25:51 <elmiko> i don't think it _needs_ to be a separate host, so i don't have a problem recommending installing to a cloud server(was instance)
18:26:21 <tmckay> elmiko, tiny issue that I've seen in a case with Sahara running inside the stack -- the credentials for the admin tenant for the stack are inside of the sahara.conf on the VM.  Maybe not a big deal, but the creds are there in addition to only being on the openstack controller node (if you run Sahara as an openstack service)
18:26:23 <elmiko> alazarev: well, i think that _we_ need to recommend something as a team.
18:27:16 <elmiko> i don't see any difference in deploying sahara in single or distributed mode into cloud servers.
18:27:17 <alazarev> elmiko, as I see sahara controller is usually run on all openstack controller nodes, never saw other deployment
18:27:41 <mattf> alazarev, +1
18:27:42 <elmiko> alazarev: so, on the same machines as say a nova controller?
18:27:59 <alazarev> elmiko, yeap
18:28:08 <elmiko> ok, but here's the question. _why_ does it need to be deployed on controller nodes?
18:28:09 <mattf> alazarev, at worst it's a vm on the controller node.
18:28:30 <mattf> we have a dep on the neutron l3-agent somewhere, right?
18:28:37 <elmiko> afaik it just needs access to the control plane
18:28:42 <alazarev> elmiko, why not?
18:28:49 <mattf> for netns exec
18:29:30 <tmckay> elmiko, doesn't have to be.  but if you run it somewhere else, not deployed by openstack, you have to add it to the service catalog.  (it may be that I'm not up on my commercial deployments, mostly dev env)
18:29:43 <alazarev> it needs access to control place and to VMs ssh
18:29:48 <elmiko> alazarev: from the brief conversation i had with bdpayne in openstack-security i think he prefers recommending it installed as an instance because it make security auditing easier. (that's my impression, he had to go before we could finish)
18:30:34 <elmiko> mattf: couldn't it still access neutron if it was inside a vm?
18:31:26 <mattf> elmiko, i don't have details cached in anymore. to handle namespaces you need to be able to netns exec, which means living in the same kernel instance
18:31:27 <alazarev> elmiko, which part of neutron? it would be hard to use netns for example
18:31:40 <egafford> tmckay: +1; I've had success running Sahara from a separate node. elmiko: I believe it is certainly TripleO's strategy to allow "optional" services like Sahara to run in or outside of controller nodes whenever possible, so if we are strictly dependent in prod on being a process on the controller node itself, this'll be important to know.
18:32:45 <elmiko> ok, so netns access is one solid check in favor of needing to be deployed on a controller instead of a cloud server. does that sound accurate? and can anyone point me towards documentation?
18:33:45 <alazarev> elmiko, also we need admin access to keystone, it can be restricted to private network
18:33:47 <tmckay> so the next question becomes, is there a way to allow netns access from a cloud server, and if there is does that create a competing sec issue
18:34:00 <tmckay> that makes the whole thing not worth doing
18:34:23 <SergeyLukjanov> sorry folks, need to disconnect right now
18:34:24 <elmiko> i think part of this stems from the impression that sahara is an application that runs on top of the cloud
18:34:29 <SergeyLukjanov> #chair alazarev
18:34:30 <openstack> Current chairs: SergeyLukjanov alazarev
18:34:31 <tmckay> alazarev, +1, that touches on the "admin cred" issue I mentioned earlier
18:34:35 <SergeyLukjanov> #chair tmckay
18:34:36 <openstack> Current chairs: SergeyLukjanov alazarev tmckay
18:34:55 <mattf> elmiko, i don't think we have a doc that talks about the deployment considerations when using namespaces. we just mention there's a config you need to set.
18:34:58 <elmiko> alazarev, tmckay, as long as the vm had a route to the identity service it shouldn't matter where it runs
18:35:16 <mattf> SergeyLukjanov, ciao
18:35:41 <alazarev> elmiko, yes, but this will require additional efforts to make such route
18:35:43 <tmckay> elmiko, I think his point is that if you put it on the controller, you *could* restrict keystone to private net
18:36:06 <elmiko> alazarev: agreed
18:36:09 <mattf> heh - https://ask.openstack.org/en/question/56906/saharacant-login-to-nodes/
18:36:15 <elmiko> but it could be done
18:36:28 <elmiko> mattf: nice
18:37:58 <alazarev> netns is not a silver bullet since it can't be used for HA mode
18:38:22 <elmiko> netns seems like a corner case to me, but it's a configuration that might not work well within an instance
18:38:31 <alazarev> this is why I've started indirect access feature
18:38:36 <elmiko> when would you _need_ to use netns?
18:38:52 <elmiko> indirect access seems like a security win to me
18:40:06 <alazarev> elmiko, the only thing I don't like in current implementation is "ssh over ssh' which is too slow, port forwarding would be much faster
18:40:14 <elmiko> yea
18:40:14 <mattf> indirect access is just awesome
18:40:22 <mattf> <3 it from the beginning
18:41:00 <elmiko> so, is our recommendation that operators install sahara on a controller node as opposed to a cloud instance?
18:41:37 <elmiko> bdpayne: are you available?
18:42:16 <alazarev> elmiko, I believe so, all installations I saw were in such way
18:42:20 <tmckay> I think we would have to try the cloud instance as a reference architecture and see how it works
18:42:45 <elmiko> ok, i can try it out
18:42:46 <tmckay> but to date, I agree with alazarev.  I always thought Sahara was meant to be on the controller nodes
18:43:13 <elmiko> right, we've always installed it on controller nodes. but for a security recommendation is it _required_ to be on controller nodes, that's the real question.
18:43:40 <egafford> elmiko: If the admin creds weren't on each Sahara node, I'd say it'd be an easy win to suggest Sahara be run in cloud nodes (assuming your config doesn't require netns). Given that the admin creds are on each Sahara node, though, it does seem a lot dodgier of a proposition.
18:43:45 <elmiko> i think the argument is that running things on cloud instances provides an easier way to manage the controllers.
18:43:47 <alazarev> elmiko, I don't think it is _required_
18:43:50 <tmckay> elmiko, if the cloud instance model is viable and it has demonstrable security benefits, maybe we recommend it.  Or maybe we present it as an option with pros and conds
18:44:19 <elmiko> i think this is a recommendation we will need to have in the sec.doc.
18:44:57 <elmiko> egafford: i guess i don't see much difference between the creds being on a vm in the cloud or on a controller attached to the cloud. exploiting either machine would be  bad.
18:45:24 <elmiko> keep in mind, i'm not suggesting creating images with the creds preloaded.
18:45:31 <bdpayne> elmiko, I'm here now
18:45:51 <egafford> elmiko: Right, that would be absurd. Don't think anyone's suggesting that.
18:46:13 <elmiko> bdpayne: awesome, we are having some difficulty with the concept of running sahara on a controller as opposed to a cloud instance. would you be able to talk about the pros/cons of those deployments from a sec perspective?
18:46:40 <elmiko> bdpayne: by default we have always run on controllers
18:47:02 <bdpayne> hey, I was just reading the backlogs
18:47:07 <bdpayne> why are admin creds needed?
18:47:29 <bdpayne> I would argue that you don't want the admin creds in an instance
18:47:33 <tmckay> I forget just what errors you get without them.  I think it's for auth checks
18:47:38 <bdpayne> so that may be what pushes it back onto the control plane
18:47:40 <tmckay> alazarev, ^^ do you know?
18:47:58 <elmiko> at the least we need admin creds for creating trusts
18:48:03 <bdpayne> but if you don't need them, all the better and I'd certainly argue for putting this in an instance
18:48:06 <tmckay> that is true
18:48:27 <bdpayne> could the trusts be setup once at install time and then just used?
18:48:31 <bdpayne> or is this an ongoing need?
18:48:41 <elmiko> ongoing, we create them dynamically under some configs
18:49:11 * tmckay should document the errors that arise when admin creds are missing
18:49:29 <bdpayne> interesting
18:49:52 <bdpayne> ok, so more generally, of course running a service on the control plane has much greater security impact
18:50:03 <bdpayne> if that service is compromised, then it could potentially be a stepping stone to more sensitive parts of the cloud
18:50:10 <bdpayne> this is why I tend to prefer things in instances
18:50:26 <bdpayne> but, it sounds like that may not be viable with the current design
18:50:36 <bdpayne> it may be useful to lay out some of these considerations in the security guide
18:50:42 <alazarev> what do we mean by 'admin' here? sahara admin or openstack admin?
18:50:56 <bdpayne> and explain under which situations someone may be able to deploy as an instance
18:50:58 <bdpayne> in case people want to do that
18:51:09 <bdpayne> I'm talking about openstack admin
18:51:11 <elmiko> i think we'll need to do more testing on the instance model
18:51:32 <alazarev> admin is used to update sahara objects in background, to create trusts, etc.
18:51:55 <elmiko> there is also a config option that uses net namespaces and we're not sure if those will work in a cloud instance
18:53:04 <bdpayne> so yeah, I think laying out the security concerns and showing how to best mitigate them is the right path for the security guide
18:53:14 <bdpayne> even if the end result is something on the control plane
18:53:52 <elmiko> ok, i'll have to do more digging around the sec doc to see if i can leverage the advice there for control plane installations
18:54:58 <elmiko> bdpayne: thanks for the advice, it gives me some food for thought
18:55:10 <bdpayne> np!
18:55:41 <elmiko> only 5 min left, should we talk about tmckay's questions?
18:56:13 <tmckay> alright, opinions on spark assembly class conflicts for swift access.  Short answer, people in spark are working on it, but I  don't know when it will be in a release.  Looks like maybe 1.3
18:56:39 <tmckay> so, are we okay with the current fix, to patch the classpath and hope it works?  Or do we carry our own spark assembly for DIB for a little while?
18:56:58 <elmiko> tough question
18:57:01 <tmckay> my gut is to wait for 1.3, and fix on fail if someone comes up with a  use case that shows we need a new assembly
18:57:11 <elmiko> i'm ok with that
18:57:16 <mattf> me too
18:57:17 <crobertsrh> +1 for wait
18:57:21 <alazarev> I'm ok with that too
18:57:49 <tmckay> okay.  it works, but we know it could break, but we haven't seen that.  Works for me.
18:57:51 <tmckay> thanks.
18:57:53 <tmckay> Other question
18:58:03 <huichun> tmckay: Hi tmckay,  i have replied your email about the masternotfound error
18:58:12 <tmckay> we left launch_command.py out of MANIFEST.in
18:58:15 <tmckay> huichun, thanks
18:58:45 <tmckay> launch_command.py lives in edp/resources, so I'm guessing it's not in a distro in Juno
18:58:56 <tmckay> do we need to backport that fix?
18:59:01 <alazarev> 1 min left
18:59:03 <elmiko> i think so
18:59:13 <tmckay> what is the best way to check, with setuptools on a juno branch and bdist?
18:59:18 <tmckay> or sdist?
18:59:32 <egafford> tmckay: Yes, we should.
18:59:32 <tmckay> makes me think that spark edp in Juno can't be working
18:59:37 <elmiko> just install to a clean virtualenv and look in the site-packages
18:59:39 <tmckay> but we haven't heard that
18:59:53 <egafford> On the RH side: https://bugzilla.redhat.com/show_bug.cgi?id=1184522
18:59:59 <alazarev> let's switch to #openstack-sahara
19:00:01 <alazarev> #endmeeting