14:00:32 <SergeyLukjanov> #startmeeting sahara
14:00:33 <openstack> Meeting started Thu Dec 18 14:00:32 2014 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:37 <openstack> The meeting name has been set to 'sahara'
14:02:02 <SergeyLukjanov> #topic sahara@horizon status (croberts, NikitaKonovalov)
14:02:08 <SergeyLukjanov> crobertsrh, NikitaKonovalov, please
14:02:20 <SergeyLukjanov> any good things happening?
14:02:25 <crobertsrh> Still a few patches waiting for review.
14:02:26 <NikitaKonovalov> our changes are getting some attention
14:02:35 <NikitaKonovalov> from Horizon team
14:03:03 <crobertsrh> I'm still waiting for meaningful input from UX for UI rework ("wizards").  Will reping them again today.
14:03:08 <NikitaKonovalov> so we'll probably get Job Executions table working properly soon
14:03:34 <SergeyLukjanov> crobertsrh, do you have some proposal for them?
14:03:40 <alazarev> two of my horizon patched were merged
14:03:54 <aignatov> o/
14:04:14 <alazarev> two other are still on review
14:04:28 <crobertsrh> I was hoping they would have proposals for me.  In the background, I'm starting a few thing though.  I will mock them up soon regardless of their input or lack thereof.
14:04:58 <SergeyLukjanov> crobertsrh, got it
14:05:57 <SergeyLukjanov> #topic News / updates
14:06:47 <crobertsrh> Specs for template editing and default templates are up for comments.  Still some work needed there, but I wanted to get comments before vacation time.
14:07:09 <crobertsrh> Skeleton blueprint for UI wizards is up as well
14:07:19 <elmiko> i'm progressing further into creating the content for the data processing chapter of the security guide. i'm using https://etherpad.openstack.org/p/sahara-security-guide-notes as a place to collect ideas, so if anyone could take a look and post comments as the content grows that would be great. aside from that, reviews, and a few minor bug fixes.
14:07:24 <alazarev> I was busy doing teampest tests for sahara, here is result: https://review.openstack.org/#/c/142632/
14:07:24 <SergeyLukjanov> crobertsrh, that's really cool
14:07:27 <alazarev> please review
14:07:37 <aignatov> well, I’ve uploaded my latest version of new pig example https://review.openstack.org/#/c/141782/
14:07:38 <alazarev> *tempest
14:08:04 <aignatov> will add new job execution to integratin tests
14:08:14 <alazarev> *teampest would be cool :)
14:08:43 <weiting> About Pre-built CDH Plugin Image, cloudera is asking for a place to put their EULA and let the user accept the license.
14:08:44 <SergeyLukjanov> elmiko, so, we'll have a chapter in openstack re sahara security?
14:08:50 <elmiko> SergeyLukjanov: yes
14:08:57 <weiting> Is there any suggestion for this?
14:09:09 <SergeyLukjanov> weiting, heh, there is no way right now to do it
14:09:40 <SergeyLukjanov> weiting, thanks for the update
14:10:02 <weiting> HDP image don't need to do this?
14:10:53 <weiting> I mean is there any other similar experience for this?
14:11:06 <SergeyLukjanov> weiting, hwx was ok with publishing their images
14:11:10 <elmiko> weiting: hdp images don't do any sort of eula agreement
14:11:53 <weiting> Ok, got it. So maybe the CDH image cannot publish at this moment.
14:12:21 <SergeyLukjanov> we should think about the mechanism to show license for plugins
14:12:23 <alazarev> weiting, at which stage should user accept agreement?
14:12:27 * mattf waves
14:12:54 <SergeyLukjanov> mattf, morning :)
14:13:00 <weiting> They are asking to show the EULA when the user use it at first time.
14:13:16 * mattf has to update his calendar
14:13:28 <mattf> eula for what?
14:13:35 * mattf finds chat log
14:13:36 <elmiko> weiting: would that be the first time a user creates a cluster from those images?
14:13:41 <weiting> End User License Agreement
14:13:47 <aignatov> mattf: for cdh deployment
14:14:15 <tosky> (hi, the new time!)
14:14:27 <alazarev> weiting, for cluster or for each instance?
14:14:48 <SergeyLukjanov> tosky, yup, it's alt time each second week
14:14:52 <weiting> Yes, but I think it should be difficult to put a license when creating a cluster
14:14:58 <alazarev> if we add new tab with agreement to UI... will this work?
14:15:10 <mattf> weiting, who's requesting the eula?
14:15:16 <mattf> legal, prod, eng?
14:15:24 <weiting> Just need one time to show and let the user accept it.
14:16:01 <mattf> seems very wrong, but assuming it's necessary
14:16:02 <crobertsrh> Right, just a one-time thing.  I think cluster create time is good.
14:16:04 <weiting> Cloudera ask to put the eula
14:16:05 <mattf> can they accept a eula before downloading / creating an image ?
14:16:20 <SergeyLukjanov> mattf, it'll be the best option
14:16:33 <mattf> a eula is very anti-automation
14:16:49 <SergeyLukjanov> mattf, yeah...
14:16:59 <mattf> and sahara is all about automation
14:17:11 <mattf> strikes me as a fundamental conflict
14:17:13 <SergeyLukjanov> but we still have dib
14:17:14 <weiting> Ok, I can ask them about to put the eula before downloading the image.
14:17:20 <crobertsrh> What about launches from the CLI?
14:17:23 <mattf> i'm curious who's asking for this because they might not have enough context
14:17:30 <aignatov> alazarev’s approach would also work
14:17:46 <mattf> weiting, do they have a eula before access their yum repositories?
14:18:33 <mattf> alazarev, aignatov, it's weird tho. who should be accepting the eula? if an org buys openstack & cloudera, does each user accept or does IT?
14:18:48 <weiting> I don't think they have, but I'm not sure.
14:19:37 <mattf> is cloudera going to be producing the images?
14:20:29 <weiting> No, it's based on sahara-image-elements
14:21:25 <mattf> imho the oracle jdk eula accept shouldn't be in the elements
14:21:32 <SergeyLukjanov> so, looks like we need to iterate on it offline
14:21:33 <mattf> i'm wary of adding another
14:21:39 <mattf> SergeyLukjanov, wise
14:21:53 <SergeyLukjanov> mattf, there is a switch to openjdk patch proposed
14:22:09 <mattf> SergeyLukjanov, yeah, i +'d it, it's currently -workflow
14:22:28 <SergeyLukjanov> do we have any other topics to chat about?
14:22:41 <crobertsrh> For template editing, is it acceptable (or possible the *right*) thing to disallow editing of templates that are in use by a cluster.  This would be similar to how we don't allow in-use templates to be edited.  It would avoid quite a bit of work with keeping old versions of templates around.
14:22:47 <SergeyLukjanov> mattf, sreshetniak is on vacation now and will finish it next week I think
14:22:53 <SergeyLukjanov> #topic Open discussion
14:22:54 <mattf> i'd like to discuss plugins w/ 3rd party deps
14:23:06 <elmiko> i'd like to discuss security
14:23:59 <mattf> elmiko, you go first
14:24:03 <SergeyLukjanov> crobertsrh, I think so
14:24:24 <alazarev> quick question, is any particular reason why we recommend using floating IPs for devstack with nova network? (this regarding https://review.openstack.org/#/c/142616/)
14:24:27 <crobertsrh> awesome.  I will rework the spec to reflect that, if nobody disagrees.
14:24:56 <weiting> I will ask Cloudera if the eula can be put during downloading.
14:25:03 <weiting> And we can discuss it offline.
14:25:06 <SergeyLukjanov> crobertsrh, but I think that it's ok to edit template that is in use only by other template
14:25:16 <SergeyLukjanov> crobertsrh, I mean node group tmpl and cluster tmpl
14:25:17 <crobertsrh> Yes, I agree with that.
14:25:25 <elmiko> ok, i'm curious about our default position with regards to sahara security. namely do we have any positions surround stack operation?
14:26:02 <elmiko> in specific things like, separation of users into projects, cluster sharing, data management, these types of topics.
14:26:57 <alazarev> crobertsrh, I don't understand why we disallow in-use template to be edited
14:27:23 <elmiko> i'm asking because as i assemble the chapter for the sec guide, i'm left wondering if we have done any work to create a baseline expectation for our users with respect to how they should configure a sahara enabled stack?
14:27:47 <crobertsrh> alazarev:  Editing an in-use template requires us to save the old template and update the reference in the cluster...so that we can still accurately show the templates that formed that cluster.
14:27:54 <alazarev> we allow clusters and job to be deleted while there are job executions... what wrong with editing template while cluster is running
14:28:16 <SergeyLukjanov> alazarev, because when we have cluster online from template - the template is in fact a spec for cluster and that means users expect that if they use the same template the same cluster wil be provisioned
14:28:19 <alazarev> crobertsrh, we copy all data to cluster, template is for reference only
14:28:55 <crobertsrh> The links in the UI go to the template itself
14:28:59 <SergeyLukjanov> alazarev, job exec is just an event about starting job and job results are most probably aren't stored in the cluster
14:29:22 <SergeyLukjanov> alazarev, templates are for creating the same cluster
14:29:51 <crobertsrh> I think it's essentially the same reason we don't let users delete templates that are in-use by a cluster.
14:31:03 <alazarev> SergeyLukjanov, I could imagine scenario when users always create cluster using template X, and admins tune settings for it in the same time
14:31:29 <SergeyLukjanov> crobertsrh, yup
14:31:41 <alazarev> crobertsrh, yeap, and I don't understand that either
14:31:53 <SergeyLukjanov> alazarev, the question of dissallow in-use templates deletion and editing was discussed when we've been adding the templates
14:32:11 <SergeyLukjanov> alazarev, and we even decided to not implement editing
14:33:16 <crobertsrh> alazarev:  So you're thinking that when a template is edited, we would also push those changes (processes, configs, etc) to the cluster right away?
14:33:16 <alazarev> SergeyLukjanov, may be it is time to discuss more :)
14:33:20 <SergeyLukjanov> elmiko, honestly it's a very difficult question about security guidelines
14:33:35 <elmiko> SergeyLukjanov: yea... ;)
14:33:52 <SergeyLukjanov> alazarev, I'm still very sure that we shouldn't allow to edit and delete in-use templates, crobertsrh is for it too as I see
14:34:44 <crobertsrh> I think it's consistent with everything else we do (to not allow it) and it avoids many possible complications.
14:34:45 <alazarev> crobertsrh, no, I mean that reference in cluster is for information only, I don't see much trouble if template was edited from time of cluster creation
14:35:29 <alazarev> crobertsrh, agree that it is consistent, I mean approach in general
14:35:46 <crobertsrh> you say consistent, but not great
14:36:37 <alazarev> it makes things complicated, sometimes
14:37:16 <crobertsrh> Ok.  I will update the spec for now and we can continue the conversation there.  I don't want to eat-up the whole meeting with other topics waiting.
14:37:59 <alazarev> but I agree, this is a big topic for design sessions, not for IRC chat
14:38:09 <SergeyLukjanov> mattf, what's about plugins with 3rd party deps?
14:39:52 <mattf> i'm curious what the team's view is about supporting plugins that don't operate w/o a 3rd party (non openstack) dependency
14:40:02 <mattf> for instance, the cdh plugin and cm_api
14:40:45 <mattf> should sahara only support plugins that are single sourced (openstack.org)?
14:40:59 <SergeyLukjanov> mattf, IMO we just should document it, enable by default, but add a warning while starting sahara about missed dependency for plugin
14:41:22 <SergeyLukjanov> (enable by default was about cdh)
14:41:31 <elmiko> imo, i'm ok with plugins that have 3rd party deps, but i think the 3rd parties should help to ensure that the deps are available downstream
14:42:08 <elmiko> SergeyLukjanov: enable by default creates issues for the downstream repackage though
14:42:24 <mattf> SergeyLukjanov, i've been thinking about it as: user has to do a non-openstack install step to get the dep, it's reasonable to also have a config step
14:42:37 <SergeyLukjanov> elmiko, if we'll add a warn and lack of the dependency will not break sahara than it's ok IMO
14:42:48 <mattf> i'm curious if folks know what other projects are doing w/ this situation
14:43:03 <weiting> Actually, Cloudera is asking to put cm_api to CDH Plugin source code.
14:43:07 <elmiko> SergeyLukjanov: ok, agreed with that
14:43:22 <SergeyLukjanov> weiting, oh, interesting
14:43:28 <mattf> interesting indeed
14:43:39 <weiting> But I think it's really hard for maintenance.
14:43:40 <SergeyLukjanov> weiting, what's the argument for doing it?
14:43:46 <mattf> i think we should have an opinion as a project, as well as discuss the cloudera case
14:44:14 <kchen> who will take the charge of maintaining that? I cannot image
14:44:32 <kchen> maintaining cm_api in cdh plugin
14:44:36 <weiting> It's open source and they accept to put it in Sahara.
14:44:41 <elmiko> kchen: good question
14:44:46 <mattf> openstack has a long history of copy-libs, but also has a well defined process for them
14:45:14 <tosky> would it be a copy, while keeping the development of the library outside?
14:45:23 <mattf> weiting, each new upstream cm_api release could be copied into the plugin?
14:45:57 <tosky> I'm not sure this is the right approach (distributions will use the upstream one as much as possible)
14:46:16 <weiting> Yes. That's what they are asking for.
14:46:43 <mattf> tosky, alternatives seem to be not enabling the cdh plugin or trying to get cm_api into openstack
14:46:43 <weiting> Yes, I agree with tosky.
14:46:49 <mattf> those aren't good results imho
14:48:09 <weiting> I think Cloudera don't have the resource to maintain cm_api, so tthey are asking for this.
14:48:49 <mattf> weiting, is cm_api something cloudera uses elsewhere or has support for their customers to use?
14:49:25 <weiting> Yes, it's open to support for their customer.
14:49:27 <SergeyLukjanov> probably we should re-spin this question next meeting when sreshetnyak will be available too
14:49:54 <weiting> But they are focusing on java based cm_api.
14:50:52 <mattf> it's almost another swift-fs situation
14:50:54 <SergeyLukjanov> as I remember sreshetnyak was talking about having very simple cm api implemented in the plugin
14:51:02 <SergeyLukjanov> like it was done for intel plugin
14:52:40 <mattf> the next meeting i'll be at is 8 jan
14:54:08 <huichunlu> Good option, we currently use just few of cm_api call
14:54:50 <SergeyLukjanov> huichunlu, yeah, in fact we don't need it, only some basic funcs
14:54:56 <SergeyLukjanov> mattf, ack
14:55:07 <SergeyLukjanov> in RU NY holidays are Jan 1-11
14:55:39 <SergeyLukjanov> btw I'll be on meeting 8 jan, but not sure about other RU folks
14:55:45 <mattf> ok, let's table for the time being
14:57:29 <SergeyLukjanov> a few mins left
14:57:34 <SergeyLukjanov> anything else to discuss?
14:57:45 <elmiko> i just want to bring this up again, i'd be happy to have any opinions, criticism, questions, etc...
14:57:49 <elmiko> #link https://etherpad.openstack.org/p/sahara-security-guide-notes
14:58:15 <alazarev> "the next meeting i'll be at is 8 jan" - the same for me, and probably for the most of others
14:58:19 <elmiko> i'm going to try and extrapolate our default sec. position from the available guides and my own common sense, so input is good =)
14:58:26 <SergeyLukjanov> elmiko, last min promotions :)
14:58:40 <kchen> what is the clock of next meeting?
14:58:52 <elmiko> kchen: 1600UTC
14:58:55 <kchen> ok
14:59:05 <SergeyLukjanov> okay, thanks folks
14:59:15 <SergeyLukjanov> have a good day/night/smth else
14:59:18 <SergeyLukjanov> #ndmeeting
14:59:20 <aignatov> elmiko: I’ll take a look on your spec
14:59:23 <SergeyLukjanov> #endmeeting