18:04:21 <elmiko> #startmeeting sahara
18:04:23 <openstack> Meeting started Thu Sep  3 18:04:21 2015 UTC and is due to finish in 60 minutes.  The chair is elmiko. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:04:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:04:26 <openstack> The meeting name has been set to 'sahara'
18:04:30 <elmiko> #char egafford tmckay
18:04:37 <elmiko> #chair egafford tmckay
18:04:38 <openstack> Current chairs: egafford elmiko tmckay
18:05:04 <elmiko> #topic sahara@horizon status (crobertsrh, NikitaKonovalov)
18:05:05 <tmckay> but I can ... hmmm, topics
18:05:13 <tmckay> alright, elmiko :)
18:05:22 <egafford> elmiko: cool, was looking for the controls. We can take it from here; thanks.
18:05:27 <vgridnev> several horizon changes got merged
18:05:27 <elmiko> ack
18:06:12 <tmckay> egafford, forgot that elmiko was not going to be here :)
18:06:13 <crobertsrh> Yeah, some merged, some unmerged
18:06:20 <egafford> I'm going to try for an FFE on https://review.openstack.org/#/c/209683/, so anyone who has a moment to review will receive my appreciation.
18:06:21 <crobertsrh> Please review "our" stuff as much as possible.
18:06:40 <crobertsrh> Yes, especially review ^^^, I think it would be great to have.
18:06:41 <vgridnev> agreed, need more revies
18:06:53 <NikitaKonovalov> ok, It's time make a slight adjustment to out agenda I think, It will make more sense if we replace me with vgridnev for this topic because he is now doing more contribution to horizon from Mirantis side
18:07:28 <vgridnev> NikitaKonovalov, yep
18:07:44 <NikitaKonovalov> I'll make an edit to the wiki
18:08:14 <huichun> does it mean that if I want my current patch merged into L,  I should add the patch link to FFE etherpad?
18:08:14 <egafford> NikitaKonovalov: Just did. :)
18:08:35 <NikitaKonovalov> ok, thanks egafford
18:08:46 <elmiko> thanks NikitaKonovalov
18:08:51 <elmiko> er egafford
18:09:25 <elmiko> #topic News / updates
18:09:35 <egafford> crobertsrh: Good question: what is the best and most politically useful path to take for FFEs in horizon?
18:10:18 <vgridnev> egafford, e-mail to openstack-dev?
18:10:45 <crobertsrh> Yeah, that's probably the best place to start
18:10:52 <tmckay> +1
18:10:53 <crobertsrh> it keeps things official
18:11:29 <egafford> vgridnev, crobertsrh: Sure; just wondering if there were specific people we had contact with or any other communication channels.
18:11:33 <crobertsrh> Again...we need to be reviewing/using that patch as much as possible.
18:11:33 <elmiko> i'm at sec. midcycle, some nice progress on credential distribution to nodes and api fuzzing. i'll have some good topics for summit or whenever we want talk about this.
18:11:54 <elmiko> also, doc progress is moving along. we need more reviews (especially from cores), and there are still docs to pickup
18:12:07 <vgridnev> egafford, openstack-horizon probably can be used for that
18:12:36 <huichun> egafford: Hi Ethan, so I should add my patch link to the FFE etherpad?
18:13:01 <egafford> huichun: Is it a UI patch or a sahara patch?
18:13:08 <huichun> Sahara patch
18:13:38 <huichun> scheduler edp, long times, current has +2
18:16:07 <egafford> huichun: Looking into current policy; I don't know about an FFE etherpad, but sending a mail to the Sahara list is a sensible start (just like Horizon.) We all know that patch has been going for a while
18:16:14 <egafford> Other statuses?
18:16:43 <vgridnev> nothing from me
18:16:47 <egafford> I've been adding doc, reviewing doc, fixing doc, and working on tripleo integration. I think I'll (finally) have patch sets up for that by EONW.
18:17:32 <vgridnev> oh, no, finally dropped many lines of code: https://review.openstack.org/#/q/status:open+project:openstack/sahara+branch:master+topic:bp/drop-hadoop-1,n,z
18:17:38 <sreshetnyak> I'm working on wait condition support
18:18:06 <huichun> egafford:  ok I will send a email to mailing list for help
18:19:50 <huichun> ops I find that etherpad
18:20:10 <elmiko> any other news or updates?
18:20:18 <egafford> I've also been thinking about a summit working session around alternative options for image generation, to allow us to give our customers an easier/more reliable process to generate their own images (and maybe even allow us to use the same code to spin up clusters from "clean" images as we use for image generation outside of sahara.)
18:20:49 <crobertsrh> egafford:  I like it.  +1.  Certainly worth thinking about.
18:20:52 <huichun> https://etherpad.openstack.org/p/sahara-liberty-ffes
18:21:16 <egafford> Pino doesn't seem to be about today, but he's been doing some great work r/t OpenStack image generation that's worth real discussion as a team. I think it could put that process in a much better place.
18:21:37 <NikitaKonovalov> egafford: from my experience customers are also asking for the opposite, like have an image that works totally offline
18:21:46 <sreshetnyak> saharaclient and sahara liberty-3 has been released
18:22:23 <egafford> NikitaKonovalov: Yup, but to have that, and make it work for everyone, they'll still want an easier image generation flow than the one we have.
18:22:46 <egafford> (This is getting into general discussion.) Any more statuses? sreshetnyak: Cool!
18:23:14 <egafford> I guess that takes care of the "client release plans" agenda item.
18:24:11 <egafford> #topic doc days progress
18:24:33 * elmiko rifles through links
18:24:38 <crobertsrh> If everyone here took just 1 doc to look at, we'd almost be done.
18:24:45 <egafford> :)
18:24:55 <elmiko> crobertsrh: +1^e6000
18:25:09 <vgridnev> Since I was on vacations, I should take 2 or 3
18:25:12 <elmiko> #link https://etherpad.openstack.org/p/sahara-liberty-doc-update-day
18:25:19 <egafford> https://etherpad.openstack.org/p/sahara-liberty-doc-update-day: we're making good progress, but yeah, if everyone can grab just a few, it makes things nicer.
18:25:39 <huichun> Will update docs after meeting
18:25:42 <elmiko> we also need reviews, especially from cores. there are several patches that are close to merge
18:25:54 <crobertsrh> vgridnev:  Yes...you are refreshed from vacation.  2 or 3 at a minimum for you!
18:26:03 <elmiko> all in all though, we are making good progress. many thanks to everyone who is involved =)
18:26:34 <egafford> Gosh, I shudder to think how much code / how many reviews vgridnev could do when he's more refreshed than usual. ;)
18:27:21 <vgridnev> egafford, good joke
18:27:32 <huichun> elmiko: hi Michael, does that etherpad for that use? adding the patch link to it
18:27:43 <huichun> https://etherpad.openstack.org/p/sahara-liberty-ffes
18:27:53 <egafford> vgridnev: More of a compliment masquerading as a joke, really.
18:29:21 <elmiko> huichun: i think that etherpad will just be for new features which need exceptions. not docs
18:29:50 <egafford> huichun, elmiko: +1. Doc cleanup seems to be a test cycle-friendly task.
18:29:58 <elmiko> but, if you have features which need exemptions, then follow the information in #link https://wiki.openstack.org/wiki/FeatureFreeze and add it to the etherpad
18:30:09 <elmiko> egafford: agreed
18:30:56 <egafford> Any more status on this?
18:31:54 <egafford> sreshetnyak: Anything else to talk about on the client?
18:32:21 <egafford> sreshetnyak: We released it; not sure we need a full-on topic about that, unless there were issues.
18:32:41 <egafford> #topic Liberty-3 and FFEs
18:32:51 <sreshetnyak> egafford: nothing
18:32:55 <egafford> Cool.
18:33:13 <egafford> Okay, so huichun's got the one FFE request. Any others currently known?
18:33:28 <elmiko> i would like to put up an exemption for the improved secret storage
18:33:49 <egafford> elmiko: Ack; you've been clear about that for a while (which is solid.)
18:33:58 <elmiko> i have the first patch ready to go which containst the sahara key manager, castellan integration, tests, and proxy user passwords integrated
18:34:02 <vgridnev> Drop of hadoop 1 should done, I suppose
18:34:08 <elmiko> i need to create some user docs, then i will post the review as WIP
18:34:10 <sreshetnyak> from me it's ambari plugin and heat wait conditions
18:35:18 <egafford> Nothing in our projects for me.
18:35:29 <NikitaKonovalov> vgridnev: agree but I'm sure that it's a real feature, I guess none will blame us for merging this w/o a ffe
18:36:11 <egafford> NikitaKonovalov: Probably still best to file the FFE and just approve it almost automatically, so that there'll be a record.
18:36:28 <NikitaKonovalov> egafford: yep, makes sense
18:36:39 <egafford> NikitaKonovalov: Although that's a good point; if anyone has QE working this, be sure to let them know not to test anything built on Hadoop 1.
18:36:46 <vgridnev> sreshetnyak, I think that we can add wait conditions and ambari plugin to list of FFE it will be good tested
18:37:05 <vgridnev> if it will be good tested*
18:38:42 <sreshetnyak> vgridnev: I will add it to list of FFE
18:38:52 <egafford> Anything else here? Good to know the likely set; let's try to get those filed on the list by next meeting.
18:39:22 <tmckay> nothing from me.  egafford, thank you for hosting.  I am in a multi-day fight with openstack and I (hope) am about to win
18:39:31 <huichun> hi all, I am preparing the topic for Sahara EDP log improvement, so do you think I can make it as a topic @submmit,  currently Sahara EDP has no log detail info for the end users, and sometimes user can not debug his own jar file, just know the result of job, not why failed, he need to login on the VM and find the log himself
18:39:57 <egafford> tmckay: I feel your pain. I'm almost at the end of the Overcloud War of 2015. Fight on, my brother.
18:40:17 <egafford> huichun: Yeah, this is really important.
18:40:33 <tmckay> huichun, sounds good to me.  We can cover it at the meetup if not before
18:40:52 <NikitaKonovalov> huichun: do you want to make job logs available through the API?
18:41:30 <huichun> NikitaKonovalov: yes
18:41:49 <egafford> Okay, let's burn through some other agenda items and come back to this in open discussion, if that's okay with everyone. Very important, though, huichun.
18:41:50 <huichun> and different log level
18:42:02 <egafford> #topic Sahara / Zaqar integration for guest agents implementation
18:43:02 <elmiko> imo, we need to setup time to talk with the trove team at summit for the purpose of discussing guest agents and ssh access to worker nodes
18:43:16 <egafford> elmiko and I had a chat this AM with the Zaqar PTL about an overview of what we're hoping; it sounds like tenant isolation and signed URLs will give us a lot of the sec benefits we're looking for if the sec working group is for the overall approach.
18:43:32 <egafford> elmiko: Ack, +{1, 2, any positive number}
18:43:35 <elmiko> they are considering moving to ssh, and we are considering guest agent. we could definitely do some "lessons learned" each way
18:43:50 <tmckay> that is funny
18:44:01 <elmiko> yea
18:44:03 <egafford> We have a lot of the same use cases and problems, and it'd be great to converge on an architecture (and if possible, on common code.)
18:44:18 <elmiko> +1
18:44:26 <NikitaKonovalov> +1
18:45:12 <egafford> So yeah, Zaqar/Trove/Sahara three-way meeting at summit seems to be the exact right answer.
18:45:25 <egafford> elmiko: From sec mid-cycle, any discussion of this set of concerns yet?
18:46:56 <egafford> Okay, as of this AM there hadn't been much. In the interests of at least 10 minutes of general discussion:
18:46:57 <egafford> #topic Future of Sahara code inside the Horizon
18:47:10 <egafford> crobertsrh, vgridnev: News here?
18:47:21 <crobertsrh> No news
18:47:23 <egafford> (Is this topic needed in the meeting any more?)
18:47:29 <crobertsrh> I'm still +1 for moving out
18:47:33 <vgridnev> egafford, yep
18:47:41 <crobertsrh> Also +1 for not needed in this meeting.
18:47:51 <egafford> vgridnev: Cool.
18:48:01 <egafford> #topic  Ceilometer at the gate
18:48:08 <crobertsrh> If I get some spare cycles, I'll experiment with having our bits in our own repo.
18:48:19 <egafford> crobertsrh: +a lot really.
18:48:36 <vgridnev> elmiko, any news in this topic?
18:49:03 <NikitaKonovalov> crobertsrh: again, I guess Trove UI is suffering from the same issues, sharing some experince might be usefull
18:49:25 <elmiko> nothing from me, i haven't been following ceilometer aside from the email that cdent sent
18:49:29 <crobertsrh> NikitaKonovalov:  Yes.  I'm also talking to Trove people and working on their reviews.
18:49:52 <egafford> Okay, fair enough.
18:49:57 <egafford> #topic Open Discussion
18:50:05 <egafford> Since we seem to be here anyway. :)
18:50:36 <egafford> huichun: I absolutely support a session about debugging information retrieval and presentation, for both EDP and cluster provisioning.
18:51:04 <egafford> Logging into your cluster to debug may not even be possible in all use cases.
18:51:12 <huichun> yes current sahara drop edp log to the specific engine like oozie
18:52:02 <huichun> we have event log for cluster provisioning, right?
18:52:22 <vgridnev> huichun, yep
18:52:39 <NikitaKonovalov> huichun: yes we have two-level log for steps/events
18:52:48 <egafford> I think we're likely to talk more formally about working sessions for summit soon, but I look forward to hearing your thoughts. :) We do, yes, but does that dig into, say, cloudera manager logs, or hdp logs, or the places where things are likely to break?
18:53:29 <huichun> and for edp log, I think we need log level too, INFO or ERROR, for INFO, user can see all the execution detail, sometimes the execution content is more different than the result of successful
18:53:37 <egafford> That's the kind of information customers almost always need to fix problems on cluster deployment. We're very good now at telling the user when it broke, but not often why.
18:54:36 <egafford> (Any other topics people want to discuss today? We've only got about 5 minutes.)
18:55:44 <huichun> egafford:  currently I am thinking digging log for specific engine like oozie and spark
18:55:47 <crobertsrh> nothing from me
18:55:49 <vgridnev> #link http://lists.openstack.org/pipermail/openstack-dev/2015-September/073606.html
18:55:54 <vgridnev> just to confirm
18:56:25 <egafford> vgridnev wins the FFE request race!
18:56:30 <egafford> (Thanks!)
18:56:33 <huichun> egafford: specific engine has the detail log
18:57:18 <egafford> huichun: Makes a lot of sense. We can probably make the mechanism reusable for both cluster provisioning and EDP, at least at a high level.
18:57:54 <huichun> egafford:  good idea, I will prepare more for this topic
18:58:01 <egafford> huichun: Sounds great to me!
18:58:02 <tmckay> yes, we've talked about this here and there for a long time
18:58:12 <tmckay> I think we should definitely make it a priority in the M cycle
18:58:25 <tmckay> robustness and usability, over new features (my opinion)
18:58:30 <egafford> tmckay: Yup. Hopefully M can be the Cycle of User Experience.
18:58:39 <tmckay> that would be really, really great
18:58:42 <egafford> tmckay: Very hearty +1.
18:58:48 <elmiko> 2 minutes left
18:58:49 <huichun> +1
18:58:59 <egafford> elmiko: Ack. Any last words for the meeting?
18:59:10 <elmiko> nothing from me
18:59:16 <tmckay> error reporting in general --
18:59:17 <egafford> That was to the assembly, not necessarily to elmiko.
18:59:20 <elmiko> wait.. review docs!
18:59:24 <egafford> :)
18:59:31 <egafford> Okay, thanks all!
18:59:32 <tmckay> for instance, I just launched a cluster but didn't have heat_stack_owner role
18:59:40 <tmckay> no indication in horizon at all
18:59:44 <tmckay> bye
18:59:48 <egafford> #endmeeting