18:01:55 #startmeeting sahara 18:01:56 Meeting started Thu Jun 11 18:01:55 2015 UTC and is due to finish in 60 minutes. The chair is alazarev. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:59 The meeting name has been set to 'sahara' 18:02:13 #topic sahara@horizon status (crobertsrh, NikitaKonovalov) 18:02:15 \o 18:03:11 alazarev, so Sahara has a meeting at this time, right? 18:03:34 ok, Sergey told me that there are some changes in reviewing policy 18:03:46 weiting, yes, 1 week - this time, 1 week - yearly 18:04:20 our panels, as well as trove's and heat's will be moved to contrib folder 18:04:38 and there our non-core +1 count 18:05:07 if that happens, we'll get things moving faster I think 18:05:29 NikitaKonovalov: That sounds amazing. 18:05:34 I've got no more updates on horizon actually )) 18:05:52 #topic News / updates 18:06:36 I've started working towards splitting sahara and sahara-plugins repos 18:06:39 working on scheduler EDP job , need for review for spec https://review.openstack.org/#/c/175719/ 18:07:35 We are also working on to bring Manila into Sahara EDP. 18:08:14 We had a hdfs driver that has already implemented in Manila. 18:08:26 weiting, sound great! 18:08:56 hello/ 18:09:03 hi folks 18:09:16 Job interface main patch is review-worthy; working on client changes now (then scenario tests, then smaller feature upgrades, then Horizon changes, then profit.) 18:09:44 hey folks :) 18:09:46 weiting: so another type of data source? 18:09:51 just arrived to home, completely forget about meeting :( 18:09:53 I have patches up for the service and client library parts of job binary and data source editing. Still need both specs to be approved and the job binaries part may need a few more tweaks. 18:10:06 Excited for new Horizon review policy for those Horizon changes, as https://review.openstack.org/#/c/160510/ is still in review from Kilo. :p 18:10:36 tmckay: hi Trevor ,need your review for scheduler edp spec 18:10:37 tosky, yes, this is what we would like to do. But currently Manila community is asking for a CI for hdfs driver. 18:11:10 huichuin, okay. I've been fighting with neutron on a test cluster :) Takes lots of attention 18:11:20 But we don't have the experience about CI. So we are asking for help to handle this. 18:11:58 If there is no CI for this driver, it may be removed in Manila. 18:12:00 weiting, we're thinking about Manila integration, too. We think it can also potentially work with NFS shares (because Hadoop can write to local filesystem) 18:12:34 tmckay, yes I agree. 18:12:44 2 applications, we think. 1) data, of course 2) code -- so that binaries can live in the cluster and not be copied 18:12:49 weiting, are you exposing already installed HDFS through Manila or Manila could install HDFS? 18:13:06 weiting, we should make sure we collaborate :) 18:13:40 tmckay, ok, we haven't propose the bp. Because we are struggling in Manila CI. 18:13:40 weiting, re CI you could chat with degorenko, he's now sahara-ci maintaine 18:14:03 SergeyLukjanov, yes, we have already finish HDFS driver in Manila. 18:14:20 weiting, understood. We are at the POC point -- what is possible? What makes sense from a high-level view in Sahara? 18:14:37 weiting, I mean - how it's working? it's exposing HDFS through Manila or it installs HDFS? 18:15:13 weiting, are thinking is maybe list available shares in a tenant and have a Sahara button to mount the share on the cluster nodes (from the UI) 18:15:54 SergeyLukjanov, I think it is installs HDFS. 18:16:11 weiting, hm 18:16:44 tmckay, we could deploy a separated HDFS-only cluster for it as an option 18:17:31 SergeyLukjanov, yes, that's another possibility 18:18:36 anyway, we should have some ideas over the next few weeks. Just starting. 18:18:58 https://review.openstack.org/#/c/151523/13/manila/share/drivers/hdfs/hdfs_native.py,cm here is an impl 18:19:14 looks like it just creates dirs in HDFS as a shares 18:19:38 and sets proper (not sure) ACLs for them 18:19:48 so, the HDFS itself seems to be installed manually 18:20:38 SergeyLukjanov, oh I got your point. 18:21:06 #chair SergeyLukjanov 18:21:07 Current chairs: SergeyLukjanov alazarev 18:22:51 any other updates? 18:23:31 SergeyLukjanov, https://review.openstack.org/#/c/190656/ looks like it will pass but jenkins is complaining (wrongly) I think 18:23:36 btw there is now eavesdrop site reworked - http://eavesdrop.openstack.org 18:23:51 Since we are going to have separate plugins repo I think should drop some old or deprecated plugins 18:24:25 both vanilla and hdp now have hadoop v1 support which is not popular at all 18:24:26 tmckay, yup, rechecked 18:24:43 NikitaKonovalov, please write a spec to drop them 18:25:07 and I should publish first draft of the plugins specs early next week 18:25:15 ok, I'll add that point to this one https://review.openstack.org/#/c/190613/ 18:25:47 NikitaKonovalov, no, please, create a separate spec for Hadoop 1 drop 18:26:02 @SergeyLukjanov: ok 18:26:05 and 190613 now missing ~ 100 details, so, we'll need to work on it 18:26:52 #topic Open discussion 18:27:13 I have a question to you folks to think about 18:27:20 do we need to have a virtual sprint? 18:28:04 in fact it's few dedicate days when we're all will chat in irc channel on the prepared topics trying to solve real issues 18:28:19 like the friday day on summit but using IRC 18:28:26 some projects doing it 18:28:30 so, folks, what do you think about it? 18:28:37 SergeyLukjanov: For design issues or impl? 18:29:13 egafford, for anything that could be solved by the whole team presence 18:29:28 We could probably give it a try at least once 18:30:03 probably we should try to collect list of topics to discuss 18:30:19 and if we'll have some reasonable list - make a try 18:30:31 any other thoughts? :) 18:30:33 It would be good I think to try to have some blueprints and specs ready for topics 18:30:56 SergeyLukjanov, I wonder if bluejeans would work instead of IRC 18:31:01 here is a wiki page about it with a links to the past virtual sprints - https://wiki.openstack.org/wiki/VirtualSprints 18:31:42 I think talking is faster than typing 18:32:10 but maybe connection speed is an issue 18:33:29 tmckay: BJ would work really well for untangling particularly tricky issues among subteams working on one part of a problem, but the whole team in BJ sounds tough to manage. 18:33:49 tmckay: Maybe IRC and break into ad-hoc BJ sessions as needed would be the ideal? 18:34:18 maybe 18:34:32 Have we ever considered a midcycle meetup? 18:34:40 We'd lose some centralization of the record, which is a downside. 18:34:54 mini-Summit 18:35:12 egafford: unless someone types the minutes on an etherpad 18:36:07 what is the BJ? 18:36:21 BeiJing? 18:36:39 SergeyLukjanov: It's like Skype. Video, voice, chat, screen sharing. 18:36:47 All the communication. :) 18:36:50 egafford, nice 18:37:08 as an option we could use openstack's audio conference 18:37:22 it supports callins either phone and ip 18:37:42 BJ is bluejeans 18:37:49 google hangouts? 18:37:54 openstack audio conf would be very "openstacky" of us 18:38:05 that could work too (hangouts) 18:38:54 I envision audio like our Friday sessions at Summit. We have a lot of people, but it's manageable I think 18:39:41 Our Friday session this time around was certainly not "a lot" of people 18:40:04 well, 10 maybe? at least before lunch ;-) 18:40:09 Fortunately for me, that means that not many people got to see me try to draw something on the whiteboard. 18:40:11 then someone told me it was over 18:40:41 crobertsrh, :) 18:41:18 so, we could evaluate for some more time pros and cons for running virtual sprint 18:44:22 SergeyLukjanov: I don't really see that there are a ton of cons to the idea of running it. It would certainly pull focus to one issue, but that can be good. It'd need to be focused on issues broad enough to use the whole team, which is fine. It'd bring the whole team together to work, which is great. 18:45:00 SergeyLukjanov: Only downside I can see is that it might be hard to schedule around any standing responsibilities folks have, but that's just the nature of things. 18:45:26 agree 18:45:40 SergeyLukjanov: It might not work for us, but experimentation is good. 18:48:04 yeah, it could make us able to actively (and hopefully effectiely) discuss some complex things 18:48:58 SergeyLukjanov: I guess the other downside is that if it's not prepped well enough, folks might not have the relevant knowledge on tap to start in quickly, but that can be addressed with planning. 18:48:59 anything else to chat about today? 18:49:28 SergeyLukjanov: On this topic, do you have any ideas for virtual sprints for us? 18:49:48 egafford, yeah, it's the common downside for any events like that 18:50:09 egafford, I'm thinking about discussing specs like extracting plugins, etc. 18:50:26 some complex stuff with many issues 18:50:56 and on the other side we could probably chat about some on-going things and make some things done like small improvements and etc. 18:51:02 no good concrete proposals 18:51:18 Yeah, that's a huge one, and makes a lot of sense. Taskflow from Summit springs to mind for me, too. Cool; thanks. 18:51:27 One thing that is needed is better tests for each panel in Horizon. I added a blueprint to cover adding additional tests. Anyone is free to help out on that effort as you have time. 18:51:38 Current panel tests in Horizon are rather skeletal. 18:51:39 I've just looking on how other projects doing the virtual sprints and decided to share thoughts on it to see what are folks thinking about it 18:52:26 crobertsrh, sounds like you're requesting a few weeks long hackaton :) 18:52:43 Well, in case anyone was bored and wanted to poke at Hoirzon :) 18:52:53 It's not like the patches will ever get merged anyways :) 18:53:02 hi guys~~ need your review comments for scheduler edp job https://review.openstack.org/#/c/175719/ 18:53:20 probably the new review policy will help us and both crobertsrh and NikitaKonovalov will be happy 18:53:36 hopefully 18:54:15 / 5 mins left 18:57:04 okay, looks like time to sleep 18:57:12 I mean to continue working :) 18:57:17 thank you folks! 18:57:28 (please keep thinking about virtual sprint options) 18:57:36 bye 18:57:37 Yes, we are going to sleep in China. 18:57:40 have a good rest of day 18:57:45 #endmeeting