18:00:13 #startmeeting sahara 18:00:14 Meeting started Thu Jan 8 18:00:13 2015 UTC and is due to finish in 60 minutes. The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:15 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:17 The meeting name has been set to 'sahara' 18:00:26 hello 18:00:39 I'm on vacation / holidays, so partially available ;) 18:00:41 #help 18:01:02 #chair elmiko 18:01:02 Current chairs: SergeyLukjanov elmiko 18:01:14 #link http://wiki.openstack.org/wiki/Meetings/SaharaAgenda 18:01:20 we appreciate your dedication =) 18:02:08 #topic horizon status 18:02:11 Still some reviews lingering in horizon (usual status). 18:02:33 I've put some thought and some trial code into a "wizard". 18:02:36 crobertsrh, yeah 18:02:47 crobertsrh, and how is it going? 18:02:49 hi 18:02:50 I will be meeting with a couple UX people tomorrow. Hopefully, they will be able to give some guidance. 18:03:03 crobertsrh, do we need completely custom code or could reuse some horizon stuff? 18:03:14 Not great so far really. I've decided to scrap my original approach.....on to new ideas now :) 18:03:28 My new idea will hopefully allow us to reuse a lot of what we have. 18:03:43 okay 18:03:50 anything else re horizon? 18:03:51 At least, if we do refactor some of what we have, it will be used both in and outside of wizard 18:04:04 Of course, UX people may have other better ideas. 18:04:07 crobertsrh, ack, it'll be great 18:04:11 Nothing else I can think of. 18:04:22 #topic News / updates 18:04:25 folks, please 18:04:42 (ru holidays are Jan 1-11) 18:04:43 Specs for "default templates" and "edit templates" have been merged. 18:05:05 i've been continuing work on the security doc, and researching some options in terms of using barbican. also working on a few bugs and many reviews. 18:05:09 #link https://etherpad.openstack.org/p/sahara-security-guide-notes 18:05:12 Cloudera confirm they can hold the cdh image on their website with a EULA. 18:05:13 crobertsrh, I hope to propose ACL spec next week 18:05:24 i could use any opinions/adivce/suggestions/criticisms on that etherpad please 18:05:30 Great! 18:05:38 weiting, so, it means that automation will not work? 18:05:45 ack elmiko: I've been meaning to look at that. 18:06:29 current focus on adding more integration test case on CDH plugin https://blueprints.launchpad.net/sahara/+spec/add-more-cdh-integration-tests 18:06:42 What do you mean "automation"? 18:07:46 weiting: i think he means automating the download of the image would not be possible with the EULA 18:08:23 weiting, wget 18:08:29 Oh, yes. There is still a EULA that the end user must accept it. 18:08:49 So it shouldn't support automation. 18:09:07 Any concern about that? 18:09:33 i think the concern is that sahara-image-elements may run in an automated mode, which would not be possible with the EULA 18:10:24 or that the image may be downloaded automatically 18:12:02 SergeyLukjanov: is there further concern about the EULA? 18:12:04 but at least we'll be able to publish link in docs to it 18:13:05 any more news or updates? 18:13:24 Yes, the image from Cloudera website should be used for publish purpose 18:13:57 IMO Bug / doc / spec days should be discussed on the next meeting when ru folks will be available 18:14:18 +1 18:14:24 and I don't see any more active agenda items 18:14:38 #topic Open discussion 18:15:33 #action discuss bug/doc/spec days at Jan. 15 meeting 18:15:34 need we manage the cdh versions? 18:16:19 I registered a bp on this. I think currently cdh version management is confusing. 18:16:30 hey there, I have an open discussion item 18:16:36 yes, i think that bp looks good 18:16:48 kchen, +1 18:16:49 It's been a while since we talked about how to include hadoop-openstack.jar 18:17:01 (or hadoop-swift.jar, however it's named_ 18:17:10 kchen, AFAIK sreshetniak was planning to add some versions managemens to CDH plugin 18:17:23 tmckay: hadoop-openstack 18:17:35 originally we had our jar injected into hadoop 1 and hadoop 2 from the mirantis site 18:17:39 tmckay, job is partially ready, I'll complete it next week (after holidays) 18:18:13 SergeyLukjanov, okay. To summarize: right now it is injected as an element for hadoop 1, but not for hadoop 2 18:18:17 tmckay, and it'll be published on tarballs.o.o/sahara/hadoop-swift/hadoop-swift-latest.jar 18:18:33 tmckay, hm 18:18:44 SergeyLukjanov, so will we add it back as an element for hadoop 2? 18:18:57 i think we should 18:18:58 so it means the version management for cdh is already ongoing? 18:18:59 dmitryme removed it last April since Hadoop already had a jar included :) 18:19:52 I found this btw because I am working on fixing up DIB for spark to include hadoop-openstack.jar in the image 18:20:01 for swift support 18:20:39 another hurdle, fyi, is that we need to change the CDH version or we need to include a jackson jar somehow to fix an incompatibility 18:20:44 kchen, I think you should ping sreshetniak about it, I think he has some ideas about it by not started implementing it 18:21:34 ok. I will ping him on this topic. 18:22:05 SergeyLukjanov: kchen has a bp up for it though, maybe sreshetniak could comment as well 18:23:57 sorry, I'm late, thought it is in 11am 18:24:29 alazarev: no problem, any topics? 18:25:34 not from my side, I was busy with SSL stuff and configs clean up 18:26:00 i'd like to just bring up the security guide again 18:26:01 plan - multi workers for API and signals support 18:26:33 alazarev, multi workers? 18:26:56 SergeyLukjanov, https://review.openstack.org/#/c/145601/ 18:27:42 SergeyLukjanov, if we want signals support - let's do it right ;) 18:28:06 alazarev, /me looking 18:28:08 +1 for doing it right 18:28:54 alazarev, hm, we're running API in threads 18:29:03 alazarev, it's done by flask / eventlet 18:29:15 alazarev, and it's not wsgi app 18:29:52 alazarev, and your spec is actually talking not about wsgi workers 18:29:56 SergeyLukjanov, no, I tried, hanging of one API request leads to complete API stop 18:30:46 alazarev, it was running in green threads some time ago 18:31:35 SergeyLukjanov, green threads is not a production ready solution 18:31:54 SergeyLukjanov, other projects use green threads for debug only 18:32:03 alazarev, the whole openstack is based on greenthreads 18:33:53 SergeyLukjanov, we just run wsgi.server without any threads, sleep in any api handler leads to stoping API handling 18:34:32 alazarev, we could spawn a set if API processes like it's done in some openstack projects 18:34:43 alazarev, but the most correct way is to support wsgi 18:35:13 SergeyLukjanov, I like how it is done in heat 18:35:18 alazarev, flask docs re wsgi in apache - http://flask.pocoo.org/docs/0.10/deploying/mod_wsgi/#mod-wsgi-apache 18:35:23 alazarev, link? 18:36:29 https://github.com/openstack/heat/blob/master/heat/common/wsgi.py#L248 18:39:02 heat's solution looks interesting 18:39:10 alazarev, yeah, it looks nice, but it's not a support for wsgi, it's just an eventlet wsgi server and green threads 18:39:20 alazarev, so, as you say it's not production ready :) 18:39:51 they don't use green threads 18:39:57 they fork processes 18:40:23 https://github.com/openstack/heat/blob/master/heat/common/wsgi.py#L328 18:40:34 green threads for debug only 18:40:35 alazarev, oops, bad wording, I mean that it's still evenlet's wsgi 18:41:17 alazarev, and I think that it's a bad idea to copy-paste their wsgi module 18:41:31 all openstack use eventlet wsgi, we have nothing to do with this 18:42:12 SergeyLukjanov, not all module, just Server class 18:43:02 alazarev, some projects supports real wsgi 18:43:05 alazarev, keystone at least 18:43:52 SergeyLukjanov, I'll take a look 18:44:10 alazarev, I think all services are using https://github.com/openstack/oslo-incubator/blob/master/openstack/common/service.py 18:44:16 ~15 min left 18:44:35 alazarev, and it's maintained now - https://github.com/openstack/oslo-incubator/blob/master/MAINTAINERS#L173 18:45:03 alazarev, I prefer using https://github.com/openstack/oslo-incubator/blob/master/openstack/common/service.py as common and maintained code 18:45:20 what happened with common.wsgi? They removed it some months ago 18:45:26 alazarev, https://github.com/openstack/oslo-incubator/blob/master/openstack/common/service.py#L309 18:46:03 alazarev, it was removed about year ago I think, it was depricated and we're using only json serializer from it AFAIK 18:46:12 SergeyLukjanov, yeah, look similar, I agree that using common code is better 18:46:58 alazarev, more than that, it looks like oslo's impl is much more flexible and it's used at least in nova 18:48:39 alazarev, okay, so, sounds we agreed you to try using oslo-incubator/service 18:48:51 SergeyLukjanov, yeap 18:49:02 folks, anything else? 18:49:06 yes 18:49:14 just wanted to bring up sec. guide one more time 18:49:22 alazarev, I'll review the spec itself early next week 18:49:23 more reviews for https://review.openstack.org/#/c/133590/, plz ;) 18:49:39 i could use any extra advice or opinions on our position in terms of how we recommend our users to secure their installations 18:49:39 elmiko, added to the reading list 18:49:46 SergeyLukjanov: thanks 18:50:06 elmiko, only ideas about how insecure it 18:50:17 lol 18:50:20 speaking of that 18:50:35 i want to start investigating if we could use the OSSG Bandit project 18:50:39 #link https://wiki.openstack.org/wiki/Security/Projects/Bandit 18:50:57 it's a static code analysis tool to help identify security weaknesses 18:51:27 elmiko, +! 18:51:43 seems kinda interesting 18:52:07 ok, that's it from me 18:54:16 elmiko, could you run bandit on sahara code and share results on the next meeting, please? 18:54:30 SergeyLukjanov: if i can get to it, yes =) 18:54:40 #action elmiko to run bandit against codebase 18:56:10 elmiko, thx 18:56:15 ok, 4 mins left 18:57:59 Hopefully, next week, I'll have some sort of proposal for what our wizard/guided mode might look like. I may have a few options depending on what input I get from UX people. If anyone has any ideas, please let me know. 18:58:14 crobertsrh, cool! 18:58:22 Or just feel free to shoot down whatever I come up with :) 18:58:30 lol 18:58:48 okay, thanks folks 18:58:52 #endmeeting