14:02:33 <tenobreg> #startmeeting sahara
14:02:34 <openstack> Meeting started Thu Nov 16 14:02:33 2017 UTC and is due to finish in 60 minutes.  The chair is tenobreg. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:02:38 <openstack> The meeting name has been set to 'sahara'
14:02:47 <jeremyfreudberg> o/
14:02:48 <tenobreg> sorry guys, I'm a little crazy this morning
14:02:50 <tosky> o/
14:02:54 <shuyingya> o/
14:03:03 <shuyingya> haha :)
14:03:09 <jeremyfreudberg> btw, i'll be leaving in about 20 minutes
14:03:13 <jeremyfreudberg> classes and such
14:03:23 <tenobreg> #topic News/Updates
14:03:27 <tenobreg> jeremyfreudberg: start quick
14:03:41 <jeremyfreudberg> well, biggest news from me is that I finally put some new S3 patch up
14:03:51 <jeremyfreudberg> please review it, I will be trying it out myself soon
14:04:18 <tenobreg> awesome
14:04:26 <tenobreg> I will look at it today
14:04:45 <tenobreg> I'm currently working on MapR image generation and hopefully will start working on decommission of specific node soon
14:05:27 <tosky> I'm working on downstream testing but keeping an active eye on the zuulv3 migration
14:05:33 <tenobreg> about MapR I have a question for all you, not sure if you guys saw how the whole process work, but we basically have some validators that get executed to instrument the image
14:05:37 <shuyingya> Recently I am working on test the performance of sahara and compare to EMR. and also working on baremetal support for downstream
14:05:38 <tosky> which is still tracked at https://etherpad.openstack.org/p/zuulv3-sahara
14:06:25 <tenobreg> I'm facing an issue with MapR because it copies some scripts to the image that should not be executed, only copied because they are used by other scripts
14:06:39 <tenobreg> the way the script validator works now is, it copies the scripts and it executes it
14:06:55 <tenobreg> I didn't find a way to tell the validator not to execute the script
14:07:44 <jeremyfreudberg> tenobreg, i'm not looking at it right now, but how does it find executable scripts? anything which is chmod +x (I would assume)
14:07:55 <jeremyfreudberg> if so, then you can make the scripts you don't want to run not-executable
14:08:09 <tenobreg> what do you guys think is a better way to go? force copy this at some point (still to be found) or add a new validator that simply copies the files and we can add them on the image.yaml file
14:08:42 <tenobreg> jeremyfreudberg: that would work if we actually looked for files, but what we do is, copy a file, execute chmod +x and execute the script, all in the same loop
14:08:43 <jeremyfreudberg> the second choice you just mentioned sounds okay too
14:09:04 <tenobreg> jeremyfreudberg: that is my prefered option as well
14:09:16 <tenobreg> just wanted to get some other perspective as well
14:09:43 <tosky> if the copied files have a different purpose, it makes sense to differentiate between copying the scripts that needs to be executed and copying the others
14:09:47 <tosky> somehow
14:10:22 <tosky> so either two validators, or the validator should know which scripts need to be executed and which not
14:10:24 <tenobreg> awesome, I will try and add a new validator and see how it goes
14:10:38 <shuyingya> agree with jeremy, keep it simple
14:11:07 <tenobreg> tosky: I would love to have one validator that we could pass the not executable variable to it
14:11:27 <tenobreg> but I don't see how that can be done the way it is structured now
14:11:46 <tenobreg> maybe it is something to be improved on the image gen/validation later
14:13:02 <tenobreg> I guess this is all for me
14:13:13 <tenobreg> shuyingya: any updated from your side?
14:14:34 <tenobreg> updates*
14:14:47 <shuyingya> this week I am working on baremetal support for downstream. not finish yet. And also working on test the performance of hadoop on cloud.
14:15:47 <tenobreg> shuyingya: both pretty cool. Are the results of the hadoop performance private to your company or can you share it?
14:15:58 <shuyingya> Hbase support is on the way.
14:16:10 <shuyingya> for sure, I can share with you
14:16:54 <shuyingya> and If you guys have some ideas for how to test it or what feature you may concern, please tell me
14:17:01 <tenobreg> shuyingya: awesome, that would be great.
14:17:12 <tenobreg> I will think of scenarios and send it to you
14:17:28 <tenobreg> we can discuss it later (not today because it is too late for you)
14:17:53 <shuyingya> thanks, tenobreg.
14:18:09 <tenobreg> lets move on
14:18:26 <tenobreg> #topic OpenStack Summit Sydney
14:18:56 <shuyingya> do we get some feedback from the forum?
14:19:11 <tenobreg> juts a quick update on what happened last summit: basically I presented the project update, we had a good attendance this time
14:19:36 <tenobreg> I got contact of two interested parties on Sahara, but I haven't followed up with them after the summit
14:19:51 <tenobreg> we had an interesting question about usage of GPU
14:19:59 <tenobreg> it may be something worth looking at
14:21:01 <shuyingya> sounds cool.
14:21:19 <tenobreg> also I talked with jeremyfreudberg's boss about sahara-ci and I will send her an email with our requirements (I will send to you all so you make sure it is ok)
14:21:37 <shuyingya> Sydney summit looks like lots of topic is related to k8s
14:21:52 <tenobreg> apparently she has the power to get us some machines
14:22:02 <shuyingya> wow
14:22:39 <tenobreg> shuyingya: containers has been the topic for two or three summits now. This time was k8s' time. But I believe that the main topic of this summit was edge computing
14:23:00 <jeremyfreudberg> tenobreg, yep, hopefully it works out, we all know bare metal is tricky request to satisfy but i think there is a good chance
14:23:13 <tenobreg> yeah, I'm very hopeful this time
14:23:42 <tenobreg> I have to calculate how much resource we need from ours scenarios tests
14:23:59 <tenobreg> but from what I remember we need to 64gb, 8 cores  machines
14:24:09 <tenobreg> need *two
14:24:11 <jeremyfreudberg> that's my recollection as well
14:24:25 <jeremyfreudberg> i remember evgeny saying we could cram everything into two machines
14:24:29 <tenobreg> just need the math so we have a justification for it
14:24:48 <tenobreg> so it doesn't seem like a magic number
14:24:51 <tosky> shuyingya: in case you didn't notice, RDO (and RH-OSP) can have containerized OpenStack services (including sahara-api and sahara-engine), but maybe you were talking about the opposite (spawning containers)
14:25:18 <tosky> (since Pike)
14:25:37 <jeremyfreudberg> tenobreg, yep, any math you can do would help us a bunch
14:25:58 * tenobreg hopes he remembers how to math
14:26:05 <tenobreg> :)
14:26:14 <jeremyfreudberg> btw, i need to sign off now, two things to think about are 1) new saharaclient release for template export for dashboard 2) apiv2 bindings in saharaclient
14:26:35 <jeremyfreudberg> but, anyway, logging off now, i will read the transcript later
14:26:50 <tenobreg> thanks jeremy
14:26:56 <tenobreg> lets move on
14:27:06 <tenobreg> #topic Open Discussion
14:27:34 <tenobreg> about new saharaclient, we just landed the export of NGT
14:28:07 <tenobreg> I will take a look to see if there are any hanging patches, if not I will request a release of the client by the end of the week
14:28:22 <tenobreg> #action tellesnobrega to request release of saharaclient
14:28:49 <tenobreg> about apiv2 binding in saharaclient, I'm not sure I understand the issue
14:29:08 <tenobreg> tosky or shuyingya do you know what he meant by that?
14:29:21 <tenobreg> are they missing? needing updates?
14:29:40 <tosky> do we have them?
14:29:41 * tosky checks
14:29:45 <shuyingya> tosky, thanks for your notice. What I am going to do next stage is to do openstack(sahara) on k8s.
14:29:52 <shuyingya> let me check
14:29:58 <tenobreg> not sure
14:31:00 <tosky> saharaclient/api/client.py contains only a "Client for the OpenStack Data Processing v1 API."
14:31:29 <tenobreg> ok, if they are completely missing, we need to get a jump and start putting apiv2 there
14:31:52 <tenobreg> I can't go to the project update a 3rd time and tell apiv2 is not release yet lol
14:32:06 <tenobreg> they record that stuff
14:32:28 <tenobreg> I will dig deeper, see what needs to be done and hopefully find someone to do it or do it myself
14:33:38 <tosky> yep, and maybe first make it easier to enable v2 so that we can setup at least the unittest job
14:33:54 <tenobreg> sure
14:34:04 <tenobreg> I will take a look on that too
14:35:06 <tenobreg> do we have other topics for today?
14:36:54 <tosky> just to be clear: you will need an easier way to setup the API v2 service anyway, in order to test the v2 clients
14:36:58 <tosky> that said, no other topic
14:37:35 <tenobreg> tosky: sure, I will start testing it and see how it can be improved
14:38:06 <tenobreg> Ok, I will end the meeting, we get 23 minutes back to do real work :)
14:38:18 <tenobreg> thanks all for participating
14:38:34 <tenobreg> see you all next week
14:38:38 <tenobreg> #endmeeting