Wednesday, 2016-07-20

openstackgerritTakashi Kajinami proposed openstack/storlets: WIP: Refactor daemon factory
*** cdoron has joined #openstack-storlets09:39
*** cdoron has quit IRC12:59
*** cdoron has joined #openstack-storlets13:00
eranrom_hey Doron\13:01
eranrom_kota_: How was the Hackathon?13:01
kota_eranrom: that was so great.13:02
kota_we had a lot of progressive conversation there.13:02
eranrom_kota_: Any new companies involved?13:03
kota_ah, i don't think new companies there but I met some of new contributors13:04
eranrom_kota_: nice! Do you know if Takashi will join today?13:04
kota_almostly rackers (including OSIC) and guys from IBM, maybe?13:05
eranrom_kota_: I guess the go stuff brought some new faces...13:05
kota_he said he might be absent or delayed. but never mind to start w/o him.13:05
eranrom_So Doron joined us for the beginning of the discussion, so would like to start with his topic.13:06
cdoronhi all13:06
cdoronI have recently experimented with a script for setting up the storlet engine on DEVSTACK13:07
cdoronusing vagrant13:07
cdoronWhich got me thinking about whether we can use this, or something similiar, in the storlets project13:07
cdoronOne question could be, whether we want to continue running the storlet tests on an all-in-on Swift13:08
cdoronor do we want to replace all-in-one with devstack?13:08
cdoronAnother question (orthogonal question):13:08
cdoronDo we want to run the test against a swift installation on a docker container13:09
cdoronrather than on the all-in-one13:09
cdoron(in the docker container we could either have devstack13:09
cdoronor an all-in-one swift)13:09
kota_so can I make sure the mean of all-in-one?13:10
cdoronI mean the current swift installation13:10
kota_you mean vagrant-swift-all-in-one or an instance result from s2io setup?13:10
cdoronwhich uses the scripts to install on the current machine13:10
cdoronI meant the s2io setup13:11
cdoron(I am not sure we can run vagrant on a VM, so let us forget about vagrant for the moment)13:11
cdoronIf we have a script for installing swift+storlet-engine on a docker container, people could use it to easily experiment with the storlet engine13:13
cdoronIt is less intrusive than installing swift on a regular machine13:13
eranrom_So I think we have two orthogonal questions which can be translated to perhaps 4 ways to get an s2aio like deployment:13:14
eranrom_{docker, vm} X {devstack, current swift ansbile install}13:14
cdoronvm == jenkins vm ?13:15
eranrom_well, or a dev end that a developer sets for himself13:15
eranrom_I am assuming all 4 can be tested from jenkins13:16
eranrom_kota_: do you know whether Swift today has any jenkins / gate jobs that rely on something else form SAIO?13:17
eranrom_e.g. devstack?13:18
kota_if we can use devstack template, it's easy to add the gate13:18
kota_probably, project-config has the template script for that.13:19
kota_and for SAIO thing, swiftstack sets a job for non-voting gate which is third party.13:19
kota_I an not familier with such a third party gate though.13:20
eranrom_kota_: When I am doing a 'git review' do you know which template is used for the jenkins test jobs13:20
kota_I think the template devstack will complete basic (whole?) component for any openstack projects (not sure, it might be only core project)13:20
eranrom_So it seems a good idea to have that as well.13:21
eranrom_Esp if we want to be an official project13:21
kota_eranrom: depends on project config.13:21
kota_wait...looking at project-config13:21
eranrom_Well, I guess that we can change (with infra help) the project config so that we test both against devstack template and against our current 's2aio setup' which uses 'clean' ubuntu.14_0413:22
eranrom_now waiting :-)13:22
-kota_- - gate-storlets-pep813:23
-kota_- - gate-storlets-python2713:23
-kota_- - gate-storlets-functional13:23
kota_you know :)13:23
eranrom_actually, Doron was the one who did that initial setup...13:24
kota_iirc, gate-(project name)-pep8 will run tox -e pep813:24
kota_as well as it, gate-(project)-devstack-something can be available for the gates.13:25
kota_most likely,13:25
kota_      - gate-swift-dsvm-functional13:25
kota_dsvm means devstack-virtual machine, i think13:26
eranrom_ok! so looks like (1) this can be done, and (2) this is a requirement from all projects.13:26
kota_and -functional will kick .functests probably.13:26
eranrom_kota_: make sense.13:27
eranrom_this is also what I think should be kicked for storlets13:27
eranrom_anything else on this subject?13:30
kota_I also think if we can run storlets with devstack, it seems great. On the other hand, s2io also be helpful for newbees to deploy not-all-in-one environment.13:30
eranrom_kota_: Agree. In fact I was thinking of having both.13:31
eranrom_kota_: If this is too much for each patch, perhaps we can have the devstack one as part of a pre-release test.13:31
eranrom_I mean, I think that there are such tests, but need to find out...13:32
kota_and note that the gate devstack always runs on the master, afaik13:32
eranrom_right, which is currently a problem for us :-) this brings to the next topic, unless there is more on this one.13:33
kota_need to make sure though. it might be broken affected from any other openstack project changes.13:33
kota_particuler Swift/Keystone.13:33
eranrom_right. good point13:33
eranrom_ok, Doron needed to drop.13:35
eranrom_next topic is a release.13:35
eranrom_At the time Takashi and I thought that we would do 3 things before 'cutting' a Storlets mitaka release.13:36
eranrom_1. land the range patch - done.13:36
eranrom_2. land the storlet docker request refactoring - done.13:36
eranrom_3. move the installation and test cod to work with Keystone V3 API13:37
eranrom_not done yet13:37
kota_wooo, v3 :/13:37
kota_just I'm a bad guy for keystone :P13:38
eranrom_I see :-)13:39
eranrom_Anyway, I am basically ok with this (although I am not sure yet what needs to be done), is that I think the Spark/Hadoop echo system currently does not support V3 auth13:40
eranrom_so I would13:40
eranrom_not want to break their ability to work with storlets.13:40
eranrom_That is unless a Keystone server can be used for both v2 and v3 auth at the same time.13:41
eranrom_Does being a bad guy with Keystone means you would not know :-)13:41
kota_Or doing with frustrating :-)13:43
eranrom_I see.13:43
eranrom_Perfectly understood!13:43
eranrom_Its never been easy with Keystone13:43
kota_eranrom: you konw ;)13:43
eranrom_ok, I will try to find out if it can support both. If not I would recommend to have a release with V213:45
eranrom_So that we will have something spark/hadoop can work with13:45
kota_sounds reasonable13:45
eranrom_ok, this is all from me13:45
kota_from my side... not so a big topic.13:46
kota_takashi is still working on something13:46
kota_thought for copy middleware placemnet13:47
kota_in the pipeline13:47
kota_and consideration for python supports.13:47
kota_I think nothing special for those right now.13:47
kota_probably, next week he could share thought for them13:48
eranrom_ok sure. So I guess this is all for today13:48
eranrom_Thanks very much for joining13:48
kota_ah, I forgot a special for today, say "thank you eronrom for submitting proposal for barcelona"13:49
eranrom_good luck to us.13:49
* kota_ hope so13:49
kota_ah, sorry, one more thing13:50
kota_next week meeting is the same time as today?13:50
kota_or will do at a new schedule? (takashi concerned if you could give a feedback for the schedule)13:51
kota_takashi is available 2 pm or 4 pm on Tsu in JST (the first proposal is 3 pm iirc)13:52
kota_anyway, not urgnet. we could define the schedule later though.13:53
*** cdoron has quit IRC14:09
eranrom_kota_: sorry seen this just now. Takashi confirmed he can do Tue 08:00 UTC.14:51
eranrom_If this is ok I will change the time on the wiki.14:51
eranrom_Also, I will be on vacation next week. Coming back Sunday the week after (July 31st)14:53

Generated by 2.14.0 by Marius Gedminas - find it at!