Wednesday, 2016-05-25

*** eranrom has quit IRC04:01
*** eranrom has joined #openstack-storlets06:35
*** eranrom has quit IRC07:50
*** eranrom has joined #openstack-storlets07:57
*** eranrom has quit IRC08:16
*** eranrom has joined #openstack-storlets08:23
openstackgerritEran Rom proposed openstack/storlets: WIP: Allow to run a single range request on the object server
*** openstackgerrit has quit IRC11:18
*** openstackgerrit has joined #openstack-storlets11:19
openstackgerritTakashi Kajinami proposed openstack/storlets: DO NOT MERGE: This is for testing
*** takashi has joined #openstack-storlets12:28
eranromLet's start13:01
eranromFirst thing, I apologize I did not review the recent refactoring patch13:02
eranromGot some deadlines...13:02
eranrombut will do asap13:02
takashieranrom: np13:02
eranromOther then that I wanted to share my plans for the next month (after leaving IBM)13:03
eranrom(1) I want to try and make the project official13:03
eranrom(2) send a talk to the Barcelona summit13:03
eranrom(3) renew the documentation (hopefully next week)13:03
kota_\o/ awesome13:03
eranrom(4) Plan a release.13:04
eranromI think that a release should have the following:13:04
eranrom1. Statement of which Swift we are tested with13:04
eranrom2. An official build of the SCOmmon jar13:05
eranrom3. A Docker container poeple can download that has s2aio inside so as storlet writers can quickly write and test storlets13:05
eranromWould be happy to get more opinions13:05
kota_what's SCOmmon?13:06
eranromoh, this is the .jar you need to develop a storlet. So as a storlet developer you will not need to build or even to clone the storlets repo.13:07
eranromthe next step would be to have an eclipse plugin for storlets deveopment13:08
eranromone that can work with e.g. the s2aio Docker image you download13:08
kota_gotcha sonds like template app.13:08
takashimakes sense to me :-)13:09
takashieranrom: do you have any target date for that release?13:10
takashieranrom: The reason of that question is that I'm wondering if we should deal with copy middleware before that release.13:10
eranromwell, I thought I should do this no later then the next openstack release.13:11
eranromnot sure then this release is13:11
kota_Swift will have a release soon that includes copy middleware.13:11
takashieranrom: agree13:11
kota_notmyname said in the previous meeting, he want to make it before we are working on encryption merge.13:12
takashikota_: ok13:12
eranromright, and the next OS release will be based on the Swift release with COPY so we definately need that13:12
eranromIf any of you can help with that then great. Othewise, lets push Takashi's refactoring, ranges and then I can do COPY13:13
takashieranrom: ok13:14
kota_I'm willing to be a voluntieer for that. Which patch is waiting for reviews?13:14
eranromcurrently Takashi's refactoring of the protocol classes, and my range stuff which needs a little more work13:15
takashikota_ patch 314140 and patch 31964013:15
patchbottakashi: - storlets - Refactor Storlet protocol classes13:15
patchbottakashi: - storlets - WIP: Allow to run a single range request on the ob...13:15
kota_(probably, I can do that since this weekend hopefully after my big patch for swift3 landed into master :/)13:15
eranromkota_,: congrats on landing this.13:16
takashikota_: great!13:16
eranromtakashi,: thanks for the links13:16
kota_eranrom: not yet :/ sorry my lack of English.13:16
eranromyou did mention after13:17
takashiI'm going to refactor gateway classes after that refactoring about protocols land13:17
eranromtakashi,: sounds great13:17
takashiand I hope that makes us to maintain handler codes easily :-)13:17
takashiincluding that copy work13:17
eranromtakashi,: curious to see what you have in ind, will wait for the patch.13:18
eranromGood time to talk about tomorrow's call?13:18
takashieranrom: yes13:19
eranromMy plan is to present two concrete use cases (1) the spark pushdown usecase (2) a security related use case we are exploring with an EU company that does analytics on electricity IOT data13:20
eranromand then have an open discussion13:20
eranromanything else you would like to cover?13:20
takashiI suppose that case 1 brings you a reason why you are working about patch 31964013:21
patchbottakashi: - storlets - WIP: Allow to run a single range request on the ob...13:21
kota_sonds great.13:21
eranromtakashi,: exactly :-)13:21
kota_plus, if some load map for storlets community, it would be great.13:21
kota_includes, features, position at openstack cummunity.13:21
kota_something like that, if possible.13:22
eranromkota_,: what do you mean by "some load map for storlets community"13:22
kota_I mean if we could have the discussion (or IBM plan) about the load map in the tommorow meeting, it would be great.13:23
kota_load map means...13:23
kota_like priority feature?13:23
kota_what we can expect in the next.13:24
kota_curcle back to takashi's mention above.13:24
eranromright, so I will not be able to answer that, but perhaps others on the call would be. We should have the same group as last week13:24
kota_eranrom: great13:25
eranromalright, so I guess this is it from my side13:25
kota_takashi: what did you mention about patch 319640/13:25
patchbotkota_: - storlets - WIP: Allow to run a single range request on the ob...13:25
kota_eranrom: yeah, thanks :)13:25
kota_eranrom: i know you might be impossible to say about that, that is why I said as 'if possible'13:26
takashieranrom: I saw the commit message of that patch says it is needed for spark pushdown usecase.13:27
takashisorry, not for eranrom, but for kota_13:27
kota_takashi: ah, that's interesting.13:28
takashikota_: That can improve concurrency for range request, by pushing storlet execution to storage servers from proxy servers, which is effective for access pattern generated by spark13:28
eranromkota_,: sure. no sweat13:28
takashias far as I understand13:28
eranromtakashi,: you are correct. there is an additional point: save the object->proxy bandwidth13:29
takashieranrom: yes!13:30
kota_takashi: does it act as concurrent?13:30
kota_I'm looking at storlet_handler.py13:31
takashikota_: if you have more storage servers than proxy servers.13:31
kota_ok, let me have more time to look at deeply.13:31
kota_takashi: though you might notice, recently some swift middleware got concurrent requests improvement13:32
kota_just FYI though.13:32
eranromtakashi,: you are correct. in terms of concurrency we get from #proxy concurrency to #object concurrency13:32
kota_takashi: patch 31181713:33
patchbotkota_: - swift - Allow concurrent bulk deletes (MERGED)13:33
takashikota_: for example, about bulk delete, right?13:33
kota_takashi: right13:33
takashikota_: Yes, I saw that patch, but didn't have a deep look yet.13:33
kota_takashi: np, just FYI.13:34
takashikota_: thanks! will check that.13:35
eranrombtw - are you aware of any work being done on partition power increase in Swift?13:37
kota_i think it is witll WIP, not sure though.13:38
takashieranrom: I'm not sure about the latest status, but I suppose now swift-ring-builder supports increasing part power of ring file13:38
eranrominteresting. Do you know is working on that?13:39
takashiI suppose some more works are required in swift deamons, to deal with that increasing13:39
takashiI know cschewede is working on that. I remember he was discussing about that topic with clayg at the last hackathon.13:40
eranromtakashi,ok thanks13:41
eranromAnything else for today?13:41
takashinothing special from me13:41
takashieranrom: this week I can attend to the meeting, and I'm lookong forward to hear you latest work tommorow13:42
eranromtakashi,: great!13:43
eranromso unless we have something else, I guess this is it for today. and will talk to you tomorrow.13:44
kota_yup, thanks eranrom :)13:44
takashieranrom, kota_: thanks. see you tommorow :-)13:45
eranromthanks for joining! ttyl.13:45
*** takashi has quit IRC14:14
*** eranrom has quit IRC14:24
*** takashi has joined #openstack-storlets14:45
*** takashi has quit IRC14:54
*** eranrom has joined #openstack-storlets16:39

Generated by 2.14.0 by Marius Gedminas - find it at!