Wednesday, 2016-07-13

*** takashi has joined #openstack-storlets06:46
*** openstackgerrit has quit IRC06:48
*** openstackgerrit has joined #openstack-storlets06:49
takashieranrom_: sorry for my late reply. I just want to make sure that some of us submit talks planned, because we do not have so long time before deadline comes.06:50
eranrom_no problem. So you will be submitting - right?06:50
takashieranrom_: can I ask you to submit it? I'm afraid I can't make time for that today... :-(06:52
eranrom_no problem. sec.06:52
takashieranrom_: thank you. please let me know when you create the submittion. I'll have a look about that.06:52
takashieranrom_: I think you can select my and kota_'s account using e-mail address, when adding speakers.06:53
takashieranrom_: please make sure to use my company e-mail.06:53
takashi(not gmail or other06:54
takashieranrom_: thank you again for working about that06:54
*** takashi_ has joined #openstack-storlets07:03
*** takashi has quit IRC07:03
eranrom_takashi_, kota_: I have just submitted the 101 talk, please review and chance as you see fit.07:27
takashi_eranrom_: thank you for submitting the talk. Now I can see the detail!07:52
eranrom_sure. Please review :-)07:53
takashi_eranrom_: I have one question. When we choose "Developers" as summit categories, we can choose "big data" as generic topic.07:53
eranrom_I am not sure. Seems like there is one checkbox there... but if you find a way that would be great. An alternative is to choose the category with 'big data'...07:54
takashi_eranrom_: Is it reasonable for you to change category from Architecture & Operations to Developers?07:55
eranrom_sure. I think you can also do that.07:56
eranrom_I am good this it.07:56
takashi_eranrom_: thanks07:56
takashi_eranrom: I just updated it07:58
takashi_IMO, "Big Data" topic can be more attractive for people than "Storage" topic, and also shows clearly that we are doing more that Storage.07:58
takashi_eranrom_: I updated the titol of the talk, as I mentioned in my e-mail. (I think current topic is a little different from 101) If you have any thought about it, please let me know.08:53
eranrom_takashi_: I am good with the change to big data. Regarding the title, it sounds to me too specific and poeple may think it will be of less interest if they are not involved in the project09:08
eranrom_This is not to say that 101 is the right title...09:08
takashi_eranrom_: I'm open for changing the title, but currently I think 101 is not good if we try to choose Developers category, to select big data topic.09:14
takashi_eranrom_: I think it is a temporal one. In my current thought, we should include two things in the title. (1) We'll explain what is storlet (2) We'll also describe technical detail09:15
eranrom_How about: Storlets deep dive09:16
eranrom_or storlets technical deep dive09:17
takashi_eranrom_: makes sense09:23
takashi_"Storlets deep dive" looks better to me, because we also have some topics for app developers09:25
eranrom_ok. great.09:35
*** takashi_ has quit IRC10:26
*** takashi has joined #openstack-storlets12:56
takashieranrom_: hello12:58
eranrom_I guess Kota is in the Hackathon, so its only us.12:58
takashieranrom_: I suppose so12:58
eranrom_ok. I have only 30 minutes today.12:58
eranrom_I can update on:12:59
eranrom_1. I will send soon a plan for the security feature12:59
eranrom_2. I am making progress on the Spark code, will have more updates soon13:00
takashieranrom_: looking forward to see them!13:00
eranrom_Otherwise, regarding the openstack talk, did you update the subject? Should I?13:01
takashieranrom_: I just did it13:01
takashithank you for your lot of working about it13:01
takashiI'll update if I find any possible improvoments, but IMO now our proposal is ready13:02
eranrom_alright, cool.13:02
takashieranrom_: I don't have so big update today. I'm still working about copy fix and python application support13:02
eranrom_ok, I still see -1 on the recent refactoring work. Want me to have a look?13:03
eranrom_Gerrit seems down so I do not have the patch is13:04
takashieranrom_: That is the last one of refactoring work. I saw some error in storlet range execution, but need some more time to find the cause13:04
eranrom_I see.13:04
eranrom_I will try to have a look13:04
takashieranrom_: thx13:05
takashiI saw two kinds of failure in functional testing13:05
takashi1. input range is ignored13:05
takashi2. timeout happens in storlet execution13:05
eranrom_I had 1 many times during the development of the feature.13:06
eranrom_I will have a look13:06
takashiI suppose there are some wrong handling about range parameters, and I'll have a deep look in this week, or at least in the early next week13:06
eranrom_sure, np13:06
takashieranrom_: thanks you for your help13:06
eranrom_yw (you welcome:-)13:07
eranrom_anything else for today?13:07
takashione update about copy fix13:07
takashiI'll inform you detailed information later, but I found current swift does not allow copy  with chunked get13:07
takashithat's because they do not want to allow copy of the large object, which has more objects than the listing limit of container13:08
takashi(I found the patch introduced that limitation, and it says so13:09
eranrom_Does this happen when you place storlet_handler to the left of copy?13:09
eranrom_actually it does not make sense13:09
takashito the right of copy13:09
eranrom_right :-)13:09
takashiI'm doing a PoC about the idea you wrote in the previous e-mail (detect PUT/GET caused by copy in storlet_handler, and invoke storlet in one side)13:10
eranrom_ok, so when do we encounter the Swift limitation...13:11
takashieranrom_: In my first idea, I tried to execute storlet in GET request, when processing copy13:11
takashibut if we invoke storlet copy, we loose content-length in that response right?13:12
takashithen copy middleware tried to use that GET request, to put the retrieved content to the destination, but It notice that GET response is a chunked one, and raise 4xx error.13:12
eranrom_and then the copy middleware cannot do the put - right?13:13
takashieranrom_: yes13:13
eranrom_ok now I get it.13:13
eranrom_I guess we can still invoke the storlet on the put...13:13
takashiThat logic is already introduced when copy was implemented inside swift. In previous architecture, we handle put by ourselves inside storlet_handler, and we didn't implement that limitation13:14
takashieranrom_: you are right, but I think we should make sure about the pros/cons of that change13:14
eranrom_yeah. so the alternative is to put the storlet_handler to the left of the middleware.13:14
eranrom_or perhaps fix Swift...13:14
eranrom_ok, do you have any thoughts as to the cons of invoking the storlet during the put part of the copy?13:15
takashiI'll have a deep look, and would like ask some thoughts about it by people who worked about copy_middleware13:15
eranrom_sounds great\13:15
takashiif we can reduce data size by invoking storlet, execution on proxy requires heavier traffic in GET than execution on object side13:16
takashiex. if you put your image and execute strlet copy to extract tag information to create json object13:17
takashiif we can execute on object, we only need transfer json13:17
takashiif we execute on proxy, we need to transfer image file from object to proxy13:18
eranrom_in such a case doing the put is really worse...13:18
takashieranrom_: yes13:18
eranrom_I guess it will be interesting to get the Swift community view on fixing that limitation...13:18
eranrom_well a limitation for us...13:19
takashieranrom_: that limitations is introduced in very old time, so there are some possible change in their opinion13:19
eranrom_I mean good point.13:20
takashiAt some point I'll show for swift community what is happening for use13:20
takashiabout copy13:20
eranrom_That would be great.13:21
takashiI'll let you inform if I get updates about this topic13:21
takashithat's all from my side today13:21
eranrom_ok thanks very much. One last thing:13:21
eranrom_Are you ok with moving the IRC to Tuesdays, at a much earlier time?13:22
eranrom_say my morning - your noon13:22
takashieramrom_: I have a weekly meeting until around noon JST13:23
eranrom_ok, so you prefer not to have yet another one right away I guess...13:24
takashieranrom_: If we can have a meeting after 13:00, I think it works for me13:24
eranrom_ok, great. I think this is also ok for Kota. I will send a mail to the list.13:25
takashiI'll wait your e-mail, and give me some time for checking my schedule (I can't check it from my house13:25
eranrom_sure. thanks13:25
takashiDid we talk all topics for today?13:27
eranrom_yes. Thanks for joining.13:27
takashieranrom_: thank you13:28
*** takashi has quit IRC13:31
*** openstackgerrit has quit IRC16:03
*** openstackgerrit has joined #openstack-storlets16:04

Generated by 2.14.0 by Marius Gedminas - find it at!