Wednesday, 2016-07-06

*** takashi has joined #openstack-storlets12:59
takashieranrom_: Hi13:00
eranrom_Do you know if Kota plans on joining?13:00
eranrom_I guess he does not....13:02
eranrom_Shell we start?13:02
takashieranrom_: no, but he said he was not feeling well, so I think he is absent from today's meeting13:02
eranrom_ok, hope he will get well soon.13:02
eranrom_I have two things.13:02
takashieranrom_: ok. let's talk about them first13:03
eranrom_1. Discuss how to deal with the range in the view of your comment13:03
eranrom_2. Talk about the Barcelona summit13:03
takashieranrom_: ok.13:04
eranrom_regarding 1: I will move the Range parsing to the _get_storlet_invocation_data13:04
eranrom_I wonder where we should decide on raising an exception if there is more then one range.13:05
eranrom_This is because _get_storlet_invocation_data will also be called when we run on the proxy13:05
eranrom_in which case we can have X-Storlet-Range with more then one range13:06
eranrom_one way is to pass _get_storlet_invocation_data a parameter if this is proxy or object node13:08
takashieranrom_: how about passing dictionally, which can be retrieved by r.ranges to StorletDockerRequest?13:08
takashiI mean, parsing Range header to create range dictionally in _get_storlet_invocation_data, and validate it is a single range inside DockerStorletRequest13:09
eranrom_takashi: right, but my problem is that DockerStorletRequest is called both on proxy and object invocations, and IMO looking if we have an fd or not, is a hack13:11
eranrom_So I was thinking - since _get_storlet_invocation_data will be called directly from the handlers we can pass it with an argument telling it if this is a proxy handler13:11
eranrom_or object handler13:12
eranrom_and parse the range in the first place only if on object13:12
takashior does it make sense to migrate _build_strlorlet_request, _get_storlet_invocation_data and _get_user_metadata to handler first (which now I"m working), and add that range handling only in object handler?13:14
eranrom_sounds good13:16
eranrom_so I will fix all other commetns, put in WIP and await your patch13:16
takashieranrom_: I think you don't have to put WIP now and leave range handling as it is at the moment13:17
takashieranrom_: If that migration takes some time, I'll land your patch first, and fix that range handling later in my patch13:17
eranrom_takashi: Alright, thanks. So I will leave it as is, and fix the other comments.13:18
takashieranrom_: thank you!13:18
eranrom_next item?13:18
takashieranrom_: yes13:18
eranrom_ok, so I will send today a first draft for the Barcelonba talk13:19
eranrom_Please feel free to add / change as you see fit13:19
takashieranrom_: thank you for creating the draft! I'll check it13:20
eranrom_I guess that you will also need NTT approval, and the deadline is getting close...13:20
eranrom_ok, will send it later today (will probably be tomorrow for you :-)13:20
takashiCan I make sure about the deadline date for submitting talk information.13:20
eranrom_I think it is July 15th or 17th13:21
eranrom_I know that the internal IBM deadline is over, but this does not need to worry us...13:21
takashieranrom: alright. so we should finish about talk proposal until next week.13:22
eranrom_ok, I am done :-)13:22
takashieranrom_: I'll discuss about the things of NTT side with kota_ later. I know he will join to Swift Hackathon next week, so I think we should finish almost all things in this week...13:23
eranrom_takashi: ok. So you are not going to the Hackathon?13:24
takashieranrom_: unfortunately no. kota_ is going, so you can discuss with him at Hackathon13:24
eranrom_I am not going either :-)13:25
eranrom_I have ot been involved for quite some time now...13:25
takashieranrom_: ok13:25
takashieranrom_: np13:25
eranrom_Anything from your side?13:25
takashiI have two topics today13:25
takashiwhick I added to meeting page13:26
takashi1. fix support for swift master (including copy)13:26
takashiI started working about copy fix and now considering what we should do to fix espacially copy things13:27
takashiI tested master storlet with master swift, and noticed that we can't launch proxy-server storlet_middleware try to import things which is already moved to copy middleware13:28
takashis/proxy-server storlet_middleware/proxy_server with storlet_middleware/13:28
takashifrom swift.common.constraints import check_copy_from_header13:29
takashithis is the import which causes import error13:29
eranrom_yep. I remember.13:29
takashiwhat I'd like to discuss today is, should we keep support for existing releases of swift, or only support master swift?13:30
takashiI mean, if we completely fix copy based on master swift, it does not work with existing releases (L, M) of swift13:31
eranrom_My take on this question is that we should have our releases too :-) Meaning: we could tag our current master with 'mitaka', and to work with Swift mitaka one will need13:31
eranrom_storlets 'mitaka'13:31
eranrom_thus, storlets master should support swift master13:32
eranrom_Does it make sense?13:32
takashieranrom_: makes sense13:34
eranrom_takashi: This comes to the question of storlets release - what it takes, etc. but this is a whole new subject. I will send a mail to the list on that13:35
takashieranrom_: ok. I'll reply to you on the mailing list13:36
eranrom_we can start by adding a tafg / branch point before we land the copy fix on master13:36
takashieranrom_: I noticed we should ask kota_'s opinion about this, because we is doking similar thing in swift313:36
takashieranrom_: I think we can put tag after current range patch and the migration of _build_storlet_request land13:37
eranrom_sounds like a plan!13:38
eranrom_I will mention it in the note13:38
takashieranrom_: thanks!13:38
eranrom_takashi: sure13:38
takashiCan I move to the second topic?13:38
takashiAs I told you in the last meeting I started working about python application support13:39
eranrom_yep :-)13:39
takashiI need some more thought about dependency management, but I'll start with fixing and refactoring modules in which we should different things in java and python.13:40
takashiI noticed I should update the following parts to introduce a swifthing mechanism between java and python at least13:41
eranrom_sounds great! I have few quick thoughts here - just in case you find it helpful:13:41
takashi1. validation of storlet/dependency upload13:42
takashi2. parameters passed from gateway to storlet13:42
takashi3. process management in daemon_factory13:42
takashieranrom_: it is surely helpful for me, because I'm just at a starting point now13:43
eranrom_ok, so I think that for 3, much of the existing functionality is still relevant.13:44
eranrom_I think that there is also:13:44
eranrom_4. A python equivalent to SDaemon. The point here is that the SDaemon uses thread pool, and not sure that green threads in python would be right here.13:45
eranrom_My thinking was that the python SDaemon would fork on each incoming request (up to a limit, similar to what e.g. the swift container-updater does)13:46
takashieranrom_: In my current opinion, it's not good to use green thread inside SDaemon, because green thread does not work with CPU intensive processing13:46
takashieranrom_: And as you suggested, we need to fork a process with a limit for incoming requests13:47
takashieranrom_: Thank you for your comment! I need some more time to thing about the python implementation of SDaemon, but that threading problem is one of the biggest topic which should be considered13:48
takashis/to thing/to think/13:48
eranrom_agree. Once you have a plan, let me know if you need help with the implementation13:49
takashieranrom_: Can I ask one thing about python implementation?13:49
eranrom_e.g. the ServerInDatagram python equivalent...13:49
takashithat's what I'd like to ask about now!13:50
eranrom_:-) you mean if I can do that?13:50
takashiI think we should implement some more python facade interface when we implement python SDaemon.13:50
takashiIs it right understanding?13:50
eranrom_yes, there is actually python code to write both on the python facade and the SCommon equivalent13:51
eranrom_which are basically helper classes to ease on the storlet writing.13:51
eranrom_including the invoke interface13:52
takashieranrom_: ok, thanks13:53
eranrom_if you have more questions feel free to ping here or on the list.13:53
takashieranrom_: thank you again. I'm currently thinking it is a good chance for me to learn more about sbus stuffs13:54
takashieranrom_: but will ask your help if I face some problems :-)13:54
eranrom_sure. I am glad I had a chance to refactor it before you look at it - it was a disastrous implementation13:55
takashiOne more update I missed in the topic about swift master support13:57
takashiwhen I tested current ansible script with keystone Mitaka, I faced some errors which happen because current ansible script uses v2 API.13:58
eranrom_and this is because keystone mitaka uses V3 by default - right?13:58
takashieranrom_: I think so, but need some more check13:59
takashiIMO, we should migrate from keystone v2 API to keystone v3 before we put 'mitaka' tag13:59
eranrom_This was my experience. When I wrote the storlets installation instructions I used keystone mitaka and had to change the default.13:59
eranrom_I guess you are right.14:00
takashieranrom_: ok. I'll work about it later.14:00
eranrom_I will add this as will14:00
eranrom_to the mail - that is14:00
takashieranrom_: thanks14:00
takashithat's all from my side today14:01
eranrom_thank you. lets talk before you start working on this, because it also requires some doc changes... perhaps we can do this together14:01
takashieranrom_: ok14:02
eranrom_Thanks for joining!14:02
takashieranrom_: thank you!14:02
openstackgerritEran Rom proposed openstack/storlets: Single range request on obj.
*** takashi has quit IRC16:09

Generated by 2.14.0 by Marius Gedminas - find it at!