| *** takashi has joined #openstack-storlets | 12:59 | |
| eranrom_ | Hi | 13:00 |
|---|---|---|
| takashi | eranrom_: Hi | 13:00 |
| eranrom_ | Do you know if Kota plans on joining? | 13:00 |
| eranrom_ | I guess he does not.... | 13:02 |
| eranrom_ | Shell we start? | 13:02 |
| takashi | eranrom_: no, but he said he was not feeling well, so I think he is absent from today's meeting | 13:02 |
| eranrom_ | ok, hope he will get well soon. | 13:02 |
| eranrom_ | I have two things. | 13:02 |
| takashi | eranrom_: ok. let's talk about them first | 13:03 |
| eranrom_ | 1. Discuss how to deal with the range in the view of your comment | 13:03 |
| eranrom_ | 2. Talk about the Barcelona summit | 13:03 |
| takashi | eranrom_: ok. | 13:04 |
| eranrom_ | regarding 1: I will move the Range parsing to the _get_storlet_invocation_data | 13:04 |
| eranrom_ | I wonder where we should decide on raising an exception if there is more then one range. | 13:05 |
| eranrom_ | This is because _get_storlet_invocation_data will also be called when we run on the proxy | 13:05 |
| eranrom_ | in which case we can have X-Storlet-Range with more then one range | 13:06 |
| eranrom_ | one way is to pass _get_storlet_invocation_data a parameter if this is proxy or object node | 13:08 |
| takashi | eranrom_: how about passing dictionally, which can be retrieved by r.ranges to StorletDockerRequest? | 13:08 |
| takashi | I mean, parsing Range header to create range dictionally in _get_storlet_invocation_data, and validate it is a single range inside DockerStorletRequest | 13:09 |
| eranrom_ | takashi: right, but my problem is that DockerStorletRequest is called both on proxy and object invocations, and IMO looking if we have an fd or not, is a hack | 13:11 |
| eranrom_ | So I was thinking - since _get_storlet_invocation_data will be called directly from the handlers we can pass it with an argument telling it if this is a proxy handler | 13:11 |
| eranrom_ | or object handler | 13:12 |
| eranrom_ | and parse the range in the first place only if on object | 13:12 |
| takashi | or does it make sense to migrate _build_strlorlet_request, _get_storlet_invocation_data and _get_user_metadata to handler first (which now I"m working), and add that range handling only in object handler? | 13:14 |
| eranrom_ | sounds good | 13:16 |
| eranrom_ | so I will fix all other commetns, put in WIP and await your patch | 13:16 |
| takashi | eranrom_: I think you don't have to put WIP now and leave range handling as it is at the moment | 13:17 |
| takashi | eranrom_: If that migration takes some time, I'll land your patch first, and fix that range handling later in my patch | 13:17 |
| eranrom_ | takashi: Alright, thanks. So I will leave it as is, and fix the other comments. | 13:18 |
| takashi | eranrom_: thank you! | 13:18 |
| eranrom_ | next item? | 13:18 |
| takashi | eranrom_: yes | 13:18 |
| eranrom_ | ok, so I will send today a first draft for the Barcelonba talk | 13:19 |
| eranrom_ | Please feel free to add / change as you see fit | 13:19 |
| takashi | eranrom_: thank you for creating the draft! I'll check it | 13:20 |
| eranrom_ | I guess that you will also need NTT approval, and the deadline is getting close... | 13:20 |
| eranrom_ | ok, will send it later today (will probably be tomorrow for you :-) | 13:20 |
| takashi | Can I make sure about the deadline date for submitting talk information. | 13:20 |
| takashi | ? | 13:20 |
| eranrom_ | I think it is July 15th or 17th | 13:21 |
| eranrom_ | I know that the internal IBM deadline is over, but this does not need to worry us... | 13:21 |
| takashi | eranrom: alright. so we should finish about talk proposal until next week. | 13:22 |
| eranrom_ | great | 13:22 |
| eranrom_ | ok, I am done :-) | 13:22 |
| takashi | eranrom_: I'll discuss about the things of NTT side with kota_ later. I know he will join to Swift Hackathon next week, so I think we should finish almost all things in this week... | 13:23 |
| eranrom_ | takashi: ok. So you are not going to the Hackathon? | 13:24 |
| takashi | eranrom_: unfortunately no. kota_ is going, so you can discuss with him at Hackathon | 13:24 |
| eranrom_ | I am not going either :-) | 13:25 |
| eranrom_ | I have ot been involved for quite some time now... | 13:25 |
| takashi | eranrom_: ok | 13:25 |
| takashi | eranrom_: np | 13:25 |
| eranrom_ | Anything from your side? | 13:25 |
| takashi | I have two topics today | 13:25 |
| takashi | whick I added to meeting page | 13:26 |
| eranrom_ | looking | 13:26 |
| takashi | 1. fix support for swift master (including copy) | 13:26 |
| eranrom_ | yep | 13:26 |
| takashi | I started working about copy fix and now considering what we should do to fix espacially copy things | 13:27 |
| takashi | I tested master storlet with master swift, and noticed that we can't launch proxy-server storlet_middleware try to import things which is already moved to copy middleware | 13:28 |
| takashi | s/proxy-server storlet_middleware/proxy_server with storlet_middleware/ | 13:28 |
| eranrom_ | right. | 13:29 |
| takashi | from swift.common.constraints import check_copy_from_header | 13:29 |
| takashi | this is the import which causes import error | 13:29 |
| eranrom_ | yep. I remember. | 13:29 |
| takashi | what I'd like to discuss today is, should we keep support for existing releases of swift, or only support master swift? | 13:30 |
| takashi | I mean, if we completely fix copy based on master swift, it does not work with existing releases (L, M) of swift | 13:31 |
| eranrom_ | My take on this question is that we should have our releases too :-) Meaning: we could tag our current master with 'mitaka', and to work with Swift mitaka one will need | 13:31 |
| eranrom_ | storlets 'mitaka' | 13:31 |
| eranrom_ | thus, storlets master should support swift master | 13:32 |
| eranrom_ | Does it make sense? | 13:32 |
| takashi | eranrom_: makes sense | 13:34 |
| eranrom_ | takashi: This comes to the question of storlets release - what it takes, etc. but this is a whole new subject. I will send a mail to the list on that | 13:35 |
| takashi | eranrom_: ok. I'll reply to you on the mailing list | 13:36 |
| eranrom_ | we can start by adding a tafg / branch point before we land the copy fix on master | 13:36 |
| eranrom_ | s/tafg/tag | 13:36 |
| takashi | eranrom_: I noticed we should ask kota_'s opinion about this, because we is doking similar thing in swift3 | 13:36 |
| eranrom_ | absolutly | 13:36 |
| takashi | eranrom_: I think we can put tag after current range patch and the migration of _build_storlet_request land | 13:37 |
| eranrom_ | sounds like a plan! | 13:38 |
| takashi | :-) | 13:38 |
| eranrom_ | I will mention it in the note | 13:38 |
| takashi | eranrom_: thanks! | 13:38 |
| eranrom_ | takashi: sure | 13:38 |
| takashi | Can I move to the second topic? | 13:38 |
| eranrom_ | sure | 13:38 |
| takashi | As I told you in the last meeting I started working about python application support | 13:39 |
| eranrom_ | yep :-) | 13:39 |
| takashi | I need some more thought about dependency management, but I'll start with fixing and refactoring modules in which we should different things in java and python. | 13:40 |
| takashi | I noticed I should update the following parts to introduce a swifthing mechanism between java and python at least | 13:41 |
| eranrom_ | sounds great! I have few quick thoughts here - just in case you find it helpful: | 13:41 |
| takashi | 1. validation of storlet/dependency upload | 13:42 |
| takashi | 2. parameters passed from gateway to storlet | 13:42 |
| takashi | 3. process management in daemon_factory | 13:42 |
| takashi | eranrom_: it is surely helpful for me, because I'm just at a starting point now | 13:43 |
| eranrom_ | ok, so I think that for 3, much of the existing functionality is still relevant. | 13:44 |
| eranrom_ | I think that there is also: | 13:44 |
| eranrom_ | 4. A python equivalent to SDaemon. The point here is that the SDaemon uses thread pool, and not sure that green threads in python would be right here. | 13:45 |
| eranrom_ | My thinking was that the python SDaemon would fork on each incoming request (up to a limit, similar to what e.g. the swift container-updater does) | 13:46 |
| takashi | eranrom_: In my current opinion, it's not good to use green thread inside SDaemon, because green thread does not work with CPU intensive processing | 13:46 |
| eranrom_ | exactly. | 13:47 |
| takashi | eranrom_: And as you suggested, we need to fork a process with a limit for incoming requests | 13:47 |
| eranrom_ | right. | 13:47 |
| takashi | eranrom_: Thank you for your comment! I need some more time to thing about the python implementation of SDaemon, but that threading problem is one of the biggest topic which should be considered | 13:48 |
| takashi | s/to thing/to think/ | 13:48 |
| eranrom_ | agree. Once you have a plan, let me know if you need help with the implementation | 13:49 |
| takashi | eranrom_: Can I ask one thing about python implementation? | 13:49 |
| eranrom_ | e.g. the ServerInDatagram python equivalent... | 13:49 |
| eranrom_ | sure | 13:49 |
| takashi | that's what I'd like to ask about now! | 13:50 |
| eranrom_ | :-) you mean if I can do that? | 13:50 |
| eranrom_ | sure | 13:50 |
| takashi | I think we should implement some more python facade interface when we implement python SDaemon. | 13:50 |
| takashi | Is it right understanding? | 13:50 |
| eranrom_ | yes, there is actually python code to write both on the python facade and the SCommon equivalent | 13:51 |
| eranrom_ | which are basically helper classes to ease on the storlet writing. | 13:51 |
| eranrom_ | including the invoke interface | 13:52 |
| takashi | eranrom_: ok, thanks | 13:53 |
| eranrom_ | if you have more questions feel free to ping here or on the list. | 13:53 |
| takashi | eranrom_: thank you again. I'm currently thinking it is a good chance for me to learn more about sbus stuffs | 13:54 |
| takashi | eranrom_: but will ask your help if I face some problems :-) | 13:54 |
| eranrom_ | sure. I am glad I had a chance to refactor it before you look at it - it was a disastrous implementation | 13:55 |
| takashi | :-) | 13:56 |
| takashi | One more update I missed in the topic about swift master support | 13:57 |
| takashi | when I tested current ansible script with keystone Mitaka, I faced some errors which happen because current ansible script uses v2 API. | 13:58 |
| eranrom_ | and this is because keystone mitaka uses V3 by default - right? | 13:58 |
| takashi | eranrom_: I think so, but need some more check | 13:59 |
| takashi | IMO, we should migrate from keystone v2 API to keystone v3 before we put 'mitaka' tag | 13:59 |
| eranrom_ | This was my experience. When I wrote the storlets installation instructions I used keystone mitaka and had to change the default. | 13:59 |
| eranrom_ | I guess you are right. | 14:00 |
| takashi | eranrom_: ok. I'll work about it later. | 14:00 |
| eranrom_ | I will add this as will | 14:00 |
| eranrom_ | to the mail - that is | 14:00 |
| takashi | eranrom_: thanks | 14:00 |
| takashi | that's all from my side today | 14:01 |
| eranrom_ | thank you. lets talk before you start working on this, because it also requires some doc changes... perhaps we can do this together | 14:01 |
| eranrom_ | alright. | 14:01 |
| takashi | eranrom_: ok | 14:02 |
| eranrom_ | Thanks for joining! | 14:02 |
| takashi | eranrom_: thank you! | 14:02 |
| openstackgerrit | Eran Rom proposed openstack/storlets: Single range request on obj. https://review.openstack.org/337256 | 14:22 |
| *** takashi has quit IRC | 16:09 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!