| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Cleanup code for FileManager and invocation_flow https://review.openstack.org/349834 | 01:48 |
|---|---|---|
| kota_ | just rebased | 01:49 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Creanup redundant README.md and README.rst https://review.openstack.org/347278 | 02:00 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Creanup redundant README.md and README.rst https://review.openstack.org/347278 | 02:01 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add capability for crazy things https://review.openstack.org/347258 | 02:02 |
| kota_ | meeting in 15 minutes? | 06:44 |
| kota_ | ah, ok 17:00 in JST | 07:04 |
| kota_ | so in 55 minutes. | 07:04 |
| *** takashi has joined #openstack-storlets | 07:34 | |
| eranrom | right :-) | 07:51 |
| kota_ | hello | 08:00 |
| takashi | meeting time :-) | 08:00 |
| takashi | kota_: hi | 08:00 |
| eranrom | Hi | 08:00 |
| takashi | eranrom: o/ | 08:01 |
| eranrom | From some reason I could not edit the agenda | 08:01 |
| takashi | eranrom: np | 08:01 |
| eranrom | but one subject for today is the TC IRC meeting tomorrow | 08:01 |
| eranrom | about making storlets official. | 08:01 |
| kota_ | yup | 08:01 |
| eranrom | what else do we want to discuss? | 08:01 |
| kota_ | no specific one | 08:02 |
| kota_ | from me | 08:02 |
| takashi | eranrom: I have a small topic about pipelining in proxy, but we can start with talking about becoming the official project | 08:02 |
| takashi | I think it is more important for all of us | 08:03 |
| eranrom | ok, I also would like to update on spark-storlets but lets do this at the end | 08:03 |
| eranrom | So. | 08:03 |
| eranrom | John has pinged me yesterday and told me he was asked to give a comment on the patch. He gave a very supportive comment | 08:04 |
| kota_ | which patch? | 08:04 |
| eranrom | https://review.openstack.org/#/c/353693/ | 08:04 |
| patchbot | eranrom: patch 353693 - governance - Storlets to become official - Proposed governance ... | 08:04 |
| kota_ | thanks | 08:04 |
| eranrom | kota raised yesterday the concern of our usage of c and java | 08:05 |
| takashi | eranrom: I saw that discussion log | 08:06 |
| eranrom | The best answer I can think of is that Java is a language binding for us (much like go and java and more used by the OpenStack SDK) | 08:06 |
| eranrom | and that we are also working on Python | 08:07 |
| takashi | Can we refer monasca about the use of java? https://github.com/openstack/monasca-api | 08:08 |
| eranrom | takashi: sure. It official right? | 08:09 |
| takashi | eranrom: I'm now trying to find that evidence. | 08:10 |
| takashi | https://github.com/openstack/governance/blob/master/reference/projects.yaml | 08:11 |
| takashi | We can find that monasca is listed in L2603. | 08:12 |
| eranrom | right, and the API is also listed there. | 08:13 |
| eranrom | great | 08:13 |
| eranrom | Another question we may be asked is why do we want to become official | 08:13 |
| eranrom | my answer is that its a way to gain traction and hopefully get deployers and developers try that | 08:14 |
| eranrom | thoughts? other concerns? | 08:14 |
| kota_ | no other concerns, basically we need to get official because we need to follow other openstack module (speficically swift) | 08:16 |
| takashi | eranrom: agree | 08:17 |
| kota_ | and to do so, we need offical support from openstack gevernance, e.g. summit slot to discuss, work with as cross projects, | 08:17 |
| takashi | kota_: right | 08:17 |
| eranrom | just being a devil advocate here (I do agree with you). What if we are told that we can do all this without being official? | 08:18 |
| kota_ | and i think storlets is definately different with other openstack project on the focus so it's unique. | 08:18 |
| kota_ | eranrom: good question | 08:18 |
| eranrom | kota_: right, and according to John this may play against us - I am afraid... | 08:18 |
| kota_ | thinking... | 08:19 |
| takashi | eranrom: There are some points which supports us to become an official project, IMO | 08:21 |
| eranrom | Well, I think that if we feel that the TC is reluctant due to us being different, we can always try and say that this is an advantage | 08:21 |
| takashi | 1. As kota_ mentioned, storlets is different from all of the other OpenStack projects, so should be independent | 08:22 |
| takashi | 2. storlets is gaining diversity. People from some organizations are surely working about it. | 08:23 |
| takashi | 3. We need foundation's support for internal activity, and cross project activity (with swift) | 08:23 |
| takashi | support means, for example, a room for discussion in design summit | 08:24 |
| eranrom | Agree. will keep those in mind during the discussion. that helps | 08:24 |
| takashi | 4. We already have codes which works | 08:25 |
| eranrom | and do something useful@ | 08:25 |
| eranrom | useful! | 08:25 |
| takashi | eranrom: yes | 08:25 |
| eranrom | has unittests, functional tests well integrated into OpenStack CI | 08:26 |
| takashi | eranrom: they are all currently in my mind. If I come up with other things, I'll tell them to you | 08:27 |
| eranrom | great thanks! | 08:27 |
| eranrom | are you planning on joining? | 08:28 |
| kota_ | i support takashi's thought. we are now working with other openstack project as an eco-system. to work continiously with them, official support is a good way. | 08:28 |
| kota_ | i think | 08:29 |
| takashi | As we need to be well integrated with Swift but can't be totally merged to swift, we need to stand at the similar position as swift, one independent official project. | 08:29 |
| takashi | eranrom: Can I ask the meeting time? | 08:29 |
| eranrom | sec. | 08:29 |
| kota_ | 5:00 am in jST | 08:29 |
| kota_ | tommorow morning | 08:30 |
| takashi | kota_: ok | 08:30 |
| kota_ | tomorrow | 08:30 |
| eranrom | kota_: thanks. here is the link to their meeting page: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee | 08:30 |
| takashi | eranrom: will join | 08:31 |
| eranrom | great! | 08:31 |
| takashi | need to set alarm clock | 08:31 |
| takashi | :-) | 08:31 |
| eranrom | :-) | 08:31 |
| eranrom | Please do not feel obligated. | 08:31 |
| kota_ | me too | 08:32 |
| eranrom | you too :-) pls do not feel obligated | 08:32 |
| eranrom | next topic? | 08:32 |
| takashi | eranrom: ok | 08:33 |
| eranrom | proxy piplining? | 08:33 |
| takashi | eranrom: ok | 08:33 |
| takashi | after merging copy fix, we noticed two things | 08:33 |
| takashi | 1st, In patch 347225, we assumed that copy middleware is listed in pipeline | 08:34 |
| patchbot | takashi: https://review.openstack.org/#/c/347225/ - storlets - Fix COPY request (MERGED) | 08:34 |
| takashi | but currently proxy automatically adds copy in its middleware, even if it is not listed in pipeline configration. | 08:35 |
| takashi | s/its middleware/its pipeline/ | 08:35 |
| takashi | like gatekeeper or so on | 08:35 |
| eranrom | gotcha | 08:35 |
| takashi | Akihito is now working about it, but we need to think about that point. | 08:36 |
| kota_ | https://github.com/openstack/swift/blob/master/swift/proxy/server.py#L541 | 08:36 |
| kota_ | ref for the pipeline modification code, just fyi | 08:36 |
| takashi | kota_: thx! | 08:36 |
| eranrom | thx. | 08:36 |
| takashi | The second one is about quota | 08:37 |
| takashi | the patch 347225 introduces a little difference in the behavior about quota, in Liberty swift and Mitaka swift | 08:38 |
| patchbot | takashi: https://review.openstack.org/#/c/347225/ - storlets - Fix COPY request (MERGED) | 08:38 |
| takashi | In liberty swift, we place storlet_handler in the left of slo, and as the result we place storlet_handler in the right of quota | 08:38 |
| takashi | In Mitaka swift, we place storlet_handler in the left of copy middleware, which is placed also in the left of quota middlewares | 08:39 |
| takashi | As the result, we place storlet_handler in the left of quota when using Mitaka swift | 08:39 |
| eranrom | and this is a problem because we create an object without a content-length? | 08:40 |
| takashi | In liberty swift, we checked a size quota besed on the content-length of "input object" | 08:40 |
| takashi | In Mitaka swift, we don't check a size quota, because we remove content-length before passing it to quota middleware | 08:41 |
| takashi | I'm wondering which is more desiable bahavior | 08:41 |
| eranrom | well, what if the user uploads a file with no content-length (using chunked transfer encoding). I think this is possible.... | 08:42 |
| takashi | eranrom: Swift does not check size quota. Swift only checks count quota. | 08:42 |
| kota_ | takashi: plus, check the exisitng volume w/o incomming request length | 08:43 |
| takashi | kota_: right, thanks | 08:43 |
| kota_ | if the total size of objects in the container exseeded the quota limit, it will reject the request. | 08:44 |
| eranrom | I guess I am misunderstanding something. If a user can upload an object without a content-length isn't it the same as our middleware being to the left of the quota middleware? | 08:44 |
| eranrom | kota_: ok gotcha. | 08:45 |
| kota_ | eranrom: true | 08:45 |
| takashi | eranrom: but previously our middleware was placed in the right of quota. | 08:45 |
| takashi | My question is, should we move storlet_middleware in the left of quota middlewares when using it with liberty swift? | 08:46 |
| eranrom | ok gotcha. So if I got kota right this does not matter really. and if its simpler then lets do it. | 08:47 |
| takashi | eranrom: ok | 08:47 |
| eranrom | regarding the first point - you probably know better then I am - if we can tell the version of swift, perhaps we can decide upon the version instead of looking for the copy middleware | 08:48 |
| takashi | eranrom: that can be one solution | 08:49 |
| takashi | eranrom: I'm currently checking if we need to add copy middleware explicitly in the proxy pipelining, when it does not appeard in conf file. | 08:50 |
| takashi | If we need it, we need to check the verison of swift, and update proxy-server.conf based on that information. | 08:50 |
| kota_ | sounds reasonable | 08:51 |
| eranrom | takashi: sounds good to me | 08:51 |
| takashi | eranrom, kota_: thx. will update you later, after some more deep looking | 08:51 |
| eranrom | ok. thanks Takashi. next item? | 08:52 |
| takashi | that's all from my side today | 08:52 |
| takashi | eranrom: yes | 08:52 |
| eranrom | ok, so I am working on a new repository called spark-storlets which allows spark to connect to Swift with storlets. Its already in github and I will update the documentation soon. | 08:53 |
| eranrom | While working on that I found a bug that is closely related to the discussion you had at the time of controlling what is happening after the storlet | 08:53 |
| takashi | eranrom: so cool! | 08:53 |
| eranrom | has started streaming and our middleware is non in control anymore | 08:54 |
| eranrom | s/non/not | 08:54 |
| eranrom | here is my thinking: | 08:54 |
| kota_ | cool | 08:54 |
| eranrom | We need to pass the storlet an additional fd, call it stderr | 08:54 |
| eranrom | whenever the storlet runs into a critical situation, it uses stderr to "say that" | 08:55 |
| kota_ | what does 'critical situation' mean? | 08:56 |
| kota_ | sounds like atomic operation? or something secret? | 08:56 |
| eranrom | Out iterator will "monitor" the other end of stderr, and once it sees a critical event it will do whatever cleanup (in eventlet?) is needed and somehow break the connection | 08:56 |
| eranrom | critical - say it fails to read the input | 08:56 |
| eranrom | or runs into an exception that makes its output worthless | 08:57 |
| eranrom | meaning something that the client should know about | 08:57 |
| eranrom | The problem I ran into, was that the read within the storlet failed, so I did not consume all the data, and the client did not know about it | 08:58 |
| eranrom | that is the storlet did not consume all the data | 08:58 |
| takashi | eranrom: We need to think the way to return that error to clients. | 08:58 |
| eranrom | takashi: right, what happens today if there is a disk error after the object server already started to stream the data back? | 08:59 |
| eranrom | that is, we already answered 200OK started to stream back the data and then, oops, bad sector... | 08:59 |
| takashi | eranrom: the proxy server will try to connect the next object server to get the remaining data | 09:00 |
| takashi | eranrom: However, in storlets, we can not do that way, because we can't indicate 'response range'. | 09:00 |
| eranrom | takashi: right. what if all replicas are down? will the client experience a timeout? | 09:01 |
| takashi | eranrom: I think so | 09:01 |
| kota_ | i'm wondering that we can solve the problem? | 09:01 |
| kota_ | sounds like same thing if swift got the sector error on disk while reading body... | 09:01 |
| eranrom | kota_: well, a timeout is better then nothing... | 09:02 |
| eranrom | at least the client knows an error occured. | 09:02 |
| kota_ | it might be detected with comparing content-length though. | 09:02 |
| takashi | kota_: but we can't get content-length for storlet execution result | 09:02 |
| kota_ | hmm.... | 09:02 |
| kota_ | yes | 09:02 |
| eranrom | :-) | 09:02 |
| kota_ | i know | 09:02 |
| eranrom | I thought I will look into HTTP ref to see what can be done in such a case? | 09:03 |
| eranrom | case/ | 09:03 |
| eranrom | case. | 09:03 |
| kota_ | i think transfer-encoding has a syntax too | 09:03 |
| kota_ | so if the socket closed with EOF, basically client can find the error ocurred. | 09:04 |
| eranrom | right. I think that when using chunked transfer encoding one can send headers 'inline' the body | 09:04 |
| kota_ | unless i am dreaming | 09:04 |
| eranrom | :-) How can the client tell the EOF is not a legit close... | 09:04 |
| eranrom | ? | 09:04 |
| kota_ | IIRC, 0 bytes chunks is for the EOF in transfer encoding | 09:04 |
| kota_ | I saw the RFC when developing EC thing on upstream swift... | 09:05 |
| eranrom | ok, will look into it. At any rate currently the WSGI framework hides this from us. | 09:05 |
| eranrom | so we are not directly writing into the 'raw socket' | 09:06 |
| eranrom | but perhaps the protocol does give something to work with. | 09:06 |
| takashi | eranrom: yes | 09:06 |
| eranrom | ok, so this was my update. :-) | 09:06 |
| eranrom | Anything else? | 09:07 |
| takashi | eranrom: I'll looking forward to seeing your doc update. | 09:07 |
| takashi | nothing from my side | 09:07 |
| kota_ | ok, i'll try to make sure my thought. | 09:08 |
| takashi | s/I'll/I'm/ | 09:08 |
| eranrom | takashi: sure, will update as soon as its there. | 09:08 |
| takashi | eranrom: :-) | 09:09 |
| kota_ | :-) | 09:09 |
| kota_ | my daughter calling me to sit down dinner table :/ | 09:09 |
| eranrom | bonappetite :-) | 09:10 |
| eranrom | Thanks for joining | 09:10 |
| takashi | thank you | 09:10 |
| kota_ | thank you | 09:10 |
| takashi | kota_: have a nice dinner | 09:10 |
| kota_ | takashi: :-) | 09:11 |
| kota_ | bye | 09:11 |
| *** eranrom has quit IRC | 09:20 | |
| *** eranrom has joined #openstack-storlets | 09:30 | |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Refactor SBusDatagram https://review.openstack.org/349348 | 10:46 |
| *** takashi has quit IRC | 11:29 | |
| openstackgerrit | Doron Chen proposed openstack/storlets: remove pushing of local docker images https://review.openstack.org/355918 | 13:25 |
| openstackgerrit | Doron Chen proposed openstack/storlets: Install Swift and the Storlet Engine on a Docker container. https://review.openstack.org/352356 | 18:00 |
| *** jroll has joined #openstack-storlets | 21:01 | |
| jroll | eranrom: have y'all looked at zerovm at all? | 21:01 |
| jroll | they tried to do something similar using google (chrome's?) NACL thing, was super interesting | 21:01 |
| jroll | basically restricts syscalls to a known set of things | 21:01 |
| kota_ | jroll: sounds interesting | 21:03 |
| kota_ | jroll: http://www.zerovm.org/ is correct to link that? | 21:04 |
| jroll | kota_: indeed, I played with it a bit a couple years ago | 21:04 |
| jroll | http://www.zerovm.org/ | 21:04 |
| jroll | yes | 21:04 |
| kota_ | jroll: eranrom might be going to offline becuase it's midnight on his timezone. | 21:04 |
| jroll | sure, no worries | 21:05 |
| jroll | just wanted to point it out :) | 21:05 |
| kota_ | jroll: ok, I7ll take a look | 21:05 |
| kota_ | I'll | 21:05 |
| eranrom | hi I am here. | 21:05 |
| eranrom | Yes I have been looking deeply into zerovm | 21:05 |
| eranrom | A very cool project. Since they have been acquired by rackspace they have gone silent. | 21:06 |
| eranrom | thanks for pointing | 21:06 |
| jroll | yeah, it's sad, dunno what happened to them | 21:07 |
| jroll | (and I work for rackspace!) | 21:07 |
| eranrom | :-) | 21:07 |
| kota_ | jroll: nice! are you in San Antonio? I visited Rackspace HQ in the last month to join Swift mid-cycle | 21:08 |
| jroll | kota_: nope, I work from home :) | 21:09 |
| kota_ | jroll: got it | 21:09 |
| notmyname | AFAIK most (all?) of the people working on zeroVM at rackspace have since left the company | 21:09 |
| eranrom | jroll: in a nutshell the difference would be: zerovm is perhaps more secured compared to Linux containers but has a proprietary tool chain for code development | 21:09 |
| jroll | indeed | 21:10 |
| eranrom | notmyname: I met their founder in IL, just before he moved the team to Rackspace HQ to work on cloud files. So I guess he left the company. | 21:10 |
| notmyname | eranrom: (or anyone) is there any sort of logo or similar for storlets? | 23:33 |
| notmyname | what's on the storlets tshirt? ;-) | 23:33 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!