| kota_ | morning | 01:29 |
|---|---|---|
| kota_ | takashi: if you are available, please ping me to talk about something I'd make sure. | 01:30 |
| *** mattoliverau has quit IRC | 02:58 | |
| *** matt6434 has joined #openstack-storlets | 03:06 | |
| *** matt6434 is now known as mattoliverau | 03:30 | |
| *** takashi has joined #openstack-storlets | 04:06 | |
| takashi | hello | 04:06 |
| takashi | kota_: ping | 04:07 |
| takashi | do we have next weekly meeting today? or tommorow? | 04:07 |
| kota_ | hi, takashi | 04:07 |
| takashi | hi, kota_ | 04:07 |
| kota_ | no idea for now | 04:07 |
| takashi | kota_: ok | 04:08 |
| kota_ | do you have a time? | 04:11 |
| kota_ | takashi | 04:11 |
| takashi | kota_: yes | 04:13 |
| kota_ | ok, wait | 04:13 |
| takashi | waiting :-) | 04:24 |
| kota_ | ah, sorry, I pingged to messenger | 04:24 |
| kota_ | it sems you're absent in messenger | 04:24 |
| kota_ | ok, let's talk here | 04:24 |
| kota_ | so | 04:24 |
| takashi | kota_: looking | 04:24 |
| kota_ | https://github.com/openstack/storlets/blob/master/Engine/swift/storlet_gateway/gateways/docker/runtime.py#L707-L715 | 04:24 |
| kota_ | I'm curious the code above because I didn't get understood for the dead lock case | 04:25 |
| kota_ | and more, AFAIK, eventlet.spawn_n doesn't resucue such a situation but... does it work actually? | 04:26 |
| takashi | kota_: ok | 04:26 |
| takashi | you mean, eventlet doesn't work for socket io? | 04:27 |
| kota_ | no | 04:27 |
| kota_ | eventlet.spawn_n will return the thread instance to run | 04:27 |
| kota_ | and we need to make a call like thead.run() | 04:28 |
| kota_ | for the execution, right? | 04:28 |
| takashi | http://eventlet.net/doc/modules/greenthread.html | 04:29 |
| takashi | it seems that we don't need to execute run in spawn_n case | 04:29 |
| takashi | actually it works | 04:29 |
| kota_ | ah, it will run if the context would be switched? | 04:30 |
| takashi | kota_: yes | 04:30 |
| -kota_- >>> import eventlet | 04:30 | |
| -kota_- >>> def test() | 04:30 | |
| -kota_- File "<stdin>", line 1 | 04:30 | |
| -kota_- def test() | 04:30 | |
| -kota_- ^ | 04:30 | |
| -kota_- SyntaxError: invalid syntax | 04:30 | |
| -kota_- >>> def test(): | 04:30 | |
| -kota_- ... print 'hoge' | 04:31 | |
| -kota_- ... | 04:31 | |
| -kota_- >>> eventlet.spawn_n(test) | 04:31 | |
| -kota_- <greenlet.greenlet object at 0x7fa51dee1870> | 04:31 | |
| -kota_- >>> eventlet.sleep() | 04:31 | |
| -kota_- hoge | 04:31 | |
| kota_ | gotcha, this works. | 04:31 |
| takashi | eventlet.swapn returns thread instance, which should be run, but swapn_n immediately spawn green thread, and returns greenlet object to get return value | 04:31 |
| takashi | s/swapn/spawn/ | 04:31 |
| kota_ | ok, it will happen if non-blocking IO waiting something | 04:32 |
| kota_ | it seems probably... read_metadata? | 04:33 |
| kota_ | so I'm bit confused because it expects to use eventlet non-blocking IO but in other place, we call a bunch of os calls like os.read. | 04:34 |
| kota_ | in outside of eventlet thread... | 04:34 |
| kota_ | but, ok. it seems to work anyway, FWIW. | 04:35 |
| kota_ | thanks! | 04:35 |
| takashi | kota_: if storlet application performes streaming processing, it expect recieving data from swift at the same time it is sending data to swift | 04:38 |
| takashi | kota_: in that case, if we continue to send body from swift side to container side without reading any response, at some point storlet inside container stucks because it can't send data back to swift | 04:39 |
| kota_ | so it's curious | 04:40 |
| kota_ | storlet handler attempts to send it (maybe in buffer?) | 04:41 |
| kota_ | if container doesn't drain anything, that does not finish? | 04:42 |
| takashi | kota_: it will wait until read content is ready with timeout | 04:43 |
| takashi | s/it/storlet_handler | 04:44 |
| kota_ | read? | 04:44 |
| kota_ | I'm missing something | 04:44 |
| takashi | s/read content/read metadata/ | 04:44 |
| takashi | https://github.com/openstack/storlets/blob/master/Engine/swift/storlet_gateway/gateways/docker/runtime.py#L673 | 04:44 |
| kota_ | can i make sure the protocol? | 04:45 |
| takashi | after lauching write thread, storlet_handler will wait until it gets ready for read metadata from fd | 04:45 |
| kota_ | yes | 04:45 |
| kota_ | my question is why writing thread halt while writing data if the reader at container doesn't anything? | 04:46 |
| kota_ | (or the reader at container might be in just sending metadata...though) | 04:47 |
| kota_ | ah, | 04:48 |
| kota_ | the protocol deson't define the case like | 04:48 |
| kota_ | doesn't | 04:48 |
| kota_ | when, we should read metadata and whether client can send back the metadata *before* closing the recieving write buffer. | 04:49 |
| kota_ | that seems a problem | 04:50 |
| kota_ | because... | 04:50 |
| kota_ | if a huge object was feeded to a storlet app which does nothing (no read or slow read) and just send back the metadata. | 04:51 |
| kota_ | what's happen actually? | 04:51 |
| kota_ | it eat the huge memory bytes? | 04:52 |
| takashi | let me make sure | 04:52 |
| kota_ | ok | 04:53 |
| kota_ | and, IMO, we should figure out around the eventlet call because | 04:53 |
| kota_ | regading with the eventlet behavior, | 04:53 |
| takashi | as we discussed, sending contents from storlet_handler to container is executed in the new eventlet thread | 04:53 |
| kota_ | we don't ensure the data sending right now unless we know eventlet hub behavior deepely | 04:54 |
| takashi | and recieving contents from container is executed in the main thread(?), which is the different thread from sending one. | 04:54 |
| kota_ | it's a problem for maintainance | 04:54 |
| kota_ | i think, it should be obviously in the same i/o context swiftch for read/write and then, we got the final response of read, it should kill all coroutines in the thread, right? | 04:55 |
| takashi | kota_: yes | 04:56 |
| takashi | kota_: so problem is, we do nothing about writing thread after we launch it, right? | 04:56 |
| takashi | we need to manage it lifecycle. | 04:56 |
| kota_ | takashi: yes, i think. | 04:56 |
| takashi | s/it/its/ | 04:56 |
| takashi | ok, ok. I got it. | 04:57 |
| takashi | kota_: and totally agree with you | 04:57 |
| kota_ | takashi: thanks for talking :-) | 04:57 |
| takashi | kota_: if you try to fix that, you should be careful about the place where you wait that process | 04:58 |
| kota_ | yes | 04:58 |
| takashi | kota_: because reading body is handled by iterator, outside gateway module | 04:59 |
| kota_ | maybe, i can do that, because I recently figure out the event let hub impl for another purpose ;-) | 04:59 |
| takashi | kota_: ok :-) | 04:59 |
| takashi | kota_: np. thank you for bringing it up. | 05:00 |
| openstackgerrit | Merged openstack/storlets: Add small docs to understand https://review.openstack.org/345853 | 05:10 |
| takashi | I often saw curios test failures in gate job, especially in merge job :-( | 05:29 |
| takashi | previously I often saw some failures in docker pull, but recently some interruption of test execution happens in some case ... | 05:31 |
| kota_ | wooo | 06:05 |
| kota_ | takashi: https://review.openstack.org/#/c/331938/ | 06:06 |
| patchbot | kota_: patch 331938 - storlets - Set -eu for setup scripts explicitly | 06:06 |
| kota_ | this might make us easy to find the reason | 06:06 |
| kota_ | it makes the gate to halt when something wrong happen in the setup | 06:07 |
| takashi | kota_: looking | 06:07 |
| kota_ | a bunch of falures will move us into fog far from the actuall reason | 06:07 |
| kota_ | the lack point of it is we cannot set -eu for swift-intall unless we change the ibeem setup tools not to swift-init stop for clean environment. | 06:08 |
| takashi | there are also some problems about the log of daemon_factory | 06:09 |
| kota_ | swift-init all stop for clean environment will return 1 (error exit code) and it makes the gate failure :/ | 06:09 |
| takashi | currently we don't have any way to change log setting of daemon_factory, and it always generates log to syslog local0 | 06:09 |
| kota_ | might be | 06:10 |
| takashi | I'm curious wheather that behavior is very acceptable for everyone | 06:11 |
| takashi | that behavior of swift-init | 06:11 |
| takashi | for example, when we try to us pssh to launch all processes in multiple nodes, 'swift-init all' returns non 0 value, and ends up with error in pssh | 06:12 |
| takashi | we can avoid that error with explicitly give the process names to launch, but it is a little bit bother | 06:14 |
| kota_ | it seems like, swift-init all status -> if all servers running, it returns 0 otherwise 1 | 06:17 |
| kota_ | so that, "status" first and if no expected server running, do "start" seems reasonable. | 06:18 |
| takashi | kota_: ok | 06:27 |
| *** takashi has quit IRC | 07:01 | |
| *** takashi has joined #openstack-storlets | 07:43 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add unittests and cleanup codes https://review.openstack.org/346614 | 08:26 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add unittests and cleanup codes https://review.openstack.org/346614 | 08:29 |
| openstackgerrit | Akihito Takai proposed openstack/storlets: Fix COPY request https://review.openstack.org/347225 | 10:07 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add unittests and cleanup codes https://review.openstack.org/346614 | 10:42 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add unittests and cleanup codes https://review.openstack.org/346614 | 11:12 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add capability for clazy things https://review.openstack.org/347258 | 11:22 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Creanup redundant README.md and README.rst https://review.openstack.org/347278 | 11:38 |
| *** takashi has quit IRC | 13:15 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add unittests and cleanup codes https://review.openstack.org/346614 | 14:33 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add capability for clazy things https://review.openstack.org/347258 | 14:55 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Refactor dispatch_command https://review.openstack.org/345892 | 16:51 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Refactor dispatch_command https://review.openstack.org/345892 | 16:53 |
| openstackgerrit | Takashi Kajinami proposed openstack/storlets: Remove unneccesory parameters in storlet_handler https://review.openstack.org/347495 | 17:36 |
| *** openstackgerrit has quit IRC | 22:03 | |
| *** openstackgerrit has joined #openstack-storlets | 22:04 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add unittests and cleanup codes https://review.openstack.org/346614 | 22:11 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/storlets: Add capability for clazy things https://review.openstack.org/347258 | 22:12 |
| *** openstackgerrit has quit IRC | 22:48 | |
| *** openstackgerrit has joined #openstack-storlets | 22:49 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!