Wednesday, 2016-06-29

*** openstack has joined #openstack-storlets01:26
*** eranrom has joined #openstack-storlets11:49
openstackgerritTakashi Kajinami proposed openstack/storlets: WIP: Merge all flow methods in gateway module
openstackgerritTakashi Kajinami proposed openstack/storlets: WIP: Fix ansible-lint error
openstackgerritTakashi Kajinami proposed openstack/storlets: WIP: Fix ansible-lint error
*** takashi has joined #openstack-storlets12:52
takashieranrom: Hi12:52
eranromHi Takashi12:53
eranromI have placed an agenda here (just now).12:53
eranromShould we wait for Kota?12:54
takashieranrom: now checking, thanks!12:54
takashieranrom: kota_ told me he can't attend to today's meeting12:54
eranromok. np.12:55
takashieranrom: So I think we can start12:55
eranromDo you have anything else you want to discuss on top of the list>?12:55
takashieranrom: nothing special :-)12:56
eranromok :-)12:56
eranromso, are you able to work with tox on master?12:56
eranromI always get a problem with the markers (> in the deps)12:57
takashieranrom: master of storlets?12:57
takashieranrom: which version of pip do you have?12:57
takashieranrom: I'm now using pip 8.0.2, and has no error12:59
eranromperhaps I will try to downgrade.12:59
eranromok, but it works for you. This was important for me to know...12:59
eranromso next topic :-)13:00
takashieranrom: I'll try to upgrade my pip later13:00
eranromoh, I hope it will not break things for you.13:00
eranromI will dig deeper13:00
eranromno problem thanks13:00
takashiIf there are any change in pip which causes such an error, I"m afraid that it can affect other projects... :-(13:01
eranromWhile working on a new patch for local ranges in the object server, I noticed a possible problem with the DockerRequest class.13:02
eranromI think you have merge the StorletPutRequest and StorletGet request into one class.13:03
eranromthe thing is that in a GET flow we instantiate the class with the response13:03
takashieranrom: yes13:03
eranromand so we do not have the expected headers/13:03
eranromsee e.g.:13:03
eranromline 19213:05
eranromsorry line 19113:05
eranromwe do:13:05
eranromsreq = DockerStorletRequest(orig_resp, req.params,13:05
eranrom                                    data_fd=orig_resp.app_iter._fp.fileno())13:05
takashieranrom: got it. Sorry for missing that13:06
eranromno problem.13:06
eranromnote that it works since we do not really make use of that information form the request...13:07
eranromI missed it in the review :-)13:07
eranromnext topic?13:07
takashieranrom: IMO we should add at least an unit test13:07
takashieranrom: Do you plan to fix that? If no, I'll work about it soone.13:08
eranromI can fix no problem.13:09
takashieranrom: I just want to make sure that we don't do duplicated work13:09
eranromOn the other hand, you initiated the refactoring, and you may have an opinion on how to fix. If you are ok with splitting back to get request and put request then I can do that13:10
eranromand will let you review13:10
eranromThis will also be a good opportunity for me to get deeper to the unit test framework.13:10
takashieranrom: For now, I have some idea to fix the problem, as the part of my remaining refactoring13:11
eranromok, so perhaps you better do it.13:12
takashieranrom: so I'd like to fix it13:12
takashieranrom: yes13:12
takashieranrom: thanks for your finding the problem :-)13:12
eranrom:-) ok, is it a good time to discuss prioritization?13:12
takashieranrom: yes13:13
eranromSo I really want to get the ranges stuff, because people in IBM want to do measurements.13:13
eranromOn the other hand, perhaps we want to have your fix earlier13:13
eranromThe range stuff is made of 3 patches:13:14
eranrom1. refactor the datagram stuff13:14
eranrom2. separate fd metadata to system and user13:15
eranrom3. add the range stuff13:15
takashieranrom: I saw you posted splited patches on gerrit13:16
eranromHow about merging 1,213:18
eranromthen have your fix for the requests - which is needed for the 3rd patch13:18
eranromand then the ranges13:18
eranromwould that work for you?13:18
takashieranrom: makes sense13:19
takashieranrom: I think my fix becomes not so big, and I suppose we can merge it without any big conflicts with your work13:20
eranromI don't mind merging it whenever it is ready, I just did not want to push :-)13:20
takashieranrom: I'll review patch 1 and 2 tommorow :-) I had a quick look today, but need some more time for deep looking13:21
eranromsure. Thanks!13:21
eranromdid you get a change to see the mail on simplifying the docker images/13:22
takashiSorry, I haven't checked it yet13:22
eranromok, np. This is just an idea.13:23
eranromWe currently have 3 'layers' in our docker image:13:23
eranrom1. ubuntu13:23
eranrom2. JRE813:24
eranrom3. storlet stuff (SCommon, SDaemon, factory)13:24
eranromI thought that the storlet stuff could be placed outside of the image and mounted as a read only volume.13:25
eranromEasier deployment, much easier to update13:25
eranromless time from dev to test13:25
takashieranrom: sounds great!13:26
eranromthis way if for example we have 10 tenants using storlets, we do not have to update any image if we fix a bug...13:26
eranromI guess my only question is:13:27
eranromdo we keep a separate copy for eaxh docker container, or a single copy that is mounted to all docker containers.13:27
takashieranrom: a snigle copy looks better for me13:28
eranromthis is also my thinking13:29
eranromI am a little worried that we might have a versioning problem with this. but we can always have more then one 'copy' e.g. different versions13:30
eranromand we will need to maintain which accounts use which versions, but this is futuristic thing I guess13:30
takashiI'm now cuoruous about the way to make sure about python/java execution13:32
eranromyou mean in terms of java classpath?13:33
eranromand python include path?13:33
takashieranrom: also jvm and python interpreter13:34
eranromare you suggesting to have them also from the host?13:34
eranrome.g. mount /usr/lib or something to the container?13:35
takashieranrom: I need some more time for thinking, but13:36
takashifrom my current thought, at least the python interpreter to execute daemon_factory should be maintained with daemon_factory, in host side13:37
eranromI can't say it does not make sense.13:38
eranromsure less of a python versioning headache13:39
eranromwe need to make sure we are aligned with the Swift python version.13:39
eranromwhich we anyway need, and we are ok13:40
eranromThe Java part is different, because we may not have Java on the host.13:40
eranromon the other hand, what happens when/if we support python storlets...13:41
eranromneed more thinking...13:41
takashieranrom: yes13:41
eranromok, so nothing more from me :-)13:42
takashieranrom: one thing from my side13:42
takashiI just started working about ppython supprot for storlet application language13:43
eranromsuper cool!!!13:43
takashieranrom: I'm now thinking about the way to handle dependency in python case13:44
takashiIn java, we can use jar file, which gathers multiple java source codes13:44
takashibut in python, we should consider another solution now13:45
eranromI am not an expert here, but have you considered python eggs?13:45
takashiI haven't checked it yet13:46
takashiI'm now thinking about automatical installing using requirement.txt file as object, but need some more time to decide detailed design13:47
eranrominteresting. Does this mean we open up networking for the docker container?\13:49
takashieranrom: yes. or another solution is opening up networking for host node, and let host node build virtualenv, which passed to docker container by filesystem mounting13:50
takashis/passed/is passed/13:51
eranromright. Still I think that there a huge value with python even with the standard libraries13:51
eranromunlessof course you have a counter example :-)13:51
takashieranrom: yes13:52
eranromrecall that we can change the Docker image as well.13:52
takashieranrom: I'm just at a starting point, so I'll let you know after I created more detailed idea13:52
eranromsure! anyway I am vary happy that you are working in this!!!13:53
takashieranrom: :-)13:53
takashiThat's all from my side13:54
eranromok, thanks for the good discussion.13:54
takashithank you, too13:55
openstackgerritTakashi Kajinami proposed openstack/storlets: Remove DockerStorletRequest class
openstackgerritTakashi Kajinami proposed openstack/storlets: Remove unused parameters from DockerStorletRequest
openstackgerritTakashi Kajinami proposed openstack/storlets: Remove unused parameters from DockerStorletRequest
*** takashi has quit IRC14:31
*** eranrom_ has joined #openstack-storlets16:03
*** eranrom has quit IRC16:06
openstackgerritTakashi Kajinami proposed openstack/storlets: Remove unused parameters from DockerStorletRequest
openstackgerritTakashi Kajinami proposed openstack/storlets: WIP: Fix ansible-lint error
openstackgerritTakashi Kajinami proposed openstack/storlets: Fix bashate errors
*** notmyname has quit IRC19:01
*** notmyname has joined #openstack-storlets19:04

Generated by 2.14.0 by Marius Gedminas - find it at!