Wednesday, 2016-01-20

kota_notmyname: hmm...eventlet==0.16.1 doesn't work in my envrionment but i got another issue test_get_conns stucked long time.01:48
kota_notmyname: anyways, nosetests test/unit/common/ with eventlet>=0.17.1 worked for me.01:50
kota_oh shit, wrong channel01:53
kota_please ignore them01:54
*** takashi_ has joined #openstack-storlets06:42
*** eranrom has joined #openstack-storlets08:19
eranromWe are thinking of dropping query string for storlets parameters in favor of headers. The key reason being that with headers it is more natural to have 'X-Storlet-Parameter' prefix then on the quesry string.08:23
eranromAny thoughts, objectstions?08:23
takashi_eranrom: I don't.08:43
eranromtakashi,: Are you using parameters today?08:45
takashi_Using query is sometime confusing, because it is not so clear which module will use that parameter.08:45
eranromtakashi,: I absolutly Agree. In you PoC does your storlet takes parameters?08:46
takashi_eranrom: I have some plan about using parameter, because sometimes it may save me many storlets, which are not so different08:46
takashi_eranrom: In our PoC, we just started with very simple Storlet, so I think we don't have to use parameters08:47
eranromtakashi,: alright. Doron is adding this feature (parameters as headers). We will depracate the query string parameters a bit later.08:47
takashi_eranrom: ok08:48
takashi_Using parameters are very useful. we can switch the algorithm to detect features in images. we can flexibly change the size of shrinked images/videos.08:54
takashi_we don't so have to use parameters in our PoC, but I believe we can't create useful applications without parameter mechanisms08:55
*** takashi_ has quit IRC09:00
eranromtakashi,: Absolutely. We currently support paramets, and will continue to support them. The question is whether its via query string or headers. The current code supports parameters via query string, with doron's latest patch we will also support headers.10:36
openstackgerritDoron Chen proposed openstack/storlets: dd support for sending parameters through headers
openstackgerritDoron Chen proposed openstack/storlets: add support for sending parameters through headers
openstackgerritDoron Chen proposed openstack/storlets: add support for sending parameters through headers
*** openstackgerrit has quit IRC11:43
*** openstackgerrit has joined #openstack-storlets11:45
*** takashi_ has joined #openstack-storlets12:45
eranromHi. Anybody here13:00
takashi_Hi :-)13:00
eranromHi Takashi.13:00
eranromHi Kota13:01
eranromLets start13:03
eranromI do not have anything special13:03
takashi_I have some updates about fix in swift13:03
takashi_First, about patch 256201, I discussed about this patch last week with Kota, and found some items to improve.13:04
patchbottakashi_: - [WIP] Fix proxy-server's support for chunked trans...13:04
eranromtakashi,: Should I try it? or wait for some more updates?13:05
eranromtakashi,: I have seen it and it had an import of _is_chunked inside the function which I was not sure if its common13:07
takashi_eranrom: I'll update it. It's almost done, and I have to solve one item.13:07
kota_takashi_: i think the biggest issue is proxy redirect when time out happen during draining body, right?13:08
takashi_eranrom: _is_chunked should be used for Response and Request, but these two has no relations like extending13:08
eranromtakashi,: alright so as soon as I see a patch I will review and test with Storlets13:08
takashi_kota_: right13:09
takashi_In that discussion, we found some items we have to fix in storlets13:09
takashi_two items13:09
takashi_bug 1534414 is the first one, and what kota_ mentioned13:10
openstackbug 1534414 in storlets "GET with Storlet execution generates wrong content with automatic retry in proxy-server" [Undecided,New] - Assigned to Takashi Kajinami (kajinamit)13:10
kota_eranrom: swfit proxy-server has a capability to redirect to another object server to resume to get the body when the connection between proxy and object-sever disconnected13:10
kota_in transfer data.13:10
takashi_that resuming causes some problems with storlets13:11
kota_for now.13:11
kota_takashi_: oh yeah, you seems to describe in the bug report.13:12
takashi_when client get object with storlet execution, proxy-server passes it to object-server, and object-server will run storlet13:12
takashi_but when proxy gets timeout, it will send another request to another object-server, with Range header13:13
takashi_that range header will cause some problem, because object-server will not run storlet after resuming.13:13
eranromtakashi,: Good catch! interesting one. Any thought on how to fix?13:13
takashi_eranrom: yes. we think we have to return error in the next object-server13:15
eranromtakashi,: So as to fail the while request - right?13:15
kota_eranrom: right13:16
takashi_eranrom: yes. It may surpress resuming feature13:16
eranromsounds good! Do you think we should do this to any range request or only to 'redirected range requests'?13:17
kota_eranrom: we couldn't expect object-servers except one the requset get first can return such a resuming body.13:17
takashi_we can't make sure we can get remaining content with that range header now, so I think we have to prevent swift from resuming13:17
takashi_I'm talking about the same thing as kota_ discribed13:18
takashi_eranrom: I think we should do only for redirected range request13:18
kota_takashi_: in current idea, x-run-storlets will be removed at header *if storlets run there*, right?13:18
takashi_kota_: yes13:19
kota_i.e. if run at proxy, object-servers will get the request w/o run-storlets13:19
takashi_and we can detect the redirection case, by checking if we have both of x-run-storlets and range.13:19
kota_at object-server13:19
takashi_kota_: right13:20
kota_eranrom: go ahead13:20
eranromWhat happen if today we send a request with range and x-run-storlet, can it get executed on an object server? or once we see range we always execute this on the proxy? (I do not remeber)13:21
takashi_we always execute on the proxy13:21
eranromok, good.13:22
eranromSo now I get the idea :-)13:22
eranromWhat was the second issue?13:22
takashi_bug 1534455, and it's very small thing13:23
openstackbug 1534455 in storlets "Storlets should return 206 instead of 200 for Range request" [Undecided,New]
eranromok gotcha, thanks13:23
takashi_that's what we found thinking about range13:24
takashi_s/thinking/when thinking/13:24
eranromI see :-) again, good catch!13:24
eranromI need to have notifications on the bugs. My bad ...13:24
takashi_it's not so big problem, and I can fix it easily13:25
takashi_but we have to be careful about RFC when fixing it,13:25
takashi_because range header should includes start/end information13:25
kota_eranrom: you can get subscription from launchpad :)13:25
takashi_but client may get the content in different length (!= end - start)13:26
eranromeranrom: right, it just that I should have done this by now :-)13:26
eranromtakashi,: right. I guess this is where we need to be carefult abour the spec - right?13:27
takashi_eranrom: yes13:27
eranromtakashi,: I am not sure that the RFC even covers such a case.13:27
eranromtakashi,: I need to look.13:27
takashi_eranrom: unfortunately, RFC does not seem to cover range request with chunked transfering13:28
eranromtakashi,: so the options are either return 206 or 412 (unless the storlet did not change anything) or we can say that if this is not covered by the spec we can do anything :-)13:29
eranromAn alternative is to refer to x-run-storlet with range as the storlet output rather the input. so we always run the storlet on the entire object and output only the requested range13:30
eranromin which case we can return a content-length13:30
eranromand 412 if the computation does not have such a range...13:31
takashi_another alternative is replace range header with something like storlet-run-over-range13:31
eranromactually we can have both :-)13:32
takashi_with that header we can return 200 OK. that does not violate RFC13:32
eranromright: here is what I gathered so far:13:33
eranromOption 1: have a new header called x-storlet-run-over-range which will do what we do today when we have both X-run-storlet AND range headers13:34
eranromOption 2: Support X-Run_Storlet + range where the range will be the storlet output13:35
eranromWe can decide to support both ot none.13:35
eranromI currently do not see a strong use case for any of them... thoughts?13:35
takashi_eranrom: that's based on the usecase...13:37
eranromtakashi,: you mean your use case?13:37
takashi_eranrom: yes, but they are no so rigid ones. just ideas in me13:38
takashi_when we use storlet to encode movie file in video application, clients want to get the range of the output13:38
takashi_but in other cases, clients want to get the output which are generated from a part of a object13:39
kota_takashi_: on that case, it might be good run storlets at object-server :)13:39
takashi_kota_: yes13:39
eranromand have the semantics of run-storlet-on-range - right?13:40
kota_on-range or run-storlets-and-get-this-range13:41
takashi_let me make more clear about this topic13:41
takashi_so in video cases, we want to get the range of result, and "range" header is suitable for that. for client, there are no difference from getting range of the object.13:42
takashi_in the latter cases, for example when object includes may record in it and clients want to get result generated from some of theses records, x-run-storlet-over-ragnge will be suitable, I think13:43
takashi_range means the range of output, and x-run-storlet-over-range means the range of input13:44
takashi_does it make sence?13:45
eranromtakashi: Absolutley. I also agree with kota_ that for the case of get the range of result using the range header we should probably allow to run this on the object sevrer.13:45
*** takashi__ has joined #openstack-storlets13:47
*** [1]eranrom has joined #openstack-storlets13:47
[1]eranromSorry, got disconnected13:47
takashi_eranrom: np13:47
[1]eranromso I think just before disconnecting, I was thinking whether we can do both type of range semantics in the object server13:49
takashi_range of input is easier thing. that's already done.13:50
[1]eranromtakashi: agree, but is supported only on proxy. Not saying its urgent to support it also on object though.13:51
kota_eranrom: +113:52
takashi_eranrom: +113:52
[1]eranromoh, but then If I got takashi right yhen its more natural to support that via a proprietary header such as run-storlet-on-range and bypass RFC problems.13:53
[1]eranromor am I confused....13:54
takashi_eranrom: that's right13:54
[1]eranromtakashi,ok thanks!13:54
*** wbhuber_ has joined #openstack-storlets13:54
*** hrou_ has joined #openstack-storlets13:55
*** takashi has quit IRC13:56
*** wbhuber has quit IRC13:56
*** eranrom has quit IRC13:56
*** hrou has quit IRC13:56
*** [1]eranrom is now known as eranrom13:56
eranromtakashi,: so perhaps what we can do in the short term is to add support for run-storlet-on-range (which exists in a sense), and reply with 400 bad request in the case of x-run-storlet with range until we support the second semantics13:56
kota_irc seems not so stable...13:57
takashi_eranrom: and add some notes about range execution in doc13:57
kota_takashi_: +113:58
eranromI can do the doc stuff.13:58
eranromtakashi_, kota_: Do we want first to write a summary of this in the bug, and continue from there?13:58
kota_eranrom: could you please?13:59
kota_eranrom: thanks!13:59
takashi_eranrom: thanks :-)14:00
eranromthank you!14:00
takashi_that's all from me today14:00
kota_eranrom: one thing14:01
eranromkota_,go ahead14:01
kota_eranrom: when do you expect the submit deadline for us for austin summit?14:01
kota_i know, officially Feb 1st but14:01
kota_i think we should add some description for submitter each other before make it finalize.14:02
eranromkota_,: you mean improve the talk abstract?14:03
kota_eranrom: yes14:03
kota_and currently i am working on getting approval in my company.14:04
kota_oh, no14:04
kota_not improve, i think14:04
kota_ah, it might include also improve14:05
kota_all things14:05
takashi_at least, we have to add some information about presenters, as kota_ said14:06
eranromok gotcha14:06
kota_(just want to make sure schedule roughly)14:06
kota_sorry my bad for explanation.14:06
eranromno worries. so you would like to circulate speakers information between us, before we submit?14:07
eranromor the list of the actual sepakers?14:08
kota_later one14:09
eranromWell, its definately not urgent for me.14:10
kota_and plus, when will you register online.14:10
kota_i mean register as submit a talk14:10
eranromSo I need to double check that I have a green light from IBM. Once I have it, I will let you know, so that once you have a green light we can finalyze the abstract and register the speakers.14:10
kota_all right, gotcha.14:11
kota_expecting i could know it next week.14:12
eranromkota_,: Great!14:12
eranromI will update on IRC as soon as I have news14:12
kota_eranrom: thanks14:12
eranromkota_,: sure!14:13
eranromyep, thanks a lot\14:17
takashi_thanks :-)14:18
eranromgood night14:18
takashi_eranrom: you too14:21
*** takashi_ has quit IRC14:24
*** hrou_ has quit IRC15:19
*** hrou has joined #openstack-storlets15:20
*** eranrom has quit IRC17:05
*** takashi has joined #openstack-storlets18:13
*** peluse_ has joined #openstack-storlets18:18
*** takashi__ has quit IRC18:22
*** peluse has quit IRC18:22
*** hrou has quit IRC18:57
*** hrou has joined #openstack-storlets18:57

Generated by 2.14.0 by Marius Gedminas - find it at!