Thursday, 2015-01-29

*** kei_yama has joined #openstack-swift00:04
*** openstackgerrit has quit IRC00:21
*** openstackgerrit has joined #openstack-swift00:21
notmynamekota_: ok, etags on static manifests....00:26
*** dmorita has joined #openstack-swift00:26
notmynamedmorita: I'm guessing you are interested in etags too? :-)00:26
notmynamekota_: so, what was it you were saying?00:27
kota_When getting slo object, firstly swift reading the manifest and then transfer each segment contaniously.00:28
*** david-lyle is now known as david-lyle_afk00:28
*** rmcall has quit IRC00:29
kota_And I would like to define etag mean.00:29
kota_firstly, here.00:30
kota_The etag on Swift is a md5 disget must be caluclated from object body.00:30
notmynameetags for normal objects is the md5 of the contents of the object. for large objects, it's the md5 of the concatenation of the etags of the segments in the manifest00:30
kota_And then, I guess we should make the etag for large objects to be the md5 of the contents of the object.00:32
kota_Same as normal objects.00:32
kota_even though doc shows the etag for large objects is the concatenation of the etags, currently.00:34
notmynamewell, that would be nice. but it would be an API change that we can't do without versioning the API00:34
kota_Oh, I didn't notice the effect for versiong API.00:35
notmynameand I think it would be hard to do without having to read in potentially _lots_ of data. I think you'd have to store the state of the md5 hasher so it's resumable. and I'm not sure what you can do with 2 stored states. maybe it's possible. I don't know00:35
kota_yes, it's hard thing but Kazuhiro knows hash algorithms more deeply than me, and he is trying to be that the imporovement be less performance issue.00:36
kota_Currently, that is still under development and making the idea to be better so that I would like to introduce his idea in hackathon00:37
kota_Is it suitable for you?00:37
notmynameit totally doesn't surprise me that NTT says "oh, we know hashing. we can do that" :-)00:37
kota_Though, it's still just a draft and research.00:38
notmynamekota_: I think it's interesting to consider, but I don't think we can change the meaning of the etag withut versioning the api00:38
notmynameya, I agree that it's interesting research right now00:39
kota_yes, it's not for users to change the API.00:39
notmynamethere's several things that we've considered for new API versions.00:39
kota_But I guess we can add a header.00:39
kota_Probably it's related to new API :)00:40
kota_I don't know the new API in detail...00:40
notmynamethere isn't one :-)00:41
torgomatichow would you go about computing the whole-object MD5 without reading the whole object, or at a minimum, all but the first segment?00:41
zaitcevWhat's the advantage of having a hash that covers the whole body of segmented object?00:42
torgomaticI mean, yes, you can save the MD5 generator state prior to length-padding, but you still have to take the first segment's saved state and update the generator with the bytes from the segments 2..N00:42
notmynametorgomatic: md5 is in the class of hashers that the digest is the serialization of the state of the hasher, right? so maybe there's some way to store the state? I know that's possible to resume it. I don't know if that's posisble to take 2 states and do something meaningful00:42
kota_Currently, computing on object-server and then share the inter-medeate info among the segment at HEAD segments to check the etag.00:43
notmynamezaitcev: if you download a large object, there's nothing you can do on the client side to validate the contents without doing subsequent HEAD requests to all of the segments00:43
torgomatickota_: that would only work if the segments were uploaded in sequence, right? if the segments are uploaded in random order, no amount of initial-state setting would help00:44
kota_calculated when uploading a manifest00:44
kota_'PUT manifest for slo'00:45
zaitcevit could be optional... like if you use that middleware thing that unpacks tarballs, then voila you have full etag. otherwise, no change from today.00:45
zaitcevwait, that middleware does different thing00:46
torgomatickota_: so, take the unpadded MD5 of segment 1, then make a request to segment 2's object server with that unpadded md5 that spins up a new generator with the unpadded MD5 as initial state, feeds in the bytes, and responds with the new unpadded state?00:46
torgomaticsaves the network but not the disk IO, in that case00:46
kota_tagomatic: yes00:46
kota_sacrifice the I/O00:47
kota_but save network.00:47
* notmyname wants the magic drives kota_ has00:48
kota_s/"the I/O"/"the disk I/O"/00:48
torgomaticinteresting... in my (admittedly limited) experience, disk IO runs out before network does00:49
torgomaticalso the latency on such a request would be phenomenal00:49
kota_torgomatic: yes, curent issue is the latency.00:49
kota_We have already write the code to make sure the efficiency but our current implementation is too slow00:50
kota_The consideration for the performance issue is still remaining but he is tryining to make it faster by shaping the code.00:52
torgomaticWell, I wish him luck with that. However, unless he can find some way to avoid reading all those bytes off disk, I'm afrait it will not be possible to speed up very much.00:55
zaitcevmaybe some kind of stateful upload00:56
zaitcevin sequence00:56
kota_torgomatic: Thanks, I will talk with him more for the disk I/O.00:57
kota_zaitcev: ok00:58
kota_Anyways, we are going to figure out about this discussion by hackathon and then would like to talk more.00:59
*** oomichi has joined #openstack-swift01:00
kota_Thanks, notmyname, torgomatic and zaitcev!01:00
*** addnull has joined #openstack-swift01:03
*** openstackgerrit has quit IRC01:05
*** openstackgerrit has joined #openstack-swift01:05
*** gyee has quit IRC01:12
*** tellesnobrega_ has quit IRC01:15
*** nexusz99 has joined #openstack-swift01:18
*** nexusz99 has quit IRC01:37
*** nexusz99 has joined #openstack-swift01:55
*** nexusz99 has quit IRC02:04
*** tsg has quit IRC02:04
*** IRTermite has quit IRC02:07
notmynameblog post on tempurls almost done...02:09
mattoliveraunotmyname: nice, once 2.2.2 is realeased a blog post on overload would be nice, feel free to delagate that to torgomatic or clayg tho ;)02:12
notmynameoh, yeah. definitely going to happen02:12
mattoliverauSwfitstack blog is an awesome ressource to point at even for those of us who don't work there ;)02:13
notmynamethat's the idea02:13
notmynameyou've discovered our business strategy.02:13
notmynameshhhh. don't tell anyone02:13
*** haomaiwang has joined #openstack-swift02:14
mattoliveraunotmyname: oh crap, you mean this isn't a private chat.... sorry :P02:16
*** openstackgerrit has quit IRC02:20
*** openstackgerrit has joined #openstack-swift02:21
*** IRTermite has joined #openstack-swift02:25
*** jkugel has joined #openstack-swift02:32
*** addnull has quit IRC02:38
*** tellesnobrega_ has joined #openstack-swift02:43
*** occupant has quit IRC02:46
*** ahonda has quit IRC02:49
*** NM has joined #openstack-swift02:50
*** bkopilov has quit IRC03:05
*** NM has quit IRC03:06
*** jkugel has quit IRC03:07
*** jkugel has joined #openstack-swift03:09
*** jkugel has quit IRC03:09
*** jkugel has joined #openstack-swift03:10
*** openstackgerrit has quit IRC03:20
*** openstackgerrit has joined #openstack-swift03:20
*** jkugel has quit IRC03:33
*** oomichi has quit IRC04:01
*** kota_ has quit IRC04:03
*** Dieterbe has joined #openstack-swift04:14
*** silor has joined #openstack-swift04:15
*** bkopilov has joined #openstack-swift04:19
*** jrichli has joined #openstack-swift04:21
*** kei_yama has quit IRC04:28
*** addnull has joined #openstack-swift04:29
*** jrichli has quit IRC04:39
*** sileht has quit IRC04:40
*** tellesnobrega_ has quit IRC04:46
*** gvernik has joined #openstack-swift04:47
openstackgerritDaisuke Morita proposed openstack/swift: Output account-reaper's logs for PolicyError
*** ahonda has joined #openstack-swift04:57
*** gvernik has quit IRC04:58
*** jrichli has joined #openstack-swift05:01
*** SkyRocknRoll has joined #openstack-swift05:02
*** jrichli has quit IRC05:11
*** jrichli has joined #openstack-swift05:12
*** jrichli has quit IRC05:18
*** echevemaster has quit IRC05:31
*** addnull has quit IRC05:33
*** ppai has joined #openstack-swift05:48
*** oomichi_ has joined #openstack-swift05:57
*** oomichi_ has quit IRC05:57
*** gvernik has joined #openstack-swift06:01
*** zaitcev has quit IRC06:18
openstackgerritMerged openstack/swift: don't print cached dispersion if it's a lie
openstackgerritMerged openstack/swift: 2.2.2 changelog and authors update
*** gvernik has quit IRC06:19
openstackgerritMerged openstack/swift: Allow set_overload to take value as percent
mattoliveraunotmyname: ^^ they finally made it through zuul :)06:24
mattoliverauon that note, I'm calling it a day. night all.06:24
*** reed has quit IRC06:26
*** SkyRocknRoll has quit IRC06:27
*** mahatic has joined #openstack-swift06:28
*** sileht has joined #openstack-swift06:29
*** SkyRocknRoll has joined #openstack-swift06:39
*** JoshNang has quit IRC06:52
*** mahatic has quit IRC07:00
*** JoshNang has joined #openstack-swift07:01
openstackgerritDaisuke Morita proposed openstack/swift: Output account-reaper's logs for PolicyError
*** mahatic has joined #openstack-swift07:13
openstackgerritHisashi Osanai proposed openstack/swift: Allow hostnames for nodes in Rings
*** oomichi has joined #openstack-swift07:33
*** addnull has joined #openstack-swift08:06
*** geaaru has joined #openstack-swift08:16
*** geaaru_ has joined #openstack-swift08:26
*** geaaru has quit IRC08:27
*** nellysmitt has joined #openstack-swift08:35
*** chlong has quit IRC08:36
*** rledisez has joined #openstack-swift08:37
*** BobBall_AWOL is now known as BobBall09:09
*** addnull has quit IRC09:10
*** jordanP has joined #openstack-swift09:10
openstackgerritBob Ball proposed openstack/swift: Remove deprecated config variables
openstackgerritBob Ball proposed openstack/swift: Remove deprecated config variables
*** jistr has joined #openstack-swift09:16
*** acoles_away is now known as acoles09:21
*** addnull has joined #openstack-swift09:23
acolestorgomatic: thanks for the rechecks!10:05
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements
*** chlong has joined #openstack-swift10:13
*** nshaikh has joined #openstack-swift10:14
*** tellesnobrega_ has joined #openstack-swift10:17
*** addnull has quit IRC10:24
*** chlong has quit IRC10:27
hoping Donagh McCabe10:31
*** haomaiwang has quit IRC10:34
*** silor has quit IRC10:40
*** jordanP has quit IRC10:41
*** tellesnobrega_ has quit IRC10:43
*** chlong has joined #openstack-swift10:44
*** tellesnobrega_ has joined #openstack-swift11:00
*** nshaikh has left #openstack-swift11:01
*** tellesnobrega_ has quit IRC11:06
*** donagh has joined #openstack-swift11:07
*** tellesnobrega_ has joined #openstack-swift11:07
*** oomichi has quit IRC11:08
*** tellesnobrega_ has quit IRC11:11
*** NM has joined #openstack-swift11:20
donaghho: you wanted to talk?11:23
hodonagh: yes!11:24
hodonagh: I just wrote a response for your comment.11:25
hodonagh: I understand your specification is already approved so I should remove -1 with the comment.11:25
hodonagh: I want to talk (tell you) about my concern11:26
donaghho: I'm not sure "approved" is the right word...11:26
donaghIt's in the "in-progress" section11:26
hodonagh: i see.11:26
hodonagh: My concerns to implement this without policy-based RBAC are following:11:26
donaghwhich means that a number of cores believe the code work should go ahead and that it will be reviewed11:26
*** oomichi_ has joined #openstack-swift11:27
hodonagh: i believe your spec is right direction what we go :)11:27
hodonagh: From end users point of view, end users need to know both usages, one is OpenStack standard and another is Swift only. Swift's implementation will be unique even if the OSLO incubator policy engine will be graduated in Kilo.11:28
donaghho: I think it's a matter of timing. Adding composite tokens and multiple prefixes is an agreed plan.11:28
hodonagh: From Swift's community point of view, we need to maitain both if they will be landed.11:29
hodonagh: but i don't know my understanding is corrent or not...11:29
donaghI think they are ortogonal issues/features11:30
donaghThe composite token/milti-resellers is a feature designed to support Glance/Cinder use-case11:30
donaghIt's implemented on the current keystoneauth design because that's what's there now11:30
donaghConverting keystoneauth to using policy files is a future feature11:31
donaghI'm not arguing that policy should not be done....but we should not delay composite tokens/multipe-resellers to also incorporate poliy11:32
donaghDoing so would complicate acceptance11:32
hodonagh: i summarize my understanding and thought
donaghOk...let me read that11:32
hodonagh: then my concerns above come to me.11:33
*** NM1 has joined #openstack-swift11:35
donaghho: let me take those items one by one11:35
hodonagh: sorry...11:35
*** NM has quit IRC11:37
donaghItem 1: I read lines 77-79 as a more general description. Specifically, "policy engine" just means "however the service handles roles, projects and resources"!11:37
donaghThe overall direction of the service token spec was to agree how the protocol (HTTP_X_SERVICE_ROLES, etc.) was done.11:38
donaghSwift *has* a policy uses options called operator_roles and an ACL syntax11:39
*** chlong has quit IRC11:39
*** aix has joined #openstack-swift11:40
donaghRe item 2, what are you asking to correct? Is that use of the word "incubation"11:42
hodonagh: i mean oslo incubator module does not need to support X-service-token11:43
hodonagh: it's just a library it depends on how to use it from openstack compo.11:44
donaghOk, I don't know enough about the library to know how to use it.11:45
donaghI was thinking of something such as "role: swiftoperator"11:45
donagh...I had assumed this intercepted the HTTP_X_ROLES environment variable?11:46
donaghIs that how it works, or doe sit work in another way?11:46
hodonagh: for example, just a moments.11:48
hodonagh: first, swift prepares policy file. in this file we describe the rule like "role: swiftoperator and role: reselleradmin"11:49
donaghok.. I understand that part11:50
hodonagh: second: keystoneauth gets a request from a users, he passes an identity (from envrion, HTTP_X_XXX) to policy engine.11:51
donaghI see a call such as: enforcer.enforce(action, target, creds)11:51
hodonagh: then policy engine compares identity and policy file, this request can be authorized or not.11:51
*** dmorita has quit IRC11:51
hodonagh: yes that right!11:52
donaghSo environment is not passed in -- however, "creds" is passed in11:52
hodonagh: yes.11:53
donaghI see creds is a dictionary, with items such as "roles"11:53
donaghSo I assume enforcer, when it parses something such as "role:swiftoperator" knows to look at the "roles" item in the creds argument. Correct11:54
hodonagh: info from identity see _get_creds in
donaghI saw question is how enforcer works11:55
hodonagh: i will write a case with this conf
hodonagh: in this file. rules are described.11:57
*** tdasilva has quit IRC11:57
donaghI understand that part (I think).11:58
donaghHowever, lets examine just one line11:58
hodonagh: when enforcer.enforce(action, target, creds) is called,11:58
hodonagh: ok11:58
donagh'reseller_admin_roles': 'role:ResellerAdmin'11:59
donaghThat says the condition reseller_admin_roles is met if the role ResellerAdmin is present in creds.roles11:59
*** oomichi_ has quit IRC11:59
hodonagh: yes12:00
donaghOk, so enforcer has code to parse the word "role:", pick up he next item, "ResellerAdmin" and then look at the "role" item in the creds argument12:01
hodonagh: enforcer checks action first.12:01
hodonagh: like get_account in 'post_account': 'rule:allowed_user'12:02
donaghHowever, you agree there is code looking for the string "role:" ??12:03
hodonagh: he meets allowed_user then parses 'allowed_user': ('(rule:authenticated and ' ...12:03
donaghho: you still there?12:05
hodonagh: sorry for multi lines. he continusly parses til (usually) meet role like 'operator_roles': 'role:admin or role:swiftoperator',12:05
*** chlong has joined #openstack-swift12:05
hodonagh: sorry for late response. just need time to write it in english.12:06
donaghMy point is that "role:" has a specific meaning. For service tokens, we need something such as "service_role: blah"12:06
donaghBecause oslo.policy does not currently support "service_role:"  or does it?12:07
hodonagh: oslo.policy is a just library. therefore they don't have "service_role" but can support it.12:08
hodonagh: s/can support it/can work with it/12:08
donaghSorry for beign so detailed, but that was the point of the paragraph you mention in your point 212:08
hodonagh: no my explantion is wrong. let me explain: it's just one correction which i think you have a mis-understanding about oslo incubator policy check.12:10
donaghOk -- since I'm still doing updates for the spec, I'll say something such as:12:12
donagh..hhhm. actually, now that I think about it...I will simply delete the paragraph since I'm *not* using oslo.policy in the solution12:13
donaghAnyway, that's just detail. Were there other points you wanted to make?12:14
*** oomichi_ has joined #openstack-swift12:16
hodonagh: i wrote above. My concerns to implement this without policy-based RBAC are following: From end users point of view, end users need to know both usages, one is OpenStack standard and another is Swift only. Swift's implementation will be unique even if the OSLO incubator policy engine will be graduated in Kilo.  From Swift's community point of view, we need to maitain both if they will be landed.12:16
hodonagh: so I would like to contribute for Swift better so I would appricate if you could think for co-work to realize this functionality before landed.12:17
donaghI think we are talking at cross purposes12:17
hodonagh: ?12:17
*** Guest52 has joined #openstack-swift12:18
donaghI agree, and support, adding policy to keystoneauth12:18
donaghI dissagree that the work I'm doing must be based on policy12:18
donaghThe reason is that to do so makes two changes at one thime: add policy and add service token12:19
donaghGenerally, we like to make incremental changes12:19
donaghNot combine changes together12:19
*** Guest52 has quit IRC12:19
*** silor has joined #openstack-swift12:20
hodonagh: i think so too. so my proposal is we finish policy based patch first.12:20
donaghIn my estimation you won't land the policy patch for some time.12:20
*** Novtopro_ has joined #openstack-swift12:20
hodonagh: s/my proposal is/my proposal was/12:20
donaghFirstly, you don't have a a spec in swift-specs  ... so Swift core's don't know this is coming and have not discussed12:21
acolesho: sorry to barge in, but can i ask - is oslo.policy still in incubation ?12:21
donaghSecond, there are more interactions with your proposal12:21
donaghFirst you would need to land policy that matches exactly the current situation12:22
donaghThen oslo.policy needs to be updated to support service_roles12:23
hoacoles: kilo-2 (feb 5)12:23
donaghOnly then, would we be able to add composite tokens/resellers to keystoneauth12:23
donaghThe effect is to delay a feature (composite token/resellers) that's ready to go...and needed by other projects12:24
hodonagh: no oslo,policy doesn't need to care it.12:24
*** Novtopro_ has quit IRC12:25
acolesho: thanks12:25
donaghIf I'm wrong about whether or not oslo.policy needs a change, my general point still stands12:25
*** Novtopro_ has joined #openstack-swift12:25
donaghAs mentioned earlier, I think your first step is to put a proposal into swift-specs12:26
hodonagh: sorry. my proposal is too late for you12:26
donaghBy all means, you can propose in that that document that my work should wait.12:27
donaghThat allows the core reviewers to comment on that point12:27
donaghThe swift-specs process is supposed to allow reviewers to comment on all aspects of a proposal.12:28
acolesho: if oslo.policy can support service tokens then why not include it in your patch? i.e. base your patch on - the changes in 139086 to keystoneauth are not *huge*, most of the lines of change are tests and docs12:30
hoacoles: i didn't recognize donagh's patch at that time.12:31
hoacoles: not
acolesho: i mean, rebase 149930 on 137086 so 149930 will depend on the test donagh has for service tokens, then add to 149930 whatever is needed for service token in policy.json12:34
hoacoles: it possible but my idea is no. (5) in
acolesho: point (5) refers to this ???12:37
donaghho: that's the hostname support in ring files...wrong URL?12:38
hoacoles: and I need to ask the spec. regarding single-project and multi-project12:39
hodonagh: "Enable Role-based access control using oslo.policy in Swift"12:40
*** Novtopro_ has quit IRC12:40
*** Novtopro_ has joined #openstack-swift12:41
hodonagh: acoles: sorry. i need to fix the url.12:41
*** Novtopro_ has quit IRC12:42
acolesho: with 149930 are you allowing a choice of (a) continue to use role options in proxy-server.conf or (b) use policy file?12:42
donaghacoles: 149930 works with old operator_roles option in [keystoneauth]12:43
donaghor uses a new polciy_file option if present12:43
hoacoles: it can work both(proxy-server.conf or policy-file)12:43
acolesho: ok, and are you suggesting that the service token would *only* be supported if the policy-file is used?12:44
hoacoles: yes.12:44
acolesho: ok. so i am suggesting that since donagh has already done the work to support service token using proxy-server.conf then we should keep that and in a follow-on patch add support for policy-file12:46
hodonagh: acoles: . ok i understand. i just worried about impact for uses. sorry for this.12:47
acolesho: its better for users, they get service token support with either .conf file or policy-file.12:48
acolesho: also, if you can rebase 149930 on 137086 then it will demonstrate how policy-file will support service tokens, which will help us understand12:51
acolesho: btw, it must be late for you?12:52
rledisezho: sorry to step in in the middle of the discussion, but i'm very interrested in having a policy file that would allow to describe very precisely some actions12:53
rledisezho: today it's almost all or nothing (reseller or standard user)12:53
rledisezho: but it would be nice to be able to have role that can for example, only set quota12:54
rledisezho: = POST with some specific headers12:54
rledisezdo you think your proposal will allow that (not right now, but in a futur patch) ?12:54
hodonagh: acoles: thaks for your time!12:54
acolesho: you're welcome, thanks for the explanations. i have to go now for a while.12:55
donaghho: no problem. In the meantime, I may post some other comments on 14993012:56
donaghho: I need to leave now (lunch)12:58
horledisez: quota is in HTTP HEADER. if it correct, it is possible to realize it with my patch but needs some coding.13:02
hoacoles: donagh: thanks!13:03
horledisez: action, key for policy, is consists of http-method and target like post_account so it can work but to compare quota i need to put quta info in headers to credential or target.13:07
horledisez: i have to leave. good night!13:08
*** ho has quit IRC13:09
*** oomichi_ has quit IRC13:14
*** bkopilov has quit IRC13:14
*** EmilienM|afk is now known as EmilienM13:14
*** dmsimard_away is now known as dmsimard13:40
*** jordanP has joined #openstack-swift13:41
*** dmsimard is now known as dmsimard_away13:49
*** dmsimard_away is now known as dmsimard13:49
openstackgerritDonagh McCabe proposed openstack/swift: Add multiple reseller prefixes and composite tokens
*** dmsimard is now known as dmsimard_away13:49
*** tdasilva has joined #openstack-swift13:52
*** jistr has quit IRC13:58
*** jistr has joined #openstack-swift14:00
*** jkugel has joined #openstack-swift14:05
*** ppai has quit IRC14:16
*** rdaly2 has joined #openstack-swift14:27
*** jasondot_ has joined #openstack-swift14:34
*** jasondot_ has quit IRC14:45
*** mahatic has quit IRC15:00
*** omame has quit IRC15:02
*** omame has joined #openstack-swift15:03
*** jistr has quit IRC15:10
*** jistr has joined #openstack-swift15:12
*** dmsimard_away is now known as dmsimard15:21
*** mahatic has joined #openstack-swift15:21
*** NM1 has quit IRC15:29
*** NM has joined #openstack-swift15:29
*** bkopilov has joined #openstack-swift16:02
*** rdaly2 has quit IRC16:22
*** abhirc has quit IRC16:24
*** silor has quit IRC16:38
*** IRTermite1 has joined #openstack-swift16:48
*** IRTermite1 has quit IRC16:48
*** IRTermite has quit IRC16:48
*** IRTermite has joined #openstack-swift16:49
*** david-lyle_afk is now known as david-lyle16:49
notmynamegood morning16:53
*** jkremer has joined #openstack-swift16:59
mahaticnotmyname, good morning17:05
mahaticI saw the comments on the server type patch. I would like it torgomatic's way of having separate headers. But the rfc you pointed differs from that. So i'm kind of split on it.17:06
notmynamettx is tagging the 2.2.2 rc right now17:07
notmynamemahatic: ya, I haven't been able to convince torgomatic that he's wrong yet ;-)17:07
mahaticnotmyname, :D will wait on that then. I made that NotImplementedError change though. Will push it when we decide on the headers17:09
*** nellysmitt has quit IRC17:11
*** SkyRocknRoll has quit IRC17:13
*** rledisez has quit IRC17:13
notmyname2.2.2rc has just been tagged17:15
notmynamemattoliverau: tempurl feature highlight blog post
torgomaticnotmyname: mahatic: if you use a header other than "Server", you get to make up your own semantics :)17:19
notmynametorgomatic: X-Recon-Binary-Data-Payload ;-)17:20
notmynameso in this case we're specifically trying to send info about what kind of server it is back to the client. http has a server header already for that exact purpose17:21
notmynameI'm more ok with dropping the version (although I'd prefer it to be there now) than just making up another header for this17:22
notmynamerephrase: currently, I'd prefer for the version to be there. ie I haven't yet been convinced otherwise17:22
torgomaticwell, the driver for all this is to have swift-recon be able to ask "what are you?" to a Swift backend server, and we've chosen OPTIONS requests as the mechanism for the question and answer17:24
torgomaticso at some point, there will be code with an OPTIONS response in hand that has to figure out what kind of backend made the response17:24
torgomaticand, while it's not terribly difficult to do some string splitting and whatnot, it's even easier to type ".headers.get("X-Server-Type") == 'swift-account-server'"17:25
torgomaticbut if you have a strong preference for the Server header, then that's probably fine too, just more code to write17:26
*** NM has quit IRC17:27
notmynamemore or different?17:27
*** EmilienM is now known as EmilienM|afk17:27
notmynameheh, well the fun part is that we're discussing code that we aren't going to write but mahatic is :-)17:28
torgomaticthe Server header is a whitespace-separated string of component/version pairs, so it'll need a little parsing help, and probably some tests17:29
torgomaticI could see the Server header sprouting more information in the future (library versions, for example), while the hypothetical X-Swift-Server-Type wouldn't.17:30
mahaticnotmyname, I guess it's more than different17:30
notmynamerecon (currently) won't need to handle a server header with more than one component/version pair17:30
torgomaticold recon + new swift, when the our-future recon becomes old17:30
notmynametorgomatic: do you think a server header with the version would help various management systems (*wink*) with determining what should be upgraded or otherwise handled?17:30
*** jistr has quit IRC17:31
torgomaticnotmyname: just saying that a list of thingies with one thingy in it tends to evolve towards having more thingies over time :)17:31
*** NM has joined #openstack-swift17:31
notmynamesure. eventually "Server: object-server/4.28.92 recon-middeware/0.6.28 foobar-middleware/9.2 acme-proprietary/0.42"17:33
mahatictorgomatic, if I understand correctly, Server header could be used in future for other purposes and hence now go with a X-Server-Type etc?17:33
torgomaticmahatic: I have a slight preference for X-Server-Type, but notmyname has a preference for the Server header17:33
torgomaticmy preference is weak enough that you probably shouldn't pay much attention to it17:34
notmynametorgomatic: to paraphrase you, "there's an http thing that is for what we are trying to communicate, but it can be overused in the future so we shouldn't use it and instead make up a new header"17:34
notmyname(admittedly, a biased paraphrase)17:34
torgomaticnotmyname: just saying that it takes more code to parse a string than to do a straight comparison is all17:34
mahatictorgomatic, notmyname hmm, my preference is whatever makes the code easier to write ;)17:34
notmynamewe should get acoles clayg peluse cschwede to weigh in :-)17:35
torgomaticnotmyname: really, it's a weak preference, so if you have a good reason for the Server header (and it sounds like you do, because library versions are good to know), then go with thtat17:35
torgomaticalright they17:35
torgomaticI retract my preference for X-Server-Type; knowing eventlet version (someday in the future) is a good enough reason to use the Server header17:36
notmynameah, that's an interesting idea17:36
*** jodah has left #openstack-swift17:37
*** jordanP has quit IRC17:37
notmynamemahatic: ok. "Server" it is.17:37
notmynametorgomatic: with or without the version?17:37
torgomaticnotmyname: with; that's what the spec wants, no?17:37
mahaticnotmyname, torgomatic "Server: object-server/4.28.92 recon-middeware/0.6.28 foobar-middleware/9.2 acme-proprietary/0.42", will we get to use that info? I mean I'm sure there is a scenario, can you point me to it?17:37
notmynamemahatic: nah, that's just some made up future worst case17:38
notmynamemahatic: for now, it would be "Server: %s/%s" % (server_type, version)17:38
mahaticnotmyname, yeah17:38
notmynameand recon would look at server_header_value.split(' ', 1)[0].split('/'). and if that has an error, then it's unexpected anyway17:39
*** nellysmitt has joined #openstack-swift17:40
mahaticnotmyname, yeah. also where would the version be used?17:41
mahaticnotmyname, for future use?17:42
notmynamemahatic: for now, I'm not sure that it would be. or it wouldn't be required in the current feature17:42
notmynameya, for future use17:42
notmynametorgomatic: you ok with that?17:42
*** torgomatic has quit IRC17:59
*** reed has joined #openstack-swift18:00
*** torgomatic has joined #openstack-swift18:13
*** ChanServ sets mode: +v torgomatic18:13
*** Trixboxer has joined #openstack-swift18:14
openstackgerritShiva Chaitanya proposed openstack/swift-specs: Swift tiering specification
*** abhirc has joined #openstack-swift18:19
*** EmilienM|afk is now known as EmilienM18:36
*** jrichli has joined #openstack-swift18:47
*** erlon has quit IRC18:48
*** fandi has joined #openstack-swift18:48
*** openstackgerrit has quit IRC18:50
*** openstackgerrit has joined #openstack-swift18:51
tdasilvanotmyname: good job on the tempurl lightning talk..pretty funny :-)18:51
openstackgerritMerged openstack/swift: Don't set the already set Connection: close
openstackgerritMahati proposed openstack/swift: Add server type in OPTIONS response
*** lpabon has joined #openstack-swift19:25
*** reed has quit IRC19:27
*** bill_az_ has joined #openstack-swift19:29
*** tdasilva has quit IRC19:33
*** rdaly2 has joined #openstack-swift19:34
*** aix has quit IRC19:35
*** rdaly2_ has joined #openstack-swift19:47
*** tdasilva has joined #openstack-swift19:48
*** torgomatic_ has joined #openstack-swift19:50
*** rdaly2 has quit IRC19:51
*** torgomatic has quit IRC19:56
*** torgomatic_ is now known as torgomatic19:56
*** ChanServ sets mode: +v torgomatic19:56
*** jkremer has quit IRC20:01
*** openstackgerrit has quit IRC20:04
*** openstackgerrit has joined #openstack-swift20:04
*** nellysmitt has quit IRC20:07
*** nellysmitt has joined #openstack-swift20:07
*** nellysmitt has quit IRC20:12
*** bkopilov has quit IRC20:13
*** fifieldt has quit IRC20:24
*** fifieldt has joined #openstack-swift20:25
*** annegent_ has joined #openstack-swift20:29
*** os1 has joined #openstack-swift20:44
os1Does swift-recon yield cluster-wide data for memory usage, and/or network statistics?20:44
os1For instance, in the case of disk-usage, it seems to be able to yield a combined disk usage and combined capacity statistic when run.20:45
os1combined, as in, the total disk usage of the cluster, and the total capacity of the cluster.20:46
os1without having to manually add up the values that are returned.20:46
os1I'm wondering if memory-related information, and/or network-related information, can be printed this way, too?20:47
os1(for the cluster)20:48
ctennisI don't believe swift-recon gathers that info20:48
*** rdaly2 has joined #openstack-swift20:52
*** rdaly2 has quit IRC20:54
*** rdaly2_ has quit IRC20:55
notmynamesome of it is in swift-recon20:56
notmynameload average and socket usage20:57
os1Okay. When I'm querying it through curl, on the other hand,20:58
*** annegent_ has quit IRC20:58
os1curl -i http://<bind-ip>:6000/recon/mem and curl -i http://<bind-ip>:6000/recon/sockstat20:59
os1notmyname : Does that give combined (or averaged) data across the nodes in the cluster?21:00
*** tdasilva has quit IRC21:01
os1If not, then is there a way to do so?21:01
*** tdasilva has joined #openstack-swift21:02
notmynameos1: no, that would be for one server. the swift-recon CLI will aggregate across many servers (you can specify the whole cluster or a zone or a server or etc)21:05
os1notmyname : Okay, that's what it seemed like. However, what if you were to send the curl request to your HAProxy server?21:06
*** annegent_ has joined #openstack-swift21:08
notmynameyou'd get the results from whatever backend the LB chooses right?21:09
mattoliverauMorning all21:09
*** acoles is now known as acoles_away21:10
notmynamemattoliverau: hi21:10
mattoliverauacoles_away: becomes away when I come online :( one day we'll overlap a little better again! P.S Been reviewing your fast post, I like the way you encoded the timestamps, nice thinking!21:12
*** zaitcev has joined #openstack-swift21:13
*** ChanServ sets mode: +v zaitcev21:13
os1notmyname : That's what I assumed, as well.21:21
os1notmyname : Lastly, then, do you know whether the swift-recon CLI can print out aggregate data for memory usage?21:22
*** NM has quit IRC21:23
os1notmyname : I can't seem to find it in the list returned by "swift-recon -h", but it's interesting that you can retrieve /recon/mem via curl.21:23
notmynameos1: interesting. that doesn't seemed plumbed through the CLI. should be relatively easy to do so. would you be interested in working on a patch?21:26
os1notmyname : Possibly. It sounds like it would be fun to do, and I may have time to look into it.21:32
notmynameos1: let me know if you need any help getting started21:32
os1notmyname : Ok.21:32
*** geaaru_ has quit IRC21:52
*** tdasilva has quit IRC21:59
*** Nadeem has joined #openstack-swift22:04
*** chlong has quit IRC22:08
*** nellysmitt has joined #openstack-swift22:08
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements
*** dmsimard is now known as dmsimard_away22:11
*** nellysmitt has quit IRC22:13
os1Hi. Do the proxy servers ever need to use rsync ?22:13
*** dmsimard_away is now known as dmsimard22:13
*** tdasilva has joined #openstack-swift22:19
*** dmsimard is now known as dmsimard_away22:24
*** donagh has quit IRC22:32
notmynameos1: no22:35
notmynameos1: that's only needed on storage nodes22:35
openstackgerritMerged openstack/swift: Add server type in OPTIONS response
*** abhirc has quit IRC22:40
*** annegent_ has quit IRC22:42
*** lpabon has quit IRC22:51
*** openstackgerrit has quit IRC22:51
*** openstackgerrit has joined #openstack-swift22:51
*** tellesnobrega_ has joined #openstack-swift22:51
*** occupant has joined #openstack-swift22:54
*** jrichli has quit IRC22:57
*** mahatic has quit IRC23:04
*** panbalag has quit IRC23:06
*** jkugel has quit IRC23:21
*** jkugel has joined #openstack-swift23:21
*** jkugel has quit IRC23:26
*** echevemaster has joined #openstack-swift23:30
*** tdasilva has quit IRC23:33
*** chlong has joined #openstack-swift23:53
*** abhirc has joined #openstack-swift23:56
*** rdaly2_ has joined #openstack-swift23:57

Generated by 2.14.0 by Marius Gedminas - find it at!