| *** itlinux has quit IRC | 02:29 | |
| *** psachin has joined #openstack-swift | 02:59 | |
| *** rcernin has quit IRC | 07:23 | |
| *** gkadam has joined #openstack-swift | 07:52 | |
| *** ccamacho has joined #openstack-swift | 07:55 | |
| *** openstackgerrit has joined #openstack-swift | 09:38 | |
| openstackgerrit | melissaml proposed openstack/swift master: fix a typo in docstring https://review.openstack.org/624034 | 09:38 |
|---|---|---|
| *** e0ne has joined #openstack-swift | 10:03 | |
| *** e0ne has quit IRC | 10:30 | |
| *** e0ne has joined #openstack-swift | 10:36 | |
| *** e0ne has quit IRC | 11:58 | |
| *** e0ne has joined #openstack-swift | 12:00 | |
| *** e0ne has quit IRC | 12:00 | |
| *** mahatic has joined #openstack-swift | 12:01 | |
| *** ChanServ sets mode: +v mahatic | 12:01 | |
| *** NM has joined #openstack-swift | 12:11 | |
| *** jistr is now known as jistr|medchk | 12:57 | |
| *** psachin has quit IRC | 13:05 | |
| *** jistr|medchk is now known as jistr | 13:56 | |
| *** e0ne has joined #openstack-swift | 14:15 | |
| *** openstackstatus has joined #openstack-swift | 14:18 | |
| *** ChanServ sets mode: +v openstackstatus | 14:18 | |
| *** gkadam has quit IRC | 14:24 | |
| *** mvkr has quit IRC | 14:55 | |
| *** mvkr has joined #openstack-swift | 15:25 | |
| *** itlinux has joined #openstack-swift | 15:44 | |
| *** itlinux has quit IRC | 15:46 | |
| *** NM has quit IRC | 16:12 | |
| *** openstackgerrit has quit IRC | 16:35 | |
| *** psachin has joined #openstack-swift | 16:42 | |
| *** gyee has joined #openstack-swift | 16:47 | |
| *** ccamacho has quit IRC | 16:52 | |
| *** itlinux has joined #openstack-swift | 16:56 | |
| notmyname | good morning | 17:06 |
| *** openstackgerrit has joined #openstack-swift | 17:19 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: s3api: Stop mangling Authorization header for v4 signatures https://review.openstack.org/591862 | 17:19 |
| *** e0ne has quit IRC | 17:22 | |
| *** e0ne has joined #openstack-swift | 17:23 | |
| *** e0ne has quit IRC | 17:23 | |
| *** e0ne has joined #openstack-swift | 17:26 | |
| notmyname | hooray! all the s3api patches have landed! | 17:26 |
| timburke | yup! i'm excited :-) | 17:26 |
| timburke | if you wanted to get a couple more small ones in, though, there's https://review.openstack.org/#/c/621055/ and https://review.openstack.org/#/c/617043/ | 17:27 |
| patchbot | patch 621055 - swift - s3api: Look for more indications of aws-chunked up... - 3 patch sets | 17:27 |
| patchbot | patch 617043 - swift - s3token: Add an underscore to the end of reseller_... - 1 patch set | 17:27 |
| timburke | or https://review.openstack.org/#/c/591862/ or https://review.openstack.org/#/c/609168/ ... | 17:29 |
| patchbot | patch 591862 - swift - s3api: Stop mangling Authorization header for v4 s... - 6 patch sets | 17:29 |
| patchbot | patch 609168 - swift - We don't have to keep the retrieved token anymore - 2 patch sets | 17:29 |
| timburke | i'm hoping to get https://review.openstack.org/#/c/603529/ reviewed today... | 17:29 |
| patchbot | patch 603529 - swift - s3 secret caching - 10 patch sets | 17:29 |
| timburke | notmyname: i guess what i'm trying to say is, i can always find you more patches to approve :P | 17:29 |
| notmyname | lol, I got distracted in the office and couldn't type "but timburke will tell me there's a bunch more ready to land" ;-) | 17:29 |
| *** e0ne has quit IRC | 18:09 | |
| *** e0ne has joined #openstack-swift | 18:11 | |
| *** psachin has quit IRC | 18:23 | |
| cwright | Hi, we have a vendor who is trying to integrate with our S3 compatible Swift cluster (pike/2.15.1). | 18:31 |
| cwright | The vendor has working integrations with many other S3 compatible object stores, but their integration tests are failing against our cluster. | 18:31 |
| cwright | Our logs show `BadSwiftRequest: Metadata value longer than 256: x-object-meta-X-Amz-Key`, and I can see in the request that the value is a base64 encoded value that is 345 bytes. | 18:31 |
| cwright | Digging through the Swift source code I've found that it defaults to a limit of 256 bytes, but seems to be configurable via `max_meta_value_length` in `[swift-constraints]`. | 18:31 |
| cwright | For comparison, the Amazon S3 documentation states "User-defined metadata can be as large as 2 KB". | 18:31 |
| cwright | I'd like to understand the implications of raising this limit. Has anyone else here needed to raise `max_meta_value_length`? | 18:31 |
| cwright | Or can anyone explain what I may need to consider if I raise this to 2 KB? | 18:31 |
| notmyname | raising it is fine. also look out for the overall max metadata length to see if you need to raise that | 18:31 |
| notmyname | there's a config value for the max single key size and the overall total size | 18:32 |
| notmyname | ultimately, more metadata will end up using more overhead in the xfs file system. that's really the "risk" | 18:32 |
| notmyname | well, also the metadata size is *not* part of the overall reported bytes used for the account | 18:33 |
| notmyname | but if you're aware of those issues as the operator, you can take appropriate steps | 18:33 |
| notmyname | eg more ram and potentially on-disk bytes (or just more memory monitoring) for the higher inode usage. | 18:34 |
| *** e0ne has joined #openstack-swift | 18:34 | |
| cwright | notmyname: thanks so much. Would you recommend I raise the `max_meta_overall_size` setting by the same amount I raise `max_meta_value_length`? | 18:35 |
| notmyname | it's totally possible to build a fuse filesystem on top of swift where you use zero-byte objects and just store the data in the object metadata fields. it would be horribly slow and terrible for the ops running the cluster, but the customer wouldn't have any bytes used reported and could therefore have it for "free" | 18:35 |
| timburke | cwright: fwiw, the clusters i deploy go with swift_max_meta_name_length=2048, swift_max_meta_value_length=2048, swift_max_meta_overall_size=4096 by default | 18:35 |
| notmyname | that's one of the reasons we originally implemented the limits ;-) | 18:35 |
| cwright | timburke: that is very helpful, thanks so much | 18:35 |
| notmyname | cwright: no, they don't need to be the same. it's really just a knob to influence how customers organize the data. do you want them to use many metadata keys? or just one big one? | 18:36 |
| notmyname | TBH, I have no idea why you'd want them to do one over the other. but you have the knobs in the system | 18:36 |
| notmyname | cwright: oh! how's the promethius stuff working out? | 18:36 |
| cwright | notmyname: this is the only issue related to metadata length we've had so far, so we aren't exactly sure how many metadata keys they will need to use. This one in particular is important for their client side encryption integrations | 18:37 |
| cwright | notmyname: I forked the prometheus exporter project to begin updating the paths in the code and systemd configs but got slowed down trying to figure out how to build the project. I know beggars can't be choosers but just some simple build docs would be tremendously helpful :) | 18:38 |
| notmyname | cwright: that's fair. I'll pass it along | 18:39 |
| cwright | (I don't have any experience with Go yet) | 18:39 |
| cwright | notmyname: that would be great, thanks again, I really appreciate it | 18:41 |
| timburke | "client side encryption integrations" <-- yes! that's right, i'm pretty sure that was a motivating factor for why *we* increased it, too ;-) | 18:43 |
| cwright | timburke: that makes me feel even better :) | 18:44 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Treat all invalid frag indexes the same https://review.openstack.org/615990 | 18:48 |
| *** e0ne has quit IRC | 19:06 | |
| *** mvkr has quit IRC | 19:09 | |
| openstackgerrit | Merged openstack/swift master: fix a typo in docstring https://review.openstack.org/624034 | 20:10 |
| *** mvkr has joined #openstack-swift | 20:16 | |
| *** e0ne has joined #openstack-swift | 20:28 | |
| *** e0ne has quit IRC | 20:28 | |
| openstackgerrit | Merged openstack/swift master: Py3: Use BytesIO when we want bytes https://review.openstack.org/621377 | 22:07 |
| *** itlinux has quit IRC | 22:43 | |
| *** rcernin has joined #openstack-swift | 22:59 | |
| *** itlinux has joined #openstack-swift | 23:48 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!