*** early has quit IRC | 00:03 | |
openstackgerrit | Merged openstack/swift master: Fix suffix-byte-range responses for zero-byte EC objects. https://review.openstack.org/526565 | 00:07 |
---|---|---|
*** early has joined #openstack-swift | 00:16 | |
openstackgerrit | Merged openstack/python-swiftclient master: Make swiftclient respect region_name when using sessions https://review.openstack.org/568979 | 00:18 |
*** links has joined #openstack-swift | 00:44 | |
*** d0ugal__ has quit IRC | 00:58 | |
*** d0ugal__ has joined #openstack-swift | 00:59 | |
*** m_kazuhiro has joined #openstack-swift | 01:08 | |
m_kazuhiro | good morning | 01:09 |
*** lifeless has quit IRC | 01:15 | |
*** gyankum has joined #openstack-swift | 01:18 | |
*** itlinux has joined #openstack-swift | 01:30 | |
*** itlinux has quit IRC | 01:38 | |
*** itlinux has joined #openstack-swift | 01:42 | |
*** itlinux has quit IRC | 01:45 | |
*** itlinux has joined #openstack-swift | 01:52 | |
*** lifeless has joined #openstack-swift | 01:58 | |
*** amandap has quit IRC | 01:59 | |
*** amandap has joined #openstack-swift | 02:00 | |
*** amandap has quit IRC | 02:04 | |
kota_ | good morning | 02:08 |
*** lifeless_ has joined #openstack-swift | 02:14 | |
*** lifeless has quit IRC | 02:14 | |
*** psachin has joined #openstack-swift | 02:36 | |
openstackgerrit | Merged openstack/swift master: display swift services in apache2 https://review.openstack.org/442724 | 02:54 |
*** itlinux has quit IRC | 03:06 | |
*** klamath has quit IRC | 03:10 | |
*** amandap has joined #openstack-swift | 03:20 | |
*** dcourtoi_ has joined #openstack-swift | 03:39 | |
*** lifeless_ has quit IRC | 03:44 | |
*** DHE has quit IRC | 03:44 | |
*** AndyWojo has quit IRC | 03:44 | |
*** timburke has quit IRC | 03:44 | |
*** StevenK has quit IRC | 03:44 | |
*** kota_ has quit IRC | 03:44 | |
*** MooingLemur has quit IRC | 03:44 | |
*** dcourtoi has quit IRC | 03:44 | |
*** tristanC has quit IRC | 03:44 | |
*** amandap has quit IRC | 03:45 | |
*** amandap has joined #openstack-swift | 03:45 | |
*** m_kazuhiro has quit IRC | 03:52 | |
*** d0ugal__ has quit IRC | 03:52 | |
*** m_kazuhiro has joined #openstack-swift | 04:09 | |
*** d0ugal__ has joined #openstack-swift | 04:14 | |
*** itlinux has joined #openstack-swift | 04:20 | |
*** cbartz has joined #openstack-swift | 04:33 | |
*** d0ugal__ has quit IRC | 04:49 | |
*** gyee has quit IRC | 04:51 | |
*** d0ugal__ has joined #openstack-swift | 04:57 | |
*** klamath has joined #openstack-swift | 05:10 | |
*** itlinux has quit IRC | 05:11 | |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job https://review.openstack.org/564700 | 05:18 |
*** cshastri has joined #openstack-swift | 05:41 | |
*** d0ugal__ has quit IRC | 06:26 | |
*** amandap has quit IRC | 06:40 | |
*** pcaruana has joined #openstack-swift | 06:40 | |
*** hseipp has joined #openstack-swift | 06:41 | |
*** amandap has joined #openstack-swift | 06:44 | |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job https://review.openstack.org/564700 | 06:44 |
*** d0ugal__ has joined #openstack-swift | 06:48 | |
*** lifeless has joined #openstack-swift | 07:07 | |
*** rcernin has quit IRC | 07:08 | |
*** m_kazuhiro has quit IRC | 07:13 | |
*** ccamacho has quit IRC | 07:16 | |
*** ccamacho has joined #openstack-swift | 07:16 | |
*** geaaru has joined #openstack-swift | 07:20 | |
*** mvk has joined #openstack-swift | 07:24 | |
*** tesseract has joined #openstack-swift | 07:25 | |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift master: WIP: Add ceph-s3 test non-voting job https://review.openstack.org/564700 | 07:29 |
acoles | good morning | 07:42 |
*** cbartz has quit IRC | 07:45 | |
*** DHE has joined #openstack-swift | 07:47 | |
*** AndyWojo has joined #openstack-swift | 07:47 | |
*** timburke has joined #openstack-swift | 07:47 | |
*** StevenK has joined #openstack-swift | 07:47 | |
*** kota_ has joined #openstack-swift | 07:47 | |
*** MooingLemur has joined #openstack-swift | 07:47 | |
*** tristanC has joined #openstack-swift | 07:47 | |
*** barjavel.freenode.net sets mode: +vv timburke kota_ | 07:47 | |
openstackgerrit | Alistair Coles proposed openstack/swift master: Verify diff stat is unchanged when syncing only shard ranges https://review.openstack.org/570171 | 07:50 |
*** mikecmpbll has joined #openstack-swift | 08:03 | |
*** d0ugal__ has quit IRC | 08:03 | |
*** d0ugal has joined #openstack-swift | 08:03 | |
*** amandap has quit IRC | 08:21 | |
*** amandap has joined #openstack-swift | 08:21 | |
*** mvk has quit IRC | 08:29 | |
*** lifeless has quit IRC | 08:37 | |
*** mvk has joined #openstack-swift | 08:37 | |
*** armaan has joined #openstack-swift | 08:39 | |
*** lifeless has joined #openstack-swift | 08:44 | |
openstackgerrit | Merged openstack/swift master: Fix marker and end_marker descriptions in API ref https://review.openstack.org/571460 | 08:46 |
*** lifeless_ has joined #openstack-swift | 08:52 | |
*** lifeless has quit IRC | 08:52 | |
openstackgerrit | Merged openstack/swift master: Verify diff stat is unchanged when syncing only shard ranges https://review.openstack.org/570171 | 08:56 |
*** amandap has quit IRC | 09:06 | |
*** prasen has joined #openstack-swift | 09:09 | |
*** d0ugal has quit IRC | 09:33 | |
*** mvk has quit IRC | 09:33 | |
*** corvus has quit IRC | 09:33 | |
*** corvus has joined #openstack-swift | 09:34 | |
*** mvk has joined #openstack-swift | 09:34 | |
*** mvk has quit IRC | 09:42 | |
*** mvk has joined #openstack-swift | 09:42 | |
remix_tj | hello, anyone can help me with a stupid thing? I'm very noob. I'm writing a middleware and i need to change an header a user puts called "X-Something" into an header called X-Object-Meta-Somethingelse | 09:43 |
remix_tj | i'm using in my __call__ function req = Request(env). At the end of the __call__ function if something has changed, how to i pass the altered environment to the next middleware in the pipeline? | 09:45 |
remix_tj | i see several self.app(env, start_response) but seems that env gets not altered by me and environment is passed clearly. I've seen in some middleware Request.blank(req.path_info, environ=req.environ) is used. Is it right? | 09:47 |
*** amandap has joined #openstack-swift | 09:52 | |
*** d0ugal has joined #openstack-swift | 09:52 | |
acoles | remix_tj: there's an example of something similar in encrypter.py middleware https://github.com/openstack/swift/blob/3a1a198780433e6ebf2bcc3862bd63c7d033930d/swift/common/middleware/crypto/encrypter.py#L207-L280 | 09:52 |
remix_tj | acoles: thank you | 09:53 |
acoles | note how the middleware uses a subclass of WSGIContext to handle requests - that's a good pattern to follow | 09:53 |
remix_tj | uhm, it's totally different from samples i've found, where middleware was derived directly by object | 10:00 |
remix_tj | i was looking at https://docs.openstack.org/swift/latest/development_middleware.html#creating-your-own-middleware and at https://github.com/redhat-cip/swift-middleware-sample | 10:01 |
remix_tj | ok, i'll try to rewrite in an easier way | 10:01 |
*** d0ugal has quit IRC | 10:16 | |
*** d0ugal has joined #openstack-swift | 10:24 | |
acoles | remix_tj: are you using the wsgify decorator? back to you original question - if you modify req.headers then the modified headers should be passed to next middleware | 10:35 |
*** sajjadg has joined #openstack-swift | 10:37 | |
*** bkopilov_ has quit IRC | 10:39 | |
*** cbartz has joined #openstack-swift | 10:48 | |
remix_tj | acoles: yes i am | 11:01 |
*** d0ugal has quit IRC | 11:06 | |
*** armaan has quit IRC | 11:06 | |
*** armaan has joined #openstack-swift | 11:06 | |
*** armaan has quit IRC | 11:09 | |
*** armaan has joined #openstack-swift | 11:09 | |
*** armaan has quit IRC | 11:15 | |
*** armaan has joined #openstack-swift | 11:16 | |
*** bkopilov_ has joined #openstack-swift | 11:16 | |
*** d0ugal has joined #openstack-swift | 11:23 | |
*** d0ugal has quit IRC | 11:53 | |
*** d0ugal has joined #openstack-swift | 11:58 | |
*** armaan has quit IRC | 12:12 | |
*** armaan has joined #openstack-swift | 12:13 | |
*** d0ugal has quit IRC | 12:16 | |
*** hoonetorg has quit IRC | 12:18 | |
*** amandap has quit IRC | 12:29 | |
remix_tj | acoles: solved, i was confused by mixing several samples with different setups. Now i'm able to get an X-Something and convert to X-Object-Sysmeta-Somethingelse | 12:32 |
*** kei_yama has quit IRC | 12:33 | |
*** d0ugal has joined #openstack-swift | 12:34 | |
acoles | remix_tj: great | 12:36 |
remix_tj | at the moment i have a stupid middleware allowing to set X-Retention: $seconds header when creating object. Subsequent POST DELETE PUT requests are denied if request time is lower than the value of retention | 12:37 |
*** mikecmpb_ has joined #openstack-swift | 12:46 | |
*** mikecmpbll has quit IRC | 12:48 | |
acoles | remix_tj: at some point you are going to run into the problem of inconsistent object replicas i.e. when you check X-Object-Sysmeta-Something you may get an old replica of the object that doesn't have that metadata. | 12:54 |
remix_tj | does data and metadata get synced in a different time? I can also decide that retention can be set only at creation time, so if the object has been created with retention can be retained, in alternative you cannot | 13:00 |
*** mikecmpb_ has quit IRC | 13:02 | |
*** mikecmpbll has joined #openstack-swift | 13:02 | |
*** pcaruana has quit IRC | 13:15 | |
acoles | the data and metadata of a PUT request are written at same time to same file. you'd need to think about cases when sampling the existing object state returns a 404 but there are in fact existing replicas of the object, they are just not currently available. | 13:23 |
*** armaan has quit IRC | 13:28 | |
*** armaan has joined #openstack-swift | 13:28 | |
*** prasen has quit IRC | 13:30 | |
*** armaan has quit IRC | 13:32 | |
acoles | I haven't thought about this much, it's probably crazy, but your middleware could run time backwards i.e. assign decreasing x-timestamps to every request. Then swift's consistency protocols would favour the first request to an object url i.e. the first PUT. But...of course, you'd then *never* be able to delete an object :/ ...except by setting an x-delete-at on the first PUT | 13:34 |
*** links has quit IRC | 13:37 | |
*** klamath has quit IRC | 13:40 | |
*** klamath has joined #openstack-swift | 13:40 | |
*** armaan has joined #openstack-swift | 13:40 | |
*** klamath_ has joined #openstack-swift | 13:41 | |
remix_tj | acoles: you say the file has been PUT at time 0, i make a query at time 1 where replica hasn't been completed and when i try to get object info i get 404? | 13:41 |
*** klamath has quit IRC | 13:45 | |
acoles | that might happen under failure conditions when replicas are not available at time 1 | 13:46 |
*** klamath_ has quit IRC | 13:48 | |
*** klamath has joined #openstack-swift | 13:48 | |
remix_tj | acoles: but if i'm not able to get the object metadata from the backend i deny the operation, to be conservative | 13:52 |
acoles | that helps, but the PUT request may be the first and should be allowed, but only if it is the first? | 13:54 |
remix_tj | that's right | 13:59 |
remix_tj | but how i know if it is? | 13:59 |
remix_tj | good question! | 13:59 |
acoles | yeah, it's a tough problem | 13:59 |
acoles | :( | 13:59 |
acoles | so it may be that the consistency mechanisms need to be modified. for example, here's where the 'correct' on disk file for a replica policy is chosen https://github.com/openstack/swift/blob/7a7677868d4ec7693132014779909ac70d4f391d/swift/obj/diskfile.py#L2767-L2793 and you can see the most recent timestamp file is chosen | 14:01 |
remix_tj | we can limit the application of this middleware to archival purposes | 14:01 |
acoles | but a different implementation might open up each .data file and look for retention metadata and make a different decision | 14:02 |
remix_tj | and saying that doesn't work if you're making an high number of requests | 14:02 |
acoles | ^^setting appropriate user expectation is another way to deal with tough problems | 14:02 |
remix_tj | hehe | 14:03 |
acoles | :) | 14:03 |
remix_tj | is repliacing something i can't do with "you're using it in the bad way" | 14:03 |
*** klamath_ has joined #openstack-swift | 14:04 | |
acoles | a.k.a. "education" ;) | 14:05 |
*** klamath has quit IRC | 14:05 | |
acoles | this will help enable more 'custom' backends https://review.openstack.org/447129 | 14:06 |
patchbot | patch 447129 - swift - Configure diskfile per storage policy | 14:06 |
*** pcaruana has joined #openstack-swift | 14:10 | |
remix_tj | this what? your commit? | 14:12 |
acoles | the change proposed in patch 447129 enables different backend diskfile implementation to be specified for different storage policies, which makes it a little easier to introduce new diskfile implementations | 14:14 |
patchbot | https://review.openstack.org/#/c/447129/ - swift - Configure diskfile per storage policy | 14:14 |
acoles | but that would still be a lot of work compared to your middleware | 14:15 |
remix_tj | what you propose is a backend change, i, instead want simply to filter out some kind of requests | 14:16 |
*** gyankum has quit IRC | 14:18 | |
acoles | remix_tj: yes, I understand. I'm just sharing info. And it may well be that your middleware is sufficient for your use case. It comes down to the probability of failure conditions that would allow a retained object to be erroneously overwritten. It sounds like you are thinking about how to minimise that probability. | 14:19 |
*** armaan has quit IRC | 14:27 | |
*** d0ugal has quit IRC | 14:31 | |
*** hoonetorg has joined #openstack-swift | 14:31 | |
*** d0ugal has joined #openstack-swift | 14:35 | |
*** klamath_ has quit IRC | 14:35 | |
openstackgerrit | Alistair Coles proposed openstack/swift master: Always fetch listings from shards when possible https://review.openstack.org/571531 | 14:35 |
remix_tj | acoles: is there a way to have the time of the current request? i'm relying on datetime.now() but i think is not the right way to do it | 14:42 |
*** cbartz has quit IRC | 14:44 | |
acoles | remix_tj: do you mean how to get the request x-timestamp value in your middleware? | 14:48 |
*** d0ugal has quit IRC | 14:49 | |
acoles | x-timestamp is not actually set until a PUT request reaches the proxy-server app (here https://github.com/openstack/swift/blob/26418140108281ae5ac11004ebf33b7b3e08c74d/swift/proxy/controllers/base.py#L1525) , so after middleware has handled the request. | 14:50 |
acoles | x-timestamp is the 'time' of the object used internally to swift | 14:50 |
acoles | BUT IIRC middleware can set this value and the proxy then respects whatever the middleware set. | 14:51 |
acoles | and if you do that then the right way is exactly as that link above i.e. use headers.setdefault('x-timestamp', Timestamp.now().internal) | 14:53 |
acoles | and remember that a middleware to the left of yours may have already set the timestamp so if you are going to use the value then do ts = Timestamp(headers.setdefault('x-timestamp', Timestamp.now().internal)) | 14:54 |
DHE | is there a recommended (or optimal?) number of disks per server? swiftstack sells machines with 60+ but that seems excessively large... | 14:54 |
*** d0ugal has joined #openstack-swift | 14:56 | |
*** gkadam has joined #openstack-swift | 15:05 | |
*** itlinux has joined #openstack-swift | 15:16 | |
*** gkadam_ has joined #openstack-swift | 15:20 | |
notmyname | good morning | 15:23 |
*** lifeless_ has quit IRC | 15:23 | |
*** lifeless has joined #openstack-swift | 15:24 | |
*** gkadam has quit IRC | 15:24 | |
openstackgerrit | Alexandre Lécuyer proposed openstack/swift master: Configure diskfile per storage policy https://review.openstack.org/447129 | 15:24 |
openstackgerrit | Alexandre Lécuyer proposed openstack/swift master: WIP - LOSF https://review.openstack.org/571763 | 15:24 |
*** amandap has joined #openstack-swift | 15:26 | |
*** sajjadg has quit IRC | 15:34 | |
*** d0ugal has quit IRC | 15:34 | |
*** sajjadg has joined #openstack-swift | 15:35 | |
*** sajjadg has quit IRC | 15:36 | |
*** d0ugal has joined #openstack-swift | 15:41 | |
*** gkadam__ has joined #openstack-swift | 15:46 | |
*** d0ugal has quit IRC | 15:48 | |
*** gkadam_ has quit IRC | 15:49 | |
*** d0ugal has joined #openstack-swift | 15:51 | |
*** gkadam_ has joined #openstack-swift | 15:52 | |
*** gkadam__ has quit IRC | 15:55 | |
*** SkyRocknRoll has joined #openstack-swift | 15:58 | |
*** tesseract has quit IRC | 16:01 | |
*** SkyRocknRoll has quit IRC | 16:08 | |
*** SkyRocknRoll has joined #openstack-swift | 16:10 | |
*** lifeless_ has joined #openstack-swift | 16:11 | |
*** lifeless has quit IRC | 16:12 | |
*** mikecmpbll has quit IRC | 16:16 | |
*** armaan has joined #openstack-swift | 16:17 | |
*** hseipp has quit IRC | 16:24 | |
*** gyee has joined #openstack-swift | 16:29 | |
*** gkadam__ has joined #openstack-swift | 16:33 | |
*** gkadam_ has quit IRC | 16:36 | |
*** gkadam_ has joined #openstack-swift | 16:37 | |
*** gkadam__ has quit IRC | 16:40 | |
*** prasen has joined #openstack-swift | 16:41 | |
*** prasen has quit IRC | 16:42 | |
*** armaan has quit IRC | 16:50 | |
timburke | good morning | 16:55 |
*** gkadam__ has joined #openstack-swift | 16:57 | |
*** gkadam has joined #openstack-swift | 17:00 | |
*** gkadam_ has quit IRC | 17:00 | |
openstackgerrit | John Dickinson proposed openstack/swift master: made changelog section breaks more consistent https://review.openstack.org/540978 | 17:01 |
*** gkadam__ has quit IRC | 17:02 | |
*** psachin has quit IRC | 17:08 | |
*** amandap has quit IRC | 17:16 | |
*** gkadam has quit IRC | 17:18 | |
*** cshastri has quit IRC | 17:20 | |
*** mikecmpbll has joined #openstack-swift | 17:36 | |
*** mvk has quit IRC | 17:46 | |
*** SkyRocknRoll has quit IRC | 18:29 | |
*** geaaru has quit IRC | 18:38 | |
*** itlinux has quit IRC | 18:42 | |
*** itlinux has joined #openstack-swift | 18:46 | |
*** itlinux has quit IRC | 18:46 | |
*** bkopilov_ has quit IRC | 18:48 | |
*** openstackgerrit has quit IRC | 19:04 | |
timburke | looking at https://bugs.launchpad.net/swift/+bug/1774238, i'm up to 18 files changed, 173 insertions(+), 146 deletions(-) and down to just 8 failures! | 19:06 |
openstack | Launchpad bug 1774238 in OpenStack Object Storage (swift) "Troubles with unicode account names" [High,Confirmed] - Assigned to Tim Burke (1-tim-z) | 19:06 |
timburke | side note: i think "%ff" may be my new favorite substring to include in names... | 19:06 |
timburke | of course, at some point i'll have to think more about how my changes handle already-written data... which may get depressing | 19:07 |
*** amandap has joined #openstack-swift | 19:16 | |
*** amandap has quit IRC | 19:20 | |
*** itlinux has joined #openstack-swift | 19:30 | |
*** itlinux has quit IRC | 19:35 | |
*** jlvillal is now known as jlvacation | 19:43 | |
*** d0ugal has quit IRC | 19:47 | |
*** lifeless_ has quit IRC | 20:00 | |
*** lifeless has joined #openstack-swift | 20:01 | |
*** pcaruana has quit IRC | 20:03 | |
*** d0ugal has joined #openstack-swift | 20:04 | |
*** itlinux has joined #openstack-swift | 20:08 | |
*** lifeless has quit IRC | 20:50 | |
*** lifeless has joined #openstack-swift | 20:51 | |
timburke | man, tempauth does *not* handle users like user_te,st_tester very well... | 20:55 |
notmyname | timburke: definitely sounds like a "closed as 'unfortunate'" bug | 20:56 |
timburke | notmyname: yeah, i'm fine with that kind of a resolution there. but i *will* make user_tést%ff_tester work! | 20:57 |
*** d0ugal has quit IRC | 21:08 | |
*** klamath has joined #openstack-swift | 21:09 | |
*** klamath has quit IRC | 21:09 | |
*** klamath has joined #openstack-swift | 21:09 | |
*** d0ugal has joined #openstack-swift | 21:15 | |
*** itlinux_ has joined #openstack-swift | 21:41 | |
*** itlinux has quit IRC | 21:44 | |
*** geaaru has joined #openstack-swift | 21:57 | |
*** lifeless has quit IRC | 22:38 | |
*** lifeless has joined #openstack-swift | 22:40 | |
*** Sukhdev has joined #openstack-swift | 22:47 | |
*** openstackgerrit has joined #openstack-swift | 23:13 | |
openstackgerrit | Pete Zaitcev proposed openstack/swift master: Make ShardRanges and its CLI py3-compliant https://review.openstack.org/570320 | 23:13 |
*** amandap has joined #openstack-swift | 23:20 | |
*** amandap has quit IRC | 23:24 | |
*** lifeless has quit IRC | 23:36 | |
*** lifeless has joined #openstack-swift | 23:43 | |
openstackgerrit | Tim Burke proposed openstack/swift master: Improve user logging https://review.openstack.org/571051 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: Make ACLs work with Unicode in user/account names https://review.openstack.org/571048 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: Make COPY work with unicode account names https://review.openstack.org/571049 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: functests: Clean up objects better https://review.openstack.org/571050 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: copy: Stop hanging per-request data on middleware instance https://review.openstack.org/571903 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: tempauth: Send back url-encoded account names https://review.openstack.org/571904 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: Fix quoting for large objects https://review.openstack.org/571905 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: Make symlink work with Unicode account names https://review.openstack.org/571906 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: Make staticweb return URL-encoded Location headers https://review.openstack.org/571907 | 23:56 |
openstackgerrit | Tim Burke proposed openstack/swift master: Support Unicode in account and user names during func tests https://review.openstack.org/571908 | 23:56 |
timburke | there are about 5 more commits on the way related to that stupid bug... | 23:56 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!