| kota_ | hello | 00:09 |
|---|---|---|
| kota_ | onovy, timburke: exactly, the CHANGELOG is missing in v1.10 :/ | 00:09 |
| kota_ | it seems to come from the order of merge commit | 00:11 |
| kota_ | i just tagged the newest master HEAD at that time but it seemed to refer the merge commit for setup.cfg change and the order was swapped :/ | 00:12 |
| kota_ | i'll try to re-tag to correct commit, i don't know how i can do that though. | 00:13 |
| kota_ | timburke: and thanks for reviewing patch 275009 | 00:15 |
| patchbot | kota_: https://review.openstack.org/#/c/275009/ - swift3 - Fix cleanup segment container | 00:15 |
| kota_ | timburke: it's really good call. that is what I thought in the last night in my bed :) | 00:15 |
| kota_ | i think i can write the test case but the solution might be problematic due to eventual consistency. | 00:17 |
| kota_ | anyways, I'll push the failure test case before submitting the fix. | 00:17 |
| timburke | kota_: yeah, that's why i recommended just to try it and see what happens :/ it may leave some segments, but we're in a similar situation when there's a failure midway through ?multipart-manifest=delete subrequests | 00:23 |
| kota_ | timburke: exactly | 00:24 |
| kota_ | right now, i think we can fix it with extra HEAD for refer SLO manifest. If it exists, failed immediately as 409 bucket not empty. | 00:25 |
| kota_ | it still has timming issue. | 00:25 |
| kota_ | though | 00:25 |
| kota_ | and I'm realizing (wondering), can we delete the bucket which has some initiated uploads in S3? | 00:26 |
| timburke | good question...hadn't thought about that part | 00:26 |
| kota_ | ok, let's make sure :) | 00:27 |
| timburke | torgomatic's audit-watcher patch has got me thinking about the possibilities when we (eventually) see all objects. in particular, i'm thinking about letting clients add some custom metadata on objects to say "hey, i only make sense in the context of this thing. if it's deleted, clean me up as well" | 00:29 |
| timburke | where "deleted" means "no, really, there's a tombstone and everything", not just a 404 on head | 00:29 |
| kota_ | hmm | 00:31 |
| kota_ | sounds good way, I need to think more about how we can use tthe watcher though :/ | 00:32 |
| timburke | may not even be the right place for it. just a though i had | 00:33 |
| kota_ | timburk: but thanks for the info :) | 00:36 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift3: WIP: Fix cleanup segment container https://review.openstack.org/275009 | 01:09 |
| *** lyrrad has quit IRC | 01:18 | |
| *** bill_az has quit IRC | 02:55 | |
| *** chsc has joined #swift3 | 04:23 | |
| *** chsc has quit IRC | 04:27 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift3: Fix cleanup segment container https://review.openstack.org/275009 | 07:07 |
| kota_ | hmm....the fix is getting bigger far from my expectation :/ | 07:08 |
| *** m_kazuhiro has joined #swift3 | 11:11 | |
| *** m_kazuhiro has quit IRC | 11:12 | |
| onovy | kota_: i think retaging is not good idea | 12:37 |
| onovy | just wait for next release and "never mind" :) | 12:37 |
| onovy | it's just changelog... | 12:38 |
| kota_ | onovy: hmm, that's an idea and either is fine to me, no matters. | 12:56 |
| onovy | :) | 12:56 |
| *** bill_az has joined #swift3 | 13:00 | |
| *** openstackgerrit has quit IRC | 13:02 | |
| *** openstackgerrit has joined #swift3 | 13:03 | |
| *** lyrrad has joined #swift3 | 17:27 | |
| *** chsc has joined #swift3 | 18:30 | |
| openstackgerrit | Sivasathurappan Radhakrishnan proposed openstack/swift3: Fix nonprintable metadata for s3 compatibility Encoding header metadata “HTTP_X_OBJECT_META_” in request.py into UTF-8 format for non-printable values to simulate S3 behavior https://review.openstack.org/266080 | 20:50 |
| openstackgerrit | Tim Burke proposed openstack/swift3: Add base64-encoding to tests https://review.openstack.org/275930 | 21:16 |
| timburke | siva_krishnan: both are good and useful. the swift3 tests (both unit and functional) are great for catching regressions, while the s3-tests suite offers the broadest S3-compatibility testing i've found. eventually, i'd like to get the s3-tests suite running as a (non-voting) gate job, just to give us the extra coverage and offer feedback on when previously failing tests start passing | 21:33 |
| siva_krishnan | yup that is great idea timburke. It would help us to keep tab on the comaptibility level that we have with s3. I would go ahead and add test cases to both of them. | 21:36 |
| siva_krishnan | timburke, kota_ have question for you guys regarding https://review.openstack.org/159130 | 21:40 |
| siva_krishnan | In the above patch question has been raised about orphaned segments containers. I would like to how to those segments containers would be cleaned up from the system ? | 21:41 |
| kota_ | siva_krishnan: https://review.openstack.org/#/c/275009/ | 21:43 |
| siva_krishnan | Do Auditors or reapers do it ? | 21:43 |
| kota_ | that is my try to prevent orphaned container | 21:43 |
| kota_ | not sure yet, it's good thing in eventualy consistency world. | 21:44 |
| kota_ | siva_krishnan: but good point of view, thanks :) | 21:45 |
| siva_krishnan | thanks for that follow up patch. Looks great ! Had an similar idea of HEADing the base container to check its access rights but never thought about container-object-count | 21:50 |
| *** chsc has quit IRC | 22:39 | |
| kota_ | hello | 23:01 |
| kota_ | meeting? | 23:01 |
| kota_ | looks like no one around | 23:03 |
| siva_krishnan | o/ | 23:04 |
| siva_krishnan | I am in | 23:04 |
| siva_krishnan | Sorry had a meeting | 23:04 |
| kota_ | siva_krishnan: o/ | 23:04 |
| kota_ | Just I was thinking "skip this week meeting" :-) | 23:04 |
| kota_ | you have something except patch 275009? | 23:05 |
| patchbot | kota_: https://review.openstack.org/#/c/275009/ - swift3 - Fix cleanup segment container | 23:05 |
| *** chsc has joined #swift3 | 23:05 | |
| siva_krishnan | thanks for this patch Kota_! | 23:06 |
| siva_krishnan | You and timburke have been supporting me a lot last weeekend in fixing up this | 23:06 |
| siva_krishnan | I just had one question regarding this | 23:07 |
| kota_ | ok, what's? | 23:07 |
| siva_krishnan | Shouldn't Swift be allowing us not to delete container unless we delete objects from it. Shouldn't it be the same way as S3 does it ? | 23:08 |
| siva_krishnan | Was there any specific design decision behind it ? | 23:09 |
| kota_ | Swift doesn't allow to delete container if it has object inside. | 23:09 |
| kota_ | I think S3 too. | 23:09 |
| kota_ | S3 docs is here, http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html | 23:10 |
| kota_ | not sure about the design decision but considering use case | 23:10 |
| siva_krishnan | S3 doesn't allow it. I have checked it myself But when I tried it with Swift3 it allowed me to do so | 23:11 |
| kota_ | users will have a lot of objects inside the bucket | 23:11 |
| siva_krishnan | May be I should cross check it once again ? | 23:11 |
| kota_ | bucket (or container if you use swift) | 23:11 |
| kota_ | siva_krishnan: :-) | 23:12 |
| kota_ | wait... | 23:12 |
| siva_krishnan | ok | 23:12 |
| kota_ | if Swift3 allows to delete not empty bucket, it's absolutely bug i think | 23:13 |
| siva_krishnan | I tried it with segment containers | 23:13 |
| kota_ | I think we don't want to delete not empty bucket because it's too easy way to delete everything if it's unexpected. | 23:14 |
| kota_ | Deleting non-empty (important data inside) bucket with mistake -> screaming!!!!! | 23:15 |
| kota_ | I was imaging :/ | 23:16 |
| siva_krishnan | I just saw your comment to the above patch "In edge case, orphaned segment container may be remaining there but this improves the behavior in normal case and I don't want to keep this to live in gerrit more so merge tentatively and will fix the remaining issue." | 23:17 |
| siva_krishnan | this made me to raise this question | 23:17 |
| *** lyrrad has quit IRC | 23:17 | |
| kota_ | ah, ok | 23:18 |
| kota_ | the edge case is timming issue i mean | 23:18 |
| kota_ | for example | 23:18 |
| kota_ | in timestamp 0, an user have empty bucket but it have initiated multi upload | 23:19 |
| kota_ | it also has some uploaded parts in the segment container | 23:19 |
| kota_ | next timestamp 1, another use who has permission to delete that bucket requests to delete | 23:20 |
| kota_ | at the same time (i.e. timestamp 1'), an user requests "complete multiupload" | 23:21 |
| kota_ | that's edge case | 23:21 |
| kota_ | consider the handling (i called it as timestamp 2) | 23:21 |
| kota_ | s/called/call now/ | 23:21 |
| kota_ | delete request checks the x-container-object-count which is 0 -> ok let's cleanup container | 23:22 |
| siva_krishnan | ok. thanks for that info : ) | 23:23 |
| kota_ | complete multiupload requests *may* put SLO just after the delete check | 23:23 |
| kota_ | siva_krishnan: :-) | 23:23 |
| kota_ | and x-container-object-count is able to be updated asynchronously. | 23:24 |
| kota_ | not ensure everything like as trunsaction. | 23:24 |
| kota_ | that's why I added the note. | 23:24 |
| kota_ | more safer way may be that check the existence for each upload id (or referred object) | 23:26 |
| siva_krishnan | how about our meeting ? | 23:27 |
| kota_ | skipped? :P | 23:27 |
| siva_krishnan | I have been flooding the meeting with my questions | 23:27 |
| kota_ | no worries | 23:27 |
| siva_krishnan | :) | 23:28 |
| kota_ | ok, closing today, I think we had enough conversations at recent IRC so assuming not so a big (urgent) topic is. | 23:30 |
| kota_ | siva_krishnan: thanks for attending :) | 23:31 |
| siva_krishnan | Hopefully we will have more people attending next week's meeting :) | 23:32 |
| kota_ | I hope so :) | 23:33 |
| *** chsc has quit IRC | 23:35 | |
| *** lyrrad has joined #swift3 | 23:42 | |
| *** lyrrad has left #swift3 | 23:42 | |
| *** lyrrad has joined #swift3 | 23:43 | |
| timburke | sorry i missed it. conflicting meeting, and then an internet outage :( | 23:52 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!