| *** ukaynar has joined #openstack-swift | 00:11 | |
| *** brimestone has quit IRC | 00:13 | |
| *** openstackgerrit has joined #openstack-swift | 00:36 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift feature/deep: make sharding 100x faster https://review.openstack.org/564374 | 00:36 |
|---|---|---|
| *** ukaynar has quit IRC | 00:46 | |
| *** vint_bra has joined #openstack-swift | 00:58 | |
| openstackgerrit | Merged openstack/swift master: Make reconstructor go faster with --override-devices https://review.openstack.org/555951 | 01:00 |
| *** vint_bra has quit IRC | 01:23 | |
| kota_ | oh, sounds awesome. sharding is being faster 100x!? | 01:31 |
| DHE | it's just a checkbox. now we make it the default. | 01:33 |
| DHE | :) | 01:33 |
| notmyname | lol @ patch 564374 | 01:39 |
| patchbot | https://review.openstack.org/#/c/564374/ - swift (feature/deep) - make sharding 100x faster | 01:39 |
| notmyname | timburke: can we get that pushed to the qa cluster asap? | 01:41 |
| *** wangyx has joined #openstack-swift | 01:52 | |
| *** kei_yama has quit IRC | 01:53 | |
| openstackgerrit | Nguyen Hai proposed openstack/python-swiftclient master: Change python-swiftclient webpage to openstackdocstheme https://review.openstack.org/553433 | 01:53 |
| openstackgerrit | Nguyen Hai proposed openstack/python-swiftclient master: Change python-swiftclient webpage to openstackdocstheme https://review.openstack.org/553433 | 01:53 |
| *** kei_yama has joined #openstack-swift | 01:57 | |
| *** AndyWojo_ has joined #openstack-swift | 02:08 | |
| *** viks_ has joined #openstack-swift | 02:08 | |
| *** early` has joined #openstack-swift | 02:09 | |
| *** ndk- has joined #openstack-swift | 02:12 | |
| *** AndyWojo has quit IRC | 02:12 | |
| *** early has quit IRC | 02:12 | |
| *** viks has quit IRC | 02:12 | |
| *** blair has quit IRC | 02:12 | |
| *** d0ugal has quit IRC | 02:12 | |
| *** ndk_ has quit IRC | 02:12 | |
| *** AndyWojo_ is now known as AndyWojo | 02:12 | |
| *** viks_ is now known as viks | 02:12 | |
| *** blair has joined #openstack-swift | 02:13 | |
| *** d0ugal has joined #openstack-swift | 02:13 | |
| *** Renich has quit IRC | 02:16 | |
| *** gkadam has joined #openstack-swift | 02:26 | |
| *** psachin has joined #openstack-swift | 02:49 | |
| *** armaan has quit IRC | 03:16 | |
| *** armaan has joined #openstack-swift | 03:17 | |
| *** dr_gogeta86 has quit IRC | 03:28 | |
| *** dr_gogeta86 has joined #openstack-swift | 03:30 | |
| *** dr_gogeta86 has quit IRC | 03:30 | |
| *** dr_gogeta86 has joined #openstack-swift | 03:30 | |
| *** kei_yama has quit IRC | 03:36 | |
| *** kei_yama has joined #openstack-swift | 03:40 | |
| *** bkopilov has quit IRC | 03:45 | |
| openstackgerrit | Merged openstack/swift feature/deep: make sharding 100x faster https://review.openstack.org/564374 | 03:54 |
| timburke | clayg: one thought on ^^^ -- do we need the db version check to switch between 'deleted' and '+deleted'? | 04:20 |
| *** armaan has quit IRC | 04:22 | |
| *** armaan has joined #openstack-swift | 04:23 | |
| *** gkadam has quit IRC | 04:45 | |
| *** ccamacho has quit IRC | 04:48 | |
| *** links has joined #openstack-swift | 04:53 | |
| *** gkadam has joined #openstack-swift | 05:29 | |
| *** bkopilov has joined #openstack-swift | 05:29 | |
| *** d0ugal has quit IRC | 05:46 | |
| *** geaaru has quit IRC | 05:48 | |
| *** armaan has quit IRC | 06:12 | |
| *** armaan has joined #openstack-swift | 06:13 | |
| *** armaan has quit IRC | 06:13 | |
| *** d0ugal has joined #openstack-swift | 06:13 | |
| *** armaan has joined #openstack-swift | 06:13 | |
| *** kei_yama has quit IRC | 06:31 | |
| *** kei_yama has joined #openstack-swift | 06:44 | |
| *** cbartz has joined #openstack-swift | 06:46 | |
| *** tesseract has joined #openstack-swift | 06:58 | |
| *** tesseract has quit IRC | 07:00 | |
| *** tesseract has joined #openstack-swift | 07:02 | |
| *** ccamacho has joined #openstack-swift | 07:03 | |
| *** ntt has quit IRC | 07:13 | |
| *** ntt has joined #openstack-swift | 07:20 | |
| *** silor has joined #openstack-swift | 07:22 | |
| *** guest has joined #openstack-swift | 07:26 | |
| guest | {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} | 07:26 |
| *** pcaruana has joined #openstack-swift | 07:31 | |
| *** rcernin has quit IRC | 07:32 | |
| guest | .{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} | 07:33 |
| acoles | good morning | 07:40 |
| *** geaaru has joined #openstack-swift | 07:41 | |
| acoles | aha, while I sleep, clayg fixes everything | 07:42 |
| *** threestrands_ has quit IRC | 07:52 | |
| *** gkadam has quit IRC | 07:56 | |
| *** ccamacho has quit IRC | 08:08 | |
| *** ccamacho has joined #openstack-swift | 08:11 | |
| *** gkadam has joined #openstack-swift | 08:20 | |
| *** cbartz has quit IRC | 08:30 | |
| *** Renich has joined #openstack-swift | 08:50 | |
| *** jmccarthy has joined #openstack-swift | 08:52 | |
| *** jmccarthy has left #openstack-swift | 08:53 | |
| *** cbartz has joined #openstack-swift | 08:57 | |
| *** hoonetorg has quit IRC | 08:58 | |
| *** guest has quit IRC | 09:08 | |
| openstackgerrit | Tovin Seven proposed openstack/swift master: OSprofiler in OpenStack Swift https://review.openstack.org/468316 | 09:12 |
| *** hoonetorg has joined #openstack-swift | 09:15 | |
| *** Renich has quit IRC | 09:39 | |
| *** Fr33m1nd has joined #openstack-swift | 09:41 | |
| *** mtreinish has quit IRC | 09:41 | |
| *** mtreinish has joined #openstack-swift | 09:47 | |
| *** silor1 has joined #openstack-swift | 09:56 | |
| *** silor has quit IRC | 09:58 | |
| *** silor1 is now known as silor | 09:58 | |
| *** cbartz has quit IRC | 09:59 | |
| *** armaan has quit IRC | 10:03 | |
| *** ccamacho has quit IRC | 10:07 | |
| *** cbartz has joined #openstack-swift | 10:07 | |
| openstackgerrit | Merged openstack/swift feature/deep: Refactor finding sharding and shrinking candidates https://review.openstack.org/564277 | 10:08 |
| *** ccamacho has joined #openstack-swift | 10:08 | |
| *** ccamacho has quit IRC | 10:08 | |
| *** ccamacho has joined #openstack-swift | 10:08 | |
| *** cbartz has quit IRC | 10:18 | |
| *** Renich has joined #openstack-swift | 10:20 | |
| *** silor has quit IRC | 10:24 | |
| *** Fr33m1nd has quit IRC | 10:28 | |
| *** cbartz has joined #openstack-swift | 10:28 | |
| *** cbartz has quit IRC | 10:44 | |
| *** threestrands_ has joined #openstack-swift | 11:01 | |
| *** Fr33m1nd has joined #openstack-swift | 11:03 | |
| *** Fr33m1nd has quit IRC | 11:04 | |
| *** armaan has joined #openstack-swift | 11:12 | |
| *** vint_bra has joined #openstack-swift | 11:35 | |
| *** wangyx has quit IRC | 11:36 | |
| *** bkopilov has quit IRC | 11:38 | |
| *** armaan has quit IRC | 11:45 | |
| *** StevenK has quit IRC | 11:55 | |
| *** wangyx has joined #openstack-swift | 11:55 | |
| *** StevenK has joined #openstack-swift | 11:56 | |
| *** vint_bra has quit IRC | 12:01 | |
| *** hseipp has joined #openstack-swift | 12:06 | |
| *** Fr33m1nd has joined #openstack-swift | 12:13 | |
| *** Fr33m1nd has quit IRC | 12:18 | |
| *** kei_yama has quit IRC | 12:34 | |
| *** psachin has quit IRC | 12:38 | |
| *** vint_bra has joined #openstack-swift | 12:42 | |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Stop internal client txn_id bleeding into sharder logs https://review.openstack.org/564488 | 12:47 |
| *** threestrands_ has quit IRC | 12:58 | |
| *** bkopilov has joined #openstack-swift | 13:23 | |
| *** brimestone has joined #openstack-swift | 13:27 | |
| *** mvk has quit IRC | 13:29 | |
| *** brimestone has quit IRC | 13:33 | |
| *** vint_bra has quit IRC | 13:40 | |
| *** ediardo has quit IRC | 13:44 | |
| *** mvk has joined #openstack-swift | 13:58 | |
| *** armaan has joined #openstack-swift | 14:24 | |
| *** bkopilov has quit IRC | 14:32 | |
| *** bkopilov has joined #openstack-swift | 14:35 | |
| *** pcaruana has quit IRC | 14:42 | |
| *** armaan has quit IRC | 14:58 | |
| *** silor has joined #openstack-swift | 14:59 | |
| *** early` has quit IRC | 15:03 | |
| *** jistr|mtgs is now known as jistr | 15:04 | |
| *** links has quit IRC | 15:06 | |
| -openstackstatus- NOTICE: We've successfully troubleshooted the issue that prevented paste.openstack.org from loading and it's now back online, thank you for your patience. | 15:06 | |
| *** wangyx has quit IRC | 15:09 | |
| *** early has joined #openstack-swift | 15:15 | |
| *** silor has quit IRC | 15:21 | |
| notmyname | good morning | 15:26 |
| notmyname | rledisez: et al: the sharding speed is going a lot faster in the cluster with the big DB, now that patch 564374 landed | 15:26 |
| patchbot | https://review.openstack.org/#/c/564374/ - swift (feature/deep) - make sharding 100x faster (MERGED) | 15:26 |
| rledisez | notmyname: nice, that's the best "2-lines patch" ever! | 15:27 |
| notmyname | :-) | 15:28 |
| DHE | as a layperson who does know SQL, what values besides 0 and 1 can 'deleted' take anyway such that this would help so much? | 15:31 |
| DHE | (or is it an index thing?) | 15:31 |
| *** zaitcev_ has joined #openstack-swift | 15:33 | |
| *** ChanServ sets mode: +v zaitcev_ | 15:33 | |
| *** zaitcev has quit IRC | 15:37 | |
| *** gyee has joined #openstack-swift | 15:40 | |
| timburke | good morning | 15:43 |
| timburke | acoles: isn't that just always the case? ;-) | 15:43 |
| acoles | timburke: yep, so typical | 15:44 |
| acoles | :) | 15:44 |
| timburke | DHE: it's an index thing. deleted will only ever be 0 or 1, but since it wasn't in the query, sqlite didn't know to use the index | 15:44 |
| *** gkadam has quit IRC | 15:53 | |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Reclaim deleted shard range rows after 2 * reclaim_age https://review.openstack.org/564540 | 15:55 |
| *** silor has joined #openstack-swift | 16:02 | |
| *** d0ugal has quit IRC | 16:03 | |
| *** d0ugal has joined #openstack-swift | 16:04 | |
| notmyname | cschwede: mattoliverau: thanks for the s3api reviews so far | 16:17 |
| *** geaaru has quit IRC | 16:27 | |
| *** brimestone has joined #openstack-swift | 16:27 | |
| *** silor has quit IRC | 16:45 | |
| *** germs has joined #openstack-swift | 16:50 | |
| *** germs has quit IRC | 16:50 | |
| *** germs has joined #openstack-swift | 16:50 | |
| DHE | timburke: interesting... not what I expected... | 16:53 |
| *** ediardo has joined #openstack-swift | 16:55 | |
| timburke | the trouble is that sqlite doesn't know what kinds of values deleted might have -- as far as it's concerned, that could take any of 4 billion values, so it assumes the index wouldn't be useful | 16:55 |
| *** silor has joined #openstack-swift | 16:56 | |
| *** ChanServ changes topic to "https://bugs.not.mn/project/Swift | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/ | Meetings: https://wiki.openstack.org/wiki/Meetings/Swift | Review Dashboard: http://not.mn/reviews.html" | 16:56 | |
| DHE | depends on the index type. I'm used to mysql/pgsql which use btree indexes which are useful for searches on only 'a' when the index was built on (a,b) | 16:56 |
| timburke | (really, it assumes it could take *any value* -- as i recall, it trusts the developer rather than impose a strict adherence to the schema) | 16:56 |
| DHE | but for a hash index, yeah I can see that | 16:56 |
| timburke | sqlite also uses btrees, as i recall. since it's a (deleted, name) index, you need to be searching off deleted as well as name to get the benefit | 16:57 |
| DHE | ah, the commit message says "(name, deleted) index" so that's what confused me | 16:59 |
| notmyname | acoles: clayg: timburke: tdasilva: would next tuesday work for a video chat about feature/deep? how about 9am pacific/1600utc ? | 16:59 |
| *** hseipp has quit IRC | 17:00 | |
| acoles | notmyname: ok for me | 17:00 |
| timburke | ah, yeah. didn't notice the error in the message -- too happy that it fixed the problem! | 17:01 |
| clayg | 9-9:30 is when I walk my kids to school. I’ll could try to drop them off early and join late. So mostly wfm. | 17:01 |
| notmyname | I keep forgetting how late your kids start | 17:02 |
| timburke | i can probably make that work. assuming no baby yet | 17:02 |
| clayg | DHE: i probably could have been more clear in the commit. Thanks for asking for clarity. | 17:04 |
| *** germs has quit IRC | 17:13 | |
| *** germs has joined #openstack-swift | 17:13 | |
| *** germs has quit IRC | 17:13 | |
| *** germs has joined #openstack-swift | 17:13 | |
| notmyname | acoles: when I make feature/deep-review, I'll need to give it a commit sha on master from which to branch. is there any reason to not use HEAD of master? | 17:24 |
| *** germs has quit IRC | 17:24 | |
| timburke | oh yeah... i should do another merge of master to feature/deep... i think there are some conflicts in utils... | 17:25 |
| *** germs has joined #openstack-swift | 17:25 | |
| *** germs has quit IRC | 17:25 | |
| *** germs has joined #openstack-swift | 17:25 | |
| acoles | notmyname: not that I am aware of. My plan was to merge from master into feature/deep once s3api lands, then propose patches to feature/deep-review. Make sense? | 17:25 |
| notmyname | yep. makes sense | 17:26 |
| acoles | or...I'll wait for timburke to do the first part :) | 17:26 |
| *** germs has quit IRC | 17:26 | |
| *** germs has joined #openstack-swift | 17:26 | |
| *** germs has quit IRC | 17:26 | |
| *** germs has joined #openstack-swift | 17:26 | |
| notmyname | I'll wait on creating deep-review until s3api lands (or at least give it another 24 hours or so). it *shouldn't* take too long for that to be created | 17:27 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Remove unused code for passing shard ranges via the pending file https://review.openstack.org/564570 | 17:41 |
| *** germs has quit IRC | 17:46 | |
| *** germs has joined #openstack-swift | 17:47 | |
| *** germs has quit IRC | 17:47 | |
| *** germs has joined #openstack-swift | 17:47 | |
| *** geaaru has joined #openstack-swift | 17:51 | |
| *** mvk has quit IRC | 17:55 | |
| *** brimestoned has joined #openstack-swift | 17:57 | |
| *** brimestone has quit IRC | 17:59 | |
| *** tesseract has quit IRC | 18:24 | |
| *** mvk has joined #openstack-swift | 18:31 | |
| *** hseipp has joined #openstack-swift | 19:09 | |
| *** brimestoned has quit IRC | 19:17 | |
| *** brimestoned has joined #openstack-swift | 19:18 | |
| *** armaan has joined #openstack-swift | 19:19 | |
| *** armaan has quit IRC | 19:22 | |
| *** ediardo has quit IRC | 19:23 | |
| *** ukaynar has joined #openstack-swift | 19:37 | |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Skip commits when recording progress for logs/recon https://review.openstack.org/564271 | 19:45 |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Catch more exceptions when recording progress https://review.openstack.org/564605 | 19:45 |
| *** silor has quit IRC | 19:50 | |
| *** geaaru_ has joined #openstack-swift | 19:58 | |
| *** hseipp has quit IRC | 19:59 | |
| *** ukaynar has quit IRC | 19:59 | |
| *** geaaru has quit IRC | 20:00 | |
| *** ukaynar has joined #openstack-swift | 20:00 | |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Clean up logs a bit https://review.openstack.org/564613 | 20:01 |
| *** geaaru__ has joined #openstack-swift | 20:04 | |
| *** NM has joined #openstack-swift | 20:04 | |
| *** geaaru_ has quit IRC | 20:06 | |
| clayg | i was able to duplicate the failures on patch 564488 - so I'm going to look at those | 20:07 |
| patchbot | https://review.openstack.org/#/c/564488/ - swift (feature/deep) - Stop internal client txn_id bleeding into sharder ... | 20:07 |
| timburke | oh yeah, that reminds me of https://review.openstack.org/#/c/507706/ -- i'd originally done that with the sharder in mind | 20:08 |
| patchbot | patch 507706 - swift - Clear logger txn_id after making requests | 20:08 |
| *** geaaru_ has joined #openstack-swift | 20:09 | |
| *** geaaru__ has quit IRC | 20:12 | |
| *** geaaru__ has joined #openstack-swift | 20:16 | |
| *** geaaru_ has quit IRC | 20:18 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift feature/deep: Stop internal client txn_id bleeding into sharder logs https://review.openstack.org/564488 | 20:18 |
| clayg | torgomatic: what do you think about making internal_client.make_request *always* do a spawn().wait() so that the request happens in a different thread_id and the loggers thread_locals txn_id never pick up the reference to subrequest's info and cram it on to subsequent log messages? | 20:21 |
| clayg | torgomatic: i ask in the context of patch 564488 | 20:29 |
| patchbot | https://review.openstack.org/#/c/564488/ - swift (feature/deep) - Stop internal client txn_id bleeding into sharder ... | 20:29 |
| acoles | timburke: we both had same thought... my drive by here https://review.openstack.org/#/c/564540/1/swift/container/sharder.py@1133 vs https://review.openstack.org/#/c/564613/1 | 20:34 |
| patchbot | patch 564540 - swift (feature/deep) - Reclaim deleted shard range rows after 2 * reclaim... | 20:34 |
| patchbot | patch 564613 - swift (feature/deep) - Clean up logs a bit | 20:34 |
| openstackgerrit | Alistair Coles proposed openstack/swift feature/deep: Reclaim deleted shard range rows after 2 * reclaim_age https://review.openstack.org/564540 | 20:36 |
| timburke | i think i'd like internal_client better if it took on the log_name of whoever created it :-/ | 20:37 |
| timburke | acoles: nice! yeah -- either's fine by me :-) | 20:37 |
| acoles | timburke: you won | 20:38 |
| acoles | pulled it out of my patch, +A on yours | 20:38 |
| *** geaaru__ has quit IRC | 21:02 | |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Commit from pending file less often https://review.openstack.org/564630 | 21:27 |
| timburke | acoles: ^^^ might make https://review.openstack.org/#/c/564271/ mostly unnecessary? *shrug* | 21:28 |
| patchbot | patch 564271 - swift (feature/deep) - Skip commits when recording progress for logs/recon | 21:28 |
| timburke | there's still that call to _make_stats_info though... | 21:29 |
| *** NM has quit IRC | 21:48 | |
| *** NM has joined #openstack-swift | 21:58 | |
| timburke | so how icky would it be to scribble down some reference to the parent container's sharding context (in incoming sync points, maybe?) when cleaving to a (sub-)shard once we're ready to replicate? then if replication fails, when we come 'round to cleave it again... we skip straight to replication if it matches our cleave context at that time | 22:06 |
| timburke | acoles: clayg: mattoliverau: ^^^ | 22:07 |
| *** rcernin has joined #openstack-swift | 22:23 | |
| mattoliverau | morning | 22:37 |
| mattoliverau | timburke: I think I need a coffee before I can think that through :) | 22:38 |
| *** NM has quit IRC | 22:41 | |
| *** kei_yama has joined #openstack-swift | 23:20 | |
| *** ukaynar has quit IRC | 23:25 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift master: Refactor replicators' statistics https://review.openstack.org/564646 | 23:29 |
| *** brimestoned has quit IRC | 23:35 | |
| openstackgerrit | Merged openstack/swift feature/deep: Remove unused code for passing shard ranges via the pending file https://review.openstack.org/564570 | 23:44 |
| *** brimestone has joined #openstack-swift | 23:44 | |
| *** gyee has quit IRC | 23:54 | |
| openstackgerrit | Tim Burke proposed openstack/swift feature/deep: Parameterize SQL args https://review.openstack.org/564651 | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!