| timburke | there was another session today, mostly talking sharding, moving beyond md5 for validation, liberasurecode, and starting to ponder the implications of eventlet removal | 00:00 |
|---|---|---|
| timburke | i should be better about updating #openinfra-events | 00:01 |
| timburke | kota, no worries! rest up | 00:01 |
| kota | timburke: thx again. hope the ptg goes excellent | 00:01 |
| DHE | I'm hoping to attend... | 00:17 |
| opendevreview | Matthew Oliver proposed openstack/swift master: Added a swift-graph-shardranges cli script / tool https://review.opendev.org/c/openstack/swift/+/775066 | 02:35 |
| mattoliver | ^ version of the shardrange graph tool that also colors deleted shardranges in red. | 02:37 |
| mcape | Hi All! | 06:52 |
| mcape | Recently I did added some new disks to our swift cluster of ~300Tb and some 300m documents. | 06:52 |
| mcape | And then there was a spike of errors in the logs that looked like: | 06:53 |
| mcape | [worker 1/1 pid=43347] <f+++++++++ 2f3/4b9b1ba80cb7366f0ee71ced5e9152f3/1605695055.99690.data | 06:53 |
| mcape | maybe a few thousand lines like that | 06:53 |
| mcape | Seems that it is rsync that is creating files on the remote replicas. | 06:53 |
| mcape | But why this log is severity: err? It seems that it is not an error at all. | 06:53 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu: probe tests: capture_logs option for run_custom_daemon https://review.opendev.org/c/openstack/swift/+/933098 | 12:07 |
| timburke | mcape, presumably toward the end of all that there'll be some actual error -- when there's an error, we log all the rsync output at error | 12:57 |
| timburke | fwiw, you can set log_rsync_transfers=false in your object-replicator config to turn off all the transfers -- it tends to be needlessly verbose | 12:58 |
| timburke | we might want to consider changing https://github.com/openstack/swift/blob/master/swift/obj/replicator.py#L416 to `if result.startswith('<') or not ret_val:` | 12:58 |
| timburke | meeting time, if we want to discuss anything before the ops feedback session in an hour -- https://meetpad.opendev.org/p/swift-ptg-epoxy | 13:01 |
| mattoliver | Be right there | 13:02 |
| mcape | timburke, thanks a lot for your answer!! | 13:56 |
| acoles | timburke: let's try to make progress on this one https://review.opendev.org/c/openstack/swift/+/887908 | 13:59 |
| patch-bot | patch 887908 - swift - Stop using cgi.parse_header - 5 patch sets | 13:59 |
| timburke | ops feedback session now in https://meetpad.opendev.org/p/swift-epoxy-ops-feedback | 14:02 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: refactor test for x-delete-at w/t part_num and x-open-expired https://review.opendev.org/c/openstack/swift/+/933061 | 15:40 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: WIP mpu: don't cleanup stale replica of versioned object https://review.opendev.org/c/openstack/swift/+/933122 | 15:53 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu: probe tests: capture_logs option for run_custom_daemon https://review.opendev.org/c/openstack/swift/+/933098 | 16:12 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: s3 compat test: null version only becomes version if overwritten https://review.opendev.org/c/openstack/swift/+/933041 | 16:15 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu-auditor: audit sessions https://review.opendev.org/c/openstack/swift/+/931517 | 16:16 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu-auditor: audit manifests https://review.opendev.org/c/openstack/swift/+/932352 | 16:16 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu: improve error response handling https://review.opendev.org/c/openstack/swift/+/933039 | 16:16 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: mpu: cleanup after specific version is deleted https://review.opendev.org/c/openstack/swift/+/933040 | 16:16 |
| opendevreview | Alistair Coles proposed openstack/swift feature/mpu: WIP mpu: don't cleanup stale replica of versioned object https://review.opendev.org/c/openstack/swift/+/933122 | 16:16 |
| fulecorafa | o/ | 21:02 |
| opendevreview | Tim Burke proposed openstack/swift master: Add swift-ring-info utility https://review.opendev.org/c/openstack/swift/+/857234 | 21:20 |
| opendevreview | Tim Burke proposed openstack/swift master: sq? ring-info: Work on builders, too https://review.opendev.org/c/openstack/swift/+/933156 | 21:20 |
| opendevreview | Tim Burke proposed openstack/swift master: ring-builder: Add revert subcommand https://review.opendev.org/c/openstack/swift/+/933157 | 21:20 |
| timburke | DHE, i wonder if ^^^ might help with your ring-composer issues -- if we ensure the version continues to march forward even as the state is rolling back, the composer ought to still be happy | 21:21 |
| opendevreview | Tim Burke proposed openstack/swift master: sq? ring-info: Work on builders, too https://review.opendev.org/c/openstack/swift/+/933156 | 21:36 |
| opendevreview | Tim Burke proposed openstack/swift master: ring-builder: Add revert subcommand https://review.opendev.org/c/openstack/swift/+/933157 | 21:36 |
| timburke | uh oh. i think i maybe needed to wait on https://review.opendev.org/c/openstack/pyeclib/+/930171 until after we had a release with https://review.opendev.org/c/openstack/liberasurecode/+/929193 ... | 21:49 |
| patch-bot | patch 930171 - pyeclib - Add close/destroy methods; clean up descriptors (MERGED) - 5 patch sets | 21:49 |
| patch-bot | patch 929193 - liberasurecode - built-in rs_vand: De-init tables only when last de... (MERGED) - 3 patch sets | 21:49 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: refactor test for x-delete-at w/t part_num and x-open-expired https://review.opendev.org/c/openstack/swift/+/933061 | 21:59 |
| timburke | really weird thing is that https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L512 still has the old version, but the log for https://zuul.opendev.org/t/openstack/build/8ab820800bde418dbbae18721e2859de/console shows the new one | 22:16 |
| timburke | looks like there's separate install_deps and install_package_deps, and the second one is running unconstrained?? | 22:16 |
| timburke | why *do* we have that -U on https://github.com/openstack/swift/blob/master/tox.ini#L13 ? | 22:17 |
| timburke | seems to go back like 11 years: https://github.com/openstack/swift/commit/9a2bd790 | 22:20 |
| clarkb | I think the reason for it is to force pip to install newer packages if they are available. It was a common problem in the past for people to run a target and have it succeed but ci would fail or vice versa due to mismatches in dep installs | 22:20 |
| clarkb | I don't knwo if it is necessary with the use of constraints (I think it may be if you have a delta to the constraints in your venv) | 22:20 |
| clarkb | the alternative would be to encourage people to use tox -r when they get unexpected results | 22:21 |
| DHE | OOOoo... | 22:35 |
| opendevreview | Tim Burke proposed openstack/pyeclib master: Only really destroy instances on new-enough liberasurecode https://review.opendev.org/c/openstack/pyeclib/+/933163 | 22:38 |
| opendevreview | Tim Burke proposed openstack/swift master: CI: Stop upgrading direct dependencies past constraints https://review.opendev.org/c/openstack/swift/+/933164 | 22:42 |
| timburke | well, fwiw, none of cinder, glance, keystone, neutron, or nova include that -U flag (at least, not any more) | 22:53 |
| timburke | at the same time, it's not like i *mind* catching (some) trouble early like happened here... maybe "test with constraints for most of the packages, but latest for direct deps" is a reasonable path to take... | 22:55 |
| clarkb | it is possible that changes to pip have made it unnecessary | 22:55 |
| timburke | 11 years is a long time, and a lot of pip upgrades :-) | 22:55 |
| clarkb | one way to test it would be to run tox, then manually downgrade/upgrade a dep in the resulting venv then rerun tox without the -U and then redo the test with the -U and see if you get the same result | 22:55 |
| clarkb | yup exactly | 22:55 |
| opendevreview | Tim Burke proposed openstack/pyeclib master: Only really destroy instances on new-enough liberasurecode https://review.opendev.org/c/openstack/pyeclib/+/933163 | 23:20 |
| opendevreview | Tim Burke proposed openstack/liberasurecode master: Release 1.6.5 https://review.opendev.org/c/openstack/liberasurecode/+/933167 | 23:42 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!