| *** zaitcev has joined #openstack-swift | 00:11 | |
| *** ChanServ sets mode: +v zaitcev | 00:11 | |
| *** gyee has quit IRC | 00:38 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Get functional/tests.py running under py3 https://review.opendev.org/642520 | 00:44 |
|---|---|---|
| openstackgerrit | Tim Burke proposed openstack/swift master: Add py37 func test job https://review.opendev.org/645856 | 00:44 |
| openstackgerrit | Tim Burke proposed openstack/swift master: py3: Cover account/container func tests https://review.opendev.org/645388 | 00:44 |
| openstackgerrit | Tim Burke proposed openstack/swift master: py3: Bring functional/test_object.py under test; add func-ec-py37 job https://review.opendev.org/645895 | 00:44 |
| openstackgerrit | Tim Burke proposed openstack/swift master: py3: port staticweb and domain_remap func tests https://review.opendev.org/647923 | 00:44 |
| openstackgerrit | Tim Burke proposed openstack/swift master: py3: port dlo func tests https://review.opendev.org/642920 | 00:44 |
| openstackgerrit | Tim Burke proposed openstack/swift master: py3: finish porting proxy/test_server.py https://review.opendev.org/657700 | 00:47 |
| openstackgerrit | Tim Burke proposed openstack/swift master: py3: port proxy/test_mem_server.py https://review.opendev.org/657705 | 00:47 |
| *** NM has joined #openstack-swift | 01:02 | |
| *** NM has quit IRC | 01:26 | |
| *** zaitcev_ has joined #openstack-swift | 01:31 | |
| *** ChanServ sets mode: +v zaitcev_ | 01:31 | |
| *** baojg has joined #openstack-swift | 01:32 | |
| *** zaitcev has quit IRC | 01:34 | |
| *** zaitcev_ has quit IRC | 02:00 | |
| *** zaitcev_ has joined #openstack-swift | 02:11 | |
| *** ChanServ sets mode: +v zaitcev_ | 02:11 | |
| *** zaitcev__ has joined #openstack-swift | 02:13 | |
| *** ChanServ sets mode: +v zaitcev__ | 02:13 | |
| *** zaitcev_ has quit IRC | 02:16 | |
| *** itlinux has joined #openstack-swift | 02:28 | |
| *** itlinux has quit IRC | 02:38 | |
| *** zaitcev__ has quit IRC | 03:24 | |
| *** psachin has joined #openstack-swift | 03:34 | |
| *** zaitcev__ has joined #openstack-swift | 03:36 | |
| *** ChanServ sets mode: +v zaitcev__ | 03:36 | |
| *** zaitcev__ has quit IRC | 03:36 | |
| *** e0ne has joined #openstack-swift | 03:47 | |
| *** e0ne has quit IRC | 04:46 | |
| *** samueldmq has quit IRC | 05:19 | |
| *** tesseract has joined #openstack-swift | 07:09 | |
| *** rdejoux has joined #openstack-swift | 07:16 | |
| *** [diablo] has quit IRC | 07:42 | |
| *** dr_gogeta86 has joined #openstack-swift | 07:58 | |
| *** [diablo] has joined #openstack-swift | 07:59 | |
| *** mikecmpbll has joined #openstack-swift | 08:07 | |
| *** e0ne has joined #openstack-swift | 08:14 | |
| *** e0ne has quit IRC | 08:17 | |
| *** tkajinam has quit IRC | 08:30 | |
| *** gmann_pto has joined #openstack-swift | 08:59 | |
| *** tdasilva has quit IRC | 09:23 | |
| *** e0ne has joined #openstack-swift | 09:38 | |
| *** e0ne has quit IRC | 09:40 | |
| *** e0ne has joined #openstack-swift | 09:56 | |
| *** e0ne has quit IRC | 09:59 | |
| *** mugsie has quit IRC | 10:25 | |
| *** mugsie has joined #openstack-swift | 10:35 | |
| *** mugsie has quit IRC | 10:35 | |
| *** mugsie has joined #openstack-swift | 10:36 | |
| *** mugsie has quit IRC | 10:38 | |
| *** mugsie has joined #openstack-swift | 10:39 | |
| *** tesseract has quit IRC | 10:40 | |
| *** tesseract has joined #openstack-swift | 10:41 | |
| *** tesseract has quit IRC | 10:45 | |
| *** tesseract has joined #openstack-swift | 10:45 | |
| *** NM has joined #openstack-swift | 11:39 | |
| *** NM has quit IRC | 11:42 | |
| *** gmann_pto has quit IRC | 11:50 | |
| *** zigo has quit IRC | 11:59 | |
| *** baojg has quit IRC | 12:08 | |
| *** zigo has joined #openstack-swift | 12:24 | |
| *** NM has joined #openstack-swift | 12:27 | |
| *** tdasilva has joined #openstack-swift | 12:52 | |
| *** ChanServ sets mode: +v tdasilva | 12:52 | |
| *** NM has quit IRC | 13:06 | |
| *** tdasilva has quit IRC | 13:12 | |
| *** NM has joined #openstack-swift | 13:14 | |
| *** samueldmq has joined #openstack-swift | 13:16 | |
| *** baojg has joined #openstack-swift | 13:20 | |
| *** tesseract has quit IRC | 13:35 | |
| *** tesseract has joined #openstack-swift | 13:35 | |
| *** altlogbot_2 has quit IRC | 13:43 | |
| *** altlogbot_0 has joined #openstack-swift | 13:45 | |
| *** psachin has quit IRC | 13:51 | |
| *** tdasilva has joined #openstack-swift | 13:58 | |
| *** ChanServ sets mode: +v tdasilva | 13:58 | |
| *** itlinux has joined #openstack-swift | 14:17 | |
| *** ccamacho has joined #openstack-swift | 14:21 | |
| *** itlinux has quit IRC | 14:23 | |
| *** tesseract has quit IRC | 14:24 | |
| *** tdasilva has quit IRC | 14:25 | |
| *** mvkr has quit IRC | 14:37 | |
| *** BjoernT has joined #openstack-swift | 14:39 | |
| *** BjoernT has quit IRC | 14:40 | |
| *** tesseract has joined #openstack-swift | 14:50 | |
| *** tesseract has quit IRC | 14:51 | |
| *** tesseract has joined #openstack-swift | 14:51 | |
| *** BjoernT has joined #openstack-swift | 14:54 | |
| *** tesseract has quit IRC | 15:01 | |
| *** tesseract has joined #openstack-swift | 15:03 | |
| *** NM has quit IRC | 15:04 | |
| *** tesseract has quit IRC | 15:11 | |
| *** tesseract has joined #openstack-swift | 15:16 | |
| *** tesseract has quit IRC | 15:18 | |
| *** tesseract has joined #openstack-swift | 15:20 | |
| alecuyer | hey kota_, I have tried to run the ansible playbooks starting from clay's work (https://github.com/swiftstack/vagrant-swift-all-in-one/pull/80) | 15:33 |
| alecuyer | The first problem I see is that the object-rpcmanager fails to start the golang binary, because it cannot find the leveldb and snappy libs, which are in /usr/local/lib | 15:34 |
| alecuyer | This can be worked around using LD_LIBRARY_PATH or adding /usr/local/lib to a new file in /etc/ld.so.conf.d/ (until we figure out how to set rpath with go modules) | 15:35 |
| *** baojg has quit IRC | 15:37 | |
| alecuyer | After that, I got level=fatal msg="write udp 127.0.0.1:37037->127.0.0.1:8125: write: connection refused" socket=/srv/node1/sdb1/losf/rpc.socket | 15:37 |
| alecuyer | which happens because the go module picked the wrong statsd commit, (it tries to check if there's a statsd server listening and exits if it doesn't find one) | 15:38 |
| alecuyer | So, I still have some reading to do on go modules | 15:39 |
| *** mvkr has joined #openstack-swift | 15:42 | |
| *** hugokuo has joined #openstack-swift | 15:50 | |
| alecuyer | ok so that second problem is because rpc.go has the import as "github.com/alecuyer/statsd" when it should be "github.com/alecuyer/statsd/v2". Then it all works | 15:50 |
| *** timburke has joined #openstack-swift | 15:51 | |
| *** ChanServ sets mode: +v timburke | 15:51 | |
| kota_ | alecuyer: sounds nice. could you make a commit for gerrit review? | 15:52 |
| alecuyer | kota_: yes, running a few more tests in the VM and I'll do that | 15:54 |
| kota_ | alecuyer: great, thanks! | 15:54 |
| alecuyer | let's see if it works ;) | 15:54 |
| *** gyee has joined #openstack-swift | 15:54 | |
| kota_ | :D | 15:54 |
| *** mvkr has quit IRC | 15:59 | |
| *** tesseract has quit IRC | 16:01 | |
| alecuyer | hmm I don't think we even try to start swift-object-rpcmanager ? (I only find "swift-init main start") | 16:03 |
| timburke | :D p 657705 gets us to 83% unit test coverage! | 16:05 |
| patchbot | https://review.opendev.org/#/c/657705/ - swift - py3: port proxy/test_mem_server.py - 3 patch sets | 16:05 |
| alecuyer | ouch nevermind, I can't read.. | 16:10 |
| *** BjoernT has quit IRC | 16:26 | |
| openstackgerrit | Alexandre Lécuyer proposed openstack/swift feature/losf: WIP: setup losf policy for in-process functional testing https://review.opendev.org/657075 | 16:34 |
| openstackgerrit | Alexandre Lécuyer proposed openstack/swift feature/losf: Change losf functest to run on saio setup https://review.opendev.org/657153 | 16:34 |
| openstackgerrit | Alexandre Lécuyer proposed openstack/swift feature/losf: Use statsd v2, set LD_LIBRARY_PATH for index server https://review.opendev.org/658152 | 16:34 |
| *** irclogbot_3 has quit IRC | 16:46 | |
| *** irclogbot_2 has joined #openstack-swift | 16:47 | |
| *** NM has joined #openstack-swift | 16:57 | |
| *** tdasilva has joined #openstack-swift | 17:15 | |
| *** ChanServ sets mode: +v tdasilva | 17:16 | |
| *** mikecmpbll has quit IRC | 17:32 | |
| *** rdejoux has quit IRC | 17:44 | |
| *** BjoernT has joined #openstack-swift | 17:55 | |
| *** NM has quit IRC | 18:01 | |
| *** NM has joined #openstack-swift | 18:02 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: We don't have to keep the retrieved token anymore https://review.opendev.org/609168 | 18:21 |
| *** BjoernT has quit IRC | 18:34 | |
| *** NM has quit IRC | 18:39 | |
| *** BjoernT has joined #openstack-swift | 18:44 | |
| clayg | @timburke what do you think about pulling the MERGE_ITEMS interface and object-expirer-async-delete-row-handling into orthogonal patches with the s3api bulk delete feature dependent on them? Then we could discuss merging the base primitives maybe ahead of the next bit? | 19:10 |
| timburke | clayg, that was exactly my thought as well. having a tool (even if it starts out just for operators!) that async-deletes a container or some prefix within a container seems valuable separate from the s3api change | 19:14 |
| timburke | *and* it side-steps the "but what if you delete the now-empty container" problem because *it doesn't need to update the container right away* -- let the expirer's delete take care of getting listings cleaned up | 19:15 |
| clayg | that would be sick! | 19:15 |
| clayg | oh right, yeah 👍 | 19:16 |
| clayg | but maybe still use merge items to bulk insert the deletes? | 19:16 |
| timburke | i think i feel justified in still having a new UPDATE verb for that... since the idea would be that we some day want it as a client-facing thing... | 19:17 |
| clayg | is update merge items? i thought... *not* client thing - just like... *InternalClient* ??? | 19:21 |
| timburke | i did realize that there are some more funny things going on with storage policies though... like, what happens if you get a split-brain in the expirer queue container? the reconciler will never be able to sort those rows out because there are no actual objects to move... | 19:21 |
| clayg | so the reconciler will recocile tombstones - presumably it could reconcile pending delete rows as well.. eventually | 19:22 |
| timburke | yeah, for a start. and since InternalClient is a type of proxy, i'd still want UPDATE | 19:22 |
| timburke | at some point, i'd want this (async delete) to be more than an operator feature | 19:22 |
| *** mathiasb has joined #openstack-swift | 19:22 | |
| clayg | but that's not what'd we'd be doing right away right - in this change we're suggesting the expirer just cleans up the bulk-inserted rows? | 19:22 |
| *** mikecmpbll has joined #openstack-swift | 19:23 | |
| clayg | yeah that works, UPDATE as internal client only verb to facilitate stuffing rows into expirer queue - sounds like a great operator feature!!! | 19:23 |
| timburke | yeah, so, three patches: 1, add ability to bulk-update rows from an InternalClient. this should probably be a new UPDATE verb, as i don't think IC knows about replication interfaces | 19:24 |
| timburke | 2, operator feature to take a container and a (possibly empty) prefix, spin up an IC, list everything in that prefix and drop it in an expiring queue. as part of it, expirer learns how to delete without an X-Delete-At | 19:25 |
| timburke | 3, s3api patch | 19:26 |
| timburke | which still may or may not end up landing | 19:26 |
| clayg | yeah, part 3 is optional 🤣 | 19:26 |
| clayg | but the first two sound like nothing but sexy operator hawtness | 19:27 |
| clayg | you could maybe squash 1 & 2 - depending on volume | 19:27 |
| timburke | oh, and 2b, be able to mark a container as getting reaped so it'll 410 via proxy-server and get cleaned up in the background. definitely needs more thought/design though | 19:28 |
| timburke | 2 puts us a long way toward being able to do that though. just gotta figure out how to make it flip to *actually* getting deleted... | 19:29 |
| timburke | i seem to remember us getting into trouble with sharding, deleting containers, and having them lose all their metadata... so i think it'd need to be a sysmeta flag or something... | 19:30 |
| openstackgerrit | Pete Zaitcev proposed openstack/swift master: py3: slo https://review.opendev.org/650507 | 19:33 |
| clayg | yeah I think problem of 2b is really more like a pre-req for 3, like a 3a, and 3b is something about accounting, maybe the deleted=2 flag | 19:34 |
| *** zaitcev has joined #openstack-swift | 19:35 | |
| *** ChanServ sets mode: +v zaitcev | 19:35 | |
| *** BjoernT has quit IRC | 19:45 | |
| timburke | clayg, nah, 2b and 3 are separate -- the s3api takes a list of files to ax, not marking the whole bucket deleted (or even some prefix) | 19:47 |
| clayg | @timburke sure, but it also marks them as "already deleted" in the container - which is really where all the trouble starts 🤷♂️ | 19:51 |
| timburke | i'm thinking 2b doesn't need to do that though | 19:52 |
| timburke | account reaper sure doesn't do it | 19:53 |
| clayg | ah... ok... I see your strategy I think then - as long as comes separate/after 2 I'm happy 😁 | 19:54 |
| clayg | cause like you could say "empty prefix async delete" and still just have the container there until it finishes processing (and if you add a *new* object you can't reap the container, and if you overwrite a name that hasn't been reaped yet it will get gobbled) | 19:55 |
| clayg | but being able to lock our writes while reaping (in the special case) would be a cool next step (do I have it right?) | 19:56 |
| *** e0ne has joined #openstack-swift | 20:09 | |
| *** e0ne has quit IRC | 20:10 | |
| *** NM has joined #openstack-swift | 20:25 | |
| *** zaitcev_ has joined #openstack-swift | 20:53 | |
| *** ChanServ sets mode: +v zaitcev_ | 20:53 | |
| *** zaitcev has quit IRC | 20:56 | |
| alecuyer | kota_: the functests have passed! but "retry limit" in swift-dvsm-functional, I'll take a look tomorrow | 21:10 |
| *** dasp has quit IRC | 21:52 | |
| *** zaitcev_ has quit IRC | 21:54 | |
| *** zaitcev_ has joined #openstack-swift | 22:06 | |
| *** ChanServ sets mode: +v zaitcev_ | 22:06 | |
| *** NM has quit IRC | 22:09 | |
| *** NM has joined #openstack-swift | 22:16 | |
| *** NM has quit IRC | 22:16 | |
| *** nottrobin has quit IRC | 22:35 | |
| *** nottrobin has joined #openstack-swift | 22:36 | |
| mattoliverau | morning | 22:52 |
| *** tkajinam has joined #openstack-swift | 23:03 | |
| *** samueldmq has quit IRC | 23:31 | |
| *** zaitcev_ has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!