Monday, 2019-02-11

*** mikecmpbll has joined #openstack-swift00:45
*** psachin has joined #openstack-swift03:04
*** mikecmpbll has quit IRC03:42
*** hoonetorg has joined #openstack-swift06:22
*** e0ne has joined #openstack-swift07:23
*** psachin has quit IRC07:45
*** e0ne has quit IRC07:47
*** e0ne has joined #openstack-swift08:00
*** gkadam has joined #openstack-swift08:01
*** e0ne has quit IRC08:05
*** ccamacho has joined #openstack-swift08:06
*** mikecmpbll has joined #openstack-swift08:22
*** mikecmpbll has quit IRC08:40
*** mikecmpbll has joined #openstack-swift09:02
*** hseipp has joined #openstack-swift09:56
*** ybunker has joined #openstack-swift14:18
ybunkerhi all, quick question... how can i access to .expiring_objects account? i would like to know if an object that has disappeared from a cluster was being hitting by the object-expirer daemon or not, I can't find the object on the logs and the customer has confirm us that until last week the object GETs work fine14:25
rledisezybunker: you should be able to check your object-expirer logs, something like:14:28
rledisezFeb 11 15:26:46 object-expirer: - - 11/Feb/2019/14/26/46 DELETE /v1/AUTH_… HTTP/1.0 204 - Swift%20Object%20Expirer - - - - txe99ea7dc975144e7bcf3a-005c618626 - 0.6796 - - 1549895206.308103085 1549895206.987672091 014:28
rledisezvar/log/swift/object-expirer.log by default IIRW14:29
ybunkerrledisez:..mmm i don't have the object-expirer.log file... also the process was running, and doing some grep on the proxy logs, can't find any DELETE operation14:31
*** ybunker has quit IRC14:36
*** hoonetorg has quit IRC15:09
*** hoonetorg has joined #openstack-swift15:27
*** ybunker has joined #openstack-swift15:35
*** gkadam has quit IRC15:49
*** ybunker has quit IRC15:58
openstackgerritClay Gerrard proposed openstack/swift master: Display crypto data/metadata details in swift-object-info
*** e0ne has joined #openstack-swift16:04
*** e0ne has quit IRC16:17
*** onovy has quit IRC16:19
*** onovy has joined #openstack-swift16:19
*** e0ne has joined #openstack-swift16:21
*** e0ne has quit IRC16:26
notmynamegood morning16:33
*** e0ne has joined #openstack-swift16:36
*** ybunker has joined #openstack-swift17:16
*** e0ne has quit IRC17:20
openstackgerritClay Gerrard proposed openstack/swift master: Display crypto data/metadata details in swift-object-info
*** gyee has joined #openstack-swift17:24
timburkeclayg: so one thought i had over the weekend about the reconstructor change... with a two-region EC policy, four primaries do a local-remote-remote sync17:40
*** mikecmpbll has quit IRC17:41
timburkei really *like* the idea that every node should be responsible for *a* remote sync... and if we get up to three or more regions, I'm not so worried... but I wonder if it might be better *in this particular case* to wrap back around to the start of *our* region when we're the first/last node for the region17:43
claygyes, I think it would be interesting to optimize the frag duplicated EC rebalance/reconstruct case17:45
claygwhat's the biggest win of all the things we can do?  probably trying to do a sync from the duplicate instead of rebuild?17:46
claygunless you want to optimize for local traffic over wan... hrmmm...17:46
clayghonestly probably if we have bandwidth for multi-region cluster enhancements we should look at metadata requests17:47
*** zaitcev has joined #openstack-swift17:50
*** ChanServ sets mode: +v zaitcev17:50
timburkethe one special case i just mentioned should be cheap -- direct copy of a matching frag seems like it'd do more good17:52
timburkewhat do you mean "metadata requests"?17:52
timburkeclayg: is the reconstructor at all read-affinity-aware?17:55
claygno.  it blasts concurrent requests to all primaries18:04
claygbasically when we added ec duplication we did no optimization for reconstructor - it "worked" so it was left to do what it already did18:04
*** hseipp has quit IRC18:12
*** mikecmpbll has joined #openstack-swift18:26
timburkeboo! sharder probe tests are using the raw swiftclient.client.delete_object api instead of at least using swiftclient.client.Connection.delete_object which would include retries :-(
*** openstackgerrit has quit IRC18:51
*** ybunker has quit IRC18:55
*** mvkr has quit IRC19:10
*** e0ne has joined #openstack-swift19:41
*** mikecmpbll has quit IRC19:45
*** gyee has quit IRC19:52
*** gyee has joined #openstack-swift20:00
*** e0ne has quit IRC20:08
claygtimburke: I think if we could get probetests to run with a 20s node_timeout we'd see a lot less failures...20:35
claygbut tdasilva says it just uses the default/saio config - i want like a probe/gate override... I think...20:36
timburkeclayg: ?20:36
claygoh neato - so we just need to find somewhere in one of playbooks to put it20:37
claygprobably just add a new `crudini` command here ->
timburkeoh yeah -- that works, too20:43
*** openstackgerrit has joined #openstack-swift20:43
openstackgerritClay Gerrard proposed openstack/swift master: Increase probetest multi-node timeout
claygwell see if that multi-node job has the bigger timeout 🤷20:44
*** itlinux has joined #openstack-swift21:22
*** e0ne has joined #openstack-swift21:34
*** itlinux has quit IRC21:35
*** e0ne has quit IRC21:41
openstackgerritClay Gerrard proposed openstack/swift master: Increase node_timeout in gate
claygso i guess i was confused about all the different jobs we have and which setups go where - maybe there is better - maybe there's some more playbooks we should update?  🤷‍♂️21:49
tdasilvaclayg: yeah, i think our jobs definition is a bit messy atm, i have this TODO about organizing them a bit better, but haven't had a chance to get to it21:53
tdasilvabasically i'd like to eventually change them to something similar we did with libec21:53
claygoh?  is libec nice?21:54
tdasilvayeah, it's nice in the sense that we defined roles:
tdasilvaand now we should be able to re-use those same roles in pyeclib21:55
timburkehuh. i'm trying to decide if this is *also* resource-constrained-gate:
timburkei don't think i've seen *that* one before22:50
*** tkajinam has joined #openstack-swift22:57
*** mvkr has joined #openstack-swift23:08
*** mikecmpbll has joined #openstack-swift23:15
*** rchurch_ has joined #openstack-swift23:21
*** rchurch has quit IRC23:23
*** jistr_ has joined #openstack-swift23:39
*** onovy has quit IRC23:46
*** jistr has quit IRC23:46
*** thyrst has quit IRC23:46
*** fyx has quit IRC23:46
*** coreycb has quit IRC23:46

Generated by 2.15.3 by Marius Gedminas - find it at!