Thursday, 2019-01-24

mattoliverautimburke: sure. if where is a boring session, I'll use the chance to review it ;)00:30
mattoliverau*there00:30
mattoliverausigh, I can't le type00:30
timburkeyay! only, wait, you guys keep raving about how great lca is... you won't get to this until you get back to the hotel, will you!?00:30
mattoliveraupotentailly that is true, and the penguin dinner is tonight.. so might be evevn later ;)00:31
mattoliveraubut will get to it when I can00:31
timburkethanks mattoliverau!00:32
zaitcev        dest_brokers = {}  # map shard range -> broker02:54
zaitcev                dest_brokers[dest_shard_range] = destination02:54
zaitcevTypeError: unhashable type: 'ShardRange'02:54
zaitcevThanks, Matt02:54
zaitcev(ironically this time)02:54
zaitcevhow about I just use id() and move on02:55
openstackgerritzhouxinyong proposed openstack/slogging master: Change openstack-dev to openstack-discuss in setup.cfg  https://review.openstack.org/63289003:06
*** psachin has joined #openstack-swift03:09
openstackgerritzhouxinyong proposed openstack/swift-specs master: Change openstack-dev to openstack-discuss in setup.cfg  https://review.openstack.org/63289603:14
*** baojg has quit IRC03:24
*** baojg has joined #openstack-swift03:43
*** spsurya has joined #openstack-swift04:33
*** gyee has quit IRC05:01
*** baojg has quit IRC05:02
*** tkajinam has quit IRC05:02
*** e0ne has joined #openstack-swift05:03
*** e0ne has quit IRC05:03
*** baojg has joined #openstack-swift05:04
*** baojg has quit IRC05:04
*** baojg has joined #openstack-swift05:05
openstackgerritPete Zaitcev proposed openstack/swift master: py3: port the container sharder  https://review.openstack.org/57032005:07
*** tkajinam has joined #openstack-swift05:07
zaitcevWhat a nightmare. It better be ending soon.05:10
zaitcevtimburke, mattoliverau: I'm going to look at some of the reviews you guys were talking about recently. You want anything, add me to reviewer list, so the website shows the patch in the list.05:15
*** tkajinam has quit IRC05:18
*** tkajinam has joined #openstack-swift05:18
*** zaitcev has quit IRC06:10
*** ccamacho has quit IRC06:26
*** ccamacho has joined #openstack-swift07:12
*** baojg has quit IRC07:15
*** baojg has joined #openstack-swift07:16
*** baojg has quit IRC07:16
*** baojg has joined #openstack-swift07:17
*** baojg has quit IRC07:25
*** baojg has joined #openstack-swift07:25
*** e0ne has joined #openstack-swift07:35
*** tkajinam_ has joined #openstack-swift08:04
*** tkajinam has quit IRC08:06
*** rcernin has quit IRC08:29
*** tkajinam_ has quit IRC08:31
*** baojg has quit IRC08:43
*** baojg has joined #openstack-swift08:45
*** ianychoi has quit IRC08:55
*** hseipp has joined #openstack-swift09:14
*** e0ne has quit IRC09:41
*** e0ne has joined #openstack-swift09:56
*** mikecmpbll has joined #openstack-swift10:08
*** mikecmpb_ has joined #openstack-swift10:33
*** mikecmpbll has quit IRC10:34
*** mikecmpb_ is now known as mikecmpbll11:41
*** ybunker has joined #openstack-swift12:28
ybunkerwhat's the recommended kernel version for ubuntu 16.04.5 LTS where rsync has no problems or oom at all :) ?12:30
*** takamatsu has joined #openstack-swift12:31
*** psachin has quit IRC12:32
*** pcaruana has quit IRC12:40
*** e0ne has quit IRC12:59
*** hseipp has quit IRC13:02
*** pcaruana has joined #openstack-swift13:20
*** e0ne has joined #openstack-swift13:20
*** baojg has quit IRC13:28
admin6Hi team, I can’t reach to make swift-dispersion-report works with keystone auth v3. Authentication seems to work but the proxy log en error :  proxy: tenant mismatch: AUTH_Dev != Prod (txn: txf960300182c04233abc2e-005c49c0ef) (client_ip: 127.0.0.1). Do you have any suggestion ?13:59
*** e0ne has quit IRC14:14
*** baojg has joined #openstack-swift14:31
*** ccamacho has quit IRC14:32
*** e0ne has joined #openstack-swift14:34
*** ccamacho has joined #openstack-swift14:35
*** takamatsu has quit IRC15:00
*** kukacz_ has quit IRC15:20
*** kukacz has joined #openstack-swift15:31
*** pcaruana has quit IRC15:35
*** kukacz has quit IRC15:42
*** ianychoi has joined #openstack-swift15:44
*** kukacz has joined #openstack-swift15:46
*** pcaruana has joined #openstack-swift15:55
*** kukacz has quit IRC15:58
*** kukacz has joined #openstack-swift16:00
*** gyee has joined #openstack-swift16:06
*** e0ne has quit IRC16:09
*** takamatsu has joined #openstack-swift16:12
*** gyee has quit IRC16:14
*** ccamacho has quit IRC16:23
*** kukacz has quit IRC16:24
*** gyee has joined #openstack-swift16:25
*** kukacz has joined #openstack-swift16:26
*** admin6 has left #openstack-swift16:31
*** pcaruana has quit IRC16:48
ybunkeris there a way to throttle the obj-replicator process so I can leave it running all day long with just a few latency impact?, I try setting the rsync_bwlimit to 10 for example on the object-server.conf file, but it seems that is not working at all16:49
*** takamatsu has quit IRC16:54
*** zaitcev has joined #openstack-swift17:07
*** ChanServ sets mode: +v zaitcev17:07
*** mikecmpbll has quit IRC17:25
*** ccamacho has joined #openstack-swift17:28
*** e0ne has joined #openstack-swift17:59
*** e0ne has quit IRC18:13
*** takamatsu has joined #openstack-swift18:15
*** e0ne has joined #openstack-swift18:27
DHEthat's a bandwidth usage setting though. a filesystem crawl will still impact disk performance even though rsync isn't really sending anything.18:32
*** ccamacho has quit IRC18:38
*** mikecmpbll has joined #openstack-swift18:45
ybunkergot it, and is there a way to handle that? to put some limitation18:48
DHEnot to rsync directly, but if you have replicator jobs set up with ionice settings they will be inherited down to rsync on the local machine18:58
DHEon the side receiving the incoming connection you might need to have xinetd run a wrapper to set ionice before launching rsync or such18:59
DHEI don't know if there's a better way18:59
*** e0ne has quit IRC19:02
zigoWe have a patch for the encryption issue we found out.19:04
zigowe will open bug and patch soonish.19:05
DHEXFS needs a metadata disk feature...19:09
*** takamatsu has quit IRC19:09
*** baojg has quit IRC19:22
zigoDhe: that is not the issue.19:25
*** e0ne has joined #openstack-swift19:26
DHEsorry no that's in relation to what ybunker is asking19:38
timburke:-( https://bugs.launchpad.net/swift/+bug/1784753 is still a thing...19:40
openstackLaunchpad bug 1784753 in OpenStack Object Storage (swift) "Add ratelimiting to more background daemons" [Undecided,New]19:40
timburkewe got some daemons to do more ratelimiting, but still need object-reconstructor and object-replicator ... and idk that we'll be able to throttle rsync real well in the replicator if your cluster's full enough that there tends to be a bunch of objects per suffix...19:41
*** mahatic has quit IRC19:42
zigoDHE: Your thoughts about this one? https://salsa.debian.org/openstack-team/services/swift/blob/debian/rocky/debian/patches/fix-decryption-of-object-if-metadata-is-missing.patch19:51
zigoPlease let us test this before we open the bug and review...19:51
DHEoh I don't work for the project. sorry.19:52
*** e0ne has quit IRC19:54
zigo:)19:56
timburkezigo: out of curiosity, what were the two different paths? how did the object come to be at the *wrong* path?19:58
timburkeat first glance, patch seems sane enough -- i'd probably want to log a warning or something about the path mismatch. makes me very glad we stuck path in the key_id!19:59
*** ybunker has quit IRC20:06
*** rcernin has joined #openstack-swift21:02
*** itlinux has joined #openstack-swift21:19
*** baojg has joined #openstack-swift21:23
*** baojg has quit IRC21:27
zigotimburke: Somehow, the specs tells that what should be in use for the decryption, is the object name. However, it's not what's in use when encrypting, it's a truncated version of the object name (or path...).22:05
zigotimburke: When the object name is large enough, then they don't match.22:06
zigoThat's the first problem.22:06
timburkeeep!22:06
zigoThe 2nd one, is that in some cases, we have no metadata recorded in the object itself.22:06
zigoIt appears it's only saved in the container.22:06
zigoWe haven't figured why exactly.22:07
zigoBut it doesn't really mater, because even if we figure out where the issue is, we still need to have the (soon) proposed patch in order to recover wrongly written objects.22:08
timburkeno metadata at all? or it's missing some of the encryption metadata? if you run swift-object-info on one of the affected .data files, what does it say?22:11
*** rcernin has quit IRC22:11
*** rcernin has joined #openstack-swift22:12
timburkemakes me happy we had the outcome we did on the review: https://review.openstack.org/#/c/328208/11/swift/common/middleware/keymaster.py@8522:20
patchbotpatch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption (MERGED) - 16 patch sets22:20
*** itlinux has quit IRC22:38
*** baojg has joined #openstack-swift22:53
*** tkajinam has joined #openstack-swift22:57
*** spsurya has quit IRC23:04
*** baojg has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!