Thursday, 2019-12-12

*** gyee has quit IRC01:45
*** psachin has joined #openstack-swift03:35
openstackgerritThiago da Silva proposed openstack/swift master: Skip container-sync when versioning enabled  https://review.opendev.org/69813906:14
*** pcaruana has joined #openstack-swift06:18
*** gkadam has joined #openstack-swift07:13
*** gkadam has quit IRC07:13
*** tesseract has joined #openstack-swift07:59
*** tkajinam has quit IRC08:04
*** rdejoux has joined #openstack-swift08:33
*** corvus has quit IRC08:40
*** tonyb has quit IRC08:56
*** tonyb has joined #openstack-swift08:56
*** corvus has joined #openstack-swift08:59
*** rpittau|afk is now known as rpittau09:00
*** ccamacho has joined #openstack-swift09:35
*** ccamacho is now known as ccamacho|pto09:48
*** mikecmpbll has joined #openstack-swift10:27
openstackgerritThiago da Silva proposed openstack/swift master: Skip container-sync when versioning enabled  https://review.opendev.org/69813910:45
*** rcernin has quit IRC10:56
*** pcaruana has quit IRC11:03
*** pcaruana has joined #openstack-swift11:06
*** FlorianFa has joined #openstack-swift13:22
*** pcaruana has quit IRC13:57
henriqueof  Soppose my controllers are down, is there a way to recover my backups from a wsift node?14:32
*** pcaruana has joined #openstack-swift14:36
DHEnot unless it's marked as a public container (sounds unlikely), or modifying a proxy-server to remove authentication requirements (not recommended!)14:51
DHEkeystone should be redundant if that's a concern14:51
*** ccamacho|pto has quit IRC14:54
*** henriqueof has quit IRC15:08
*** henriqueof1 has joined #openstack-swift15:08
claygsorry i missed the meeting yesterday, just read logs15:35
*** gyee has joined #openstack-swift16:05
*** irclogbot_2 has quit IRC16:14
*** irclogbot_1 has joined #openstack-swift16:16
*** tesseract has quit IRC16:48
*** rdejoux has quit IRC16:56
*** efried is now known as efried_afk17:08
*** mikecmpbll has quit IRC17:39
*** rpittau is now known as rpittau|afk17:49
*** psachin has quit IRC18:09
*** gmann is now known as gmann_afk18:53
*** efried_afk is now known as efried19:38
*** pcaruana has quit IRC20:19
openstackgerritTim Burke proposed openstack/swift master: Add normalize_etag() helper function  https://review.opendev.org/69879920:32
openstackgerritTim Burke proposed openstack/swift master: Allow clients to send quoted ETags for static links  https://review.opendev.org/69880020:32
claygtimburke: ^ and those are orthogonal to the new proxy option?20:36
claygoh oh, i see - you're gunna rebase 👍20:37
timburkeyup! just breaking them out as that option seems like it may instead become a per-container setting implemented in middleware20:37
timburkeyeah, maybe rebase, maybe abandon -- either way, those seem like independent changes that could merge on their own merits20:37
DHEhow bad is this action?  cd /srv/node ; for i in slot*; do swift-object-replicator -v -o -d $i /etc/swift/object-server.conf & sleep 1 ; done20:57
DHEbackground: a region with only 2 hosts (most disks on a single host) was expanded to 5 hosts. this command was run on that "most disks" host20:59
timburkerunning replicators is rarely "bad"21:00
timburkemakes me wonder why we're running it as a once-off thing, though. and why we don't just set replicator_workers to the number of disks21:01
timburkeis there any bad behavior you're observing that's prompting the question?21:02
DHEI set concurrency=4, workers=4 so maybe that needs to be cranked21:03
DHEnumber of disks is 3021:03
DHErunning as a one-off just to get the partitions redistributed quickly21:04
timburkei could see logs getting noisy or disks getting overloaded if you've got multiple replicators trying to process the same disk at the same time... but shouldn't be anything that'd put durability at risk, say21:04
timburkewhat's your replication cycle time like?21:05
DHEdefaults21:05
DHEthis is my first big replication event and I felt like it was underperforming for what should have been a pretty easy port saturation...21:07
openstackgerritThiago da Silva proposed openstack/swift master: Skip container-sync when versioning enabled  https://review.opendev.org/69813921:25
openstackgerritClay Gerrard proposed openstack/python-swiftclient master: object versioning features  https://review.opendev.org/69187721:35
claygDHE: workers == # disks with a concurrency of 2-4 is normally enough to get the job done - it helps if the receiving end has lots of head room to accept rsync connections21:39
claygif this is your "first big replication event" - it's totally normal to be looking at network graphs and disk utilization and doing some tuning/optimization of settings21:40
claygalso reasonable to stop/slowdown the background replicators and play with settings using smaller targeted runs until you get a feel for what you want to do21:41
claygi.e. your for loop is very good and normal way to be thinking about the replicators21:41
claygthe trick is to get a sense for what your cluster can do and find some settings you can run with so you don't have to monkey around with every rebalance21:42
claygalthough TBH - we very often turn on handoffs_only after a big ring change to get things moving21:42
*** dosaboy has quit IRC22:01
*** aluria has quit IRC22:02
*** henriqueof1 has quit IRC22:08
mattoliveraumorning22:13
tdasilvamattoliverau: o/22:15
*** gmann_afk is now known as gmann22:29
openstackgerritClay Gerrard proposed openstack/swift master: WSGI server workers must drop_privledges  https://review.opendev.org/69856323:02
*** tkajinam has joined #openstack-swift23:06
*** irclogbot_1 has quit IRC23:11
*** irclogbot_2 has joined #openstack-swift23:13
*** dosaboy has joined #openstack-swift23:23
*** diablo_rojo has quit IRC23:24
*** dosaboy has quit IRC23:29

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!