Wednesday, 2016-03-16

*** haomaiwa_ has quit IRC00:01
*** haomaiwang has joined #openstack-swift00:01
*** baojg has quit IRC00:01
*** mmcardle has quit IRC00:02
ho_good morning!00:03
openstackgerritClay Gerrard proposed openstack/swift: wip: make auditor cleanup stale rsync tempfiles  https://review.openstack.org/29317700:09
*** Jeffrey4l has joined #openstack-swift00:12
*** garthb has quit IRC00:13
*** StraubTW has joined #openstack-swift00:16
*** openstack has joined #openstack-swift00:25
takashigood morning00:25
pdardeaugood morning takashi00:26
takashipdardeau: good morning :-)00:26
brianclineclayg: "stale rsync droppings" is probably the best thing i've read so far this week00:33
*** lyrrad has quit IRC00:50
*** Jeffrey4l has quit IRC00:53
*** gyee has quit IRC00:54
*** zaitcev has quit IRC00:59
*** haomaiwang has quit IRC01:01
*** haomaiwang has joined #openstack-swift01:01
*** rickyrem1 has joined #openstack-swift01:05
*** david-lyle has joined #openstack-swift01:06
*** rickyrem2 has joined #openstack-swift01:12
*** rickyrem1 has quit IRC01:16
*** dmorita has quit IRC01:21
*** haomaiwang has quit IRC01:23
*** dmorita has joined #openstack-swift01:23
*** dmorita_ has joined #openstack-swift01:28
*** dmorita__ has joined #openstack-swift01:30
*** dmorita has quit IRC01:30
*** MVenesio has joined #openstack-swift01:32
*** dmorita_ has quit IRC01:33
*** MVenesio has quit IRC01:36
*** tsg has quit IRC01:36
*** esker has joined #openstack-swift01:37
*** NM has joined #openstack-swift01:38
*** panda has quit IRC01:40
*** panda has joined #openstack-swift01:41
*** StraubTW has quit IRC01:41
*** CaioBrentano has joined #openstack-swift01:44
*** _JZ_ has quit IRC01:45
*** _JZ_ has joined #openstack-swift01:47
*** CaioBrentano has quit IRC01:48
*** _JZ_ has quit IRC01:52
*** mingdang1 has joined #openstack-swift01:56
*** Lickitysplitted_ has quit IRC01:56
*** baojg has joined #openstack-swift01:59
*** Lickitysplitted has joined #openstack-swift01:59
*** mingdang1 has quit IRC02:00
*** haomaiwang has joined #openstack-swift02:00
*** haomaiwang has quit IRC02:01
*** haomaiwa_ has joined #openstack-swift02:01
openstackgerritMerged openstack/swift: Faster suffix invalidations on object PUT/DELETE  https://review.openstack.org/28661802:08
*** baojg has quit IRC02:17
*** baojg has joined #openstack-swift02:20
*** dmorita__ has quit IRC02:20
*** StraubTW has joined #openstack-swift02:23
*** dmorita has joined #openstack-swift02:26
*** StraubTW has quit IRC02:40
*** NM has quit IRC02:56
*** Jeffrey4l has joined #openstack-swift02:57
*** haomaiwa_ has quit IRC03:01
*** haomaiwang has joined #openstack-swift03:01
*** natarej__ has joined #openstack-swift03:04
*** rickyrem2 has quit IRC03:08
*** baojg has quit IRC03:08
*** natarej_ has quit IRC03:08
*** baojg has joined #openstack-swift03:10
*** esker has quit IRC03:11
*** dmorita has quit IRC03:11
*** dmorita has joined #openstack-swift03:24
*** dmorita has quit IRC03:28
*** MVenesio has joined #openstack-swift03:32
*** proteusguy__ has quit IRC03:36
*** MVenesio has quit IRC03:37
*** baojg has quit IRC03:42
*** baojg has joined #openstack-swift03:43
kota_hello03:47
*** aswadr_ has joined #openstack-swift03:47
mattoliveraukota_: o/03:47
kota_mattoliverau: o/03:47
* mattoliverau is finally back03:47
*** proteusguy__ has joined #openstack-swift03:53
*** trifon has joined #openstack-swift03:53
*** natarej__ has quit IRC03:55
*** natarej__ has joined #openstack-swift03:56
*** haomaiwang has quit IRC04:01
*** esker has joined #openstack-swift04:01
*** haomaiwang has joined #openstack-swift04:01
mahaticgood morning04:03
mattoliveraumahatic: o/04:03
mahaticmattoliverau: you're back? is the baby out?04:05
*** esker has quit IRC04:06
mattoliveraumahatic: nope. hospital was final visit before baby. Making sure everything is ok. Bec (due to reasons) will need to be induced. So looks like that'll happen very early Friday morning my time (if it doesn't come before hand). So looks like the swiftlet will be here by/on the weekend :)04:08
aswadr_morning!04:08
mahaticmattoliverau: awesome :)04:08
mattoliverauaswadr_: o/04:08
*** links has joined #openstack-swift04:10
aswadr_o/04:10
*** klrmn has quit IRC04:15
*** ppai has joined #openstack-swift04:19
*** dmorita has joined #openstack-swift04:25
*** natarej__ has quit IRC04:28
*** natarej has joined #openstack-swift04:28
*** dmorita has quit IRC04:30
*** treaki__ has joined #openstack-swift04:30
*** treaki_ has quit IRC04:34
*** baojg has quit IRC04:45
*** asettle has quit IRC04:50
*** haomaiwang has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:01
*** asettle has joined #openstack-swift05:11
*** dmorita has joined #openstack-swift05:27
*** dmorita has quit IRC05:31
*** Jeffrey4l has quit IRC05:32
*** esker has joined #openstack-swift05:33
*** MVenesio has joined #openstack-swift05:33
*** MVenesio has quit IRC05:37
*** esker has quit IRC05:38
*** panda has quit IRC05:40
*** panda has joined #openstack-swift05:41
*** david-lyle has quit IRC05:45
*** david-lyle has joined #openstack-swift05:46
openstackgerritMatthew Oliver proposed openstack/swift: Add concurrent reads option to proxy  https://review.openstack.org/11771006:00
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-swift06:01
*** andreaponza has joined #openstack-swift06:11
*** stantonnet has quit IRC06:15
*** mingdang1 has joined #openstack-swift06:40
*** ChubYann has quit IRC06:45
*** stantonnet has joined #openstack-swift06:49
*** asettle has quit IRC06:49
*** esker has joined #openstack-swift06:49
*** dmorita has joined #openstack-swift06:54
*** stantonnet has quit IRC06:54
*** esker has quit IRC06:54
*** stantonnet has joined #openstack-swift06:58
*** dmorita has quit IRC06:58
*** haomaiwang has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** mingdang1 has quit IRC07:04
*** tesseract has joined #openstack-swift07:16
*** tesseract is now known as Guest6496707:17
*** Guest64967 has quit IRC07:21
*** esker has joined #openstack-swift07:23
*** AJaeger has joined #openstack-swift07:25
AJaegerswift cores, https://review.openstack.org/#/c/292719/ is the translation import. It has one +A already but did not merge due to the CI problems yesterday. Could somebody just add another +A, please?07:26
patchbotAJaeger: patch 292719 - swift - Imported Translations from Zanata07:26
*** esker has quit IRC07:28
*** wanghua has joined #openstack-swift07:29
*** MVenesio has joined #openstack-swift07:34
*** tesseract- has joined #openstack-swift07:35
*** MVenesio has quit IRC07:38
*** chlong has quit IRC07:42
*** mariusv_ has quit IRC07:44
*** mariusv has joined #openstack-swift07:45
kota_AJaeger: Is it resolved by +A again? I saw a few similer patch which got +A but not merged yet due to CI problem.07:57
*** haomaiwang has quit IRC08:01
*** haomaiwang has joined #openstack-swift08:01
*** yuywz has joined #openstack-swift08:03
*** jmccarthy has quit IRC08:04
*** yuywz has left #openstack-swift08:04
*** jmccarthy has joined #openstack-swift08:04
*** mingdang1 has joined #openstack-swift08:05
*** mingdang1 has quit IRC08:10
*** rledisez has joined #openstack-swift08:10
*** mmcardle has joined #openstack-swift08:15
*** permalac has joined #openstack-swift08:20
cschwedeAJaeger: just wanted to +A, but I noticed that the translation is misleading. Could this be fixed with a simple new patchset, or does that impact the automated import?08:24
*** ig0r_ has quit IRC08:39
AJaegerkota_: yes, another +A08:42
AJaegercschwede: if there's a problem, could yuo file a bug against the translators?08:42
AJaegerOr if you tell me what'S wrong, I might fix it quickly - for the next import08:42
cschwedeAJaeger: i submitted a comment on patch 29271908:43
patchbotcschwede: https://review.openstack.org/#/c/292719/ - swift - Imported Translations from Zanata08:43
AJaegercschwede: will forward to German translators mailing list - are you a German? Then I could copy you...08:43
cschwedeAJaeger: yep, from Hamburg, that’s why i stumbled upon this :)08:44
AJaegercschwede: Gruß aus Nürnberg ;)08:44
cschwedeAJaeger: Moin Moin aus dem Norden ;)08:45
AJaegercschwede: in general we don't proof read during import08:45
AJaegermail sent, let's discuss there...08:46
*** geaaru has joined #openstack-swift08:51
kota_I'm back to be here, and cschwede looks to suggest another point for the patch. Alright, waiting the result before adding +A.08:55
AJaegercschwede, kota_ : Problem is no new patch will be proposed since we do not propose a new patch if one is in the gate.08:56
AJaegerLet me rebase that change manually so that tomorrow a new patch gets proposed08:57
openstackgerritAndreas Jaeger proposed openstack/swift: Imported Translations from Zanata  https://review.openstack.org/29271908:57
kota_yup08:57
openstackgerritKota Tsuyuzaki proposed openstack/swift: Fix ssync related object-server docs  https://review.openstack.org/29225708:58
AJaegerbye for now! Thanks!08:58
*** AJaeger has left #openstack-swift08:58
*** kei_yama has quit IRC08:59
*** haomaiwang has quit IRC09:01
*** ppai has quit IRC09:01
*** haomaiwang has joined #openstack-swift09:01
*** stantonnet has quit IRC09:02
*** stantonnet has joined #openstack-swift09:04
*** ppai has joined #openstack-swift09:05
*** takashi has quit IRC09:07
*** esker has joined #openstack-swift09:12
*** esker has quit IRC09:16
openstackgerritoshritf proposed openstack/swift: Container-Sync to perform HEAD before PUT object on remote  https://review.openstack.org/27096109:18
*** jistr has joined #openstack-swift09:24
*** jmccarthy1 has joined #openstack-swift09:25
*** jordanP has joined #openstack-swift09:25
*** esker has joined #openstack-swift09:32
*** MVenesio has joined #openstack-swift09:35
*** esker has quit IRC09:39
*** MVenesio has quit IRC09:39
*** panda has quit IRC09:40
*** panda has joined #openstack-swift09:41
*** ho_ has quit IRC09:42
*** acoles_ is now known as acoles09:48
acolesgood morning09:50
acolesmattoliverau: good luck tomorrow!!09:50
kota_acoles: good morning09:51
acoleskota_: hi! what's happening on patch 29225709:52
patchbotacoles: https://review.openstack.org/#/c/292257/ - swift - Fix ssync related object-server docs09:52
acolesdid you just rebase?09:52
kota_acoles: looks like affected by gerrit stalling yesterday.09:52
kota_just rebased.09:53
kota_to be able to kick the merge process.09:53
*** openstackgerrit has quit IRC09:53
*** openstackgerrit_ is now known as openstackgerrit09:53
*** openstackgerrit has quit IRC09:53
acoleskota_: oic. i just aded +A again09:53
kota_acoles: thanks, again!09:54
*** openstackgerrit_ has joined #openstack-swift09:54
acoleskota_: i think it would have been ok if you had done that, since no change after my +A09:54
*** mingdang1 has joined #openstack-swift09:54
*** openstackgerrit_ is now known as openstackgerrit09:54
kota_acoles: good news :)09:54
*** openstackgerrit_ has joined #openstack-swift09:55
*** openstackgerrit has quit IRC09:55
*** openstackgerrit has joined #openstack-swift09:56
*** haomaiwang has quit IRC10:01
*** haomaiwang has joined #openstack-swift10:01
*** cdelatte has quit IRC10:19
*** ankur has joined #openstack-swift10:25
*** daemontool has joined #openstack-swift10:28
ankur.10:28
*** mkrcmari__ has quit IRC10:42
*** haomaiwang has quit IRC11:01
*** haomaiwang has joined #openstack-swift11:01
*** esker has joined #openstack-swift11:04
*** Jeffrey4l has joined #openstack-swift11:06
*** esker has quit IRC11:08
*** mkrcmari__ has joined #openstack-swift11:11
*** openstackgerrit has quit IRC11:18
*** openstackgerrit has joined #openstack-swift11:18
*** haomaiwang has quit IRC11:24
acolesnotmyname: suggest patch 258158 could be targeted for inclusion in mitaka?11:33
patchbotacoles: https://review.openstack.org/#/c/258158/ - swift - Keystone middleware deprecated option is_admin rem...11:33
*** dmorita has joined #openstack-swift11:35
*** MVenesio has joined #openstack-swift11:36
*** geaaru has quit IRC11:36
openstackgerritMerged openstack/swift: Fix ssync related object-server docs  https://review.openstack.org/29225711:37
openstackgerritMerged openstack/swift: clear pycache and remove all pyc/pyo before starting unit test  https://review.openstack.org/25427611:38
*** dmorita has quit IRC11:40
*** MVenesio has quit IRC11:41
openstackgerritAlistair Coles proposed openstack/swift: Make keystone middleware options consistent in docs  https://review.openstack.org/29338211:43
*** cdelatte has joined #openstack-swift11:44
acolesnotmyname: and that one ^^ which should be trivial11:45
*** cdelatte has quit IRC11:46
*** geaaru has joined #openstack-swift11:48
*** cdelatte has joined #openstack-swift11:48
*** ppai_ has joined #openstack-swift12:03
*** ppai has quit IRC12:05
*** haomaiwang has joined #openstack-swift12:16
*** haomaiwang has quit IRC12:20
*** NM has joined #openstack-swift12:22
*** Razva has joined #openstack-swift12:25
Razvaguys does any of you have experience with SwiftStack?12:26
*** andreaponza has quit IRC12:27
*** pcaruana has quit IRC12:28
tdasilvagood morning12:29
*** ppai_ has quit IRC12:30
Razvamornin'12:31
Razvatdasilva do you have any experience with SwiftStack?12:31
tdasilvaRazva: no, but there are a number of swiftstack employees on the channel, but they are most likely not awake yet :) they are in the west coast12:32
tdasilvaRazva: they also have a channel #swiftstack if you have specific questions12:32
RazvaGod this is so frustrating! I've gone by the book for 3 times12:32
Razvaand each time I stop at the same place, I don't have any kind of "enable" button on the node12:33
tdasilvayeah, sounds like you have specific questions on their software. I'd checkout #swiftstack channel, as they might have someone available there to answer questions...12:34
Razvado you know any other solution that can deploy Swift with ease, except swiftstack?12:35
Razvahttp://screencast.com/t/PZu6FYePnlE < I have no idea if/what is wrong and where the heck is the "Enable" button :|12:35
tdasilvaRazva: definetely not with the ease of use that swiftstack has built12:36
Razvaok, I can configure/install swift by myself, but then I need something to monitor the thing12:36
Razvaor at least get some information easy, without going to #console each time I want to check how much space is left on the disk12:36
tdasilvaRazva: but you can also checkout openstack-ansible project and the openstack tripleO project. They are both doing a lot of work to support swift deployment.12:38
*** MVenesio has joined #openstack-swift12:38
tdasilvaRazva: not sure how much this would help, but I'd also checkout the Swift Ops Runbook: http://docs.openstack.org/developer/swift/ops_runbook/index.html12:39
*** ppai_ has joined #openstack-swift12:44
*** StraubTW has joined #openstack-swift12:44
*** takashi has joined #openstack-swift12:45
*** CaioBrentano has joined #openstack-swift12:48
*** CaioBrentano has quit IRC12:48
*** vinsh_ has quit IRC12:50
*** vinsh has joined #openstack-swift12:50
*** jlvillal has quit IRC12:52
*** jlvillal has joined #openstack-swift12:52
*** StraubTW has quit IRC12:54
*** ppai_ has quit IRC13:00
cschwedeRazva: hmm, i don’t have any experience using swiftstack, but there is an enable button in your screenshot - left side on the bottom?13:01
*** Jeffrey4l has quit IRC13:02
*** rcernin has joined #openstack-swift13:03
*** BobBall has joined #openstack-swift13:03
BobBallTrying to upload to swift with a content-type / content-encoding set using openstacksdk.  What's the right way to specify these?  I'm using https://github.com/citrix-openstack/swift-uploader/blob/master/swiftuploader/upload.py#L134 which I thought should work from the unit tests, but it appears these are not set when retrieving the object13:05
*** mrmoje has joined #openstack-swift13:06
*** Jeffrey4l has joined #openstack-swift13:14
*** mingdang1 has quit IRC13:16
*** silor has joined #openstack-swift13:24
*** ig0r_ has joined #openstack-swift13:24
*** Jeffrey4l has quit IRC13:27
*** haomaiwang has joined #openstack-swift13:29
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873613:33
*** d0ugal has quit IRC13:35
*** d0ugal has joined #openstack-swift13:35
*** panda has quit IRC13:40
*** panda has joined #openstack-swift13:41
*** _JZ_ has joined #openstack-swift13:43
sgundurgood morning13:43
*** NM has quit IRC13:46
*** NM has joined #openstack-swift13:48
*** takashi has quit IRC13:56
*** haomaiwang has quit IRC14:01
*** haomaiwang has joined #openstack-swift14:01
*** dmellado is now known as dmellado|lunch14:02
*** acorwin_ has quit IRC14:06
*** StraubTW has joined #openstack-swift14:06
*** aerwin3 has quit IRC14:06
*** ntata has quit IRC14:06
*** zacksh has quit IRC14:07
*** StraubTW has quit IRC14:07
*** StraubTW has joined #openstack-swift14:07
*** aerwin3 has joined #openstack-swift14:07
*** zacksh has joined #openstack-swift14:08
*** acorwin has joined #openstack-swift14:09
*** _JZ_ has quit IRC14:12
*** ho_away has joined #openstack-swift14:20
*** mrmoje has quit IRC14:26
*** aswadr_ has quit IRC14:30
*** baojg has joined #openstack-swift14:31
*** ho_away has quit IRC14:33
*** _JZ_ has joined #openstack-swift14:56
pdardeaugood morning14:57
*** haomaiwang has quit IRC15:01
*** haomaiwang has joined #openstack-swift15:01
rledisezhello all15:03
rledisezis anyone using SSYNC on a "big" cluster? I have to use it because EC. But i get really poor performance when rebalancing. My customers upload faster than SSYNC move data :/15:04
rledisezWhen comparing to rsync which were easily reaching more than 3Gb/s per server, ssync hardly it 70Mb/s15:05
*** garthb has joined #openstack-swift15:05
rledisezI tried to change reconstructor concurrency, I played with replication_one_per_device, replication_concurrency. I wrote a patch to handle concurrency per device15:06
rledisezno significant improvement15:06
rledisezIs there any bench somewhere? Do somebody have any tips?15:06
gmmahagood morning15:08
*** trifon has quit IRC15:08
*** david-lyle has quit IRC15:15
*** david-lyle has joined #openstack-swift15:15
*** dmellado|lunch is now known as dmellado15:15
*** klrmn has joined #openstack-swift15:18
BobBallFYI in case people are reading the logs, my previous question on content-type has been identified as an openstack-sdk bug https://bugs.launchpad.net/python-openstacksdk/+bug/155811515:22
openstackLaunchpad bug 1558115 in OpenStack SDK "Swift upload using content-type broken" [Undecided,New] - Assigned to Bob Ball (bob-ball)15:22
*** david-lyle_ has joined #openstack-swift15:34
*** david-lyle has quit IRC15:34
*** mrmoje has joined #openstack-swift15:35
pdardeaugood morning15:39
*** jroll has quit IRC15:41
*** jroll has joined #openstack-swift15:41
*** jroll has quit IRC15:41
*** jroll has joined #openstack-swift15:41
*** gyee has joined #openstack-swift15:55
*** haomaiwang has quit IRC16:01
*** haomaiwang has joined #openstack-swift16:01
*** baojg has quit IRC16:03
notmynamegood monring16:17
*** david-lyle_ is now known as david-lyle16:20
*** klrmn has quit IRC16:23
*** trifon has joined #openstack-swift16:26
openstackgerritoshritf proposed openstack/swift: Per container stat. report  https://review.openstack.org/28181416:29
*** acoles is now known as acoles_16:31
notmynameremember that the meeting today is still at 2100UTC (http://time.unitarium.com/utc/21). UTC doesn't have DST, so the meeting time may have changed for your local time zone16:31
*** mrmoje has quit IRC16:34
*** lyrrad has joined #openstack-swift16:34
*** ntata has joined #openstack-swift16:37
*** links has quit IRC16:38
*** dmorita has joined #openstack-swift16:42
*** acoles_ is now known as acoles16:47
*** dmorita has quit IRC16:58
*** dmorita has joined #openstack-swift17:00
*** haomaiwang has quit IRC17:01
*** haomaiwang has joined #openstack-swift17:01
*** dmorita has quit IRC17:01
*** dmorita has joined #openstack-swift17:02
*** dmorita has quit IRC17:03
*** dmorita has joined #openstack-swift17:03
*** jistr has quit IRC17:12
*** nadeem has joined #openstack-swift17:22
*** nadeem has quit IRC17:22
*** nadeem has joined #openstack-swift17:23
*** klrmn has joined #openstack-swift17:32
notmynamehttps://review.openstack.org/#/c/279440/ is a patch from cschwede that closes a critical bug. it already has a +2 from kota_, but needs one more. can someone please take a look asap, please?17:37
patchbotnotmyname: patch 279440 - swift - Skip already checked partitions when auditing obje...17:37
*** timur has quit IRC17:39
*** timur has joined #openstack-swift17:40
*** panda has quit IRC17:40
*** panda has joined #openstack-swift17:41
notmynametdasilva: where are we with patch 156923? do you know of other work needed? is it just waiting for reviews? any chance we can land it in the next week?17:43
patchbotnotmyname: https://review.openstack.org/#/c/156923/ - swift - Refactor server side copy as middleware17:43
notmynameacoles: ^17:43
tdasilvanotmyname: just waiting for reviews :/17:43
notmynameok17:43
acolesnotmyname: and importantly waiting for second +2 on the two dependent patches17:44
tdasilvanotmyname: acoles and I feel that we have put our hands on the code too much that limits us from giving it a +2, so it would be nice to get another set of eyes on it17:44
notmynameright17:44
acolestdasilva: shhhhhh ;)17:44
tdasilvalol17:44
notmynameacoles: grrr...every time I hear about patch chains, I hate the new gerrit interface more. IMO it's pretty much impossible to see those or what order they are in with the new UI17:45
tdasilvabut yeah, it definetely would be very nice for the cryto work17:45
tdasilvanotmyname: agreed!!!17:45
acolesnotmyname: agree17:45
*** chsc has joined #openstack-swift17:46
notmynamepatch 263902 seems it might be a dependent one. hasn't landed but has +A on it17:46
patchbotnotmyname: https://review.openstack.org/#/c/263902/ - swift - Re-format the SLO manifest file on new multipart-m...17:46
tdasilvanotmyname: right, that depends on the versioned writes one17:47
notmynameah ok17:47
tdasilvawhich already has a +217:47
notmynamesee, I can't really tell that :-(17:47
tdasilvai know :/17:47
*** rledisez has quit IRC17:47
notmynameok, so patch 260179 is what needs another +2 first17:48
patchbotnotmyname: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY17:48
notmynametdasilva: so since you're just waiting on you're own +2s for copy middleware, how about looking at patch 279440? ;-)17:48
patchbotnotmyname: https://review.openstack.org/#/c/279440/ - swift - Skip already checked partitions when auditing obje...17:48
tdasilvanotmyname: was just reading up on the bug there, trying to come up to speed17:49
notmynamethanks17:49
openstackgerritJohn Dickinson proposed openstack/swift: Change schedule priority of daemon/server in config  https://review.openstack.org/23879917:52
notmyname^ fixed merge conflict17:52
*** rcernin has quit IRC17:53
notmynamehttps://wiki.openstack.org/wiki/Swift/PriorityReviews updated to help us over this week and next18:00
*** haomaiwang has quit IRC18:01
*** haomaiwang has joined #openstack-swift18:01
*** kevinc has joined #openstack-swift18:16
openstackgerritAlistair Coles proposed openstack/swift: Put correct Etag and Accept-Ranges in EC 304 response  https://review.openstack.org/29363318:18
*** tesseract- has quit IRC18:19
kevincIs there a guide to implementing swift with swauth and keystone auth on the same proxy? Which should be first in the pipeline? I am using swift (kilo) with swauth (1.1.0) and keystone but I am getting lots of token errors.18:19
jrichliacoles: good catch with Bug 1558193 - sounds like this will be a good thing to test with crypto18:19
openstackbug 1558193 in OpenStack Object Storage (swift) "Wrong etag returned with 304 GET response using EC policy" [Undecided,In progress] https://launchpad.net/bugs/1558193 - Assigned to Alistair Coles (alistair-coles)18:19
acolesnotmyname: great wiki page (priority reviews)18:20
notmynamethanks18:20
notmynamekevinc: IIRC, keystone needs to be first before other auth middleware. I can check on that...18:21
kevincthanks!18:21
kevincmy pipeline is pipeline = catch_errors gatekeeper healthcheck proxy-logging cache bulk slo dlo swift3 s3token authtoken keystone swauth staticweb proxy-logging proxy-server18:22
notmynamekevinc: I think some of those are wrong. see the sample at https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L7918:24
*** NM has quit IRC18:24
timburkekevinc: do you know which auth system issued the failing tokens? (i.e., does keystone consistently work while swauth consistently doesn't, or vice versa?) also, is delay_auth_decision set to true for keystoneauth (see https://github.com/openstack/swift/blob/2.3.0/etc/proxy-server.conf-sample#L292-L295)?18:28
timburkealso, notmyname is right; slo & dlo will need to move further right (between staticweb and proxy-logging)18:29
*** NM has joined #openstack-swift18:29
notmynamekevinc: keystone goes first18:29
*** dmorita has quit IRC18:31
kevincmy guess is that since authtoken and keystone are before swauth, keystone authentication is failing (causing the errors in the logs) but deferring to swauth. I don't believe it is preventing any valid requests from failing, just creating a lot of spam in the logs and maybe creating a lot of unnecessary connections to keystone (i don't manage that server so i don't know yet)18:32
kevincoops I don't believe it is preventing any valid requests from succeeding18:33
notmynamethat sounds like it's likely (lots of spurious connections to keystone)18:33
kevincproxy-server: Authorization failed for token and Invalid user token - deferring reject downstream18:33
timburkekevinc: ah, yeah. keystoneauth is rather noisy about that. i'd hope that we cache the fact that keystone doesn't know what to do with the token, but i'm not actually sure18:35
notmynamethere's a couple of things that annoy me about the keystone middleware. there related, and that's one of them18:35
* notmyname goes to file bugs18:35
notmynameacoles: that looks like an important patch/bug18:36
*** jmccarthy1 has quit IRC18:37
kevincI would be happy to enter a ticket for it if you think it is necessary18:37
kevincI guess I will go back to running swauth proxies and keystone proxies on different servers/ports18:38
*** jordanP has quit IRC18:40
acolesnotmyname: the etags in 304's? yeah, and the 'companion' bug 1558197 causes a tempest failure when running with EC policy as default, so hurts CI18:40
openstackbug 1558197 in OpenStack Object Storage (swift) "Accept-Ranges is not returned with 304 GET response using EC policy" [Undecided,In progress] https://launchpad.net/bugs/1558197 - Assigned to Alistair Coles (alistair-coles)18:40
claygacoles: I think lp bug #1558193 is a dupe of lp bug #149623418:43
openstackLaunchpad bug 1558193 in OpenStack Object Storage (swift) "Wrong etag returned with 304 GET response using EC policy" [Undecided,In progress] https://launchpad.net/bugs/1558193 - Assigned to Alistair Coles (alistair-coles)18:43
openstackLaunchpad bug 1496234 in OpenStack Object Storage (swift) "EC: etag of range request is not consistent" [Low,In progress] https://launchpad.net/bugs/1496234 - Assigned to Mahati Chamarthy (mahati-chamarthy)18:43
*** clyps__ is now known as dcourtois18:43
notmynamekevinc: https://bugs.launchpad.net/swift/+bug/155821618:43
openstackLaunchpad bug 1558216 in OpenStack Object Storage (swift) "keystone middleware doesn't respect reseller prefix" [Medium,New]18:43
*** dmorita has joined #openstack-swift18:46
*** dmorita has quit IRC18:46
*** dmorita_ has joined #openstack-swift18:46
*** BobBall is now known as BobBall_AWOL18:47
dcourtoisHi all, did anyone face this ssync issue ?18:48
dcourtois16:04 < rledisez> is anyone using SSYNC on a "big" cluster? I have to use it because EC. But i get really poor  performance when rebalancing. My customers upload faster than SSYNC move data :/18:48
dcourtois16:05 < rledisez> When comparing to rsync which were easily reaching more than 3Gb/s per server, ssync hardly it  70Mb/s18:48
dcourtois16:06 < rledisez> I tried to change reconstructor concurrency, I played with replication_one_per_device,  replication_concurrency. I wrote a patch to handle concurrency per device18:49
dcourtois16:06 < rledisez> no significant improvement18:49
dcourtois16:06 < rledisez> Is there any bench somewhere? Do somebody have any tips?18:49
openstackgerritMichael Barton proposed openstack/swift: go: proxy rewrite  https://review.openstack.org/29365018:54
*** amit2131 has quit IRC18:55
*** amit213 has joined #openstack-swift18:55
*** haomaiwang has quit IRC19:01
*** haomaiwa_ has joined #openstack-swift19:01
notmynamereminder that the team meeting is in 1 hour and 55 minutes from now (not 55 minutes from now). it is at 2100UTC19:04
*** mkrcmari__ has quit IRC19:05
notmynamedcourtois: I don't have a particular answer to you, other than that I've seen reports of ssync not doing well at large scale. there have been some improvements that have landed in the last several months, so I'm not sure if you're using those (what version of swift you're running).19:06
claygdcourtois: rledisez: are you mainly comparing speed or EC rebalance to replicated/rsync rebalance?19:08
dcourtoiswe are running v2.5.019:15
*** glange_ has joined #openstack-swift19:22
*** ChanServ sets mode: +v glange_19:22
*** glange_ has quit IRC19:22
*** rledisez has joined #openstack-swift19:25
rledisezclayg: i'm comparing the speed of rebalance (aka how many GB are put on a freshly inserted device in few hours)19:26
rlediseznotmyname: we are currently running 2.519:26
*** daemontool has quit IRC19:28
*** tsg has joined #openstack-swift19:31
*** awelleck has joined #openstack-swift19:32
*** tsg_ has joined #openstack-swift19:33
glangedcourtois: rledisez: you might have the biggest cluster trying to run EC -- how big is your cluster?19:33
rledisezthousands of device, around 10PB of capacity19:34
acolesclayg: those bug reports are indeed similar, but different scenarios and AFAICT they require separate fixes - the 416 raised by proxy isn't handled in _fix_response place where I fix the etag in patch 29363319:34
patchbotacoles: https://review.openstack.org/#/c/293633/ - swift - Put correct Etag and Accept-Ranges in EC 304 response19:34
Razvahey guys, does anybody have experience with swiftstack here? I've deployed a test swiftstack and I *really* cannot enable the thing. everything is partitioned, mounted and roles assigned. but no enable button19:35
claygacoles: k, thanks19:35
acolesclayg: mahatic's patch fixes the 416 Etag before it gets raised, I haven't quite understood the path it then takes19:35
*** ChubYann has joined #openstack-swift19:36
claygRazva: i'll point someone from support at you in #swiftstack19:36
acolesback later fo rmeeting19:36
*** tsg has quit IRC19:36
acolesfor*19:36
Razvaclayg ok, and if you can point somebody from sales...19:36
RazvaI'm struggling to get a pricing list for like...3 days.19:36
Razvawooohooo somebody answered the live-chat! great success!19:37
claygRazva: heh19:37
claygrledisez: that is the largest EC deployment I've heard anyone talking about - congrats19:38
claygrledisez: do you have some experience running replicated swift at that scale then?19:38
rledisezclayg: yeah, we are running bigger cluster in replicated. no problem so far, it was really smooth :)19:39
*** geaaru has quit IRC19:39
rledisezclayg: glad to be the biggest in EC. just gotta make it works now :D19:40
claygrledisez: glad to hear it - we've been working on the replicated stuff for awhile - ec has some different challanges19:40
rledisezclayg: for now, i'm seriously considering hacking in reconstructor to use rsync for REVERT job19:40
rledisezclayg: old school way19:40
claygrledisez: I've seriously considered it as well :\19:41
rledisezclayg: in a way, it's comforting that you did too. at least i'm not totally wrong19:42
claygrledisez: I think some plumbing to make handoffs_first mode work could help the revert jobs win the race against the reconstruct/update jobs coming from the other priarmy partners noticing the new primary doesn't have the fag19:42
clayggah... f*r*ag19:42
rledisezclayg: it could save use for now, but in the long run it doesn't seem safe. I know we will add capacity continuously19:44
*** kevinc_ has joined #openstack-swift19:44
rledisezclayg: i'll give it a try tomorrow, see if it gives good results19:44
rledisezclayg: but as far as i checked, we do not have much reconstruct (i dumped the jobs list in reconstructor)19:45
claygrledisez: looking forward to hearing about your results - kudos on the initiative19:45
*** swiftfan has joined #openstack-swift19:45
claygrledisez: excellent instrumentation!  I wanted to see some of those figures coming out to statsd so we could get some more learnings/expierence there19:46
*** dmorita_ has quit IRC19:46
swiftfanI'm having trouble running EC in production19:46
*** dmorita has joined #openstack-swift19:46
*** kevinc has quit IRC19:48
claygswiftfan: acoles have some patches up to address a few things - if you have any specific problems bugs would be most helpful - the more general stuff I'm sure will discussed at the design summit in Austin as more and more people are starting to deploy EC and learn about the parts that we'll want to focus for improvement19:48
*** dmorita has quit IRC19:51
*** acoles is now known as acoles_19:52
*** ig0r_ has quit IRC19:55
*** swiftfan has quit IRC19:58
*** haomaiwa_ has quit IRC20:01
*** swiftfan has joined #openstack-swift20:01
*** haomaiwang has joined #openstack-swift20:01
*** trifon has quit IRC20:01
*** Zyric has quit IRC20:09
*** Zyric has joined #openstack-swift20:14
*** dmorita has joined #openstack-swift20:18
*** silor has quit IRC20:23
openstackgerritAlistair Coles proposed openstack/swift: Put correct Etag and Accept-Ranges in EC 304 and 416 responses  https://review.openstack.org/29363320:28
*** acoles_ is now known as acoles20:30
acolesmahatic: ^^ I co-authored you on that, hope thats ok20:30
acolesnotmyname: I may be a few mins late for meeting, just leaving for home20:31
*** acoles is now known as acoles_20:31
*** diogogmt has joined #openstack-swift20:33
notmynameacoles_: ack20:39
*** asettle has joined #openstack-swift20:42
*** rickyrem has joined #openstack-swift20:43
notmynameswift meeting in 5 minutes20:54
notmynamerledisez: that's really cool (the EC cluster)20:54
*** ho_away has joined #openstack-swift20:56
kota_good morning20:57
notmynamehello kota_20:57
kota_notmyname: hello20:57
mattoliveraumorning20:59
*** mattoliverau has quit IRC20:59
*** matt6434 has joined #openstack-swift20:59
notmynamemeeting time21:00
notmynamein #openstack-meeting21:00
*** matt6434 is now known as mattoliverau21:00
*** haomaiwang has quit IRC21:01
*** ChanServ sets mode: +v mattoliverau21:01
*** haomaiwa_ has joined #openstack-swift21:01
notmynameclayg: torgomatic: available for the meeting?21:01
torgomaticnotmyname: I just joined21:02
*** acoles_ is now known as acoles21:02
openstackgerritMichael Barton proposed openstack/swift: go: proxy rewrite  https://review.openstack.org/29365021:05
*** DericHorn-HP has joined #openstack-swift21:12
*** DericHorn-HP has quit IRC21:12
openstackgerritMichael Barton proposed openstack/swift: go: proxy rewrite  https://review.openstack.org/29365021:23
*** CaioBrentano has joined #openstack-swift21:25
openstackgerritMichael Barton proposed openstack/swift: go: proxy rewrite  https://review.openstack.org/29365021:29
*** darrenc is now known as darren_afk21:31
*** trifon has joined #openstack-swift21:33
*** NM has quit IRC21:35
*** CaioBrentano has quit IRC21:40
*** panda has quit IRC21:40
*** panda has joined #openstack-swift21:41
*** darren_afk is now known as darrenc21:46
*** permalac has quit IRC21:50
*** ho_away has quit IRC21:50
*** nadeem has quit IRC21:51
timburkeacoles: now that fast POST merged, should we close https://bugs.launchpad.net/swift/+bug/1487791?21:52
openstackLaunchpad bug 1487791 in OpenStack Object Storage (swift) "POST to DLO squashes data without fast-POST" [Undecided,Confirmed]21:52
*** MVenesio has quit IRC21:54
*** awelleck has left #openstack-swift21:55
*** rickyrem has quit IRC21:56
acolestimburke: i don't believe so, that bug remains with post-as-copy mode21:59
acolestimburke: i lost track of where we were with it though, jrichli had a patch21:59
timburkeah, right. forgot that that hasn't actually gone away :(21:59
acolestimburke: and you got me wondering if it fixed as a side effect of copy middleware patch, will have to check22:00
*** haomaiwa_ has quit IRC22:01
*** haomaiwang has joined #openstack-swift22:01
*** acoles is now known as acoles_22:02
notmynamehttps://wiki.openstack.org/wiki/Swift/PriorityReviews updated22:04
*** StraubTW has quit IRC22:07
*** kevinc_ has quit IRC22:09
*** remix_tj has quit IRC22:10
*** remix_tj has joined #openstack-swift22:12
*** arch-nemesis has quit IRC22:12
*** chlong has joined #openstack-swift22:32
*** trifon has quit IRC22:33
*** rledisez has quit IRC22:40
*** Jeffrey4l has joined #openstack-swift22:49
*** asettle has quit IRC22:54
*** mrmoje has joined #openstack-swift22:56
*** haomaiwang has quit IRC23:01
*** km has joined #openstack-swift23:01
*** haomaiwa_ has joined #openstack-swift23:01
*** _JZ_ has quit IRC23:02
*** chsc has quit IRC23:32
*** tsg_ has quit IRC23:36
*** asettle has joined #openstack-swift23:37
*** diogogmt has quit IRC23:38
*** mingdang1 has joined #openstack-swift23:39
*** chlong has quit IRC23:41
*** asettle has quit IRC23:42
*** kei_yama has joined #openstack-swift23:42
*** mmcardle has quit IRC23:46
*** mingdang1 has quit IRC23:50
*** gyee has quit IRC23:53
*** MVenesio has joined #openstack-swift23:55
*** MVenesio has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!