Friday, 2017-10-06

*** mingyu has joined #openstack-swift00:14
openstackgerritTim Burke proposed openstack/swift feature/deep: Address review comments  https://review.openstack.org/50988400:29
torgomaticbriancline: with Python 2.7, we had to go to some custom ring format instead of a pickle because loading _replica2part2dev got much slower than 2.6; I think it was how 2.7 serialized it, not the loading code though00:34
claygi guess builders are un-compressed - and doings lots of `swift-ring-builder builder add ...` is already annoyingly slow - but that's mainly python startup... well I mean deserialization/reserialization is *also* slow (in addition to python startup/import)00:39
timburkeclayg: briancline: could knock out https://bugs.launchpad.net/swift/+bug/1644387 at the same time :-)00:52
openstackLaunchpad bug 1644387 in OpenStack Object Storage (swift) "Ring builder files need a consistent format" [Undecided,New]00:52
claygtimburke: so py2 => py3 is ok but py3 back to py2 is noewheno?00:54
timburkeyup. depending on which version of py200:54
timburke2.7.13, it's OK! 2.7.10? nope.00:54
claygthat's pretty neat00:55
timburkeseems like if we do something like our current ring format, then just serialize the other large bits *after* r2p2d, it might actually work out pretty well?00:57
timburkethis even means there'd be some benefit to be had *before* everything is py3, so it'd stand some chance of actually getting done00:59
*** openstackstatus has quit IRC01:14
claygyeah... we should write down (like with ascii art) what the format should look like and put it on swift ideas or something...01:16
*** openstackstatus has joined #openstack-swift01:16
*** ChanServ sets mode: +v openstackstatus01:16
clayg"and the reasons it should look like this are cause: ...."01:16
*** tovin07_ has joined #openstack-swift01:16
*** zigo has quit IRC01:27
*** gyee has quit IRC01:28
*** zigo has joined #openstack-swift01:31
openstackgerritTim Burke proposed openstack/swift master: Let clients request heartbeats during SLO PUTs  https://review.openstack.org/50930601:38
*** d0ugal has quit IRC01:45
*** d0ugal has joined #openstack-swift01:45
*** portdirect has quit IRC02:04
*** portdirect has joined #openstack-swift02:04
*** jrichli has quit IRC02:05
*** jrichli has joined #openstack-swift02:05
*** mingyu has quit IRC02:06
*** andymccr has quit IRC02:06
*** csmart has quit IRC02:06
*** SkyRocknRoll has quit IRC02:06
*** dja has quit IRC02:06
*** mingyu has joined #openstack-swift02:06
*** onovy has quit IRC02:08
*** mtreinish has quit IRC02:08
*** jistr has quit IRC02:08
*** jistr has joined #openstack-swift02:10
*** onovy has joined #openstack-swift02:10
*** seongsoocho_ has joined #openstack-swift02:10
*** mtreinish has joined #openstack-swift02:10
openstackgerritMerged openstack/swift feature/deep: split merge_items into merge_objects and merge_shard_ranges  https://review.openstack.org/50985702:11
*** kong_ has joined #openstack-swift02:11
openstackgerritMerged openstack/swift feature/deep: Add Container Sharder to source docs  https://review.openstack.org/50983002:11
*** seongsoocho has quit IRC02:12
*** kong has quit IRC02:12
*** blairo has quit IRC02:12
*** seongsoocho_ is now known as seongsoocho02:12
*** kong_ is now known as kong02:12
*** dja has joined #openstack-swift02:13
*** andymccr has joined #openstack-swift02:13
*** SkyRocknRoll has joined #openstack-swift02:14
*** mahatic has quit IRC02:16
*** csmart has joined #openstack-swift02:16
*** mahatic has joined #openstack-swift02:17
*** mingyu has quit IRC02:18
kota_good morning02:18
*** blair has joined #openstack-swift02:19
*** mingyu has joined #openstack-swift02:22
*** mgagne has quit IRC02:23
*** zacksh has quit IRC02:23
*** mgagne has joined #openstack-swift02:24
*** mgagne is now known as Guest6609802:24
*** zacksh has joined #openstack-swift02:24
*** csmart has quit IRC02:35
*** csmart has joined #openstack-swift02:38
kota_timburke: btw, could you have time to update swift3 patch? If not, I could push my suggestion and can start to release work. How do you think?02:38
*** mahatic has quit IRC02:39
*** mahatic has joined #openstack-swift02:41
*** janonymous has joined #openstack-swift03:21
*** SkyRocknRoll has quit IRC03:25
*** wes_dillingham has quit IRC03:29
*** wes_dillingham has joined #openstack-swift03:30
*** wes_dillingham has quit IRC03:30
*** links has joined #openstack-swift03:32
*** abhitechie has quit IRC03:36
openstackgerritMerged openstack/swift feature/deep: add post-sharding object PUT to probe test  https://review.openstack.org/50985803:39
*** kei_yama has quit IRC03:40
mahatichi clayg ! tombstone bugs eh03:51
*** gkadam has joined #openstack-swift04:01
*** kei_yama has joined #openstack-swift04:02
*** klrmn has quit IRC04:53
*** psachin has joined #openstack-swift05:05
*** HCLTech-SSW has joined #openstack-swift05:24
HCLTech-SSWrledisez: Hi05:24
*** ChubYann has quit IRC05:34
mattoliverauHCLTech-SSW: o/05:52
mattoliverauI haven't seen rledisez online yet, thanks for the undelete patch05:53
mattoliverauUnfortuantly I'm about to leave for the day (almost 5pm on Friday here in Oz and have family plans). Otherwise I'd love to talk about undelete.05:54
*** ChanServ sets mode: +v mattoliverau05:55
*** spectr has joined #openstack-swift05:56
*** m_kazuhiro has joined #openstack-swift06:04
*** silor has joined #openstack-swift06:10
*** skudlik has joined #openstack-swift06:12
*** rcernin has joined #openstack-swift06:17
*** pcaruana has joined #openstack-swift06:24
*** geaaru has joined #openstack-swift06:26
mattoliverauacoles: I'll be back for a team meeting in a few hours, I'll review the patch while meeting (not saying the meeting is borning or anything) :P06:34
*** HCLTech-SSW has quit IRC06:36
*** silor has quit IRC06:38
*** rcernin has quit IRC06:41
*** links has quit IRC06:42
*** rcernin has joined #openstack-swift06:43
*** links has joined #openstack-swift06:59
*** cbartz has joined #openstack-swift07:00
*** tesseract has joined #openstack-swift07:25
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Imported Translations from Zanata  https://review.openstack.org/51000007:41
*** links has quit IRC07:46
*** links has joined #openstack-swift07:59
*** tovin07_ has quit IRC08:18
*** tovin07 has joined #openstack-swift08:22
*** tovin07_ has joined #openstack-swift08:27
*** psachin has quit IRC08:34
acolesmattoliverau: no worries, it's not urgent (just removing code) and it's your Friday evening - it can wait til next week08:34
*** kei_yama has quit IRC08:34
acolesclayg: *one* day we'll fix *everything* - on that day the sun will stand still in the sky, somewhere over greenland maybe :D08:38
acolesbriancline: this may be of interest to you https://review.openstack.org/#/c/291990/ (IDK, I just speed-read my scrollback)08:40
patchbotpatch 291990 - swift - Add commands to ring-builder to change region / zone08:40
*** tovin07 has quit IRC08:40
openstackgerritHCLTech-SSW proposed openstack/swift master: Added the man page for container-sync-realms.conf  https://review.openstack.org/51001708:41
*** m_kazuhiro has quit IRC08:42
*** psachin has joined #openstack-swift09:18
*** spectr has quit IRC09:26
*** spectr has joined #openstack-swift09:34
*** spectr has quit IRC09:34
*** spectr has joined #openstack-swift09:36
*** spectr has quit IRC09:36
*** spectr has joined #openstack-swift09:37
*** spectr has quit IRC09:38
*** spectr has joined #openstack-swift09:40
*** spectr has quit IRC09:40
*** spectr has joined #openstack-swift09:41
*** spectr has quit IRC09:43
*** spectr has joined #openstack-swift09:43
*** spectr has quit IRC09:56
*** spectr has joined #openstack-swift09:57
*** spectr has quit IRC09:57
*** spectr has joined #openstack-swift09:58
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Add some assertions to test_set_sharding_states  https://review.openstack.org/51004110:00
*** tovin07_ has quit IRC10:06
*** mingyu has quit IRC10:21
*** Dinesh_Bhor has quit IRC10:35
*** mjseger has joined #openstack-swift10:40
mjsegermattoliverau: is this morning good for you?10:41
openstackgerritJan Zerebecki proposed openstack/swift master: Fix swift-ring-builder set_weight with >1 device  https://review.openstack.org/50960611:03
*** mingyu has joined #openstack-swift11:22
*** mingyu has quit IRC11:26
*** mingyu has joined #openstack-swift11:38
mattoliveraumjseger: sorry, Friday night and am a little busy. Monday morning good for you?11:40
*** dewanee has joined #openstack-swift11:45
dewaneehello there11:45
mjsegermattoliverau: ouch, forgot about day difference as well as time. monday could work. re your last email: iostat is old school, collectl does it all ;)11:45
dewaneequick question on public access enabled container11:46
dewaneeit seems that the link provided by horizon misses a trailing "/"11:47
dewaneeI gather that link gets generated by the webinterface itself, is that correct?11:48
dewanee(I don't seem to find a way to have the whole address assembled by the swift client itself)11:48
*** spectr has quit IRC12:02
*** spectr has joined #openstack-swift12:03
dewaneeinvestigating more it seeems that without a trailing slash I get a redirect to the original_url + "/"12:05
*** wes_dillingham has joined #openstack-swift12:07
dewaneebut as I have ssl termination done by haproxy the url get stripped of the https prefix12:08
*** d0ugal has quit IRC12:15
*** MVenesio has joined #openstack-swift12:21
kota_dewanee: do you mean that http://host/v1/a/c equals to http://host/v1/a/c/ ?12:28
*** spectr has quit IRC12:28
kota_dewanee: I think they work as same. And my quick check with my swift, they both returned same results.12:29
kota_i don't think it depends on if the container is public or not.12:30
dewaneekota_, apparently the problem is that we use https12:32
dewaneeso the redirect we get is wrong12:32
dewaneeif I try https://host/v1/a/c I get host/v1/a/c/ that does not work12:33
dewaneeto be more precise our situation is :12:34
kota_dewanee: it looks like you did the translation/redirect on haproxy, not swift itself?12:34
dewaneethat the case, yes12:34
dewanee*that's12:34
dewaneebut why should I want to do it on swift itself?12:35
kota_?12:36
*** spectr has joined #openstack-swift12:39
dewaneeI mean what are the advantages? Is ssl termination on the load balancer an ill conceived idea when it comes to swift?12:39
openstackgerritAlistair Coles proposed openstack/swift feature/deep: Add some assertions to test_set_sharding_states  https://review.openstack.org/51004112:41
openstackgerritAlistair Coles proposed openstack/swift feature/deep: copy put-timestamp from original to sharded db  https://review.openstack.org/51009212:41
kota_i don't think it's a bad case. I think users usually use ssl termination in front of swift proxies for speeding up the transfer.12:41
kota_can your proxy be configured the protocol prefix to add "http" for the request URI?12:43
kota_haproxy or load balancer i meant.12:43
kota_outside of your swift.12:43
dewaneethat's a good question: I would apache do that for me in other situations12:44
dewaneenot sure about haproxy capabilities12:44
dewaneethere's no way to instruct swift-proxy to do that as far as I understand,correct?12:45
kota_hmm... sorry, I'm not familiar with haproxy too but haproxy - (to) -> swift should have http protocol, i suppose.12:46
dewaneethat's the main issue. the problem is then swift -> haproxy -> client issuing a redirect12:47
thurloatLPT: don't fill up disks12:47
kota_ah... perhaps, the host could be changed. I'm not familiar with such a case but it seems the use case for cname lookup?12:49
dewaneemmh not sure I am following: let's recap for the sake of not talking about two different things :)12:50
dewanee https://host/v1/a/c -> haproxy -> swift -> http://host/v1/a/c12:51
dewaneeswift -> http://host/v1/a/c/  -> haproxy -> http://host/v1/a/c/12:52
dewaneeat this point it does not work because the client then tries to access a http on a port that uses SSL12:53
kota_so if it is able to do that haproxy can return https prefix always, it seems to be resolved12:54
dewaneewell if that's the case, yes. I'll dig into it12:54
kota_sorry, it's time over. I'm on Asian Timezone and I have to get out my office soon :/12:54
dewaneenp. thanks for your time.12:55
*** psachin has quit IRC13:05
*** mat128 has joined #openstack-swift13:17
*** gkadam has quit IRC13:18
openstackgerritOpenStack Proposal Bot proposed openstack/python-swiftclient master: Updated from global requirements  https://review.openstack.org/8925013:19
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873613:21
*** mingyu has quit IRC13:21
*** mingyu has joined #openstack-swift13:25
openstackgerritAlistair Coles proposed openstack/swift feature/deep: remove support for +/- relative object_count in shard updates  https://review.openstack.org/50986013:27
*** mingyu has quit IRC13:30
*** spectr has quit IRC13:35
*** spectr has joined #openstack-swift13:36
*** wes_dillingham has quit IRC13:45
*** wes_dillingham has joined #openstack-swift13:46
*** links has quit IRC13:48
*** d0ugal has joined #openstack-swift14:00
*** catintheroof has joined #openstack-swift14:05
*** catinthe_ has joined #openstack-swift14:06
*** catintheroof has quit IRC14:07
thurloatI'm not seeing the functionality in the ringbuilder, is it possible to reassign a disk from one physical server to another? i.e. I want to take 15/45 disks from one object server and move them to another without having to do a huge rebalance.14:09
*** spectr has quit IRC14:11
*** d0ugal has quit IRC14:15
openstackgerritAlistair Coles proposed openstack/swift feature/deep: revert _record_to_dict to same as master  https://review.openstack.org/51013214:19
*** gyee has joined #openstack-swift14:24
*** d0ugal has joined #openstack-swift14:32
*** cbartz has quit IRC14:58
*** spectr has joined #openstack-swift15:04
*** spectr has quit IRC15:04
*** links has joined #openstack-swift15:07
*** MVenesio has quit IRC15:16
*** cbartz has joined #openstack-swift15:18
*** rcernin has quit IRC15:19
*** chsc has joined #openstack-swift15:19
*** cbartz has quit IRC15:21
*** skudlik has quit IRC16:09
*** pcaruana has quit IRC16:13
*** mingyu has joined #openstack-swift16:15
*** mingyu has quit IRC16:20
*** alenavolk has joined #openstack-swift16:32
*** alenavolk has quit IRC16:33
*** wes_dillingham has quit IRC16:47
claygnothing but win from the new contributor in patch 509606 AFICT16:57
patchbothttps://review.openstack.org/#/c/509606/ - swift - Fix swift-ring-builder set_weight with >1 device16:57
claygonovy: the `.manpages` script is throwing a warning for some link in swift-recon.1 ???17:02
claygi don't really know how to render man pages or what the error means?  `130: warning [p 2, 10.0i]: cannot adjust line` ???17:03
*** openstackstatus has quit IRC17:05
*** openstack has joined #openstack-swift17:07
*** ChanServ sets mode: +o openstack17:07
*** wes_dillingham has joined #openstack-swift17:08
openstackgerritClay Gerrard proposed openstack/swift master: Remove trailing whitespace in manpage  https://review.openstack.org/51019017:11
*** klrmn has joined #openstack-swift17:19
*** vint_bra has joined #openstack-swift17:22
claygacoles: I should have included the related patch 510017 in that commit message  - but thanks for the enthusiasm17:24
patchbothttps://review.openstack.org/#/c/510017/ - swift - Added the man page for container-sync-realms.conf17:24
acolesclayg: I didn't notice the parent patch, but I did wonder what you were doing noticing those whitespaces :)17:26
openstackgerritTim Burke proposed openstack/swift feature/deep: Address review comments  https://review.openstack.org/50988417:28
*** links has quit IRC17:32
*** MVenesio has joined #openstack-swift17:40
*** MVenesio has quit IRC17:45
*** psachin has joined #openstack-swift18:29
openstackgerritMerged openstack/swift feature/deep: remove support for +/- relative object_count in shard updates  https://review.openstack.org/50986018:49
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873619:03
*** chlong has joined #openstack-swift19:06
zaitcevclayg: I'm going to embark on an ill-advised adventure tomorrow, so just in case here's an update -- since you're almost the only guy who was interested. PUT+POST is now 3-stage, but full-up testing found that at times it fails to send chunked as needed or whatever, so object server returns 411 Length Required.19:06
claygLol. Be careful!?19:07
claygAnd thanks.19:07
zaitcevI should be back by Thursday19:07
claygCool!  Good luck and have fun.19:08
zaitcevBut because of weight limits I'm only taking a cellphone, so no data19:08
claygSounds like a party!19:08
*** wes_dillingham has quit IRC19:24
*** ChubYann has joined #openstack-swift19:26
*** tesseract has quit IRC19:31
timburkehttps://github.com/openstack/swift/blob/master/test/functional/swift_test_client.py#L350 doesn't interact well with return_resp over in https://github.com/openstack/swift/blob/master/test/functional/swift_test_client.py#L966 :-(19:41
timburke...and we've got no func tests for any of bulk, do we?19:42
timburkeclayg: you know how you like to point to "Be very careful about asking an author to add a test ... before attempting to do so yourself." ? i might have to call you out on it for the func test for https://review.openstack.org/#/c/509306/19:46
patchbotpatch 509306 - swift - Let clients request heartbeats during SLO PUTs19:46
claygtimburke: sounds fair - do we not currently test SLO's?19:46
timburketotally do. we just never look at any response bodies for writes (as far as i'm aware)19:47
claygtimburke: I do think you can't merge client facing API changes without functional tests - but ... if you think it will be a challenge I could take a stab at it and let you know what I think?19:47
claygoic.  that sucks.  Sorry bro :'(19:47
timburkei'm really tempted to just stick self.response.body = self.response.read() ahead of that close...19:49
timburkebut even then, we're not really testing the heartbeating19:50
*** oshritf has joined #openstack-swift20:09
*** ChubYann has quit IRC20:10
*** oshritf has quit IRC20:12
*** oshritf has joined #openstack-swift20:17
*** catinthe_ has quit IRC20:28
*** spectr has joined #openstack-swift20:34
*** spectr has quit IRC20:36
*** ChubYann has joined #openstack-swift20:50
openstackgerritTim Burke proposed openstack/swift master: Let clients request heartbeats during SLO PUTs  https://review.openstack.org/50930620:57
*** mat128 has quit IRC21:03
openstackgerritTim Burke proposed openstack/swift feature/deep: revert _record_to_dict to same as master  https://review.openstack.org/51013221:13
openstackgerritMerged openstack/swift feature/deep: Address review comments  https://review.openstack.org/50988421:20
*** wes_dillingham has joined #openstack-swift21:23
*** wes_dillingham has quit IRC21:24
timburkeclayg: if you like p 509306 you might want to go land https://review.openstack.org/#/c/509305/ so it's out of the way21:29
patchbotpatch 509306 - swift - Let clients request heartbeats during SLO PUTs21:29
patchbotpatch 509305 - swift - Clean up SLO tests and docs21:29
timburkeand if you want to see my plan to use it in p 509321, it'd be helpful (but not strictly necessary) to have https://review.openstack.org/#/c/441520/21:29
patchbotpatch 509321 - swift3 - WIP: support long-running multipart uploads21:29
patchbotpatch 441520 - swift - Use swift3's check_signature function21:29
timburke(and thanks for already looking at https://review.openstack.org/#/c/509594/!)21:29
patchbotpatch 509594 - swift - Stop clearing params for account_autocreate responses21:29
claygoh yeah, we need to get patch 509594 landed - I'm already shipping that change :\21:35
patchbothttps://review.openstack.org/#/c/509594/ - swift - Stop clearing params for account_autocreate responses21:35
timburkethanks clayg! that first one helped minimize the test changes necessary for the heartbeating since we don't raise exceptions as often21:37
*** wes_dillingham has joined #openstack-swift21:51
*** wes_dillingham has quit IRC21:52
*** wes_dillingham has joined #openstack-swift21:53
*** wes_dillingham has quit IRC21:53
*** wes_dillingham has joined #openstack-swift21:54
*** wes_dillingham has quit IRC21:54
*** wes_dillingham has joined #openstack-swift21:55
*** wes_dillingham has quit IRC21:56
*** wes_dillingham has joined #openstack-swift21:57
*** wes_dillingham has quit IRC21:57
*** chlong has quit IRC22:14
*** oshritf has quit IRC22:42
*** mingyu has joined #openstack-swift22:48
*** mingyu has quit IRC22:52
openstackgerritMerged openstack/swift master: Clean up SLO tests and docs  https://review.openstack.org/50930522:58
*** vint_bra has quit IRC23:05
*** chsc has quit IRC23:21
thurloatlooking for some info on mucking with ring data without using swift-ring-builder, maybe my google-fu is weak but I'm not seeing much other than folks mentioning it's possible.23:36
*** chsc has joined #openstack-swift23:41
*** chsc has quit IRC23:45
openstackgerritMerged openstack/swift master: Fix swift-ring-builder set_weight with >1 device  https://review.openstack.org/50960623:49
*** mingyu has joined #openstack-swift23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!