Monday, 2015-08-03

*** chlong has joined #openstack-swift00:03
*** jasondotstar has joined #openstack-swift00:28
*** Kennan has left #openstack-swift00:29
*** ho has joined #openstack-swift00:39
*** kei_yama has quit IRC01:03
*** kei_yama has joined #openstack-swift01:04
*** DericHorn-HP has joined #openstack-swift01:06
*** jamielennox is now known as jamielennox|away01:08
*** DericHorn-HP has quit IRC01:09
*** haigang has joined #openstack-swift01:10
*** DericHorn-HP has joined #openstack-swift01:11
*** DericHorn-HP has quit IRC01:11
*** kei_yama has quit IRC01:28
*** kei_yama has joined #openstack-swift01:28
*** haigang has quit IRC01:35
*** haigang has joined #openstack-swift01:36
*** kota_ has joined #openstack-swift01:59
*** ChanServ sets mode: +v kota_01:59
kota_good morning02:01
*** haomaiw__ has joined #openstack-swift02:09
*** haomaiwang has quit IRC02:11
mattoliverau kota_: good morning!02:20
*** haomaiwang has joined #openstack-swift02:25
*** haomaiw__ has quit IRC02:28
charzmorning03:03
*** jrichli has joined #openstack-swift03:07
*** alejandrito has quit IRC03:12
*** sanchitmalhotra has joined #openstack-swift03:13
kota_mattoliverau, charz: morning :D03:17
*** sanchitmalhotra1 has joined #openstack-swift03:22
*** sanchitmalhotra has quit IRC03:23
mattoliveraucharz: morning03:30
*** sanchitmalhotra has joined #openstack-swift03:44
*** sanchitmalhotra1 has quit IRC03:47
*** jrichli has quit IRC03:49
*** ppai has joined #openstack-swift04:00
*** DericHorn-HP has joined #openstack-swift04:05
*** sanchitmalhotra1 has joined #openstack-swift04:08
*** sanchitmalhotra has quit IRC04:10
*** sanchitmalhotra has joined #openstack-swift04:24
*** sanchitmalhotra1 has quit IRC04:26
*** jamielennox|away is now known as jamielennox04:38
*** sanchitmalhotra1 has joined #openstack-swift04:42
*** sanchitmalhotra has quit IRC04:44
*** DericHorn-HP has quit IRC04:53
*** sanchitmalhotra has joined #openstack-swift05:01
*** sanchitmalhotra1 has quit IRC05:03
*** sanchitmalhotra1 has joined #openstack-swift05:14
*** sanchitmalhotra has quit IRC05:16
*** jome has joined #openstack-swift05:18
*** madhuri has joined #openstack-swift05:22
*** mahatic has joined #openstack-swift05:30
*** ig0r_ has joined #openstack-swift05:33
*** SkyRocknRoll has joined #openstack-swift05:38
*** nexusz99 has joined #openstack-swift05:42
*** DericHorn-HP has joined #openstack-swift05:47
*** DericHorn-HP has quit IRC05:51
*** ig0r_ has quit IRC05:53
*** ig0r_ has joined #openstack-swift05:56
*** madhuri has quit IRC05:57
*** haigang has quit IRC06:03
*** haigang has joined #openstack-swift06:04
*** pcaruana has quit IRC06:09
*** Kennan has joined #openstack-swift06:11
*** haigang has quit IRC06:16
*** haigang has joined #openstack-swift06:20
*** nakagawamsa has joined #openstack-swift06:23
openstackgerritHiroshi Miura proposed openstack/python-swiftclient: change deprecated assertEquals to assertEqual  https://review.openstack.org/20833106:26
openstackgerritHiroshi Miura proposed openstack/python-swiftclient: fix old style class definition(H238)  https://review.openstack.org/20833206:27
openstackgerritHiroshi Miura proposed openstack/python-swiftclient: fix H101 TODO(name)  https://review.openstack.org/20833306:28
*** geaaru has joined #openstack-swift07:03
*** jome has quit IRC07:07
*** rledisez has joined #openstack-swift07:07
*** SkyRocknRoll has quit IRC07:25
*** chlong has quit IRC07:38
*** pcaruana has joined #openstack-swift07:40
*** sanchitmalhotra has joined #openstack-swift07:44
*** sanchitmalhotra1 has quit IRC07:46
*** hrou has quit IRC07:47
*** jordanP has joined #openstack-swift07:47
*** jamielennox is now known as jamielennox|away07:57
*** jistr has joined #openstack-swift08:15
*** aix has joined #openstack-swift08:18
*** mahatic has quit IRC08:30
*** mahatic has joined #openstack-swift08:38
*** jordanP has quit IRC08:39
*** sanchitmalhotra1 has joined #openstack-swift08:54
*** sanchitmalhotra has quit IRC08:56
openstackgerritHiroshi Miura proposed openstack/python-swiftclient: fix import order for H306  https://review.openstack.org/20837608:58
*** ho has quit IRC08:58
*** julian has joined #openstack-swift09:12
*** julian is now known as Guest1312409:13
Guest13124Hi everybody. I have a problem uploading a file with formpost middleware. When I print the request.body within my own middleware, the content seems to be modified and there is a corrupt image saved into swift. I have read I can access a file like object with request.body_fil but this do not work also. Any tips where I can find  an example for this?09:15
Guest13124Have already found this: http://stackoverflow.com/questions/1783383/how-do-i-copy-wsgi-input-if-i-want-to-process-post-data-more-than-once09:15
Guest13124Which sounds similar to my problem?09:15
*** sanchitmalhotra has joined #openstack-swift09:20
*** jordanP has joined #openstack-swift09:21
*** sanchitmalhotra1 has quit IRC09:22
openstackgerritAlistair Coles proposed openstack/swift: Make test_proxy work independent of env vars  https://review.openstack.org/18875609:24
acolescschwede: torgomatic: you already +2'd this ^^ but it needed a trivial change to satisfy stricter pep8 checks09:34
*** Guest13124 has quit IRC09:42
*** marzif_ has quit IRC09:46
*** marzif_ has joined #openstack-swift09:47
*** marzif_ has quit IRC09:48
*** marzif has joined #openstack-swift09:57
*** aix has quit IRC10:06
*** Kennan has quit IRC10:07
*** logan2 has quit IRC10:08
*** HenryG has quit IRC10:08
*** blair has quit IRC10:08
*** HenryG has joined #openstack-swift10:08
*** logan2 has joined #openstack-swift10:08
*** Kennan has joined #openstack-swift10:09
*** sanchitmalhotra1 has joined #openstack-swift10:31
*** sanchitmalhotra has quit IRC10:33
*** haigang has quit IRC10:44
*** haigang has joined #openstack-swift10:48
*** pcaruana has quit IRC10:57
*** marzif has quit IRC11:09
*** marzif has joined #openstack-swift11:10
*** pcaruana has joined #openstack-swift11:14
*** blair has joined #openstack-swift11:24
openstackgerritKota Tsuyuzaki proposed openstack/swift: WIP: fixups for thiago  https://review.openstack.org/20843911:27
*** sanchitmalhotra has joined #openstack-swift11:27
*** marzif has quit IRC11:27
*** sanchitmalhotra1 has quit IRC11:29
*** kota_ has quit IRC11:38
*** marcusvrn has joined #openstack-swift11:49
*** ChanServ sets mode: +v tdasilva12:00
*** kei_yama has quit IRC12:02
*** ppai has quit IRC12:03
*** km has quit IRC12:05
*** jome has joined #openstack-swift12:11
*** ppai has joined #openstack-swift12:17
*** sanchitmalhotra1 has joined #openstack-swift12:20
*** sanchitmalhotra has quit IRC12:22
*** jasondotstar has quit IRC12:22
*** marzif has joined #openstack-swift12:24
*** marzif has quit IRC12:24
*** marzif has joined #openstack-swift12:25
*** marzif_ has joined #openstack-swift12:32
*** jasondotstar has joined #openstack-swift12:32
*** daemontool_ has joined #openstack-swift12:33
*** marzif has quit IRC12:33
*** chlong has joined #openstack-swift12:35
*** nadeem has joined #openstack-swift12:36
*** SkyRocknRoll has joined #openstack-swift12:37
*** ppai has quit IRC12:38
openstackgerritNicolas Trangez proposed openstack/swift: Use entrypoints for storage policy implementation lookups  https://review.openstack.org/20610512:45
*** daemontool_ is now known as marzif12:46
*** nakagawamsa has quit IRC13:02
*** NM has joined #openstack-swift13:02
*** lpabon has joined #openstack-swift13:05
*** breitz has joined #openstack-swift13:08
*** aix has joined #openstack-swift13:11
*** janonymous_ has joined #openstack-swift13:14
*** SkyRocknRoll has quit IRC13:14
janonymous_Hi Guys, i think this review is not moving to gate instead of  +2s and A: https://review.openstack.org/#/c/200873/13:15
*** haigang has quit IRC13:17
acolesjanonymous_: ok i added another +A to try to move it to gate queue.13:20
janonymous_acoles: Thanks ..!13:20
*** zul has joined #openstack-swift13:21
*** haomaiw__ has joined #openstack-swift13:24
*** ig0r_ has quit IRC13:26
*** haomaiwang has quit IRC13:27
*** bapalm has joined #openstack-swift13:27
acolesjanonymous_: seemed to work.13:30
janonymous_Yes .. :)13:31
*** ig0r__ has joined #openstack-swift13:33
openstackgerritAlistair Coles proposed openstack/swift: Handle removed suffix dirs the same as empty suffix dirs  https://review.openstack.org/18567913:38
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org has been restarted in an attempt to improve performance.13:41
*** jistr is now known as jistr|mtg13:41
*** marzif_ has quit IRC13:43
*** ig0r__ has quit IRC13:44
*** marzif_ has joined #openstack-swift13:44
*** h00327910__ has quit IRC13:48
*** bapalm_ has joined #openstack-swift13:51
*** bapalm has quit IRC13:54
*** annegentle has joined #openstack-swift13:58
*** haomaiw__ has quit IRC14:01
*** vinsh has quit IRC14:01
*** nadeem has quit IRC14:01
*** haomaiwang has joined #openstack-swift14:02
*** nadeem has joined #openstack-swift14:02
*** jrichli has joined #openstack-swift14:02
openstackgerritAlistair Coles proposed openstack/swift: Test that get_hashes ignores only removed hash dir  https://review.openstack.org/20849414:05
*** ig0r__ has joined #openstack-swift14:14
*** mahatic_ has joined #openstack-swift14:18
*** mahatic has quit IRC14:18
*** jistr|mtg is now known as jistr14:20
*** janonymous_ has quit IRC14:37
openstackgerritAlistair Coles proposed openstack/swift: Merge branch 'master' into merge-master  https://review.openstack.org/20851314:40
openstackgerritAlistair Coles proposed openstack/swift: Merge branch 'master' into feature/crypto  https://review.openstack.org/20851314:40
* eikke hates imports-with-side-effects14:41
*** jlhinson has joined #openstack-swift14:41
*** jlhinson_ has joined #openstack-swift14:43
*** jlhinson has quit IRC14:43
acolesjrichli: ^^ when the merge of master lands on feature/crypto there will some style things to clean up due to the pep8 checks on master becoming stricter14:45
jrichliacoles: understood, thx14:46
acolesjrichli: not too much http://paste.openstack.org/show/406767/14:46
*** jlhinson_ has quit IRC14:48
*** vinsh has joined #openstack-swift14:49
acolesjrichli: i figured its worth getting that change and the diskfile refactor onto feature/crypto sooner rather than later.14:49
jrichliacoles: sounds good, thank you!14:50
*** abaptist has joined #openstack-swift14:54
openstackgerritMerged openstack/swift: sys.exc_type/exc_value/exc_traceback are Deprecated  https://review.openstack.org/20087314:54
*** barra204 has joined #openstack-swift14:57
*** jlhinson has joined #openstack-swift14:58
*** hrou has joined #openstack-swift14:58
*** minwoob has joined #openstack-swift14:59
*** zaitcev has joined #openstack-swift15:01
*** ChanServ sets mode: +v zaitcev15:01
*** haomaiwang has quit IRC15:01
*** dustins has joined #openstack-swift15:01
*** haomaiwang has joined #openstack-swift15:02
*** jasondotstar has quit IRC15:02
*** ig0r__ has quit IRC15:09
*** barra204 has quit IRC15:09
*** pcaruana has quit IRC15:13
*** thumpba has joined #openstack-swift15:14
*** shakamunyi has joined #openstack-swift15:14
*** jasondotstar has joined #openstack-swift15:15
*** shakamunyi is now known as barra20415:16
*** barra204 has quit IRC15:26
acolesjrichli: looking at the cryto etherpad section "Overall metadata length can break constraints", its only broken for account/containers right? For objects the current fixes are sufficient?15:34
jrichliacoles: tbh, I dont remember.  I will look at that in a moment.  in a call right now15:34
acolesjrichli: k, no rush15:35
*** silor has joined #openstack-swift15:38
*** barra204 has joined #openstack-swift15:40
*** abaptist has quit IRC15:41
*** gyee has joined #openstack-swift15:47
*** openstackgerrit_ has joined #openstack-swift15:53
*** abaptist has joined #openstack-swift15:54
*** jasondotstar has quit IRC15:54
*** haomaiwang has quit IRC16:01
*** lpabon has quit IRC16:01
*** jome has quit IRC16:02
*** haomaiwang has joined #openstack-swift16:02
*** ig0r_ has joined #openstack-swift16:08
*** lcurtis has joined #openstack-swift16:15
charzclayg: peluse I can repo this issue on VMs (https://bugs.launchpad.net/swift/+bug/1469094), please take a look the gist.16:16
openstackLaunchpad bug 1469094 in OpenStack Object Storage (swift) "Missing .durable files of FAs can cause rebuildable objects to be inaccessible" [Medium,Confirmed]16:16
pelusecharz, awesome!  will do16:18
charzpeluse: cool!16:18
*** dmorita has joined #openstack-swift16:24
pelusecharz, OK, just commented on the big.  Wow man, really, really great job w/debug and documentation there.  I think we should be able to get line of sight on this one pretty quickly now, thanks!!!!16:25
*** jlhinson has quit IRC16:25
pelusebug that is, not big :)16:25
*** marzif_ has quit IRC16:25
*** tab___ has joined #openstack-swift16:27
*** jistr has quit IRC16:28
*** dustins has quit IRC16:28
*** rledisez has quit IRC16:28
charzpeluse: That's great, I'll do other tests(different object sizes) to see the root cause is it or there still have others.16:31
peluseawesome, thanks again.16:31
charzpeluse: have a good day, nite16:31
*** jlhinson has joined #openstack-swift16:34
*** jordanP has quit IRC16:37
notmynamegood morning16:38
*** dustins has joined #openstack-swift16:39
*** kutija has joined #openstack-swift16:48
*** ig0r_ has quit IRC16:49
*** jasondot_ has joined #openstack-swift16:51
*** jasondot_ has quit IRC16:52
*** jasondot_ has joined #openstack-swift16:52
*** jasondot_ is now known as jasondotstar_16:54
*** jasondotstar_ is now known as jasondotstar__16:54
*** jasondotstar__ is now known as jasondotstar_16:54
*** ig0r_ has joined #openstack-swift16:54
*** ig0r_ has quit IRC16:55
claygcharz: great work16:56
claygnotmyname: good morning16:56
pelusegood morning to both of you16:56
*** bi_fa_fu has joined #openstack-swift16:57
*** haomaiwang has quit IRC17:01
*** haomaiwang has joined #openstack-swift17:02
*** tongli has joined #openstack-swift17:03
*** ig0r_ has joined #openstack-swift17:04
*** pcaruana has joined #openstack-swift17:05
*** zhill has joined #openstack-swift17:07
notmynameone thing I missed when I was traveling last week was my keyboard. it's nice to be back in the office this week :-)17:07
breitzI submitted a simple test bug just now (Bug #1481019).  Since I'm new to contributing I'd like to use this bug to learn the process.17:11
openstackbug 1481019 in OpenStack Object Storage (swift) "docstring in test_denied_DELETE_of_versioned_object confuses test output" [Undecided,New] https://launchpad.net/bugs/148101917:11
breitzheh - that one ^^   :-)17:12
notmynamebreitz: great. so do you know what needs to happen next?17:12
breitznotmyname: not really17:12
breitzI *think* I do - but I need to make sure.17:13
notmynamebreitz: I'll walk you throught he process. but my boss just grabbed me to chat about something. give me a few minutes :-)17:16
breitzsure np.  thanks!17:16
*** mahatic_ has quit IRC17:18
*** DericHorn-HP has joined #openstack-swift17:20
*** abaptist has quit IRC17:22
*** ig0r_ has quit IRC17:25
*** DericHorn-HP_ has joined #openstack-swift17:25
*** DericHorn-HP has quit IRC17:25
*** DericHorn-HP_ is now known as DericHorn-HP17:25
*** chlong has quit IRC17:31
*** hrou has quit IRC17:32
*** ig0r_ has joined #openstack-swift17:33
*** jasondotstar_ has quit IRC17:34
acolesjrichli: left some comments on patch 193749 but gotta run now.17:43
patchbotacoles: https://review.openstack.org/#/c/193749/17:43
*** ig0r_ has quit IRC17:44
*** acoles is now known as acoles_away17:45
*** NM has quit IRC17:49
*** NM has joined #openstack-swift17:56
notmynamebreitz: back17:58
breitzhi17:59
notmynamethe funny thing is that he wanted to talk about some stuff we're working on the help people get involved in swift ;-)17:59
breitzgood timing17:59
notmynamebreitz: ok, so you've identified a problem and you filed a bug about it17:59
notmynamethanks for filing the bug17:59
breitzsure - np17:59
notmynamebased on what you write there, it seems that you already have dev environment set up. is that right?18:00
breitzyes18:00
notmynameie you've run tests, so you must have the code somewhere18:00
notmynamecool18:00
*** haomaiwang has quit IRC18:01
breitzubuntu - devstack18:01
notmynamehow did you set it up? did you set up the Swift All in One (SAIO)? are you running in a VM? via a cloud provider18:01
notmynameah ok18:02
breitztwo ways - solaris bits - and ubuntu devstack18:02
*** haomaiwang has joined #openstack-swift18:02
breitzyes - running in a VM18:02
notmynamedevstackw ill be ok for a little while (and for your change that you're looking at now), but there's a better way to have a swift dev environment. for now, though, let's skip over that part and move on to some of the mundane setup pieces18:03
breitzk - but that part I'd be really interested in learning more about (not using devstack)18:04
notmynameok. we'll get there :-)18:04
breitzsure - thanks18:04
notmynamehave you installed git-review yet? if so, you need to run `git review -s` in your local copy of the repo18:04
breitzyes18:04
notmynameok, great18:04
notmynameand so you've set up the appropriate launchpad accounts (of course, you've submitted a bug!) and signed the CLA in gerrit?18:04
breitzyes I believe so18:05
notmynamegreat :-)18:05
notmyname(you're making this easy, so far)18:05
notmynamehow comfortable are you with git?18:05
breitzcomfortable18:05
notmynameok18:05
breitzI'm sure I'm not a power user - but I understand how to use it18:06
notmynamethe general idea of code contributions is that you make a change and propose it. it gets reviewed by other people, and when it's good, it lands. by convention, that requires 2 people who have commit rights (core-reviewers) to approve the change18:07
breitzright - got it18:07
*** openstackgerrit_ has quit IRC18:07
notmynamespecifically in openstack projects and the configuration of the gerrit code review tool we use, each change is one commit. and if you have to make updates to it, then you'll amend the first change and resubmit it18:08
notmynameso you propose a change, I review and make suggestions, you fix it and resubmit it as one ammended commit, not as a commit on top of the first one18:08
notmynamemake sense?18:08
breitzI think so yes - so each patch is a complete entity - not a collection of patches that you would need to apply in order18:09
*** abaptist has joined #openstack-swift18:09
notmynamethe way gerrit ties it all together is with a Change-Id that is in the commit message. the `git review -s` you ran to set up the gerrit remote also installed a post-commit hook to make sure there is a change-id in the commit message18:09
notmynameso when you amend a commit, you keep the same change-id, and it all works18:09
breitzcool18:09
breitzI was wondering how that works (from the git mechanics point of view)18:10
notmynameit's not complicated, but it is different enough from some other common methods that it's worth pointing out up front18:10
notmynameok, so let's get down to the patch itself18:10
notmynameat this point, it's type to type18:11
notmynameI'd recommend creating a local branch in which to do your patch18:11
breitzsure18:11
notmynamemake the change, commit it. make sure there's just one commit on that branch, and type `git review`18:11
notmynamethat will submit it to gerrit for review18:11
notmyname(which behind the scenes is pushing a reference to the gerrit remote)18:12
breitzk - I assume a test stage in there would be good.  make change, commit, test, git review?18:12
notmynameyes, of course :-)18:12
breitzk - just making sure.  :-)18:13
notmynameso I'd expect that you will run tests on a patch before you submit it18:13
notmynamebut the CI system will also run tests against the patch18:13
notmynamein reality, no human actually will commit code to master. humans say "this should land" but it only happens after the bots run the tests and they pass18:13
breitzright - is there a way to run CI on your own - or does that typically happen post git review time?18:13
notmynameboth18:14
notmynamein your source repo that has your change checked out (on your dev environment), there are a couple of ways to run tests18:15
notmynameone is to use `tox` and the other is to use the provided scripts. they do pretty much the same things18:15
notmynamethe local scripts are ./.unittests ./.functests and ./.probetests18:15
notmynameonly the first 2 will work on a devstack environment (which is the reason for my comment earlier)18:16
breitzok - good to know18:16
notmynameunit tests test small sections of code. functests are functional tests that test the API as a black-box. and probe tests are for checking that various components in a cluster interact as expected18:17
notmynamenow, in this case, you're talking about a docstring change. so testing requirements are pretty minimal ;-)18:18
breitzright - which is sort of why I picked to bug - so I could focus on the process, and learn more about testing as well along the way18:18
notmynameonce you've made your changes and seen that the patch works as expected and you've submitted it for review...18:18
notmynameat that point, others in the community will review it18:19
notmynameand to review it, they will pull down your change, run it, check that it does what it says and its a good change, and leave a review18:19
breitzright18:21
*** Kennan has quit IRC18:21
notmynameat this point I think it's up to you to start typing :-)18:22
breitzk - thanks!  I'm interested in learning more about how to setup a dev env outside of devstack.  Is there docs/wiki that I can read?18:23
breitzalso - I assume I can just assign the bug to myself?  Or does somebody else need to do that18:23
notmynameyes18:25
*** abaptist has quit IRC18:25
notmynamethe launchpad stuff is good for trackign what's going on, but don't consider it as a permission slip to work on stuff :-)18:25
notmynameie you don't have to have something with all the right values in the right places before you can work on something18:26
breitzsure18:27
notmynameor as I've phrased it in other places, they are tools to facilitate communication, not a goal in themselves. so as long as communication happens, we're all good :-)18:27
breitzk - I appreciate the help.  I'll go off and type, type, type.18:28
notmynameas my australian friends say, "typey typey"18:28
jrichliacoles: thanks for reviewing!18:28
*** Kennan has joined #openstack-swift18:29
breitzheh18:29
notmynameok, for other swift dev environments18:30
notmynameoh, actually, here's another thing. it just was added a few days ago :-) http://docs.openstack.org/developer/swift/first_contribution_swift.html18:31
notmynamethe swift dev environment is the SAIO (Swift All in One)18:31
notmynamedocs to set it up are at http://docs.openstack.org/developer/swift/development_saio.html18:31
breitzah - cool.  thanks18:31
notmynameand there is also https://github.com/swiftstack/vagrant-swift-all-in-one that automates it all for you and takes care of some config options18:32
breitzI'll snapshot my vm and give that a try18:32
breitzI really appreciate all the help18:33
notmynamehappy to help18:33
*** abaptist has joined #openstack-swift18:34
*** mahatic has joined #openstack-swift18:45
*** shakamunyi has joined #openstack-swift18:46
*** mahatic has quit IRC18:46
openstackgerritpaul luse proposed openstack/swift: Add ability for GET path to see/select alternate frag archs  https://review.openstack.org/20716518:48
peluse   clayg, ^ obj server test as requested.  acoles_away minor nits addressed.  Still need to add the alt headers to HEAD but that won't change whats in there now (just add to it)18:49
*** haomaiwang has quit IRC19:01
*** haomaiwang has joined #openstack-swift19:02
openstackgerritpaul luse proposed openstack/swift: Add ability for GET path to see/select alternate frag archs  https://review.openstack.org/20716519:08
peluseclayg, acoles_away ^ there's everything (HEAD stuff too)19:08
*** zul has quit IRC19:11
*** lpabon has joined #openstack-swift19:11
*** tab___ has quit IRC19:11
*** bi_fa_fu has quit IRC19:12
*** kutija_ has joined #openstack-swift19:14
lcurtishello all...when someone has a moment...i have a bunch of async_pending objects from a few days ago that seem to be orphaned...19:16
lcurtisany thoughts on how to rectify?19:16
*** kutija has quit IRC19:16
*** jasondot_ has joined #openstack-swift19:17
*** michaeljk has joined #openstack-swift19:18
*** jome has joined #openstack-swift19:20
*** jasondot_ has quit IRC19:26
*** abaptist has quit IRC19:33
minwoobIsn't a primary node both primary and a handoff, if it contains a handoff fragment that belongs on a partner primary?19:36
minwoobBasically, it serves the function of *both* a primary, and a handoff node, if that is the case?19:37
minwoobOr is there something unique about such nodes?19:37
*** geaaru has quit IRC19:38
peluseminwoob, for a given hash a single node can't be a primary and a handoff at the same time.  when handoff nodes are iterrated they exlucde primaries19:39
peluseminwoob, but that doesn't mean that a primary for obj 1 can't be a handoff for obj 2...19:40
minwoobpeluse: I see. How is it possible, then, for a fragment for primary x to end up on partner primary y?19:41
peluseheh, I was just about to explain that...19:41
peluserebalance19:42
minwoobAhhh. Okay.19:43
minwoobThen it's just a new permanent location.19:43
pelusethere are a bunch of crazy things you can do chaning the ring around before a rebalance is totally done that can result in fragments all over th epalce19:43
*** michaeljk has quit IRC19:43
peluseand the reconstructor has the job of figuring who is supposed to be where and getting them back the node that is best (node index = fragment index)19:44
*** jome has quit IRC19:44
*** bi_fa_fu has joined #openstack-swift19:45
peluseminwoob, great question though19:45
*** silor has quit IRC19:45
minwoobpeluse: Thanks for the helpful explanation :-)19:46
minwoobBtw, has it ever been considered to architect the rebalancing process so that only the partitions being moved to an added node are actually shuffled around?19:50
minwoob... or is that not possible?19:50
pelusethat's a question for one of the other cores... I have succesfully avoided the guts of the ring code so far :)19:51
* peluse outta here for a few hours...19:51
minwoobI see.19:52
minwoobFor instance, Ceph is implemented that way, I believe.19:53
*** sanchitmalhotra has joined #openstack-swift19:59
*** mfalatic has joined #openstack-swift20:01
*** haomaiwang has quit IRC20:01
*** sanchitmalhotra1 has quit IRC20:01
*** haomaiwang has joined #openstack-swift20:02
redboI'm pretty sure that's how it works, or else I don't know what you mean.20:03
redboit tries to move as few partitions as possible20:03
*** dustins has quit IRC20:05
redbosomeone needs to give our algorithm a catchy name and write up a white paper20:06
openstackgerritMerged openstack/swift: go: add ability to lock device/drive with file on file system  https://review.openstack.org/20378920:07
minwoobI could be wrong about that. But that was from my last impression on it.20:08
notmynameunique-as-possible?20:10
notmyname(which today is more of distributed-as-balanceable)20:10
notmynamehttps://www.youtube.com/watch?v=JUXRJaZMOb0&list=PLIr7I80Leee5NpoYTd9ffNvWq0pG18CN3&index=120:11
MooingLemurif you have two zones and three replicas, adding storage to one zone results in additional inter-zone movement to keep things as balanced as possible (at least in the old ring code that doesn't have the overload parameter).  Has this changed?20:16
notmynamelcurtis: the object updater clears those. check that it's running. you could stop it on a machine and run it explicitly to see what's happening20:17
notmynameMooingLemur: correct, in general. it's a good idea to keep zones pretty evenly balanced20:18
lcurtisthanks notmyname...was just about to report back that was what i did...and they are cleared20:18
lcurtisthanks!20:18
MooingLemurbut I suppose that all depends on uniqueness of data that exists on a host anyway.  If it can, I suppose it would prefer to move the data off of zone 1 that it could add directly to the host with increased storage in zone 2.20:19
MooingLemurOr at least should :)20:19
MooingLemurIt really sounds like it can get messy, and you end up with conflicting objectives that a rebalance would have to satisfy.  I haven't looked much at the rebalancing code :)20:20
claygminwoob: redbo: notmyname: torgomatic: we of course try to move as few partitions as possible - it's a design goal for the rebalance algorithm - but we've discovered some scenarios in the past where there was room to improve (e.g. 2328983b)20:23
*** bi_fa_fu has quit IRC20:24
*** bi_fa_fu has joined #openstack-swift20:24
redbobut you should only discuss the straightforward case in the white paper20:25
claygredbo: oh good call20:25
*** bi_fa_fu has quit IRC20:26
openstackgerritMerged openstack/swift: Make test_proxy work independent of env vars  https://review.openstack.org/18875620:26
clayg^ well that sucks - gholt (and acoles, and I) worked really hard on that test -> https://review.openstack.org/#/c/102598/20:34
claygafter a few bugs with a similar "make sure we use the args" test - we seem to have decided to man up and make an integration test - but idk, i guess as long as we don't have anymore regressions in that area20:35
claygI agree requiring the env var was not great, but it seems like you could fix that by just stubbing the environment20:35
claygoh well20:35
*** ig0r_ has joined #openstack-swift20:36
*** hrou has joined #openstack-swift20:44
*** breitz has quit IRC20:50
*** ig0r__ has joined #openstack-swift20:50
*** breitz has joined #openstack-swift20:50
*** ig0r_ has quit IRC20:50
*** lpabon has quit IRC20:53
*** annegentle has quit IRC20:54
*** thumpba has quit IRC20:56
*** bapalm_ has quit IRC21:00
*** haomaiwang has quit IRC21:01
*** haomaiwang has joined #openstack-swift21:01
*** jrichli has quit IRC21:02
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873621:03
*** lpabon has joined #openstack-swift21:15
*** ig0r__ has quit IRC21:21
*** lpabon has quit IRC21:32
*** lpabon has joined #openstack-swift21:33
*** lpabon has quit IRC21:35
*** annegentle has joined #openstack-swift21:47
*** breitz has quit IRC21:58
*** tongli has quit IRC21:59
*** haomaiwang has quit IRC22:01
*** haomaiwang has joined #openstack-swift22:02
*** breitz has joined #openstack-swift22:06
*** sanchitmalhotra1 has joined #openstack-swift22:09
*** sanchitmalhotra has quit IRC22:11
*** minwoob has quit IRC22:12
*** NM has quit IRC22:18
*** thumpba has joined #openstack-swift22:19
*** jlhinson has quit IRC22:21
claygdfg: I'd appreciate your thoughts on patch 208106 - if those fixups make sense to you maybe we could merge it with patch 20805722:23
patchbotclayg: https://review.openstack.org/#/c/208106/22:23
*** DericHorn-HP has quit IRC22:24
mattoliverauMorning22:25
openstackgerritHamdi Roumani proposed openstack/swift: Doc instructions for post rebase steps  https://review.openstack.org/20866922:26
*** thumpba has quit IRC22:29
*** breitz has quit IRC22:34
*** breitz1 has joined #openstack-swift22:34
*** annegentle has quit IRC22:35
lcurtishmm...is there a utility to get total number of objects in a swift cluster?22:36
openstackgerritBrian Reitz proposed openstack/swift: Convert docstring to a comment in test_denied_DELETE_of_versioned_object unit test.  https://review.openstack.org/20867522:40
*** thumpba has joined #openstack-swift22:41
*** thumpba has quit IRC22:41
*** annegentle has joined #openstack-swift22:43
*** nadeem has quit IRC22:45
*** shakamunyi has quit IRC22:51
notmynamebreitz1: I see your patch ^22:55
*** km has joined #openstack-swift22:55
breitz1yep22:56
*** breitz1 is now known as breitz22:56
breitzyour help got me this far.  :-)22:57
notmynamebreitz: the patch looks fine. I didn't know that nose was pickup up the docstrings. good thing to keep in mind22:59
notmynamebreitz: in this case, since it's "just" a docstring to comment patch in a test (and thus very low risk), I will go ahead and approve it without another review. for most changes though, it will take 2 +2 votes from core reviewers before a patch lands23:00
notmynameeventually zuul will get around to running the tests and jenkins will leave the test results as comments in gerrit. once it goes through that a couple of times, it will get merged23:00
*** haomaiwang has quit IRC23:01
breitzright - I need to figure out how to follow all of that around (as far as a progress tracking sort of thing - for my own education)23:01
notmynamebreitz: http://status.openstack.org/zuul/ and find the patch id of your patch23:01
*** haomaiwa_ has joined #openstack-swift23:02
breitzmy patch id is 208675 right?23:02
notmynamecorrect23:02
breitzk - thanks23:02
breitzI still have a list of a bunch of test env things I want to try (SAIO) but post this patch it would be awesome to find something else to work on in swift23:06
notmynamethere's a few places to look for things to do, but a lot of it comes down to "what do you want to work on?"23:07
notmynamehttps://wiki.openstack.org/wiki/Swift/ideas is a list of ideas of stuff to work on23:07
breitzthanks - I can go read that23:08
notmynamehttps://wiki.openstack.org/wiki/Swift/PriorityReviews has a list of some of the bigger things that are ongoing, including some notes from the last summit23:08
breitzk - nice23:08
notmynameand in the channel topic there's a "review dashboard" link that has open patches that need reviews. the starred section of the top are the ones that are most important to review23:08
breitzah ok. cool23:09
notmynamebreitz: what brings you to swift? how'd you find it and are you using it today?23:11
breitzI work on openstack for solaris.  I've been involved in storage as a kernel developer for years.23:12
*** aix has quit IRC23:12
notmynamecool23:12
breitzI want to help contribute to the product - and learn as much as I can23:12
notmynamehave you seen any problems running swift on solaris?23:12
notmyname(and by "...for solaris", I assume you work for oracle?)23:13
breitzyes - I work for Oracle.  No I've not seen issues in what I've done with it yet - but I'm just starting to learn all of this (I'm new to the openstack group in solaris as well)23:14
breitzrelatively new anyway.  :-)23:14
breitzthanks again for all of your help.23:16
breitzoff to dinner - biab23:16
notmynamebreitz: welcome to swift23:16
*** darrenc has quit IRC23:23
*** darrenc has joined #openstack-swift23:23
*** annegentle has quit IRC23:24
*** kei_yama has joined #openstack-swift23:25
*** jlhinson has joined #openstack-swift23:33
*** marcusvrn has quit IRC23:33
*** darrenc has quit IRC23:35
*** darrenc has joined #openstack-swift23:35
*** ho has joined #openstack-swift23:36
breitzthanks!23:42
*** DericHorn-HP has joined #openstack-swift23:42
*** jrichli has joined #openstack-swift23:42
pelusehmmm, did something hppen w/the tests on pep8/flake8 in the last few days?  I have a patch that passes locally, and passed in gerrit within the last few days, that now fails in gerrit for what looks like no valid pep8 reason at all (and that file didn't change since the last time I submitted it)?23:43
*** darrenc is now known as darrenc_afk23:45
notmynameyay23:46
notmynamepeluse: I hope not, but it's possible23:46
peluseargh23:47
notmynamepeluse: with https://review.openstack.org/#/c/205977/23:47
notmynamepeluse: see clayg's comment at teh bottom23:48
peluseso I'm to yell at clayg now huh?23:48
notmynameseems that way23:49
*** jrichli has quit IRC23:50
hrounotmyname, peluse - and one more thing, if you use tox, use -r to update the tox env (as this will rebuild the tox env with the new packages that are required)23:50
pelusehrou, OK thanks.23:59
pelusenow if pip install would just work....23:59
hrouanytime ! 'sudo pip install ..' ? : )23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!