Tuesday, 2014-06-10

*** zhiyan_ is now known as zhiyan00:10
*** matsuhashi has joined #openstack-swift00:27
*** NM has joined #openstack-swift00:29
*** dmorita has joined #openstack-swift00:36
*** NM has quit IRC00:38
*** shri has quit IRC00:49
*** dmsimard_away is now known as dmsimard00:51
*** mwstorer has quit IRC00:51
*** lpabon has quit IRC01:06
*** haomaiwang has joined #openstack-swift01:10
*** haomaiw__ has joined #openstack-swift01:12
*** mkollaro has quit IRC01:14
*** haomaiwang has quit IRC01:15
*** dmsimard is now known as dmsimard_away01:18
*** mkollaro has joined #openstack-swift01:19
*** diegows has quit IRC01:28
*** nosnos has joined #openstack-swift01:33
*** kashyapk has joined #openstack-swift01:49
*** kashyapk has quit IRC01:55
*** nsquare has quit IRC02:21
*** mkollaro has quit IRC02:27
*** byeager_away is now known as byeager_02:29
*** byeager_ is now known as byeager02:32
*** byeager is now known as byeager_away02:38
*** byeager_away is now known as byeager02:39
*** byeager_ has joined #openstack-swift02:44
*** byeager is now known as byeager_away02:45
*** bkopilov has quit IRC02:46
*** byeager_ has quit IRC02:46
*** byeager_away is now known as byeager02:46
*** byeager is now known as byeager_away02:47
anticwdfg: do you know much about how authorize(...) is called for xLO segments?03:13
anticwspecifically i see a situation where a user has permissions for the DLO but not the segment ... authorized gets called for a segment (i assume) and environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed'03:15
notmynameanticw: that's been under development recently. specifically to answer the question of what happens when a token expires in the middle of serving a *LO03:16
anticwthat i'm less worried about03:17
notmynameanticw: but in general, the user needs read on the manifest and read/listings on the segments container IIRC03:17
anticwthe issue i have is a non-admin user who has access to the manifest object but not the segments ... we end up in code paths that explode stack-traces03:17
anticwnotmyname: sure, so where is the code that walks the manifest and calls into authorize?03:18
notmynameI think the proxy object controller GETorHEAD03:19
anticwi guess from _get_container_listing03:19
anticwdlo is done as middleware magically inserted into the pipeline03:19
notmynameoh yeah. sorry I don't have the code in front of me at the moment03:19
anticwso afaict dlo.py : handle_request -> get_or_head_response03:20
*** zhiyan is now known as zhiyan_03:20
portanteanticw: what version of swift are you talking about?03:20
portantegreat ...03:22
anticwdlo.py : GetContext : _get_container_listing  does     con_resp = con_req.get_response(self.dlo.app)03:22
anticwwill that be done with the same acl/security behavior for non-admin as the actual  dlo/slo fetch itself?03:23
anticwit's entirely possible this is a local thinko here ... i'm trying to get someone to test older swift03:23
portanteso, perhaps we should start with the stack traces you are seeing, can you provide any examples?03:24
anticwi can show you where i think it would also blow-up with keystone if that's useful03:24
anticw./swift/common/middleware/keystoneauth.py  , see  def authorize(...)03:25
anticw  env_identity = self._integral_keystone_identity(env)03:25
anticwlook at def _integral_keystone_identity03:25
portanteokay, why would that blow up?03:25
anticwthe very first non-comment   if environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed'03:25
anticwfor a dlo segment afiact environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed'03:26
*** zhiyan_ is now known as zhiyan03:26
anticwso we hit the return right away03:26
anticw(i'm a bit out on a limb here i admit ... i don't have an env to confirm this)03:26
anticwand if we return (naked) it's None03:26
anticwso back in authorize(...) we then go and do      tenant_id, tenant_name = env_identity['tenant']03:26
anticwwhere env_identity = None03:26
*** haomaiw__ has quit IRC03:27
*** haomaiwang has joined #openstack-swift03:27
anticwportante: what's not clear to me is how the auth stuff works for dlo's03:28
portanteso right now, we say place dlo/slo to the left of auth middleware in the pipeline, afaict03:28
anticwsegments still have to be authenticated03:29
portanteso that means each sub-request made my dlo/slo will be checked for authentication and authorization03:29
anticwor else i can create a manifest pointing to data that i don't have access to03:29
portantethe proxy server app does the authorization03:29
portanteso the middleware, once authenticated, places an env.authorize method for the proxy server to invoke at the proper time03:29
anticwyes, so back at 20:22 (scroll-back)   i think that's the container get to list the dlo elements ?03:30
anticwand it's not clear what the auth situation is there ... does   con_req.get_response(...)  look and smell enough like the original request that the authentication works as for the original object?03:30
portanteyes, it tries to do that03:31
anticwso then environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed'     or   ==   ?03:31
portanteif this is a new request, one where tempauth or keystoneauth has not seen before, it won't have an x-auth-token header on the request, so it won't be able to operate on the DLO object03:31
portanteonce the client has authenticated, each DLO REST API invocation using the x-auth-token previously received by the client should work just fine03:32
anticwexcept the stacktrace shows we get None .... which makes me think environ.get('HTTP_X_IDENTITY_STATUS') != 'Confirmed'03:33
portantecan you share that stacktrace?03:34
anticwsorta, sec...     tenant_id, tenant_name = env_identity['tenant']#012TypeError: 'NoneType' object has no attribute '__getitem__'03:35
anticwis the relevant part03:35
portantefilename and line number, if you have it?03:36
portanteline ...03:36
anticwline 18303:36
anticwsorry, i tried explaning this above but probably didn't put enough detail03:36
portantethat just looks like a but in keystoneauth.py03:37
anticwbut as you indicated if the container listing get is done as for the original manifest get ... won't HTTP_X_IDENTITY_STATUS be set correctly?03:38
portanteSo DLO only copies over the X-AUTH-TOKEN ...03:40
anticwah yes, i see what you mean03:41
portanteso keystoneauth.py has a bug, it should not stack trace there, if no x-identity-status exits, the request should be denied, not a stacktrace03:41
anticwportante: that won't work03:41
anticwi did that initially03:41
anticwso what then i get 403 not 500 for when i don't have perms03:41
anticwbut when i *do* have perms i also get 40303:41
torgomaticI believe there is a patch proposed to fix that by clever application of functools.partial, but it has not been approved03:41
anticwso we need to copy over more env ... let me quickly try that now03:41
portanteDLO somehow needs to copy all the headers from the original request so that it includes what the client03:41
portantetorgomatic: do you know what patch name or number?03:42
portanteanticw: let's look at the patch torgomatic is talking about, sec ...03:42
torgomaticI'm on a smartphone at the moment so gerrit is basically pure annoyance03:42
*** nosnos has quit IRC03:43
notmynametorgomatic: that doesn't sound very "smart"03:43
portantetorgomatic: could you be talking about https://review.openstack.org/92165, "xLO bug with auth tokens expiring during download"03:43
torgomaticnotmyname: to be fair, neither do I much of the time ;)03:44
torgomaticYeah, i think that's the patch.03:44
anticwgerrit is pretty nasty on a phone03:45
anticw+            environ['swift.authorize'] = functools.partial(03:45
anticw+                self.authorize, env_identity)03:45
anticwso in this case the dlo segment stuff happens after the auth middleware ... but the callbacks from the original callback?03:47
portanteanticw: so yes, in that patch you move dlo to right of all the auth middleware, and the authorize() method is a closure over the original identity information03:49
portanteavoiding any changes to the proxy server to support03:50
anticwawesome, i'm going to try that in a few03:50
portantethe change to the proxy-server is to place dlo after known auth middleware when it is not specified03:50
anticwthanks very much03:50
portanteI hope it works! :)03:50
anticwme too!03:50
portanteping dfg with feedback on it, as well03:51
anticwi'm not sure if it matters, but that patch doesn't consider keystone doing anon authentication04:08
zaitcevI didn't realize Keystone even had such a thing.04:09
anticwit changes environ['swift.authorize'] for known identity ... and the arguments to the function then ... but none for anon04:09
*** madhuri has joined #openstack-swift04:09
portanteanticw: doesn't it use a different auth method then?04:13
portanteself.authorize vs self.authorize_anonymous04:13
anticwno, it should be ok ... for the anon method we don't change the auth function04:13
anticwright, but i don't know when self.authorize_anonymous is used ... it won't be affected in this case04:13
*** bkopilov has joined #openstack-swift04:13
anticwthough ... i hope that doesn't mean something like anon access for xLOs aren't checked04:13
portanteit is used when there is no identity04:13
portanteso this is a two phase kind-a thing, right?04:14
anticwright ... so we're sure this doesn't allow a rogue xLO to provide access when there is no identity aren't we?04:14
portantethe middleware is invoked via __call__ when the pipeline is being processed04:14
portantebut the swift.authorize environment method is invoked later when the proxy-server's __call__ method is invoked at the end of the pipeline04:15
portanteso that in that patch, keystoneauth's __call__ method is setting up self.authorize() to always have a good "identity" objectd, if one was present to begin with, otherwise it uses the self.authorize_anonymous() method04:16
*** kashyapk has joined #openstack-swift04:22
*** nosnos has joined #openstack-swift04:24
*** kashyapk has quit IRC04:26
*** ppai has joined #openstack-swift04:26
*** kashyapk has joined #openstack-swift04:26
*** nshaikh has joined #openstack-swift05:02
*** gyee has quit IRC05:38
*** kashyapk has quit IRC05:50
*** kashyapk has joined #openstack-swift05:50
*** Longgeek_ has joined #openstack-swift06:06
*** kashyapk has quit IRC06:08
*** kashyapk has joined #openstack-swift06:08
*** nthacker has joined #openstack-swift06:11
nthackerhi guys, i wanted to know if there's a reason that FakeLogger in the unit test framework doesnt have a working flush to clear the FakeLogger's buffer06:12
nthackerit seems i can use close, but close sounds like I shouldnt use it again06:12
nthackeralthough the close routine doesnt appear to stop me from using it again06:13
nthackerbasically i want to call FakeLogger::_clear06:13
*** jamie_h has joined #openstack-swift06:26
*** openfly has quit IRC06:27
*** mmcardle has joined #openstack-swift06:51
*** nosnos has quit IRC06:59
*** matsuhashi has quit IRC07:01
*** matsuhas_ has joined #openstack-swift07:01
*** mkollaro has joined #openstack-swift07:02
*** kashyapk has quit IRC07:03
*** kashyapk has joined #openstack-swift07:03
*** nosnos has joined #openstack-swift07:05
*** kashyapk has quit IRC07:08
mattoliverauNight all07:12
*** zaitcev has quit IRC07:16
*** foexle has joined #openstack-swift07:17
*** roock has quit IRC07:30
*** kashyapk has joined #openstack-swift07:36
*** kashyapk has quit IRC07:56
*** kashyapk has joined #openstack-swift07:56
*** kashyapk has quit IRC08:01
*** occupant has quit IRC08:04
*** occupant has joined #openstack-swift08:04
*** kashyapk has joined #openstack-swift08:06
*** nthacker has quit IRC08:08
*** openfly has joined #openstack-swift08:24
*** matsuhas_ has quit IRC08:38
*** kashyapk has quit IRC08:38
*** kashyapk has joined #openstack-swift08:39
*** nosnos has quit IRC08:40
*** mkollaro has quit IRC08:40
*** nosnos has joined #openstack-swift08:43
*** nosnos has quit IRC08:44
*** matsuhashi has joined #openstack-swift08:44
*** mlipchuk has joined #openstack-swift08:52
*** nosnos has joined #openstack-swift09:03
*** zhiyan is now known as zhiyan_09:18
*** ahale_ has quit IRC09:31
*** ahale has joined #openstack-swift09:32
*** nacim has joined #openstack-swift09:35
*** dmorita has quit IRC09:41
*** kashyapk has quit IRC09:49
*** kashyapk has joined #openstack-swift09:50
*** kashyapk has quit IRC09:54
*** jamie_h has quit IRC10:17
*** jamie_h has joined #openstack-swift10:18
*** omame has quit IRC10:28
*** jamie_h_ has joined #openstack-swift10:29
*** omame has joined #openstack-swift10:29
*** jamie_h has quit IRC10:32
*** kashyapk has joined #openstack-swift10:34
*** ekarlso has quit IRC10:57
*** ekarlso has joined #openstack-swift10:58
*** dmsimard_away is now known as dmsimard11:17
*** kashyapk has quit IRC11:36
*** kashyapk has joined #openstack-swift11:37
*** miqui has quit IRC11:38
*** diegows has joined #openstack-swift11:41
*** mkollaro has joined #openstack-swift11:54
*** ppai has quit IRC11:58
*** mkollaro1 has joined #openstack-swift12:00
*** mkollaro has quit IRC12:00
*** shakayumi has joined #openstack-swift12:10
*** kashyapk has quit IRC12:13
*** kashyapk has joined #openstack-swift12:13
*** kashyapk has quit IRC12:19
*** Longgeek_ has quit IRC12:25
*** mmcardle has quit IRC12:35
*** Ju has quit IRC12:35
*** mmcardle has joined #openstack-swift12:46
openstackgerritPeter Portante proposed a change to openstack/swift: xLO bug with auth tokens expiring during download.  https://review.openstack.org/9216512:48
portantedfg: ^^^ just a couple fixups on the commit message12:50
*** mkollaro has joined #openstack-swift12:52
*** kashyapk has joined #openstack-swift12:53
*** mkollaro1 has quit IRC12:56
*** bkopilov has quit IRC12:59
*** miqui has joined #openstack-swift13:05
*** mlipchuk has quit IRC13:29
*** matsuhashi has quit IRC13:39
*** judd7 has joined #openstack-swift13:40
*** kashyapk has quit IRC13:41
*** grapsus_ is now known as grapsus13:49
*** grapsus is now known as grapsus_13:50
*** nosnos has quit IRC13:50
*** mlipchuk has joined #openstack-swift13:51
*** Ju_ has joined #openstack-swift14:06
creihtnotmyname, clayg:  What is the current timeline look like for policies?14:14
*** miqui has quit IRC14:23
*** miqui has joined #openstack-swift14:23
*** miqui has quit IRC14:23
*** miqui has joined #openstack-swift14:23
*** mlipchuk has quit IRC14:24
*** donagh has joined #openstack-swift14:27
openstackgerritOpenStack Proposal Bot proposed a change to openstack/python-swiftclient: Updated from global requirements  https://review.openstack.org/8925014:40
openstackgerritOpenStack Proposal Bot proposed a change to openstack/swift: Updated from global requirements  https://review.openstack.org/8873614:40
*** NM has joined #openstack-swift14:40
*** mlipchuk has joined #openstack-swift14:51
*** diegows has quit IRC14:52
openstackgerritpaul luse proposed a change to openstack/swift: Mock time for audit test 'test_sleeper'  https://review.openstack.org/9853815:03
notmynamecreiht: current plan: merge this week and then RC for 2 weeks. I'm working the non-code parts for this now15:04
notmynameI'm hoping that the 2-dimentional time patch will be proposed today, so please look at it when you can :-)15:05
creihtahh cool, any other outstanding patches besides that15:06
notmynameSP related? are you comfortable with the rest that's there?15:08
*** kevinc_ has joined #openstack-swift15:09
creihtnotmyname: yeah SP related... just wanted to see if there are any other outstanding things coming in before I made another pass through15:10
openstackgerritpaul luse proposed a change to openstack/swift: Change assertCalledWith to assert_called_with  https://review.openstack.org/9912415:11
*** mkollaro has quit IRC15:12
*** mkollaro has joined #openstack-swift15:13
*** mlipchuk has quit IRC15:23
*** nshaikh has quit IRC15:30
*** gyee has joined #openstack-swift15:31
*** mkollaro has quit IRC15:35
*** nacim has quit IRC15:43
claygcreiht: torgomatic and yuan have some updates to the bin scripts (get-object-info and swift-get-nodes)15:43
*** kevinc_ has quit IRC15:53
*** byeager_away is now known as byeager15:53
acolesnotmyname: or clayg: which patch in the chain will the 2D time stuff show up in? or is it a new patch?15:58
*** kevinc_ has joined #openstack-swift16:00
*** mlipchuk has joined #openstack-swift16:01
claygacoles: i haven't really decided yet16:02
claygacoles: suggestion?16:02
claygit's all over the map -> https://gist.github.com/clayg/66bb3c049b629fb0eca616:03
acolesclayg: nah, i just want to make sure i look out for it16:03
*** byeager is now known as byeager_away16:09
* clayg was hoping acoles had a plan since this whole thing is his fault anyway16:09
* peluse_ smiles16:09
acolesclayg: i need more time :)16:10
claygpeluse_: well not the *whole* thing - giant big portions of this are peluse_'s fault16:10
claygacoles: but at least he bought me a beer16:10
peluse_and that's really what its all about!16:10
acolesclayg: come to uk and i'll buy you beer16:12
peluse_hey, I'm in!16:12
* acoles doesn't mention that beer here comes warm16:12
claygafter getting probetests and functests passing with the new timestamp format - does anyone want to guess how many unittest failures are left?16:15
peluse_double digits or more?16:16
*** shakayumi is now known as shakayumi_afk16:17
clayg~60 if i force the new format even when the offset is 0 and equal to the normalized form - but surprisingly only 16 if i treat the zero offset as the normalized form (which is what'd you do for upgrades)16:18
claygthen there's gunna have to be an audit16:18
*** shakayumi_afk has quit IRC16:19
*** diegows has joined #openstack-swift16:19
*** mwstorer has joined #openstack-swift16:22
*** mwstorer has quit IRC16:23
anticwis there a (simple) way to run a subset of the tests against a real swift cluster?16:25
clayganticw: functests run against a real cluster16:26
clayganticw: you can do standard nose test selection at that point16:26
*** diegows has quit IRC16:26
anticwclayg: awesome, i was hoping you would say that16:27
clayganticw: but functests are fast - you should run all of them16:27
anticwrunning them all would be fine, i'll poke about and see if i can get them working16:28
*** diegows has joined #openstack-swift16:38
*** shri has joined #openstack-swift16:39
portantenotmyname, clayg: hopefully SP patches won't be merged today or wednesdays, as I am behind here at the office reviewing them, and in general because we are out for the second half of the day celebrating the RHEL 7 launch16:44
notmynameportante: oh I see how it is. go party instead of get storage policies reviewed ;-)16:50
portanteyes!  glad I communicated that well!16:50
torgomaticis that sarcasm or instructions? because I'm 100% on-board with those being instructions.16:50
peluse_lord knows I'll do my part :)16:51
notmynameI've been chatting with mordred about how to get the stuff merged. I think we won't be clicking merge today or tomorrow, but targeting thursday (with the flexibility of friday or monday) is reasonable I think16:53
*** NM has quit IRC16:53
notmynameand we'll have an upstream branch that clayg will propose the stuff to after the patches are reviewed. then we'll only gate the merge commit instead of all the 27 patches16:53
notmynamemordred: ^^ please confirm :-)16:54
acolesnotmyname: sounds good16:54
notmynameso far, we've got soft QA commitments from HP, RAX, Red Hat, softlayer (thanks briancline!), and I'll be reaching out to the NeCTAR guys today16:55
notmynamepeluse_: mlanner and erik are putting the new mobos in right now16:57
peluse_notmyname: sweet16:58
notmynamepeluse_: they aren't cursing your name too much ;-)16:59
*** shri1 has joined #openstack-swift17:01
*** shri1 has quit IRC17:02
mordrednotmyname: yes. the actually mechanics may vary slightly, but that's the idea17:03
*** shri has quit IRC17:04
*** nsquare has joined #openstack-swift17:09
peluse_notmyname:  remind them that I'm just a SW guy.. :)17:11
*** judd7 has quit IRC17:18
*** NM has joined #openstack-swift17:22
*** tdasilva has joined #openstack-swift17:30
claygi'm having a bit of exestential crisis with the timestamp tests... there's lots of tests that send in denormalized x-timestamps and the assert they get them back - which is sorta ok, but then like i want to make sure those methods get called for realzy's with normalized timestamps - or force them to re-normalize "just in case"17:34
claygwhich seems wasteful17:34
claygmaybe better safe than sorry?17:35
claygi'll think about it on the train - bbiab17:35
*** mmcardle has quit IRC17:36
*** bkopilov has joined #openstack-swift17:49
kevinc_When expanding into a new rack, is it common practice to add the new servers that will be in that rack into a new zone?17:51
*** haomaiwang has quit IRC17:52
notmynamekevinc_: yes that's common. but do whatever is right for your deployment. it's common to treat a rack as a zone in swift, since it normally is a physical failure domain (eg single ToR switch or power source)17:53
kevinc_ok, thanks that is what I thought. Is there any downside to having 3 object copies over 5 zones? or is it best to expand it multiples of 3?17:55
notmynamekevinc_: no. keep three replicas (good price vs durability/availability metric) and have as many zones as you have failure domains17:57
notmynamekevinc_: in fact, swift gets better if you have more zones than replicas17:57
notmyname(in both statements above, I'm not considering global clusters)17:58
notmynamekevinc_: swift doesn't have any preferences for having a multiple of the replica count number of zones17:59
kevinc_Perfect, thank you very much that was exactly what I needed to know :)17:59
notmynameso github is having some problems18:07
*** kevinc_ has quit IRC18:11
*** kevinc_ has joined #openstack-swift18:16
*** shri has joined #openstack-swift18:21
*** mlipchuk has quit IRC18:28
claygboo github18:36
*** zaitcev has joined #openstack-swift18:43
*** ChanServ sets mode: +v zaitcev18:43
anticwportante / dfg :  i'm using a slight variant of the xLO fixes as proposed in a test lab and so far it's looking very good18:54
anticwnotmyname: i'm thinking we should try to get this in for 1.13.2 - as it stands we have a regression between 1.12.0 and 1.13.x18:54
*** gyee has quit IRC19:23
*** kr4zy has joined #openstack-swift19:30
kr4zyanyone here have experience setting up haproxy for ssl encrypted swift?19:30
notmynamekr4zy: what's your question?19:31
zaitcevanticw: can you post the slight differences you mentioned?19:31
* briancline would also be interested in the regression details19:32
kr4zyI have two load balancers running haproxy. I have two proxy servers running Apache2 reverse proxy for SSL termination. I am trying to set up load balancing and HA for them. Both my HAProxies are using SSL Passthrough. Currently, I have to execute swift list twice to see the containers initially. I am not sure what is causing the problem.19:34
kr4zyhere's my haproxy setting: https://gist.github.com/anonymous/08eae701569b753c4c9619:37
*** byeager has joined #openstack-swift19:40
brianclinekr4zy: silly question but I have to ask anyway: the sums for the account and container rings on both proxies match, right?19:40
kr4zyI also noticed that the client is always going to one haproxy. Is my configuration wrong?19:41
brianclinemy initial thought is that's probably due to sticking via payload_lv19:45
brianclinesince you're specifying round robin, are you sure you want to stick at all?19:47
kr4zyI am new to HAProxy. My goal is to set up some kind of load balancing and have HA19:48
brianclineahh, well I'd simplify your swift backend block - try commenting out the stick-table, stick, acl, tcp-request and tcp-response statements and reload19:53
*** kevinc_ has quit IRC19:56
brianclineas for the container listing issue, do you see the same thing if you hit both of the proxies directly?19:57
brianclineso uh... swift config vs. devstack config question -20:00
*** NM has quit IRC20:00
*** NM has joined #openstack-swift20:01
brianclinethe sample proxy config says to put formpost before the auth filter in the pipeline, however devstack's swift_config is actually placing it *after* tempauth/keystoneauth in the pipeline it sets up20:02
notmynamebriancline: I'd defer to the sample config20:02
brianclinenotmyname: yeah, I trusted that more -- however with us having gating devstack jobs, I'm curious if we're overriding that or if it's perhaps gone undetected in the tests20:04
notmynameAFAIK there is no test that validates the order of the pipeline (handwavy about the gatekeeper)20:04
portanteanticw: back, can you share your slight changes?20:06
*** NM has quit IRC20:09
*** kevinc_ has joined #openstack-swift20:09
brianclinenotmyname: sorry, I meant more on the functional test side -- do we do any formpost functional tests? I had a cursory glance but couldn't find any20:10
brianclineI see the unit tests20:10
portantebriancline: I don't think so, as the formpost middleware is not in the pipeline for the functional tests by default20:12
portantebut I might be missing something20:12
notmynamethat sounds right20:13
*** NM has joined #openstack-swift20:13
openstackgerritThiago da Silva proposed a change to openstack/swift: Refactoring functional tests  https://review.openstack.org/9264320:14
*** byeager has quit IRC20:16
kr4zybriancline: I am now using this https://gist.github.com/anonymous/a676df11d2350edfdb28. The container that I created seems to disappear after a haproxy service restart.20:17
*** miqui has quit IRC20:20
*** jamie_h has joined #openstack-swift20:23
*** jamie_h_ has quit IRC20:25
*** byeager has joined #openstack-swift20:30
*** byeager_ has joined #openstack-swift20:30
notmynameif you aren't currently looking at storage policy patches, take a look at torgomatic's small patch at https://review.openstack.org/#/c/98530/20:32
* notmyname is looking at the reviewed-but-not-merged patches and other patches that should be included in the release with storage policies20:33
*** goodes- has quit IRC20:33
*** MooingLemur has quit IRC20:33
*** esmute has quit IRC20:33
*** cschwede_ has quit IRC20:33
*** jokke_ has quit IRC20:33
*** jokke_ has joined #openstack-swift20:33
*** esmute has joined #openstack-swift20:33
*** MooingLemur has joined #openstack-swift20:33
notmynameanticw: BTW, that xLO patch is reviewed, but hasn't been merged because of the soft freeze with SP. it will land for the next release with storage policies20:33
*** goodes has joined #openstack-swift20:33
*** byeager has quit IRC20:34
*** cschwede has joined #openstack-swift20:35
*** byeager_ has quit IRC20:37
*** byeager has joined #openstack-swift20:37
*** jokke__ has joined #openstack-swift20:38
notmynamealso https://review.openstack.org/#/c/97828/20:43
notmynameactually, just take a look at the bottom of https://wiki.openstack.org/wiki/Swift/PriorityReviews for a set of patches that needs one more review20:46
notmynameplease don't merge them yet, but approve them if they are ready20:47
notmynamewe'll have a merge fest between thursday and monday to get everything landed for the RC20:47
*** gyee has joined #openstack-swift20:49
*** gyee has quit IRC20:51
*** gyee has joined #openstack-swift20:53
*** jokke_ has quit IRC20:54
*** jamie_h has quit IRC20:54
*** omame has quit IRC20:54
*** openstackgerrit has quit IRC20:54
*** hugokuo has quit IRC20:54
*** kevinc_ has quit IRC21:00
*** foexle has quit IRC21:03
*** kevinc_ has joined #openstack-swift21:04
*** omame has joined #openstack-swift21:11
*** openstackgerrit has joined #openstack-swift21:11
*** hugokuo has joined #openstack-swift21:11
*** yuan has quit IRC21:13
*** nthacker has joined #openstack-swift21:13
*** yuan has joined #openstack-swift21:13
*** nthacker_ has joined #openstack-swift21:19
nthacker_Hi All - i want to use unittest to verify if an exception is raised. So i would use assertRaises, but I have a problem here. maybe this is a basic python question21:20
nthacker_the snippet of code is:21:20
nthacker_except OSError as e:21:20
nthacker_if e.errno != errno.ENOTDIR21:20
nthacker_now to check if the exception is raised, it appears i have to inject an object that will enter into the if loop21:21
nthacker_and the error that the object should generate cannot be an ENOTDIR21:21
nthacker_it also cannot be an ENOENT21:21
nthacker_so now how will i inject a different errno to therefore check if this exception is raised21:22
zaitcevset a scope variable21:22
nthacker_zaitcev: I dont follow - could you elaborate21:22
nthacker_or show me an example?21:22
zaitcevexcept OSError as e:21:23
zaitcev    was_raised=True21:23
nthacker_zaitcev: the snippet of code that i showed is the one against which i am running  a unittest. basically this is a suggestion from peluse_ - you can  see it here: https://review.openstack.org/#/c/97148/21:24
nthacker_so if i was to add a scope variable to the snippet, i'd be modifying the code i am testing against21:25
zaitcevOkay. Why cannot you put  raise OSError(errno.EPERM) into your mocker?21:27
nthackerzaitcev, ah ok, so that requires i use python mock correct? im new to that too, but i did read something about it21:29
nthackerzaitcev, sorry im using multiple clients, but this is still me21:29
*** nthacker_ has quit IRC21:30
*** NM has quit IRC21:30
anticwnotmyname: next release being?21:32
notmynameanticw: soon21:32
notmynameone of the numbers in the existing version string will get one bigger ;-)21:33
anticwportante: slight changes are required because we don't actually use keystone21:33
anticwnotmyname: i meant are we talking 1.14.x which is a big jump ... or 1.13.2 ?21:33
anticwi think small fixes justify the latter21:33
notmynameanticw: the next release will include storage policies and the existing reviewed-but-not-yet-merged-because-of-freeze patches. the scope of the work in this release, as discussed in-person at the hackathon, will result in a 2.0 version string21:35
notmynamewhich I'm not yet broadly proclaiming21:35
anticwi can live with it either way - but i would argue that 1.13.x has regressed since 1.12 in this context21:36
anticwso surely there is reason to consider a 1.13.2?21:36
zaitcevnthacker: I'll be darned. You have to do  raise OSError(errno.EPERM, "moo"), or else the first argument goes to strerr not errno.21:36
notmynameanticw: is the regression documented in a LP bug?21:36
anticw(i don't much care - neither outcome helps me directly, we've done our version of things and will sit on that for a bit)21:36
anticwnotmyname: i don't think so - i'm not sure if anyone other than me hit it21:36
anticwthough the xLO token-expiration i assume is21:37
anticw(diff bug, same fix)21:37
notmynameanticw: I'd appreciate it if you'd document it there. that patch references only https://bugs.launchpad.net/swift/+bug/131513321:37
nthackerzaitcev, im confused - are you proposing that the mock test raise the assert so the unittest may catch it?21:37
notmynameand having it in LP makes it helpful to others who might see it. also those trying to repro it21:38
zaitcevnthacker: what else could you be doing? My first answer presumed that you allowed it to happen naturally, but you didn't like it.21:38
anticwnotmyname: i will in a few, working over the URL stream21:39
notmynameanticw: thanks21:40
anticwnotmyname: i don't think commenting on the original bug makes sense, so unless you prefer that i think it's better to make a new bug21:40
nthackerzaitcev, OK maybe i have to work a bit more with my head down to understand how i can raise this assert using mock. The way I was trying to handle it, was by giving the target routine an object that would generate neither an ENOENT, nor an ENOTDIR. So anything in the list of errno's except the latter 2, however that seems more complicated than your suggestion21:40
notmynameanticw: agreed. a new bug seems the right thing21:41
kr4zyis it possible to implement ha for haproxy doing ssl passthrough? My current haproxy ssl passthrough doesn't failover to the second proxy server.21:47
anticwnotmyname: https://bugs.launchpad.net/swift/+bug/132869921:48
anticwprobably needs rewording a little21:48
notmynameanticw: is this only for keystone or does it affect tempauth too?21:50
peluse_nthacker:  I think what zaitcev was suggesting would be to mock lisdir just for the one test where you want to cover that LOC, obviously mocking the function under test itself would make no sense :)  I can send you an example if you want21:50
nthackerpeluse_, ah ok! that now makes sense21:51
nthackerpeluse_, no i get it now - thanks for clearing that bit - was like an aha moment :)21:52
anticwnotmyname: i think only keystone21:52
anticwnotmyname: and probably ssauth :-)21:52
notmynameanticw: the manifest container and the segments container have different ACLs? that's what triggers it?21:53
notmynameor is it that the request is now mangled such that the original creds aren't active and therefore it fails21:54
anticwactually, that's not required21:54
anticwi thought it was21:54
peluse_nthacker: cool, no problem. only other small thing is that you'll only want your mocked func to raise when its passed the datadir_path.  Let me know how it goes, have fun!21:54
anticweven if you have the same ACLs21:54
anticwthe lookup-logic for keystone breaks for the segment access21:54
anticwdetails in the scroll-back ... i'm happy to go over it again but if people here prefer we can take this to /msg or elsewhere ?21:55
*** kevinc_ has quit IRC21:56
notmynameanticw: I'll check on my saio. (also, it's not like there is a flood of chatter in here)21:57
anticwnotmyname: in short, dlo.py: https://github.com/openstack/swift/blob/master/swift/common/middleware/dlo.py#L3721:58
anticwsome headers are copied, not all ... for the dlo segment access21:58
anticwhttps://github.com/openstack/swift/blob/master/swift/common/middleware/keystoneauth.py#L180 calls https://github.com/openstack/swift/blob/master/swift/common/middleware/keystoneauth.py#L13521:58
anticwwhich has a check on HTTP_X_IDENTITY_STATUS - that won't exist in the case of a segment access21:59
anticwso that code does a naked return - which is a None ...  authorize then tries to do:  tenant_id, tenant_name = env_identity['tenant']21:59
anticwand None['xxx']  blows up21:59
anticwlmk if you need more details, not sure how best to explain it tersely and accurately22:00
notmynameI think that was pretty good :-)22:02
*** byeager has quit IRC22:05
*** byeager has joined #openstack-swift22:05
*** byeager has quit IRC22:09
*** byeager has joined #openstack-swift22:11
*** byeager has quit IRC22:14
*** byeager has joined #openstack-swift22:14
notmynametorgomatic: ^^ FYI and please review22:17
notmynameanticw: what's the pipeline order you have? dlo before or after auth?22:21
*** byeager has quit IRC22:26
*** byeager has joined #openstack-swift22:27
notmynamewe had 3 patches proposed for a one character patch to the proxy config man page22:30
peluse_good attention to detail :)22:35
*** kr4zy has quit IRC22:38
portanteanticw: the xLO patch fixes that as well22:40
portantethat can no longer happen with that code22:40
* notmyname has his head around it now (thanks for the patience torgomatic) and understands22:41
notmynameand also, since this was a regression during icehouse and the bug is present in 1.13.1 (icehouse), I'm leaning towards backporting it. assuming the openstack gods are with me there22:42
* portante gotta head home, will be available later22:42
*** kevinc_ has joined #openstack-swift22:43
*** dmsimard is now known as dmsimard_away22:43
*** byeager_ has joined #openstack-swift22:45
*** byeager_ has quit IRC22:46
*** byeager has quit IRC22:48
notmynamemattoliverau: good morning22:56
notmynamemattoliverau: thanks for doing so many reviews!22:56
mattoliveraunotmyname: You welcome, In atlanta you guys mentioned you were in need of more reviews especially with all the storage policy code coming through. So thought I'd do what I can to help :)22:58
notmynamemuch appreciated. I know it's hard to participate when you're on the opposite time zone of most of us. please let me know how I can make it easier23:00
mattoliveraunotmyname: thanks, it is and I really appreciate that! but at least I'm awake when most of you sleep, so once I'm up to speed of all things swift I'll be able to help in channel during the grave yard shift :P23:02
notmynametorgomatic: https://pbs.twimg.com/media/Bn3MoZQCYAAvJmZ.jpg23:22
*** kevinc_ has quit IRC23:39
notmynametorgomatic: `git show c2744caac` and `git show f63b58f5b`  you'd think we'd learn :-)23:42
torgomaticI KEEP TRYING :)23:42
notmynameanticw: FWIW, swiftstackauth isn't affected :-)23:47
notmynameAlex_Gaynor wins the award for vague commit messages ;-) `git show 032f0bfc7c6` and `git show 181e9496f`23:51
* torgomatic is going to start using http://whatthecommit.com/ from now on23:52
Alex_Gaynornotmyname: haha, my 9th grade english teacher once called me "the master of ambiguity"23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!