Monday, 2018-04-30

*** edmondsw has joined #openstack-keystone00:28
*** edmondsw has quit IRC00:33
empty_cupwhen operating devstack, should it run from the master branch?00:57
empty_cupis the master branch a "develop" branch?00:58
empty_cupi'm rerunning ./stack.sh while on the stable/queens branch. Originally it was situated on master.01:05
empty_cupWhat prompted this path was the user showing as unauthenticated when originally set with a password and options showing up as empty01:06
*** edmondsw has joined #openstack-keystone02:16
*** edmondsw has quit IRC02:21
*** d0ugal has quit IRC02:49
*** d0ugal has joined #openstack-keystone02:50
*** empty_cup has quit IRC03:51
*** dikonoor has joined #openstack-keystone03:51
*** pooja_jadhav has joined #openstack-keystone04:30
*** links has joined #openstack-keystone04:39
*** pcichy has quit IRC04:55
*** jaosorior has joined #openstack-keystone05:12
*** belmoreira has joined #openstack-keystone05:37
*** masber has joined #openstack-keystone05:41
*** d0ugal has quit IRC05:48
*** masuberu has joined #openstack-keystone05:51
*** edmondsw has joined #openstack-keystone05:53
*** d0ugal has joined #openstack-keystone05:54
*** masber has quit IRC05:55
*** edmondsw has quit IRC05:58
*** masuberu has quit IRC06:02
*** martinus__ has joined #openstack-keystone06:33
*** Guest1988 has joined #openstack-keystone07:10
*** Guest1988 has quit IRC07:17
*** hoonetorg has quit IRC07:41
*** edmondsw has joined #openstack-keystone07:42
*** edmondsw has quit IRC07:47
*** hoonetorg has joined #openstack-keystone07:55
*** panbalag has joined #openstack-keystone09:30
*** Horrorcat has left #openstack-keystone10:12
*** Horrorcat has joined #openstack-keystone10:17
*** Horrorcat has left #openstack-keystone10:23
*** Horrorcat has joined #openstack-keystone10:28
*** nicolasbock has joined #openstack-keystone10:30
*** raildo has joined #openstack-keystone10:45
openstackgerritMerged openstack/keystone master: Invalidate the shadow user cache when deleting a user  https://review.openstack.org/56190810:53
*** panbalag has quit IRC11:14
*** aloga has quit IRC11:14
*** dave-mccowan has joined #openstack-keystone11:27
*** dave-mccowan has quit IRC11:31
*** mvk has quit IRC11:32
*** dave-mcc_ has joined #openstack-keystone11:33
*** edmondsw has joined #openstack-keystone12:07
*** ninag has joined #openstack-keystone12:22
*** ninag has quit IRC12:22
*** mvk has joined #openstack-keystone12:28
*** panbalag has joined #openstack-keystone12:41
*** panbalag has left #openstack-keystone12:42
*** jaosorior has quit IRC12:43
*** mchlumsky has joined #openstack-keystone12:44
*** mchlumsky has quit IRC13:07
*** mchlumsky has joined #openstack-keystone13:09
*** panbalag has joined #openstack-keystone13:17
*** panbalag has left #openstack-keystone13:20
*** links has quit IRC13:22
gagehugoO/13:24
*** superdan is now known as dansmith13:29
*** ayoung has quit IRC13:30
*** belmorei_ has joined #openstack-keystone13:31
*** belmoreira has quit IRC13:34
*** jroll has quit IRC13:36
*** lbragstad has joined #openstack-keystone13:37
*** ChanServ sets mode: +o lbragstad13:37
*** jroll has joined #openstack-keystone13:38
*** jmlowe_ has quit IRC13:45
*** panbalag has joined #openstack-keystone13:47
*** panbalag has left #openstack-keystone13:47
*** r-daneel has joined #openstack-keystone13:47
hrybackio/13:55
kmallocZzzz13:56
kmalloc;)13:56
lbragstado/14:00
*** felipemonteiro has joined #openstack-keystone14:05
*** aloga has joined #openstack-keystone14:06
*** felipemonteiro_ has joined #openstack-keystone14:07
*** felipemonteiro has quit IRC14:11
openstackgerritMatt Riedemann proposed openstack/oslo.policy master: make the sphinxpolicygen extension handle multiple input/output files  https://review.openstack.org/56462714:11
*** jmlowe has joined #openstack-keystone14:21
*** panbalag has joined #openstack-keystone14:29
*** panbalag has left #openstack-keystone14:39
*** spilla has joined #openstack-keystone14:53
*** felipemonteiro_ has quit IRC15:19
mordredkmalloc, lbragstad there's a failure in the openstacksdk patch that I made to have it consume the ksa alias/discovery stuff that looks real - although it looks VERY strange15:22
kmallocHmm15:22
mordredkmalloc, lbragstad: tl;dr - it looks like cinder endpoints in devstack are being discovered incorrectly - missing the version15:22
kmallocDoh!15:23
mordredI'm digging in now to figure out if it's a bug in ksa or a bug in sdk's use of ksa15:23
kmallocThis is KSA master or a release?15:23
mordredmaster15:23
mordredthis is the "test the new ksa patches with shade/sdk before landing them"15:23
kmallocOk. Good hope that means no release with the bug. ;)15:24
kmallocAlso yay for that test.15:24
mordredhttp://logs.openstack.org/94/564494/1/check/openstacksdk-functional-devstack-tips/52ba4b2/testr_results.html.gz is the failed test run15:25
knikollao/15:25
mordredkmalloc: yah. I mean, also it's a patch to remove discovery related logic from sdk and replace it with just passing parameters to ksa ... which is awesome - but it's entirely possible there's a nuance in that patch that's bong15:26
kmallocWalking teh dog, those.logs don't load well (too big) on mobile (any logs)15:26
kmallocWill look as soon as I am home.15:26
mordredkmalloc: bah. get a bigger phone15:26
kmallocIt is actual data volume15:26
kmallocThis is a pixel XL 2. Chrome just crashes with our logs15:26
kmalloc(it is also a bad phone imo)15:26
mordredwell, you could use a laptop as a phone and then you could look at logs on your phone while you walk the dog15:27
kmallocOh that is the report not the full log15:28
kmallocThat loaded ok15:28
mordredthe first failure actually looks like it's doing the right things (ignore for a sec the fact that it's a test of v2 and it's talking to v3)15:30
kmallocAhh15:30
kmallocHehe15:30
*** Horrorcat has left #openstack-keystone15:36
*** Horrorcat has joined #openstack-keystone15:41
*** ayoung has joined #openstack-keystone15:43
*** Horrorcat has left #openstack-keystone15:47
*** Horrorcat has joined #openstack-keystone15:52
*** belmorei_ has quit IRC16:03
*** Kumar has joined #openstack-keystone16:11
*** felipemonteiro has joined #openstack-keystone16:12
*** Kumar has quit IRC16:21
*** felipemonteiro_ has joined #openstack-keystone16:26
openstackgerritLance Bragstad proposed openstack/keystone master: Use the provider_api module in limit controller  https://review.openstack.org/56271216:26
openstackgerritLance Bragstad proposed openstack/keystone master: Add configuration option for enforcement models  https://review.openstack.org/56271316:26
openstackgerritLance Bragstad proposed openstack/keystone master: Add policy for limit model protection  https://review.openstack.org/56271416:26
openstackgerritLance Bragstad proposed openstack/keystone master: Implement enforcement model logic in Manager  https://review.openstack.org/56271516:26
openstackgerritLance Bragstad proposed openstack/keystone master: Expose endpoint to return enforcement model  https://review.openstack.org/56271616:26
*** felipemonteiro has quit IRC16:29
*** itlinux has joined #openstack-keystone16:33
* lbragstad goes to take a run over lunch16:43
*** felipemonteiro_ has quit IRC16:46
*** felipemonteiro_ has joined #openstack-keystone16:46
*** mchlumsky has quit IRC16:56
*** mchlumsky has joined #openstack-keystone16:58
*** mchlumsky_ has joined #openstack-keystone17:02
*** mchlumsky has quit IRC17:03
*** mchlumsky has joined #openstack-keystone17:07
*** mchlumsky_ has quit IRC17:07
*** dikonoor has quit IRC17:18
*** itlinux has quit IRC17:29
*** r-daneel has quit IRC18:06
*** r-daneel has joined #openstack-keystone18:06
*** mvk has quit IRC18:14
*** itlinux has joined #openstack-keystone18:17
*** dave-mcc_ has quit IRC18:21
*** dave-mccowan has joined #openstack-keystone18:28
*** felipemonteiro__ has joined #openstack-keystone18:29
*** sonuk has joined #openstack-keystone18:30
*** felipemonteiro_ has quit IRC18:32
*** mvk has joined #openstack-keystone18:47
lbragstaddims: just got done parsing the google doc link you added to the jwt spec18:57
lbragstadit sounds like they are iterating on jwt to provide service tokens?18:57
dimsright, this one seems to be under discussion still18:57
lbragstadso a workload is equivalent to a service then?18:58
lbragstadi'm still trying to build a mapping of terms18:59
lbragstadit appears they're looking to do nested jwts to limit the power of service scoped token?19:03
*** sonuk has quit IRC19:04
*** Horrorcat has left #openstack-keystone19:05
*** jmlowe has quit IRC19:07
*** itlinux has quit IRC19:07
dims@lbragstad : i am not sure :) we may have to go to one of their meetings may be in a week or so (this week is KubeconEU)19:08
lbragstadoh - right19:09
lbragstaddims: is there a spec for just the jwt work they did?19:09
lbragstadthis seems like it's reusing some of that to solve a different problem, so i'm just wondering if there is another document that has more context19:10
dims@lbragstad : let me check19:14
dims@lbragstad : https://github.com/kubernetes/community/pull/1460/commits/6e209490c441d8df84b6b5d8e352c0e2491a41bd may be?19:19
dims@lbragstad : was looking through notes from the container identity work group - https://docs.google.com/document/d/1uH60pNr1-jBn7N2pEcddk6-6NTnmV5qepwKUJe9tMRo/edit?ts=59a03344#heading=h.n00r55m5f4gw19:20
*** jmlowe has joined #openstack-keystone19:23
openstackgerritLance Bragstad proposed openstack/keystone master: Implement enforcement model logic in Manager  https://review.openstack.org/56271519:25
openstackgerritLance Bragstad proposed openstack/keystone master: Expose endpoint to return enforcement model  https://review.openstack.org/56271619:25
lbragstaddims: thanks19:25
*** d0ugal has quit IRC19:45
*** d0ugal has joined #openstack-keystone19:46
*** d0ugal_ has joined #openstack-keystone20:00
*** dave-mccowan has quit IRC20:02
*** d0ugal has quit IRC20:02
*** itlinux has joined #openstack-keystone20:06
*** pcichy has joined #openstack-keystone20:12
*** felipemonteiro__ has quit IRC20:12
*** dave-mccowan has joined #openstack-keystone20:17
*** itlinux has quit IRC20:17
*** d0ugal_ has quit IRC20:18
*** itlinux has joined #openstack-keystone20:19
*** d0ugal_ has joined #openstack-keystone20:19
*** jmlowe has quit IRC20:21
*** d0ugal_ has quit IRC20:36
*** itlinux has quit IRC20:36
*** raildo has quit IRC20:39
*** d0ugal_ has joined #openstack-keystone20:43
*** pcichy has quit IRC20:45
*** dave-mccowan has quit IRC20:55
lbragstadanyone around to talk limits?20:56
*** martinus__ has quit IRC21:02
*** edmondsw has quit IRC21:07
*** edmondsw_ has joined #openstack-keystone21:11
*** edmondsw_ has quit IRC21:15
*** d0ugal_ has quit IRC21:32
ayounglbragstad, I am, but I don't think I count.  Have not really thought about limites in a while.21:35
ayoungJust need a sounding board, fire away21:35
*** d0ugal_ has joined #openstack-keystone21:35
* lbragstad thanks ayoung for being his rubber duck21:37
lbragstadok21:37
lbragstadi'm reviewing the current specification for CERNs limit use cases21:37
lbragstadhttps://review.openstack.org/#/c/540803/9/specs/keystone/rocky/hierarchical-unified-limits.rst21:37
lbragstadit's expected to be a "strict" model, it in that appears to lean on the side of being explicit versus implicit21:38
*** d0ugal_ has quit IRC21:38
ayoungGood21:38
lbragstadin queens we implemented registered limits and project limits21:38
*** d0ugal_ has joined #openstack-keystone21:39
lbragstadwhere registered limits are something that need to be created before a limit can be associated to a project21:39
ayoung"A child may have no more quota than it's parent. The21:39
ayoung                        sum of all **direct children** is must be <= the21:39
ayoung                        parent. all children would have to have default quota21:39
ayoung                        equal to its parent's."21:39
ayoung    }21:39
lbragstadthey also act as a default21:39
lbragstadso an operator could registered a limit of 10 cores for the compute service for a deployment21:39
ayoungSo...management of Quota happens inside Keystone, where we have full access to all data.  Enforcement makes a call to Keystone?21:40
lbragstadyeah - pretty much, the limit and it's validation is stored within keystone21:40
ayounggo on...I have about 5 minutes21:40
lbragstadand exposed to services and consumed via a library to make it easier to adopt/understand21:40
lbragstadso - given a registered limit of 10 cores21:41
lbragstadeach project without an explicit limit "override" or project limit21:41
lbragstadwill assume a default limit of 10 core21:41
lbragstadcores*21:41
lbragstadnow - imagine you have project A, whose limit is 20 and usage is 121:41
lbragstadand A has two children projects21:41
lbragstadB and C21:41
lbragstadneither have project overrides21:42
lbragstadso - they assume the default value for cores set by the registered limit21:42
lbragstad(e.g. 10 a piece)21:42
ayounga piece?21:42
lbragstadyeah - becaues of the registered limit21:42
*** felipemonteiro has joined #openstack-keystone21:42
ayoungso the default is each project gets 10, including child projects21:42
lbragstadlet's say the parent is A in this example and it has a project limit of 20 cores21:43
lbragstad(so overriding the default21:43
ayoungand that is OK, cuz there are two, and the parent project has 2021:43
lbragstadand B and C are siblings21:43
lbragstadok - here's where it get's confusing21:43
lbragstad(for me at least)21:43
lbragstadshould you be able to create project D, which is a sibling to B and C without a project limit override?21:44
ayoungI never found a solution to this problem that I liked21:44
lbragstadbecause SUM(B.limit, C.limit, D.limit) > A.limit21:44
ayoungI would say yes21:44
lbragstadok - so let's say we can do that21:45
lbragstadwe now have B, C, and D under A21:45
ayoungSo...21:45
lbragstadall children are relaying on the default value of 10 cores and A's limit is still 2021:45
*** felipemonteiro_ has joined #openstack-keystone21:46
ayoungIf A has a quoate of 20, and  A creates B and assignes to it 10, A then has 10 remaining...right?21:46
ayoungit is tracked that way?21:46
lbragstadyeah - i think so21:46
lbragstadso - let's assume21:47
lbragstadA.usage = 121:47
lbragstadB.usage = 221:47
lbragstadC.usage = 1021:47
ayoungBut usage is not tracked in Keystone21:47
lbragstadright...21:47
ayoungonly in Nova21:47
lbragstadthis is where is get's really fuzzy21:47
lbragstadD.usage = 021:47
lbragstadlet's say you want to use 10 cores in D21:48
lbragstadand the interface for oslo.limit from the service handing out cores is:21:48
lbragstadenforce(project_usage, project_id)21:49
lbragstadso, leaving oslo.limit to use the project ID get the hierarchy of limits from keystone and evaluate the usage to say "yes" or "no"21:49
lbragstaddon't we have a circular dependency because oslo.limit still needs more usage information from the other children in the tree to make the decision?21:50
*** felipemonteiro has quit IRC21:50
lbragstadthe current method signature doesn't allow oslo.limit to determine if (A.usage + B.usage + C.usage + D.usage) <= A.limit (which it finds by querying keystone for limit information)21:51
lbragstadto me, it seems like we have to problems... 1.) there is a certain level of ambiguity in the model and 2.) requiring services to pass in *all* resources and their owning projects to oslo.limit is going to be painful21:57
*** d0ugal_ has quit IRC21:57
lbragstads/to problems/two problems/21:58
*** d0ugal_ has joined #openstack-keystone21:59
ayounglbragstad, yes, that is the problem with usage not being recorded in Keystone. That has always been the tough-to-solve problem22:04
lbragstadright22:04
*** jmlowe has joined #openstack-keystone22:05
lbragstadso - what about this22:05
* lbragstad whips out the crazy idea notebook22:05
ayoungI had thought about an idea of a resource pool22:05
ayoungso you have a unified identifier for the quota.  Additional level of indirection22:05
ayoungparent project ID could be the resource pool id by default22:06
lbragstadwhat if all limit validation operations assumed the registered limit for all projects without an explicit override?22:06
ayoungnot sure it is a solution22:06
lbragstadayoung: the resource pool?22:06
ayoungI need to parse that22:06
lbragstaddoes the resource pool just push usage the leaves of the tree?22:06
ayoungyeah, resource pool requires the projects to keep track of what project is in what resource pool22:06
ayoungI think that was why I discarded it last time It came up...let me turn to your statement22:07
lbragstadok22:07
ayoungSo...I kindof think that all quotas should be explicit, and at the project level22:07
ayounglike, if I have 100 quota, and 10 proejcts, each get 10, or each get 5 and I keep 50 at the parent or something]22:08
lbragstadso - let's back up the example22:08
lbragstadA.limit is still 2022:08
ayoungand then push back on the user to request/authorize push-down and automate reclaim22:08
lbragstadB and C are still children of A and are siblings22:08
ayoungresetting to your problems initial state?22:09
lbragstadyep22:09
ayoungq(A)=2022:09
ayoungno children22:09
ayoungcreate proejct B child of A22:09
ayoungby default, 0 quota22:09
lbragstadif we assume A.limit to be divisible by the number of children in the tree, then it's clear that project D shouldn't be created, right?22:09
ayoungexplicitly grab 5 quota from A, and now q(A)=15, q(B)=522:09
ayoungB quickly burns through quota.22:10
lbragstadwell - by default, B and C get 10 each because of the registered limit22:10
ayoungA has a policy of "allow 5 floating"22:10
ayoungso, automated, Nova could say "requiest more quota for B"22:10
ayoungand Keystone would grab 1 (or whatever) from "Floating:" and now q(A)=14 q(B)=622:11
ayoungnow once B deletes a VM, the question is how to rebalance22:12
lbragstadso - the interesting part is that the default limit established by the registered limit resource keeps the service from having to pass in all resources and owning projects when calculating usage22:12
ayoungOK...so if the default is each get 10, then the question is, would we want to split A, or force the new quota to come from outside A22:12
ayounglike...maybe A is just a polaceholder project, and should never have a VM22:13
lbragstadmaybe22:13
lbragstadit depends22:13
*** dklyle has joined #openstack-keystone22:13
lbragstadbut you'd have to go to A to update A.limit to 30 in order to create another child project under A22:13
*** rcernin has joined #openstack-keystone22:13
ayoungright22:13
ayoungOR, more likely, new projects get 0 quota, or the create-project call fails22:14
ayoungor some other reasonable failure mode22:14
lbragstadthe outcome would be the same, in that case you'd have to supply explicit overrides22:14
ayoungIf the goal is to say "give explicit quota to A, and then let that float among all of A's childre" it is one value system22:15
ayoungif the goal is "each proejct should have a strict quota" it is a different one22:15
ayoungstrict quota is easier to enforce22:15
ayoungfloating...I still suspect needs to use Keystone as a clearing house22:15
lbragstadthe more i think about it - the more i think they should be separate models22:15
ayoungand...I suspect that clearing through Keystone is OK22:15
ayoungyes, they are def separate models22:15
lbragstadbecause they affect how the service passes usage information to oslo.limit22:16
ayoungif request/approve quota are easy and light weight, then explict quota makes sense22:16
lbragstadand ideally, that should be consistent regardless of the enforcement model configured in keystone22:16
ayoungthe question quickly becomes one of churn22:17
ayoungSince Keystone can't call into Nova to free up Quota, nova has to trigger the free-action when the VM is deleted22:17
lbragstadwell - it is possible for keystone to store a limit that is higher than a project's usage for a given resource22:18
lbragstadif H.limit is 15 and H.usage is 15, you can update H.limit to be 10 if you want22:19
lbragstadthe usage enforcement check should prevent any more resources to be allocated to project H until usage is back under a limit of 1022:20
lbragstadthat can be done by an admin, or members of the project22:20
ayoungIt was this veruy discussion that got the Cinder folks to withdraw "have keystone record quota" the first time around...I want to say that discussion happened in Portland22:21
lbragstadkeystone is just storing the limit information though22:21
lbragstadwe're still not tracking the actual usage information22:21
lbragstadif the usage exceeds the limit, that's fine and should be handled by the oslo.limit library during the usage calculation22:23
lbragstadat least until the usage is back under the limit threshold22:24
ayoungYou hit my trigger word22:24
ayoung"just"22:24
lbragstadlol - it's *just* that easy ayoung22:24
ayoungSo, I think that we need to make all quotas explicit22:24
lbragstadthat makes the code easier to write i think22:25
*** jmlowe has quit IRC22:37
*** jmlowe has joined #openstack-keystone22:53
*** felipemonteiro_ has quit IRC22:56
*** felipemonteiro has joined #openstack-keystone23:02
*** felipemonteiro_ has joined #openstack-keystone23:02
*** lbragstad has quit IRC23:05
*** felipemonteiro has quit IRC23:06
*** r-daneel has quit IRC23:09
*** d0ugal__ has joined #openstack-keystone23:11
*** jmlowe has quit IRC23:13
*** d0ugal_ has quit IRC23:14
*** spilla has quit IRC23:14
*** jmlowe has joined #openstack-keystone23:15
*** felipemonteiro__ has joined #openstack-keystone23:23
*** felipemonteiro_ has quit IRC23:23
kmallocayoung: as in, no floating quotas (within children)?23:28
*** d0ugal__ has quit IRC23:29
kmallocayoung: remember, all nodes are strictly leaning on keystone for limits -- it is on the services to do "reserve" and "consume" and "free" for active quota usage -- we don't want to hold what is in use for every service.23:30

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!