Wednesday, 2015-12-02

*** e0ne_ has quit IRC00:00
notmorganjamielennox: let me know when/if you want to talk betamax more00:01
*** lhcheng has quit IRC00:03
*** richm has quit IRC00:08
*** tsymanczyk has joined #openstack-keystone00:09
*** tsymanczyk is now known as Guest4831700:09
shalehnotmorgan: crickey!! I just followed that canon lens link. They are ripping you off when they fail to include the pack mule for that price.00:12
notmorganshaleh: hah, it's quite the lens. i mean... estimation is 20-100 in existence (closer to 20)00:13
shalehnotmorgan: at 2 a year for 12 years, yeah 20 is close to accurate00:14
shalehthe f5.6 is a little disappointing00:15
shalehLease for as low as $6,102 / month00:15
*** jerrygb has joined #openstack-keystone00:16
*** EinstCrazy has joined #openstack-keystone00:16
*** btully has quit IRC00:17
*** EinstCrazy has quit IRC00:17
*** EinstCrazy has joined #openstack-keystone00:18
*** EinstCrazy has quit IRC00:22
*** btully has joined #openstack-keystone00:24
*** _zouyee has joined #openstack-keystone00:30
openstackgerritOpenStack Proposal Bot proposed openstack/keystonemiddleware: Updated from global requirements
jamielennoxlol - it's $600 just for the lens cap for that puppy00:36
*** roxanagh_ has joined #openstack-keystone00:37
*** btully has quit IRC00:38
gyeehelluva deal! :)00:40
*** roxanagh_ has quit IRC00:41
*** markvoelker has quit IRC00:45
jamielennoxnotmorgan: i sent sigmavirus a message in -dev to have a look, ideally we could use betamax like requests-mock and mock out the entire requests adapter layer00:49
notmorganjamielennox: ++00:50
notmorganjamielennox: i like it00:50
*** btully has joined #openstack-keystone00:51
jamielennoxnotmorgan: doesn't seem to be a public api today so if we have to support it in session i'm ok with that00:52
notmorganjamielennox: i think we will need to.00:54
*** shaleh has quit IRC00:54
*** ctina_ has joined #openstack-keystone00:55
*** ctina_ has quit IRC01:00
*** EinstCrazy has joined #openstack-keystone01:01
*** alex_xu has quit IRC01:09
*** alex_xu has joined #openstack-keystone01:12
*** topol has joined #openstack-keystone01:15
*** ChanServ sets mode: +v topol01:15
*** btully has quit IRC01:17
*** topol has quit IRC01:20
*** jasonsb has quit IRC01:23
*** btully has joined #openstack-keystone01:23
*** Guest48317 is now known as tsymanczyk01:26
*** yangweiwei has joined #openstack-keystone01:30
*** aginwala has quit IRC01:31
*** topol has joined #openstack-keystone01:34
*** ChanServ sets mode: +v topol01:34
*** topol_ has joined #openstack-keystone01:35
*** ChanServ sets mode: +v topol_01:35
*** btully has quit IRC01:37
*** aginwala has joined #openstack-keystone01:37
notmorganjamielennox: i was thinking maybe we should be looking to move KSA to be a traditional mount of the requests session object instead of a wrapper [with exception of the basic helper functions]01:38
notmorganjamielennox: might have been a convo i was having w/ mordred at somepoint.01:38
*** topol has quit IRC01:38
jamielennoxnotmorgan: i'm not following01:38
jamielennoxlike make it an adapter?01:38
notmorganjamielennox:yeah a traditional requests adapter01:39
*** btully has joined #openstack-keystone01:39
notmorganunless we were already doing that. but i think we weren't.01:39
jamielennoxnotmorgan: no we're not doing that01:39
jamielennoxnotmorgan: we could do some of it as an adapter, but i don't think we can do everything that way01:39
notmorgannot everything01:40
*** topol_ has quit IRC01:40
notmorganbut we can move things that way01:40
jamielennoxso we would still need something that wraps session01:40
notmorganyeah, we would still need the helper... actually01:40
notmorganwhat if we created an adapter and instead of acting on the session we acted on the Auth object (mounted) into the session01:41
notmorganso instead of asking session.get_catalog, it's session.mount(adapter), and adapter.get_catalog?01:41
jamielennoxso there's a problem i've been thinking of but have never seen anyone have01:41
*** RA has joined #openstack-keystone01:42
*** RA is now known as Guest7921201:42
jamielennoxbecause sessions and auths aren't strictly tied (you can share an auth between sessions) if it ever needed to reauthenticate it would use whatever session it was being used with at the time01:42
jamielennoxif this auth was in some way based on a SSL or other session setup that would fail01:43
jamielennoxthis is a problem for example if we are passing the user auth object down from auth_token middleware01:43
jamielennoxwe would want to do reauth of the service user via the session defined in auth_token middleware not whatever the service defined01:44
jamielennoxso i've thought of how i would redesign this if i were to do it again01:44
notmorganso part of what we discussing earlier would eliminate that reauth01:44
jamielennoxand i think session is really a connection because it shouldn't be storing cookies01:44
jamielennoxor doing anything other than connection pooling01:45
notmorganbecause there shouldn't really need to be a reauth at the service level01:45
notmorganit was already authorized01:45
jamielennoxsure, but that's a bigger problem01:47
jamielennoxbut so if i was doing it again it would be a connection object, wrapped with an auth object that you made requests through01:48
jamielennoxand the interfaces would be the same so clients wouldn't care if it was authed or not01:48
notmorgani think that was the premise behind the adapter01:48
jamielennoxbut given the effort we've put into getting people onto this system i don't know if that's worth it01:48
jamielennoxthere's a lot of stuff you loose at the adapter level01:49
notmorgansure. but for most of what we need it's there01:49
notmorganit's the helper functions that we lose01:49
notmorganand honestly, those are things we can work around01:49
*** jerrygb has quit IRC01:50
jamielennoxwell a big one we lose is redirects but we don't do that anyway01:50
notmorgansince we don't do it ;)01:50
jamielennoxbut a lot of the like get_endpoint stuff has to happen above that layer01:51
*** lhcheng has joined #openstack-keystone01:51
*** ChanServ sets mode: +v lhcheng01:51
notmorgananyway this can be shelved01:51
notmorgani think we need to focus on the stuff we were talking about earlier01:51
notmorgani think it's a massive improvement over what we have today [even just as a POC]01:52
jamielennoxyea, there's some stuff there we might be able to do, but i'm just not sure the wins are enough now01:52
notmorgani think the wins are there. mostly because we can drive the conversation in a good way01:53
jamielennoxwe still haven't finished gettting people on board with the current design, we would need a really compelling reason to uproot that all for something new01:53
notmorgani see this as a preview to where we are headed01:53
notmorganthe current design is a stepping stone, once everything is solidly on it, it'll be easier to make these kinds of adjustments01:54
notmorganesp. if KSM is handing down the auth to the services in a way they can consume.. then we just make KSM hand down a different KSA auth01:54
notmorganthat is smart enough to do things the new way. but i think showing it can all work and the benefits (even if under the hood it's a bit ugly) will help drive the support we need to make it all happen01:55
*** jerrygb has joined #openstack-keystone01:55
* notmorgan is going to continue working on the POC in either case.01:55
jamielennoxthe POC can be done with the current, but i hadn't considered doing ksa as an adapter01:55
jamielennoxit would simplify some stuff01:56
*** lhcheng has quit IRC01:56
jamielennoxlike how i was saying i don't want to use requests at all, adapter is beneath that point01:56
* notmorgan shrugs.01:57
jamielennoxwe could handle all the connection management stuff we need there, and if people want to use a requests.Session on top of that for cookie management they could do what they like01:57
*** btully has quit IRC01:57
jamielennoxnot something we can/should try and solve now01:57
stevemarGDI, new pycadf broke keystone :(01:58
notmorganstevemar: hey, i said it was gonna break things!01:58
notmorgandidn't i?01:59
stevemarnotmorgan: you did01:59
*** doug-fish has quit IRC01:59
*** topol has joined #openstack-keystone01:59
*** ChanServ sets mode: +v topol01:59
*** doug-fish has joined #openstack-keystone01:59
*** doug-fish has quit IRC01:59
*** doug-fish has joined #openstack-keystone02:00
*** topol has quit IRC02:04
*** ctracey has quit IRC02:05
*** ctracey has joined #openstack-keystone02:06
stevemarnotmorgan: looks like its just cause the context we send over is bs02:08
ayoungjamielennox, I know you are not really thinking about RDO anymore, but
jamielennoxayoung: what's instack?02:10
*** ctracey has left #openstack-keystone02:10
jamielennoxayoung: i was talking to cloudnull earlier and they are working on splitting out the ansible roles, given everything you should look at getting someone working on that02:12
jamielennoxthe roles from openstack-ansible02:12
ayoungjamielennox, heh,,,,02:13
jamielennoxthey're very ubuntu focused02:13
ayoungjamielennox, I still don't have a working manager deploy02:13
*** omkarjoshi has left #openstack-keystone02:13
jamielennoxbut they're already way more complete than the puppet scripts02:13
ayounginstack is a way to set it up02:13
jamielennox(at least for keystone)02:14
ayoungsounds good.  BUt, you know, inertia02:14
ayoungI have a new Dell T1700 that is, in theory, powerful enough to run three big VMs undercloud, controller, compute...02:15
ayoungand, wel;l...trying to find the right magik incantation to get it to install02:15
ayoungand now I can't talk to repo.fedorapeople.org02:19
ayounggah...layers upon layers...its turtles all the way down02:19
*** fangxu has quit IRC02:30
*** david-lyle has joined #openstack-keystone02:31
*** doug-fish has quit IRC02:34
*** doug-fish has joined #openstack-keystone02:34
*** aginwala has quit IRC02:35
*** topol has joined #openstack-keystone02:36
*** ChanServ sets mode: +v topol02:36
*** roxanagh_ has joined #openstack-keystone02:38
*** doug-fish has quit IRC02:39
*** aginwala has joined #openstack-keystone02:40
*** topol has quit IRC02:41
*** roxanagh_ has quit IRC02:44
*** jasonsb has joined #openstack-keystone02:51
bknudsonlooks like keystone unit tests are busted.02:54
bknudsonI won't have time to look at it today... will look tomorrow.02:55
yangweiweiThe policy.json is common to each project, but if the project has its own policy rules, Can we?02:58
*** david-lyle has quit IRC03:00
*** Guest79212 has quit IRC03:06
*** topol has joined #openstack-keystone03:13
*** ChanServ sets mode: +v topol03:13
*** gyee has quit IRC03:16
*** aginwala has quit IRC03:17
*** topol has quit IRC03:18
*** aginwala has joined #openstack-keystone03:20
*** btully has joined #openstack-keystone03:25
*** dims has quit IRC03:26
stevemarbknudson: it's cause of the new pycadf release03:30
stevemarit now has more strict validation on "id" fields03:30
stevemarwhich, in some tests, we were setting the user_id as the id, but in cause we test with a silly empty context, the user_id that we get was `None`03:32
stevemarwhich is not a valid ID as far as pycadf is concerned03:32
*** jerrygb has quit IRC03:34
*** jerrygb has joined #openstack-keystone03:34
*** spandhe has quit IRC03:36
*** jerrygb has quit IRC03:39
*** roxanagh_ has joined #openstack-keystone03:40
*** links has joined #openstack-keystone03:43
*** aginwala has quit IRC03:44
*** roxanagh_ has quit IRC03:44
openstackgerritShu Muto proposed openstack/python-keystoneclient: Delete python bytecode before every test run
*** topol has joined #openstack-keystone04:03
*** ChanServ sets mode: +v topol04:03
openstackgerritSteve Martinelli proposed openstack/pycadf: Revert "make generate_uuid return valid uuid"
*** topol has quit IRC04:08
stevemarbknudson: looked at the bug:
openstackLaunchpad bug 1521844 in OpenStack Identity (keystone) "pycadf ID validation fails for multi-domain IDs" [Critical,Triaged]04:14
stevemarnotmorgan: dolphm others, if you're around, i could use some creative ideas :\04:14
*** miyagishi_t has joined #openstack-keystone04:22
jamielennoxstevemar: not much else we can do there, it's valid for keystone to return non-uuid user_ids04:30
*** jbell8 has joined #openstack-keystone04:31
stevemari have sort of a solution04:32
*** david-lyle has joined #openstack-keystone04:44
openstackgerritSteve Martinelli proposed openstack/keystone: Ensure pycadf initiator IDs are UUID
openstackgerritSteve Martinelli proposed openstack/pycadf: Revert "make generate_uuid return valid uuid"
stevemarjamielennox: is my weak ass attempt at fixing it04:47
*** fawadkhaliq has joined #openstack-keystone04:48
openstackgerritgordon chung proposed openstack/pycadf: relax id validation
stevemarapparnently gordon also proposed a change04:50
*** aginwala has joined #openstack-keystone04:51
*** topol has joined #openstack-keystone04:52
*** ChanServ sets mode: +v topol04:52
*** aginwala has quit IRC04:56
*** topol has quit IRC04:57
*** aginwala has joined #openstack-keystone04:59
*** darrenc is now known as darrenc_afk05:10
*** markvoelker has joined #openstack-keystone05:24
*** fangxu has joined #openstack-keystone05:28
*** aginwala has quit IRC05:29
*** topol has joined #openstack-keystone05:29
*** ChanServ sets mode: +v topol05:29
*** darrenc_afk is now known as darrenc05:32
*** topol has quit IRC05:33
*** markvoelker has quit IRC05:41
*** roxanagh_ has joined #openstack-keystone05:41
*** links has quit IRC05:43
*** btully has quit IRC05:46
*** roxanagh_ has quit IRC05:47
*** rcernin has joined #openstack-keystone05:52
*** links has joined #openstack-keystone05:54
*** btully has joined #openstack-keystone06:05
*** topol has joined #openstack-keystone06:06
*** ChanServ sets mode: +v topol06:06
*** topol has quit IRC06:10
*** gildub has quit IRC06:13
*** harlowja has quit IRC06:16
*** aginwala_ has joined #openstack-keystone06:18
*** rcernin has quit IRC06:25
*** topol has joined #openstack-keystone06:42
*** ChanServ sets mode: +v topol06:42
*** roxanagh_ has joined #openstack-keystone06:43
*** spandhe has joined #openstack-keystone06:44
*** spandhe has quit IRC06:46
*** spandhe has joined #openstack-keystone06:47
*** topol has quit IRC06:47
*** roxanagh_ has quit IRC06:48
*** aginwala_ has quit IRC06:49
*** shaleh has joined #openstack-keystone06:50
*** shaleh has quit IRC06:55
*** spandhe_ has joined #openstack-keystone07:06
*** spandhe has quit IRC07:08
*** spandhe_ is now known as spandhe07:08
*** rcernin has joined #openstack-keystone07:09
*** woodster_ has quit IRC07:09
*** akscram has quit IRC07:11
*** akscram has joined #openstack-keystone07:13
*** topol has joined #openstack-keystone07:19
*** ChanServ sets mode: +v topol07:19
*** jerrygb has joined #openstack-keystone07:21
*** topol has quit IRC07:24
*** jerrygb has quit IRC07:25
*** aginwala has joined #openstack-keystone07:37
*** Nirupama has joined #openstack-keystone07:37
*** roxanagh_ has joined #openstack-keystone07:44
*** topol has joined #openstack-keystone07:44
*** ChanServ sets mode: +v topol07:44
*** spandhe has quit IRC07:46
*** topol has quit IRC07:49
*** roxanagh_ has quit IRC07:49
*** jaosorior has joined #openstack-keystone07:50
*** topol has joined #openstack-keystone08:09
*** ChanServ sets mode: +v topol08:09
*** aginwala has quit IRC08:10
*** aginwala has joined #openstack-keystone08:10
*** topol has quit IRC08:13
odyssey4meayoung FYI - OpenStack-Ansible is looking for contributors to assist implementing Fedora/etc support:
odyssey4mecc jamielennox08:28
jamielennoxodyssey4me: less up my alley these days, however given redhat owns ansible that would definetly be something ayoung should take a look at/delegate08:29
*** fawadkhaliq has quit IRC08:30
odyssey4methat would be most useful - we need someone to lead the charge from a work management standpoint - we have several volunteers signed up, but no-one to lead the work yet08:32
odyssey4mewe're happy to help with orientation and such08:32
*** btully has quit IRC08:35
*** fhubik has joined #openstack-keystone08:35
*** aginwala has quit IRC08:40
*** roxanagh_ has joined #openstack-keystone08:46
*** fawadkhaliq has joined #openstack-keystone08:49
*** roxanagh_ has quit IRC08:52
*** btully has joined #openstack-keystone08:52
*** lhcheng has joined #openstack-keystone08:54
*** ChanServ sets mode: +v lhcheng08:54
*** fangxu has quit IRC09:00
*** btully has quit IRC09:06
*** gus has quit IRC09:07
*** gus has joined #openstack-keystone09:09
*** topol has joined #openstack-keystone09:10
*** ChanServ sets mode: +v topol09:10
*** fhubik is now known as fhubik_brb09:14
*** topol has quit IRC09:15
*** yangweiwei has quit IRC09:15
*** raginbajin has joined #openstack-keystone09:17
*** aix has joined #openstack-keystone09:21
*** tyagiprince has joined #openstack-keystone09:21
tyagiprinceHey people.. I am having problem in fetching users from active directory.. when I run the openstack user list command, it takes around 15 to 20 seconds to list the users.. whereas when I open the Users tab inside the identity panel, it takes around a minute or two..09:22
tyagiprinceSo What step should I take? I am thinking about changing the horizon code and fetch only the first 100 users and then on filtering the results through horizon, the user list should update itself in horizon and still show me the best 100 results matching the criteria..09:23
*** mhickey has joined #openstack-keystone09:24
*** fhubik_brb is now known as fhubik09:29
*** henrynash has quit IRC09:31
*** jbell8 has quit IRC09:34
*** jbell8 has joined #openstack-keystone09:35
*** henrynash has joined #openstack-keystone09:35
*** ChanServ sets mode: +v henrynash09:35
*** henrynash has quit IRC09:35
*** hogepodge has quit IRC09:38
*** jbell8 has quit IRC09:39
*** hogepodge has joined #openstack-keystone09:41
*** tomoiaga has joined #openstack-keystone09:41
tomoiagaI've been looking through the code but I can't find a way to delete project extra attributes. Is that possible ? Thank you!09:42
*** henrynash has joined #openstack-keystone09:43
*** ChanServ sets mode: +v henrynash09:43
*** henrynash has quit IRC09:44
*** roxanagh_ has joined #openstack-keystone09:48
*** jistr has joined #openstack-keystone09:50
*** roxanagh_ has quit IRC09:53
*** topol has joined #openstack-keystone10:00
*** ChanServ sets mode: +v topol10:00
*** e0ne has joined #openstack-keystone10:03
*** topol has quit IRC10:04
*** fhubik is now known as fhubik_brb10:06
*** e0ne has quit IRC10:06
*** e0ne_ has joined #openstack-keystone10:07
*** e0ne_ has quit IRC10:09
*** e0ne has joined #openstack-keystone10:09
*** e0ne_ has joined #openstack-keystone10:14
*** e0ne has quit IRC10:16
*** fhubik_brb is now known as fhubik10:22
*** lxsli has joined #openstack-keystone10:30
lxsliGood morning10:30
*** e0ne_ has quit IRC10:30
lxsliCan anyone give me any background on the transition from /v2/ endpoints to /v2.0/ please?10:30
*** Nirupama has quit IRC10:34
*** fhubik is now known as fhubik_brb10:35
samueldmqlxsli: hi, I've never heard about /v2/10:36
lxslisamueldmq: huh, ok! thanks10:37
samueldmqlxsli: v3 was the newest API when I arrived here , and I've only seen conversations using /v3/ or /v2.0/10:37
*** weihan has joined #openstack-keystone10:39
*** btully has joined #openstack-keystone10:41
*** btully has quit IRC10:45
*** roxanagh_ has joined #openstack-keystone10:50
*** roxanagh_ has quit IRC10:54
*** lhcheng_ has joined #openstack-keystone11:02
*** lhcheng has quit IRC11:03
*** lhcheng has joined #openstack-keystone11:04
*** ChanServ sets mode: +v lhcheng11:04
*** lhcheng_ has quit IRC11:04
*** fhubik_brb is now known as fhubik11:12
*** topol has joined #openstack-keystone11:14
*** ChanServ sets mode: +v topol11:14
*** _zouyee has quit IRC11:17
*** topol has quit IRC11:18
*** fhubik has quit IRC11:19
*** fhubik has joined #openstack-keystone11:19
*** doug-fish has joined #openstack-keystone11:21
*** doug-fish has quit IRC11:26
*** aix has quit IRC11:34
*** jbell8 has joined #openstack-keystone11:36
*** fhubik has quit IRC11:36
*** fhubik has joined #openstack-keystone11:37
*** jbell8 has quit IRC11:40
*** fawadkhaliq has quit IRC11:40
*** roxanagh_ has joined #openstack-keystone11:50
*** topol has joined #openstack-keystone11:51
*** ChanServ sets mode: +v topol11:51
*** roxanagh_ has quit IRC11:55
*** topol has quit IRC11:55
*** EinstCrazy has quit IRC11:57
*** doug-fish has joined #openstack-keystone12:00
openstackgerritBernhard M. Wiedemann proposed openstack/python-keystoneclient: Revert "Use textwrap instead of home made implementation"
*** _zouyee has joined #openstack-keystone12:01
*** aix has joined #openstack-keystone12:04
*** rha has joined #openstack-keystone12:06
*** btully has joined #openstack-keystone12:08
*** fhubik is now known as fhubik_brb12:11
*** raildo-afk is now known as raildo12:12
*** fhubik_brb is now known as fhubik12:14
*** LiuNanke has joined #openstack-keystone12:16
*** btully has quit IRC12:19
*** EinstCrazy has joined #openstack-keystone12:22
*** LiuNanke has quit IRC12:22
*** btully has joined #openstack-keystone12:25
*** topol has joined #openstack-keystone12:28
*** ChanServ sets mode: +v topol12:28
*** LiuNanke has joined #openstack-keystone12:28
*** LiuNanke has quit IRC12:29
*** topol has quit IRC12:32
*** kiran-r has joined #openstack-keystone12:33
*** btully has quit IRC12:38
*** btully has joined #openstack-keystone12:42
*** gordc has joined #openstack-keystone12:42
*** jaosorior has quit IRC12:43
*** jaosorior has joined #openstack-keystone12:44
*** kiran-r has quit IRC12:47
openstackgerritMehdi Abaakouk (sileht) proposed openstack/keystonemiddleware: Keystoneauth shouldn't list options of other lib
*** figless has joined #openstack-keystone12:54
*** dancn has joined #openstack-keystone12:56
*** pauloewerton has joined #openstack-keystone13:01
*** figless has left #openstack-keystone13:03
*** topol has joined #openstack-keystone13:04
*** ChanServ sets mode: +v topol13:04
*** topol has quit IRC13:09
*** btully has quit IRC13:13
*** tyagiprince has quit IRC13:15
*** btully has joined #openstack-keystone13:15
openstackgerritgordon chung proposed openstack/pycadf: relax id validation
openstackgerritgordon chung proposed openstack/pycadf: relax id validation
*** thiagop-afk has joined #openstack-keystone13:22
*** topol has joined #openstack-keystone13:22
*** ChanServ sets mode: +v topol13:22
*** topol has quit IRC13:24
*** thiagop-afk is now known as thiagop13:26
dolphmlbragstad: the eBay keystone talk
*** lhinds has joined #openstack-keystone13:34
*** markvoelker has joined #openstack-keystone13:36
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation model
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation driver
openstackgerritAlexander Makarov proposed openstack/keystone: Unified delegation migration
*** ninag has joined #openstack-keystone13:40
*** btully has quit IRC13:41
*** markvoelker_ has joined #openstack-keystone13:41
*** jaosorior has quit IRC13:42
*** markvoelker has quit IRC13:42
*** btully has joined #openstack-keystone13:43
*** jaosorior has joined #openstack-keystone13:43
*** edmondsw has joined #openstack-keystone13:46
*** roxanagh_ has joined #openstack-keystone13:52
*** links has quit IRC13:53
*** josecastroleon has quit IRC13:56
*** josecastroleon has joined #openstack-keystone13:57
*** roxanagh_ has quit IRC13:57
*** jed56 has joined #openstack-keystone13:58
openstackgerritMonty Taylor proposed openstack/python-keystoneclient: Accept v2 params to v3 service create
*** dims has joined #openstack-keystone14:01
*** topol has joined #openstack-keystone14:02
*** ChanServ sets mode: +v topol14:02
*** breitz has quit IRC14:05
*** breitz has joined #openstack-keystone14:05
lbragstaddolphm nice, watching it now14:06
*** topol has quit IRC14:06
*** richm has joined #openstack-keystone14:10
*** jerrygb has joined #openstack-keystone14:10
*** btully has quit IRC14:10
*** ayoung has quit IRC14:13
*** e0ne has joined #openstack-keystone14:14
*** fawadkhaliq has joined #openstack-keystone14:15
*** btully has joined #openstack-keystone14:17
*** e0ne_ has joined #openstack-keystone14:23
*** petertr7_away is now known as petertr714:23
*** e0ne has quit IRC14:24
*** topol has joined #openstack-keystone14:26
*** ChanServ sets mode: +v topol14:26
*** topol has quit IRC14:27
*** topol has joined #openstack-keystone14:28
*** ChanServ sets mode: +v topol14:28
*** topol has quit IRC14:28
*** topol has joined #openstack-keystone14:28
*** ChanServ sets mode: +v topol14:28
*** tyagiprince has joined #openstack-keystone14:29
*** btully has quit IRC14:31
*** btully has joined #openstack-keystone14:31
*** topol has quit IRC14:33
*** josecastroleon1 has joined #openstack-keystone14:34
*** josecastroleon has quit IRC14:35
openstackgerritBoris Bobrov proposed openstack/keystone: Enable limiting in ldap for groups
openstackgerritBoris Bobrov proposed openstack/keystone: Limiting for fake LDAP
openstackgerritBoris Bobrov proposed openstack/keystone: Use @truncated in ldap for users
*** fawadkhaliq has quit IRC14:44
*** fhubik is now known as fhubik_brb14:46
xekbknudson, lbragstad, Hi14:53
*** _zouyee has quit IRC14:54
*** roxanagh_ has joined #openstack-keystone14:54
openstackgerritAlexander Makarov proposed openstack/keystone-specs: Unified delegation spec
*** josecastroleon has joined #openstack-keystone14:56
*** btully has quit IRC14:57
*** josecastroleon1 has quit IRC14:58
stevemarthanks for looking at the pycadf patch bknudson14:59
stevemarthat seems like the right place to fix it14:59
*** btully has joined #openstack-keystone14:59
*** josecastroleon1 has joined #openstack-keystone15:00
bknudsonstevemar: don't we need to fix it in keystone, too?15:00
bknudsonwe don't want warnings all over keystone logs15:00
*** roxanagh_ has quit IRC15:00
xekbknudson, lbragstad, I'm thinking about introducing a configuration variable to aid online schema migration, like "compatibility_mode=true", which could turn off the migration logic to improve performance15:00
xekwhat do you think?15:00
*** josecastroleon has quit IRC15:00
stevemarbknudson: filter out the warnings? we can't change the IDs, and we'll always fail since they are not actually UUIDs15:01
*** EinstCra_ has joined #openstack-keystone15:01
*** dancn has quit IRC15:02
lbragstadxek do we know what approach we want to take15:03
bknudsonstevemar: we could filter the warnings but then we should print out own anyways. If it's wrong to use non-uuids then we're not using pycadf correctly15:03
*** mhickey has quit IRC15:03
lbragstadxek some people yesterday were thinking the expand/contact path was the right way to go15:03
xekin my opinion, the two approaches are complementary15:04
*** EinstCrazy has quit IRC15:04
*** topol has joined #openstack-keystone15:04
*** ChanServ sets mode: +v topol15:04
*** csoukup has joined #openstack-keystone15:05
xeklbragstad, the expand/contract approach helps in that it would perform the contraction in the same release that stops using the old columns/tables15:06
bknudsonstevemar: seems like pycadf shouldn't validate that the IDs are UUIDs since it's not actually required to be a UUID15:06
bknudsonor is the ID required to be a UUID?15:06
openstackgerritShuquan Huang proposed openstack/keystone: Put py34 first in the env order of tox
bknudsonxek: I don't think deployers are going to know when to set compatibility_mode and it's going to be very difficult for devs to get the code right.15:07
stevemargordc: ^15:08
*** topol has quit IRC15:09
stevemarbknudson: lemme read the spec... i'm not sure if IDs have to be UUIDs15:09
xekbknudson, I think of describing it in the spec as one of the approaches, in the end, the developer could decide if he would like to optimize in this way15:09
stevemari think that may be an implementation choice15:09
bknudsonmaybe pycadf could have a switch or something that we could set in keystone15:10
*** btully has quit IRC15:10
bknudsonthat says that we know that IDs aren't uuids.15:10
bknudsonalthough I guess that's the same as disabling the warning.15:11
gordcbknudson: according to spec, it should be uuid15:11
gordcgranted, it's damn near impossible for us to actually do that in openstack15:11
bknudsonI think we have user IDs that are sha256 of user ID and domain ID or something15:11
*** btully has joined #openstack-keystone15:12
gordcbknudson: yeah, it's not consistent anywhere in openstack.15:12
bknudsonalso we have user IDs that come from the LDAP entry so it could be any string.15:12
gordci proposed an oslo topic on it... but i quite frankly am not that optimisitic of a person15:12
gordc1420 General requirements15:13
gordc1421  CADF Identifier type values SHALL be created to be Universally Unique Identifiers (UUIDs) so that when15:13
gordc1422 CADF data (e.g., CADF Event Records, Logs, Reports, Resources, Metrics, etc.) is federated, it will be15:13
gordc1423 uniquely identifiable to the source (e.g., cloud provider, service, etc.) that created it.15:13
gordcthat's from cadf spec.15:13
stevemarCADF Identifier type values SHALL be created to be Universally Unique Identifiers (UUIDs) so that when15:14
stevemar1422 CADF data (e.g., CADF Event Records, Logs, Reports, Resources, Metrics, etc.) is federated, it will be15:14
stevemar1423 uniquely identifiable to the source (e.g., cloud provider, service, etc.) that created it.15:14
bknudsonok, ok, I get it!15:14
gordcone more time!15:14
bknudsonso what's wrong with stevemar's keystone change?15:14
*** tyagiprince1 has joined #openstack-keystone15:14
stevemarit's a change in behavior15:14
bknudsonright, other than it's not backwards compatible15:15
bknudsonbut we already said the pycadf change wasn't backwards compatible15:15
bknudsonhow about we always set initiator.user_id to user_id15:15
stevemarwe could revert the pycadf change and break the spec, which is better than breaking customers15:15
bknudsonand only set if it's a uuid?15:15
gordcbknudson: my only concern is the more we generate new ids, the more likely you are going to have no idea what the hell that new id is15:15
*** mhickey has joined #openstack-keystone15:16
stevemarthat was another idea15:16
*** tyagiprince has quit IRC15:16
*** tyagiprince1 is now known as tyagiprince15:16
*** raginbajin has quit IRC15:16
gordcbknudson: well the patch i had was to allow you to continue to put non-uuid... but it'll probably flood your based on what you're saying15:16
*** raginbajin has joined #openstack-keystone15:17
stevemargordc: bknudson only if you have notifications turned on15:17
bknudsonit should only be printed once, but the problem is I don't think deployers are going to know what to do about it15:17
gordcbknudson: ignore it. as they do with all non-errors.lol15:18
bknudsonI'll tell them to complain to stevemar.15:18
gordcgood choice15:18
gordcbknudson: maybe a more neutral warning?15:19
mordrednotmorgan: wow, we bumped the major?15:20
gordctbh, i partly put warning there so more people would start realising we need uuid15:20
notmorganmordred: dropped py26 support15:20
*** tyagiprince has quit IRC15:20
bknudsongordc: I don't have a problem with the warning from pycadf (or even continuing to raise)15:21
notmorganmordred: even though we had no py26 code or special cases15:21
*** tyagiprince1 has joined #openstack-keystone15:21
notmorganmordred: so i am grumpy we had to bump the major... but...15:21
notmorganthe reason makes sense15:21
bknudsongordc: I think we need to figure out how keystone is supposed to work when cadf says we're supposed to use uuid15:21
gordcbknudson: cool cool.15:21
stevemarbknudson: dolphm's plan for shadow user accounts would fix this15:22
stevemarbknudson: this also breaks federated user's since their IDs are not uuid also15:22
gordcbknudson: what we did in gnocchi was we convert all non-uuid to uuid.15:22
stevemargordc: how'd you convert it?15:22
gordcknida like a display_id vs real id15:22
lbragstadthat sounds similar to the shadow user thing15:23
gordcstevemar: using uuid515:23
*** tyagiprince1 is now known as tyagiprince15:23
gordcstevemar: i'll need to find exact code.15:23
stevemargordc: hook it up15:23
gordcbut it does add another level of processing where you need to resolve stuff15:24
lbragstadgordc interesting, you just passed the exsiting ids as the strings and hashed them?15:25
gordclbragstad: basically.15:25
gordcany id comes in, we check if its uuid, if not, we hash them and assume it's that hashed version.15:25
*** ayoung has joined #openstack-keystone15:26
*** ChanServ sets mode: +v ayoung15:26
bknudsonis pycadf an oslo project?15:26
lbragstaduuid.uuid5(ns, 'my-user-id-which-is-not-uuid--at-all')15:26
bknudsonI can't tell anymore.15:26
dimsbknudson : nope15:26
gordcbknudson: it's yours :)15:27
bknudsonok, I'll go ahead and approve it15:27
*** mhickey has quit IRC15:27
*** mhickey has joined #openstack-keystone15:27
bknudsonwhy does gcb have +2 on pycadf?15:27
*** petertr7 is now known as petertr7_away15:28
bknudsonpycadf must be oslo15:28
*** josecastroleon has joined #openstack-keystone15:28
stevemarbknudson: he's oslo core and pycadf is oslo15:28
*** fawadkhaliq has joined #openstack-keystone15:28
gordcbknudson: oslo group is probably still listed... take it off i guess.15:28
*** btully has quit IRC15:28
*** markvoelker_ has quit IRC15:28
dimsbknudson :
dimsit's keystone's!15:29
*** josecastroleon1 has quit IRC15:29
*** petertr7_away is now known as petertr715:29
dims+1 to take oslo group off pycadf15:29
*** weihan has quit IRC15:30
lbragstaddolphm fyi -
bknudsonand I assume we still have a pycadf-core15:30
dolphmlbragstad: woooooooooooooo15:31
bknudsonwe'll be defaulting to fernet in not ime.15:31
notmorgandims: sure15:31
notmorgandims: oslo core was kept on as a courtesy15:31
dimsthanks notmorgan :)15:32
notmorgandims: but i mean, i'm sure stevemar wont mind if you want to drop oslo-core off15:32
notmorgancause oslo-core is kinda your thing :P15:32
dimsnotmorgan : i'd recommend looking at individuals who have helped from oslo and add them if appropriate15:32
stevemarnice job lbragstad and dolphm15:32
notmorgandims: dhellmann and gordc should be explicits as well15:33
notmorgandims: otherwise ... i don't think many folks have15:33
dimsnotmorgan : +115:33
stevemardims: you have helped review as well15:33
stevemarso +1 for dims15:33
notmorganbknudson: lets land that change! :)15:33
stevemarand of course gordc and dhellmann15:33
notmorganbknudson: >.>15:33
lbragstaddolphm stevemar i have a patch up for deprecating pki/z15:33
stevemarlbragstad: i know ;)15:34
*** spandhe has joined #openstack-keystone15:34
lbragstadstevemar that you do :) thanks for the review15:34
dolphmlbragstad: does it depend on the default in devstack?15:34
*** ChanServ has quit IRC15:34
*** raginbajin has quit IRC15:34
*** roxanaghe has quit IRC15:34
*** marekd has quit IRC15:34
*** jaosorior has quit IRC15:34
*** hogepodge has quit IRC15:34
*** Ephur has quit IRC15:34
*** ninag has quit IRC15:34
*** lhinds has quit IRC15:34
*** pauloewerton has quit IRC15:34
*** gordc has quit IRC15:34
*** rha has quit IRC15:34
*** doug-fish has quit IRC15:34
*** lhcheng has quit IRC15:34
*** alex_xu has quit IRC15:34
*** zz_john5223 has quit IRC15:34
*** dtroyer has quit IRC15:34
*** pkarikh has quit IRC15:34
*** tsufiev has quit IRC15:34
*** amakarov has quit IRC15:34
*** stevemar has quit IRC15:34
*** sileht has quit IRC15:34
*** chlong has quit IRC15:34
*** xek has quit IRC15:34
*** amit213 has quit IRC15:34
*** notmyname has quit IRC15:34
*** opilotte has quit IRC15:34
*** mkoderer has quit IRC15:34
*** crinkle has quit IRC15:34
*** reed has quit IRC15:34
*** swebb has quit IRC15:34
*** mgagne has quit IRC15:34
*** fawadkhaliq has quit IRC15:34
*** josecastroleon has quit IRC15:34
*** e0ne_ has quit IRC15:34
*** jerrygb has quit IRC15:34
*** breitz has quit IRC15:34
*** aix has quit IRC15:34
*** david-lyle has quit IRC15:34
*** chmouel has quit IRC15:34
*** dulek has quit IRC15:34
*** mdavidson has quit IRC15:34
*** anteaya has quit IRC15:35
*** elmiko has quit IRC15:35
*** x58 has quit IRC15:35
*** _fortis has quit IRC15:35
*** Daviey has quit IRC15:35
*** rmstar has quit IRC15:35
*** albertom has quit IRC15:35
*** mordred has quit IRC15:35
*** charz has quit IRC15:35
*** afazekas has quit IRC15:35
*** hrou has quit IRC15:35
*** kfox1111 has quit IRC15:35
*** lifeless has quit IRC15:35
*** breton has quit IRC15:35
*** gerhardqux has quit IRC15:35
*** Madkiss has quit IRC15:35
*** raildo has quit IRC15:35
*** hugokuo has quit IRC15:35
*** blogan has quit IRC15:35
*** grantbow has quit IRC15:35
*** htruta has quit IRC15:35
*** samueldmq has quit IRC15:35
*** ptoohill has quit IRC15:35
*** rodrigods has quit IRC15:35
*** jgriffith has quit IRC15:35
*** sirushti has quit IRC15:35
*** cburgess has quit IRC15:35
*** bknudson has quit IRC15:35
*** ir2ivps8_ has quit IRC15:35
*** sshen_ has quit IRC15:35
*** trey has quit IRC15:35
*** dstanek has quit IRC15:35
*** mtreinish has quit IRC15:35
*** zzzeek has quit IRC15:35
*** bapalm has quit IRC15:35
*** jistr has quit IRC15:35
*** jimbaker has quit IRC15:35
*** bigjools has quit IRC15:35
*** mhu has quit IRC15:35
*** boltR has quit IRC15:35
*** toddnni has quit IRC15:35
*** zhiyan has quit IRC15:35
*** ekarlso has quit IRC15:35
*** bradjones has quit IRC15:35
*** noqa_v_g1ovnie has quit IRC15:35
*** DuncanT has quit IRC15:35
*** ramishra has quit IRC15:35
*** iurygregory has quit IRC15:35
*** andreaf_ has quit IRC15:35
*** andreaf has quit IRC15:35
*** errr has quit IRC15:35
*** briancurtin has quit IRC15:35
*** EinstCra_ has quit IRC15:35
*** dims has quit IRC15:35
*** fhubik_brb has quit IRC15:35
*** tomoiaga has quit IRC15:35
*** tsymanczyk has quit IRC15:35
*** mancdaz has quit IRC15:35
*** mnaser has quit IRC15:35
*** flaper87 has quit IRC15:35
*** tjcocozz has quit IRC15:35
*** krotscheck has quit IRC15:35
*** petertr7 has quit IRC15:35
*** Tridde has quit IRC15:35
*** lxsli has quit IRC15:35
*** darrenc has quit IRC15:35
*** gissi has quit IRC15:35
*** jrist has quit IRC15:35
*** gsilvis__ has quit IRC15:35
*** ericksonfgds has quit IRC15:35
*** kragniz has quit IRC15:35
*** baffle has quit IRC15:35
*** arunkant_ has quit IRC15:35
*** arunkant has quit IRC15:35
*** david8hu has quit IRC15:35
*** d0ugal has quit IRC15:35
*** lars2 has quit IRC15:35
*** ccard has quit IRC15:35
*** zigo has quit IRC15:35
*** Nakato has quit IRC15:35
*** mjb has quit IRC15:35
*** esp has quit IRC15:35
*** timburke has quit IRC15:35
*** miguelgrinberg has quit IRC15:35
*** EmilienM has quit IRC15:35
*** telemonster has quit IRC15:35
*** notmorgan has quit IRC15:35
*** freerunner has quit IRC15:35
*** med_ has quit IRC15:35
*** jmccrory has quit IRC15:35
*** wolsen has quit IRC15:35
*** dgonzalez has quit IRC15:35
*** Dave has quit IRC15:35
*** nonameentername has quit IRC15:35
*** Anticimex has quit IRC15:35
*** tobasco has quit IRC15:35
*** jed56 has quit IRC15:35
*** serverascode has quit IRC15:35
*** johnthetubaguy has quit IRC15:35
*** hockeynut has quit IRC15:35
*** lbragstad has quit IRC15:35
*** jraim has quit IRC15:35
*** sudorandom has quit IRC15:35
*** jamielennox has quit IRC15:35
*** wasmum has quit IRC15:35
*** cloudnull has quit IRC15:35
*** d34dh0r53 has quit IRC15:35
*** dolphm has quit IRC15:35
*** eglute has quit IRC15:35
*** BrAsS_mO- has quit IRC15:35
*** dobson has quit IRC15:35
*** notmorgan has joined #openstack-keystone15:36
*** jgriffith has joined #openstack-keystone15:37
*** sirushti has joined #openstack-keystone15:37
*** cburgess has joined #openstack-keystone15:37
*** bknudson has joined #openstack-keystone15:37
*** sshen_ has joined #openstack-keystone15:37
*** ir2ivps8_ has joined #openstack-keystone15:37
*** trey has joined #openstack-keystone15:37
*** sets mode: +v bknudson15:37
*** notmorgan has quit IRC15:38
*** spandhe has quit IRC15:38
*** adelia has joined #openstack-keystone15:38
*** ninag has joined #openstack-keystone15:39
*** lhinds has joined #openstack-keystone15:39
*** pauloewerton has joined #openstack-keystone15:39
*** gordc has joined #openstack-keystone15:39
*** rha has joined #openstack-keystone15:39
*** doug-fish has joined #openstack-keystone15:39
*** lhcheng has joined #openstack-keystone15:39
*** alex_xu has joined #openstack-keystone15:39
*** zz_john5223 has joined #openstack-keystone15:39
*** dtroyer has joined #openstack-keystone15:39
*** pkarikh has joined #openstack-keystone15:39
*** tsufiev has joined #openstack-keystone15:39
*** amakarov has joined #openstack-keystone15:39
*** stevemar has joined #openstack-keystone15:39
*** sileht has joined #openstack-keystone15:39
*** chlong has joined #openstack-keystone15:39
*** xek has joined #openstack-keystone15:39
*** amit213 has joined #openstack-keystone15:39
*** notmyname has joined #openstack-keystone15:39
*** opilotte has joined #openstack-keystone15:39
*** mkoderer has joined #openstack-keystone15:39
*** swebb has joined #openstack-keystone15:39
*** crinkle has joined #openstack-keystone15:39
*** reed has joined #openstack-keystone15:39
*** mgagne has joined #openstack-keystone15:39
*** sets mode: +vo lhcheng stevemar15:39
*** jistr has joined #openstack-keystone15:39
*** jimbaker has joined #openstack-keystone15:39
*** bigjools has joined #openstack-keystone15:39
*** mhu has joined #openstack-keystone15:39
*** boltR has joined #openstack-keystone15:39
*** toddnni has joined #openstack-keystone15:39
*** zhiyan has joined #openstack-keystone15:39
*** ekarlso has joined #openstack-keystone15:39
*** bradjones has joined #openstack-keystone15:39
*** noqa_v_g1ovnie has joined #openstack-keystone15:39
*** DuncanT has joined #openstack-keystone15:39
*** ramishra has joined #openstack-keystone15:39
*** iurygregory has joined #openstack-keystone15:39
*** andreaf_ has joined #openstack-keystone15:39
*** andreaf has joined #openstack-keystone15:39
*** marekd has joined #openstack-keystone15:39
*** sets mode: +v marekd15:39
*** btully has joined #openstack-keystone15:39
*** raginbajin has joined #openstack-keystone15:40
*** zigo has joined #openstack-keystone15:40
*** Nakato has joined #openstack-keystone15:40
*** mjb has joined #openstack-keystone15:40
*** esp has joined #openstack-keystone15:40
*** timburke has joined #openstack-keystone15:40
*** miguelgrinberg has joined #openstack-keystone15:40
*** EmilienM has joined #openstack-keystone15:40
*** telemonster has joined #openstack-keystone15:40
*** freerunner has joined #openstack-keystone15:40
*** med_ has joined #openstack-keystone15:40
*** jmccrory has joined #openstack-keystone15:40
*** wolsen has joined #openstack-keystone15:40
*** dgonzalez has joined #openstack-keystone15:40
*** Dave has joined #openstack-keystone15:40
*** nonameentername has joined #openstack-keystone15:40
*** Anticimex has joined #openstack-keystone15:40
*** tobasco has joined #openstack-keystone15:40
*** iurygregory has quit IRC15:41
*** alex_xu has quit IRC15:41
*** jaosorior has joined #openstack-keystone15:42
*** lhinds has quit IRC15:42
stevemarkeystone broke everyone this morning15:42
*** roxanaghe has joined #openstack-keystone15:42
stevemarincluding itself15:42
*** dstanek has joined #openstack-keystone15:42
*** mtreinish has joined #openstack-keystone15:42
*** zzzeek has joined #openstack-keystone15:42
*** bapalm has joined #openstack-keystone15:42
*** sets mode: +v dstanek15:42
*** errr has joined #openstack-keystone15:42
*** briancurtin has joined #openstack-keystone15:42
*** hogepodge has joined #openstack-keystone15:42
*** Ephur has joined #openstack-keystone15:42
*** johnthetubaguy has joined #openstack-keystone15:43
*** jed56 has joined #openstack-keystone15:43
*** serverascode has joined #openstack-keystone15:43
*** hockeynut has joined #openstack-keystone15:43
*** lbragstad has joined #openstack-keystone15:43
*** jraim has joined #openstack-keystone15:43
*** sudorandom has joined #openstack-keystone15:43
*** jamielennox has joined #openstack-keystone15:43
*** wasmum has joined #openstack-keystone15:43
*** cloudnull has joined #openstack-keystone15:43
*** d34dh0r53 has joined #openstack-keystone15:43
*** dolphm has joined #openstack-keystone15:43
*** eglute has joined #openstack-keystone15:43
*** BrAsS_mO- has joined #openstack-keystone15:43
*** dobson has joined #openstack-keystone15:43
*** sets mode: +vo jamielennox dolphm15:43
*** topol has joined #openstack-keystone15:44
*** sigmavirus24 has joined #openstack-keystone15:46
*** ctina has joined #openstack-keystone15:51
*** btully has quit IRC15:51
*** aix has joined #openstack-keystone15:51
*** _fortis has joined #openstack-keystone15:51
dstanekit felt like i was cut off from the world for a bit15:51
openstackgerritLance Bragstad proposed openstack/keystone: Deprecate the pki and pkiz token providers.
lbragstaddolphm if we are recommending fernet *or* uuid, then I don't think it does15:53
bknudsonwe should change one of the devstack jobs to use fernet15:54
*** ayoung has quit IRC15:54
bknudsonmaybe there's a devstack job using pki?15:54
lbragstadbknudson not sure, where do I look for that?15:54
dolphmbknudson: not that i'm aware of15:54
bknudsonlbragstad: you could check the logs in the gate-tempest-dsvm-* jobs in keystone reviews15:55
*** markvoelker has joined #openstack-keystone15:55
*** slberger has joined #openstack-keystone15:55
bknudsonprobably best to ask in -qa how they figure out what options to send to devstack.15:55
*** elmiko has joined #openstack-keystone15:58
*** iurygregory has joined #openstack-keystone15:59
*** d0ugal has joined #openstack-keystone15:59
*** jrist has joined #openstack-keystone15:59
*** chmouel has joined #openstack-keystone15:59
*** fawadkhaliq has joined #openstack-keystone15:59
*** josecastroleon has joined #openstack-keystone15:59
*** e0ne_ has joined #openstack-keystone15:59
*** jerrygb has joined #openstack-keystone15:59
*** breitz has joined #openstack-keystone15:59
*** david-lyle has joined #openstack-keystone15:59
*** dulek has joined #openstack-keystone15:59
*** mdavidson has joined #openstack-keystone15:59
*** anteaya has joined #openstack-keystone15:59
*** x58 has joined #openstack-keystone15:59
*** Daviey has joined #openstack-keystone15:59
*** rmstar has joined #openstack-keystone15:59
*** albertom has joined #openstack-keystone15:59
*** mordred has joined #openstack-keystone15:59
*** charz has joined #openstack-keystone15:59
*** kfox1111 has joined #openstack-keystone15:59
*** lifeless has joined #openstack-keystone15:59
*** breton has joined #openstack-keystone15:59
*** gerhardqux has joined #openstack-keystone15:59
*** Madkiss has joined #openstack-keystone15:59
*** raildo has joined #openstack-keystone15:59
*** hugokuo has joined #openstack-keystone15:59
*** blogan has joined #openstack-keystone15:59
*** grantbow has joined #openstack-keystone15:59
*** htruta has joined #openstack-keystone15:59
*** samueldmq has joined #openstack-keystone15:59
*** ptoohill has joined #openstack-keystone15:59
*** rodrigods has joined #openstack-keystone15:59
lbragstadxek have you worked with the expand/contact method much?15:59
*** rcernin has quit IRC15:59
lbragstadxek i like the expand/contact approach from the perspective that we don't have to keep things around for several releases.16:00
lbragstadthat is appealing, just because it's less context to have to remember over a longer period of time16:00
openstackgerritMehdi Abaakouk (sileht) proposed openstack/keystonemiddleware: Keystonemiddleware shouldn't list options of other lib
lbragstadand if I were an operator, I'd consider that a plus16:01
dstaneklbragstad: that was my main concern with the proposal16:01
xeklbragstad, the problem is it doesn't help much, here is a discussion in neutron that a colleague of mine continued
dstanekkeeping cruft for a year or year and a half16:02
*** mhickey has quit IRC16:02
*** kragniz has joined #openstack-keystone16:03
xeklbragstad, so to really shorten the timeframe, we would have to introduce something like that configuration value, and to shorten the upgrade to one release, multiple versions of sqlalchemy models should be supported16:03
lbragstadxek so, in order to do that we would have to know which model to used based on schema?16:05
lbragstadxek try and load a model, if it fails try and load another one, so on and so forth16:06
bknudsonwe can get the version of the schema from sqlalchemy16:06
dstaneklbragstad: loading the model won't fail in many cases. you just won't get all of the data16:06
xeklbragstad, yes, but it would also have to be configured somewhere, so we know which schema to use before we get a DB error16:06
lbragstadwhat if the model requires something that has been removed?16:06
bknudsonos then you could load the model based on the migration #.16:07
lbragstadwill we ever hit that case?16:07
dstanekbknudson: ++16:07
*** ctina has quit IRC16:07
xekbknudson, oh, your'e right16:07
bknudsonso you don't need a config option, just check the migration #.16:07
*** ctina has joined #openstack-keystone16:07
bknudsonumm... maybe not... have to think about that some more16:08
dstanekxek: lbragstad: bknudson: so that means we may have to have our versioned drivers keep their old models after all?16:08
bknudsondstanek: I was thinking about that too, since we can use the old sql driver...16:08
dstanekcouldn't this be as simple as having the correct versioned driver in the config?16:08
lbragstadso, instead of specifying the sql driver, you'd specify the versioned driver?16:09
*** iurygregory has quit IRC16:09
*** d0ugal has quit IRC16:09
*** jrist has quit IRC16:09
*** chmouel has quit IRC16:09
*** fawadkhaliq has quit IRC16:09
*** josecastroleon has quit IRC16:09
*** e0ne_ has quit IRC16:09
*** jerrygb has quit IRC16:09
*** breitz has quit IRC16:09
*** david-lyle has quit IRC16:09
*** dulek has quit IRC16:09
*** mdavidson has quit IRC16:09
*** anteaya has quit IRC16:09
*** x58 has quit IRC16:09
*** Daviey has quit IRC16:09
*** rmstar has quit IRC16:09
*** albertom has quit IRC16:09
*** mordred has quit IRC16:09
*** charz has quit IRC16:09
*** kfox1111 has quit IRC16:09
*** lifeless has quit IRC16:09
*** breton has quit IRC16:09
*** gerhardqux has quit IRC16:09
*** Madkiss has quit IRC16:09
*** raildo has quit IRC16:09
*** hugokuo has quit IRC16:09
*** blogan has quit IRC16:09
*** grantbow has quit IRC16:09
*** htruta has quit IRC16:09
*** samueldmq has quit IRC16:09
*** ptoohill has quit IRC16:09
*** rodrigods has quit IRC16:09
bknudsonthe new driver might still have to do some work until the migration is complete.16:09
dstaneklbragstad: yes16:09
dstanekbknudson: not the driver but the manager16:10
dstanekor actually an adapter16:10
dstanekit already has to know how to deal with the difference16:10
lbragstadthe driver will have to be smart enough to handle both schemas16:10
xekdstanek, not the old models, but the intermediate models, which migrate the data16:10
dstaneklbragstad: no, only it's schema16:10
xekbut only in case of migrating a column to a new type16:10
lbragstadso what if we take this as an example -
bknudsonalso, we might be overarchitecting this. maybe we don't change the schema enough that it's actually going to be a problem.16:11
xekbknudson, I'm of the same opinion16:12
lbragstadbknudson what do you mean?16:12
dstanekbknudson: ++16:12
lbragstadnot being able to change types?16:12
dstanekif this gets too complicated we'll just break it in worse ways16:13
bknudsonlbragstad: if there's only 1 or 2 times that we're planning on a backwards-incompatible change then it's not that big of a deal to carry some cruft for a release.16:13
lbragstadbknudson oh, i see what you're saying16:13
gordchow do i avoid this in my test: oslo_config.cfg.RequiredOptError: value required for option: keystone_authtoken.auth-url16:13
lbragstadbknudson so we would be opting for *not* the expand/contact model?16:14
bknudsonlbragstad: is a good example, so would be interesting to know how online schema migration would handle it.16:14
bknudsonlbragstad: yes, then we wouldn't worry about expand/contract.16:14
dstanekxek: so the overall goal is to require restarts to load new keystone code, but not have to take the database down?16:14
xekdstanek, yes16:15
dstanekxek: so do we need to tell people to stop using MySQL? doesn't it table lock in lots of cases? like adding or changing an index16:15
xekdstanek, but restarting one by one, so there may be 2 versions running simultaneously, like L+M16:15
bknudsonI think the goal would be to do 1) run keystone-manage db_sync with new code, 2) ansible-deploy new keystone code to new notes and retire old instances16:15
bknudsoneventually all your instances are running new code16:16
xekdstanek, I heard it does, but I think operators have some ways to work around those issues, or at least know about them16:16
dstanekbknudson: i think you may be better off moving all to new code first and then doing the migration16:16
*** EinstCrazy has joined #openstack-keystone16:16
dstanekthe code can understand the current and future versions of the schema16:16
dstanekxek: i don't think you can work around them. i think the database needs to stop (if it's mysql)16:17
xekwith the unit forbidding backward incomatible changes in place, an operator could even downgrade the service, without downgrade scripts16:17
xek*unit test16:18
*** EinstCrazy has quit IRC16:18
*** woodster_ has joined #openstack-keystone16:18
bknudsondstanek: how would keystone know when the migration is complete? is it going to be monitoring the schema version?16:18
dstanekxek: no, because we can still interpret the data differently16:18
bknudsonyou'd need a new column if the data is interpreted differently16:19
bknudsonand you'd need to maintain both columns for a release.16:19
dstanekbknudson: the way you do it in a nosql database is looking at the data. in lbragstad's example the data from the same column would have different type16:19
dstanekbknudson: that is the traditional way with sql databases16:20
bknudsonthen we should switch to nosql16:20
bknudsonbut that's not what we're doing16:20
bknudsonin sql you can't have some rows where the column is INTEGER and other rows where the column is DATETIME16:21
dstanekbknudson: exactly16:21
bknudsonI'm not sure how sql-based code would be able to handle the column type changing under it.16:22
dstanekso in lbragstad's example you'd have issued_before as a datetime and issued_before2 as an integer16:22
dstanekwrites would go into the new column and reads would deal with both16:23
lbragstadbknudson ++ i think updating the application layer first makes sense16:23
dstaneka migration would move the data16:23
xekbknudson, there would have to be some abstraction layer, like another object, which would replace the abstracted sqlalchemy model16:23
*** jbell8 has joined #openstack-keystone16:23
dstaneklbragstad: yep, that's what i said above. code first and that wold handle old and new schema16:23
xekbknudson, ^16:24
bknudsondstanek: but I don't get how you're going to change the schema while the instances are running?16:24
dstanekbut it doesn't solve the issue of not bringing mysql down16:24
lbragstadyeah, so that code has to be smart enough to look in the old place if the new place doesn't have anything in it16:24
dstanekbknudson: you would just add the column16:24
lbragstadbring mysql down is a case by case thing16:24
bknudsonthen keystone would somehow discover that the column was added?16:25
lbragstaddepending on what you're doing to the schema16:25
xekdstanek, I don't think all mysql db backends have these problems16:25
xekdstanek, but I run postgres in production, so I wouldn't know16:25
dstanekxek: it's a mysql server thing, or at least it use to be (not just a table type thing)16:25
dstanekwe'd have to do some research since i haven't had to mess with mysql in at least 5 years16:26
lbragstadalso, what do we consider "bringing mysql down"?16:26
bknudsonwhy do you have to restart mysql?16:26
lbragstadwhat about locking the table?16:26
lbragstadbecause if we can't lock a table, then that throws alter table statements out the window right?16:27
dstaneklbragstad: if you lock a table then the better experience would be to bring the service down and give 500s than have somethings work and others not16:27
lbragstadyeah, depending on the table you're locking16:28
dstanekbknudson: you don't have to restart, but in the past mysql would table lock16:28
dstanekfrom way back in the day
bknudsonyes, schema changes typically lock the table16:28
xekdstanek, I just googled that there are some some options of altering the schema without bringing mysql down
lbragstadxek dstanek bknudson do you think it would make sense to detail that more in the spec? the expectation between lock and bringing down?16:29
bknudsonI think with the work we're doing we really shouldn't have much in the database16:30
lbragstadbknudson you mean the stuff that keystone stores?16:30
bknudsonlbragstad: right16:30
lbragstadyay to tokens being gone!16:30
dstanekxek: the question is "is all that worth it?"16:31
lbragstadodyssey4me had some good feedback at the summit around this too16:31
lbragstadmaybe cloudnull if he's available?16:32
bknudsonI don't want my cloud to have to be taken down during upgrades.16:32
lbragstadthat seems to be the general concensus16:32
dstanekxek: where i use to work we bought companies and migrated them to our systems. the code would understand how to get the data from both systems during the migration and we'd have to do incremental data migrations because the data could change in the source system16:32
dstanekit got complicated16:32
lbragstaddstanek you needed something to migrate the delta between the two, right16:33
dstaneklbragstad: yes16:33
lbragstadthat's hairy16:33
bknudsonwhy don't we keep users and groups in nosql.16:33
lbragstadwhat about revocation events?16:33
bknudsonand tokens and revocation events16:33
stevemarbknudson: agree/disagree, keystone fix for pycadf warnings is not critical to m1?16:34
dstanekoption 3 is what i am use to, but if would require us to deal with some interesting issues16:34
lbragstadlol, so the catalog is the only thing still in sql16:34
stevemaronce we depend on the new library...16:34
bknudsonstevemar: I don't think the fix for warnings is critical to m116:34
*** ccard has joined #openstack-keystone16:34
*** lars2 has joined #openstack-keystone16:34
*** david8hu has joined #openstack-keystone16:34
*** arunkant has joined #openstack-keystone16:34
*** arunkant_ has joined #openstack-keystone16:34
*** baffle has joined #openstack-keystone16:34
*** ericksonsantos has joined #openstack-keystone16:34
*** gsilvis__ has joined #openstack-keystone16:34
*** gissi has joined #openstack-keystone16:34
*** darrenc has joined #openstack-keystone16:34
*** Tridde has joined #openstack-keystone16:34
*** petertr7 has joined #openstack-keystone16:34
*** krotscheck has joined #openstack-keystone16:34
*** tjcocozz has joined #openstack-keystone16:34
*** flaper87 has joined #openstack-keystone16:34
*** mancdaz has joined #openstack-keystone16:34
*** tsymanczyk has joined #openstack-keystone16:34
*** lxsli has joined #openstack-keystone16:34
*** hrou has joined #openstack-keystone16:34
*** notmorgan has joined #openstack-keystone16:34
*** afazekas has joined #openstack-keystone16:34
*** alex_xu_ has joined #openstack-keystone16:34
*** mnaser has joined #openstack-keystone16:34
*** lhinds has joined #openstack-keystone16:34
*** diazjf has joined #openstack-keystone16:34
*** d0ugal has joined #openstack-keystone16:34
*** jrist has joined #openstack-keystone16:34
*** chmouel has joined #openstack-keystone16:34
*** fawadkhaliq has joined #openstack-keystone16:34
*** jerrygb has joined #openstack-keystone16:34
*** breitz has joined #openstack-keystone16:34
*** david-lyle has joined #openstack-keystone16:34
*** dulek has joined #openstack-keystone16:34
*** mdavidson has joined #openstack-keystone16:34
*** anteaya has joined #openstack-keystone16:34
*** x58 has joined #openstack-keystone16:34
*** Daviey has joined #openstack-keystone16:34
*** rmstar has joined #openstack-keystone16:34
*** albertom has joined #openstack-keystone16:34
*** mordred has joined #openstack-keystone16:34
*** charz has joined #openstack-keystone16:34
*** kfox1111 has joined #openstack-keystone16:34
*** lifeless has joined #openstack-keystone16:34
*** breton has joined #openstack-keystone16:34
*** gerhardqux has joined #openstack-keystone16:34
*** Madkiss has joined #openstack-keystone16:34
*** raildo has joined #openstack-keystone16:34
*** hugokuo has joined #openstack-keystone16:34
*** blogan has joined #openstack-keystone16:34
*** grantbow has joined #openstack-keystone16:34
*** htruta has joined #openstack-keystone16:34
*** samueldmq has joined #openstack-keystone16:34
*** ptoohill has joined #openstack-keystone16:34
*** rodrigods has joined #openstack-keystone16:34
stevemarcool. i will ask dhellmann to release a new version of it and bump the requirements16:34
dstanekbknudson: is there already an expectation that a cloud needs nosql?16:34
lbragstadcloudnull from an operator perspective - have you had much experience with expand/contract models?16:34
bknudsonit's webscale16:35
lbragstadcloudnull versus adding and removing things over long running time periods16:35
dstanekmongodb it is!16:35
bknudsonwe can't even use foreign key constraints16:36
* lbragstad waves goodbye to referential integrity 16:36
lbragstadas keystone, do we care about one of those more than the other/16:37
lbragstadas keystone operators and developers16:37
dstaneki'm interested in hearing what odyssey4me and cloudnull think about it16:38
odyssey4meo/ sorry, in another meeting simultaneously16:39
lbragstad++ from a dev perspective it sounds like we want the ability to be flexible, but I've never scaled mongodb or anything of the like16:39
xekodyssey4me, +1 a previous version of the spec16:39
dstanekwith the second table approach we had a way to tell the code that the data was being migrated so that it wouldn't update records in the old system and instead would migrate on demand when it needed data that wasn't yet migrated16:39
odyssey4meI haven't re-read the spec in detail recently. Conceptually I'm in favour of implementing some sort of mechanism. Are there any specific questions on the table?16:39
lbragstadodyssey4me do you have experience with an expand/contract db migration model?16:40
cloudnulldstanek:  I have not used much of the expand / contract capabilities, the only service that has that implemented, for liberty, was neutron (best of my knowledge). I think expand/contract makes a lot of sense for maintaining overall uptime and supporting a deployment for long periods of time.16:40
lbragstadversus having long running tables that get cleaned up after a long period of time?16:40
xekodyssey4me, neutron has an expand/contract approach, but it doesn't yet support rolling upgrades16:40
cloudnull^ that.16:41
dstanekcloudnull: doesn't expand/contract introduce downtime anyway?16:41
cloudnullhowever i think if work is being done on the models / upgrade working toward rolling upgrades is the way to go16:41
openstackgerritHenrique Truta proposed openstack/keystone: API support for project cascade delete
openstackgerritHenrique Truta proposed openstack/keystone: Manager support for project delete cascade
openstackgerritHenrique Truta proposed openstack/keystone: Add backend support for deleting a projects list
odyssey4meyeah, the discussion in the spec around expand/contract versus the long running tables has been interesting16:42
*** jaosorior has quit IRC16:42
cloudnulldstanek:  it does for larger migrations however work is being done to get to rolling upgrades16:42
lbragstadcloudnull odyssey4me this is the example we're working with -
bknudsonif you have to maintain 2 copies of the data that's going to affect your performance for the entire release16:43
odyssey4mewhile I don't have specific operational experience either way, expand/contract seems to be beneficial to developers and operators... for operators it's all about minimising down-time and making it predictable... the nova approach isn't so nice for developers, but is very nice for operators as it's no down time16:43
bknudsonand we seem to have enough performance problems already16:43
lbragstadthe other thing to think about, is if you have to add a new column, but the original column has references to it.16:44
lbragstadthat's something i get hung up on16:44
dstanekodyssey4me: how is it no downtime?16:44
*** d0ugal has quit IRC16:45
*** jrist has quit IRC16:45
*** chmouel has quit IRC16:45
*** fawadkhaliq has quit IRC16:45
*** jerrygb has quit IRC16:45
*** breitz has quit IRC16:45
*** david-lyle has quit IRC16:45
*** dulek has quit IRC16:45
*** mdavidson has quit IRC16:45
*** anteaya has quit IRC16:45
*** x58 has quit IRC16:45
*** Daviey has quit IRC16:45
*** rmstar has quit IRC16:45
*** albertom has quit IRC16:45
*** mordred has quit IRC16:45
*** charz has quit IRC16:45
*** kfox1111 has quit IRC16:45
*** lifeless has quit IRC16:45
*** breton has quit IRC16:45
*** gerhardqux has quit IRC16:45
*** Madkiss has quit IRC16:45
*** raildo has quit IRC16:45
*** hugokuo has quit IRC16:45
*** blogan has quit IRC16:45
*** grantbow has quit IRC16:45
*** htruta has quit IRC16:45
*** samueldmq has quit IRC16:45
*** ptoohill has quit IRC16:45
*** rodrigods has quit IRC16:45
*** diazjf has quit IRC16:45
*** lhinds has quit IRC16:45
*** alex_xu_ has quit IRC16:45
*** afazekas has quit IRC16:45
*** notmorgan has quit IRC16:45
*** hrou has quit IRC16:45
*** mnaser has quit IRC16:45
*** tsymanczyk has quit IRC16:45
*** mancdaz has quit IRC16:45
*** flaper87 has quit IRC16:45
*** tjcocozz has quit IRC16:45
*** krotscheck has quit IRC16:45
*** petertr7 has quit IRC16:45
*** Tridde has quit IRC16:45
*** lxsli has quit IRC16:45
*** darrenc has quit IRC16:45
*** gissi has quit IRC16:45
*** gsilvis__ has quit IRC16:45
*** ericksonsantos has quit IRC16:45
*** baffle has quit IRC16:45
*** arunkant_ has quit IRC16:45
*** arunkant has quit IRC16:45
*** david8hu has quit IRC16:45
*** lars2 has quit IRC16:45
*** ccard has quit IRC16:45
dstanekodyssey4me: i think what they've solved is the need to update DB and code at the same time and not DB upgrades without downtime16:45
odyssey4meperhaps it's also good to bear in mind the type of service nova is, versus keystone - nova has many, many agents all over the place... keystone simply has clients16:46
odyssey4meI think16:46
*** rodrigods has joined #openstack-keystone16:46
*** ptoohill has joined #openstack-keystone16:46
*** samueldmq has joined #openstack-keystone16:46
*** htruta has joined #openstack-keystone16:46
*** grantbow has joined #openstack-keystone16:46
*** blogan has joined #openstack-keystone16:46
*** hugokuo has joined #openstack-keystone16:46
*** raildo has joined #openstack-keystone16:46
*** Madkiss has joined #openstack-keystone16:46
*** gerhardqux has joined #openstack-keystone16:46
*** breton has joined #openstack-keystone16:46
*** lifeless has joined #openstack-keystone16:46
*** kfox1111 has joined #openstack-keystone16:46
*** charz has joined #openstack-keystone16:46
*** mordred has joined #openstack-keystone16:46
*** albertom has joined #openstack-keystone16:46
*** rmstar has joined #openstack-keystone16:46
*** Daviey has joined #openstack-keystone16:46
*** x58 has joined #openstack-keystone16:46
*** anteaya has joined #openstack-keystone16:46
*** mdavidson has joined #openstack-keystone16:46
*** dulek has joined #openstack-keystone16:46
*** david-lyle has joined #openstack-keystone16:46
*** breitz has joined #openstack-keystone16:46
*** jerrygb has joined #openstack-keystone16:46
*** fawadkhaliq has joined #openstack-keystone16:46
*** chmouel has joined #openstack-keystone16:46
*** jrist has joined #openstack-keystone16:46
*** d0ugal has joined #openstack-keystone16:46
*** diazjf has joined #openstack-keystone16:46
*** lhinds has joined #openstack-keystone16:46
*** mnaser has joined #openstack-keystone16:46
*** alex_xu_ has joined #openstack-keystone16:46
*** afazekas has joined #openstack-keystone16:46
*** notmorgan has joined #openstack-keystone16:46
*** hrou has joined #openstack-keystone16:46
*** lxsli has joined #openstack-keystone16:46
*** tsymanczyk has joined #openstack-keystone16:46
*** mancdaz has joined #openstack-keystone16:46
*** flaper87 has joined #openstack-keystone16:46
*** tjcocozz has joined #openstack-keystone16:46
*** krotscheck has joined #openstack-keystone16:46
*** petertr7 has joined #openstack-keystone16:46
*** Tridde has joined #openstack-keystone16:46
*** darrenc has joined #openstack-keystone16:46
*** gissi has joined #openstack-keystone16:46
*** gsilvis__ has joined #openstack-keystone16:46
*** ericksonsantos has joined #openstack-keystone16:46
*** baffle has joined #openstack-keystone16:46
*** arunkant_ has joined #openstack-keystone16:46
*** arunkant has joined #openstack-keystone16:46
*** david8hu has joined #openstack-keystone16:46
*** lars2 has joined #openstack-keystone16:46
*** ccard has joined #openstack-keystone16:46
stevemarodyssey4me: you got it16:46
dstanekodyssey4me: for example adding an index (i think) will result in some downtime for the duration of the alter16:46
odyssey4medstanek I am happy to be wrong. :) I haven't yet had enough time to really dig into the real details of both approaches.16:47
stevemarodyssey4me: i tihnk mfisch upgrading keystone in < 20 minutes,, and just had a tiny planned outage16:47
lbragstadthat depends on various SLAs16:47
dstanekodyssey4me: i need to see if my assumptions are still true - but 'alter table' use to lock the table during the execution16:48
odyssey4meI'm happy for people to add additional reasearch here:
notmorganok looking16:48
*** topol has quit IRC16:48
*** topol has joined #openstack-keystone16:48
*** aix has quit IRC16:48
*** aix has joined #openstack-keystone16:48
*** elmiko has quit IRC16:48
*** elmiko has joined #openstack-keystone16:48
*** woodster_ has quit IRC16:48
*** woodster_ has joined #openstack-keystone16:48
notmorganoooh and de-netsplit here16:48
*** btully has joined #openstack-keystone16:48
odyssey4meit's a planning etherpad for putting together a framework in OSA - that'll eventually make its way into a spec.16:49
odyssey4methe references would be of general interest16:50
bknudsondstanek: realistically, don't we only care about locking the token table?16:50
notmorgankrotscheck: X-Auth-Token is the actual auth-token sent (both used to/from keystone and to endpoints), X-Identity-Status, X-Roles, X-Service-Catalog, X-User-Id, X-Tenant-Id, are all Middleware driven16:50
*** iurygregory has joined #openstack-keystone16:50
lbragstadthe revocation table gets big, too16:50
*** roxanagh_ has joined #openstack-keystone16:50
bknudsony, revocations, too would be a problem since you couldn't delete tokens.16:50
dstaneklbragstad: so it looks like it's true as of a year ago16:50
notmorgankrotscheck: those other ones are set when passing down to the application under middleware and are translated to HTTP_X_*16:50
lbragstaddstanek yeah16:51
notmorgankrotscheck: so the underlying app can access them16:51
notmorgankrotscheck: so, not sure the difference between allow and expose.16:51
krotschecknotmorgan: Coooooool. So the middleware headers are request only?16:51
dstanekbknudson: depends if the lock blocks reads because we read all kinds of stuff when generating token responses16:51
notmorgankrotscheck: the middleware headers are stripped by keystonemiddleware and then repopulated based on the auth token... NOW...16:51
bknudsony, role assignments16:51
krotschecknotmorgan: Allow is "I want to send you this header, may I" from the client. Expose is "I'm sending you this response, and here's the headers you're allowed to read"16:51
notmorgankrotscheck: i am trying to change that16:51
bknudsonI assume reads would be blocked16:51
dstanekbknudson: me too16:52
lbragstadso if doing an ALTER statement locks the table, and we opt to create a new table and migrate data, we run into possible performance issues, as well we referential integrity head aches16:52
*** chmouel has quit IRC16:52
notmorgankrotscheck: so we only authorize a request once at the edge, so you still will want to accept them and let keystonemiddleware strip them/repopulate them16:52
lbragstadbknudson dstanek reads would be blocked, at least according to that link16:52
dstanekxek: lbragstad: bknudson: my point is that we need to know what we are solving because I'd hate to add tons of complexity only to realize we still have downtime16:52
*** d0ugal has quit IRC16:52
notmorgankrotscheck: auth-token will be used in both cases16:52
lbragstaddstanek ++16:52
xekI know that in case of postgres, there is also a lock, but it's brief in cases like adding a column16:52
lbragstaddstanek I think we want to solve the "no-downtime" problem16:53
krotschecknotmorgan: Yes, I will always want to accept them, because else the CORS middleware will block the request because security. (urm... yeah. Because that's secure?)16:53
notmorgankrotscheck: yeah haha.16:53
krotschecknotmorgan: But those middleware headers aren't sent back, yes?16:53
xekand in case of the tool I found for mysql, it operates on a copy of the table, so the old table is accessible the whole time
lbragstadI, as an operator, want to be able to upgrade my keystone deployment without an interruption of service16:53
lbragstadcloudnull odyssey4me right? ^16:53
notmorgankrotscheck: they should not be part of the response16:53
dstanekxek: usually in mysql is fast. the issues we had were generating indexes on really large tables that were replicated16:53
notmorgankrotscheck: they should only be part of the "what is sent" to the application16:53
krotschecknotmorgan: Same with X-Auth-Token? (it's not in the response?)16:54
notmorgankrotscheck: depends on where you use X-Auth-token and X-Subject-token and X-Service-Token16:54
notmorganthose are valid to get as responses from keystone16:54
notmorganbut to apps behind middleware it should not be part of the response.16:54
notmorganfrom the app*16:54
notmorgankrotscheck: here is the (very basic) diagram of what i'm working towards, and i think it works with what you've laid out16:55
krotschecknotmorgan: Right. I'm talking about twhat the user agent (browser) sees.16:56
cloudnulllbragstad: yes. however as an operator I understand that some downtime maybe required.16:56
lbragstadcloudnull what would those cases be to you?16:56
lbragstadcloudnull or what cases are acceptable to have downtime?16:56
notmorgankrotscheck: with the goal that the browser/agent/horizon lives outside [perhaps]16:56
*** timcline has joined #openstack-keystone16:57
krotscheckYep. That makes sense.16:57
notmorgankrotscheck: i'm actually setitng up a PoC of this now so i can hack on it some and see if i can get everything to talk sanely through the API layer then make it so projects don't re-auth all the time16:58
notmorgankrotscheck: :)16:58
notmorgankrotscheck: for svc->svc that is16:58
notmorgankrotscheck: also it's an excuse to learn OSAD, Shade, and Ansible better16:59
notmorgankrotscheck: i might bug you to poke at it once i have it stood up16:59
krotschecknotmorgan: Let me know how that goes, I have to write a js-keystonelib pretty soon.16:59
odyssey4melbragstad Generally speaking, Operators like things to be predictable. If down time is required, then it is - but it should be easy for me to understand how much down time is required and what the risk is of failure or that time being extended. Every instance of down time has to be justified when planning changes and often has ripple effects.16:59
* notmorgan has been writing LUA integration for keystonemiddleware to run in HAProxy17:00
notmorgankrotscheck: ^_^17:00
*** btully has quit IRC17:00
notmorgankrotscheck: if i'm lucky i can do this for $50/mo or so [i need to get a real lab]17:00
odyssey4meSo I would say that down time is undesirable, but can happen when necessary. When it is necessary then it must be for a predictable period.17:00
*** btully has joined #openstack-keystone17:01
krotschecknotmorgan: Ya know, our entire team needs a hardware lab.17:01
lbragstadodyssey4me makes sense17:01
krotschecknotmorgan: Ok, here's an update, does that all make sense?
*** jistr has quit IRC17:01
odyssey4melbragstad That's where the expand/contract mechanism can help - do what can be done online and minimise the offline time.17:01
openstackgerritHenrique Truta proposed openstack/keystone: Restricting domain_id update
*** pgbridge has joined #openstack-keystone17:02
odyssey4menotmorgan I saw your tweet on that - nifty work. :)17:02
bknudsonyou might not be offline, but operations are all locked up waiting for the schema change17:02
xekodyssey4me, I think data migration is what can be more unpredictable, so they should be done online rather then offline17:03
odyssey4mexek does data migration involve more risk?17:03
*** d0ugal has joined #openstack-keystone17:03
xekodyssey4me, well I think it's more unpredictable then schema changes, because it depends how much data you have to migrate17:04
odyssey4merisk of failure, data loss, etc?17:04
xekodyssey4me, so you have to think about it case by case17:04
xekodyssey4me, I was thinking about the execution time17:04
*** ChanServ has joined #openstack-keystone17:04
*** sets mode: +o ChanServ17:04
openstackgerritMerged openstack/pycadf: relax id validation
*** ChanServ sets mode: +v topol17:05
*** mnaser has quit IRC17:05
*** mnaser has joined #openstack-keystone17:05
*** e0ne has joined #openstack-keystone17:05
*** rderose has joined #openstack-keystone17:05
*** d0ugal is now known as Guest6334017:05
*** notmorgan is now known as Guest6639117:05
cloudnulllbragstad: i dont have a specific case that makes downtime "acceptable". however if we cant get the a rolling upgrade 100% then I understand.17:05
*** btully is now known as Guest8704917:05
*** chmouel has joined #openstack-keystone17:06
cloudnullbut i think a rolling upgrade should be the target (even if its a moving one)17:06
*** Guest66391 is now known as morganfainberg17:07
*** morganfainberg has quit IRC17:07
*** morganfainberg has joined #openstack-keystone17:07
*** ChanServ sets mode: +v morganfainberg17:07
*** morganfainberg is now known as notmorgan17:07
*** tyagiprince has quit IRC17:07
openstackgerritSteve Martinelli proposed openstack/keystone: Ensure pycadf initiator IDs are UUID
xekmaybe we could merge the unit test, and at least require that the incompatible schema changes would be listed in the release notes17:08
xekI think some knowledgeable DBAs could work around those with triggers17:08
*** belmoreira has joined #openstack-keystone17:08
*** tyagiprince has joined #openstack-keystone17:09
lbragstadbknudson dstanek thoughts? ^17:09
dstanekxek: i do like that idea17:10
bknudsonA unit test shouldn't cause problems, so I'm fine with that.17:10
lbragstadxek think we could rework the spec for that scope?17:11
*** rcernin has joined #openstack-keystone17:11
lbragstadspecifically detailing the responsibility on the dba after they read the release notes17:11
lbragstadit could also use a rename17:12
lbragstadsince it not necessarily an online-migration anymore17:12
lbragstadits more or less a primer for a real solution to online-migrations17:12
*** gyee has joined #openstack-keystone17:12
*** ChanServ sets mode: +v gyee17:12
*** Guest63340 is now known as d0ugal17:12
*** d0ugal has quit IRC17:13
*** d0ugal has joined #openstack-keystone17:13
*** jed56 has quit IRC17:13
xeklbragstad, ok, do you know maybe of someone with DB skills that could review it?17:13
lbragstadxek I thought that was you!17:15
xekwell, I'm not entirely sure all changes could be worked around in this way17:16
xekand I don't know about mysql17:17
*** Guest87049 has quit IRC17:17
lbragstadxek if you know of someone that is a dba, it would be nice to get their opinion17:19
*** EinstCrazy has joined #openstack-keystone17:19
lbragstadxek otherwise I'd value any operator feedback17:19
*** diazjf has quit IRC17:19
xekdstanek, bknudson, lbragstad, summing up, we are worried mainly about performance?17:20
lbragstadxek i think we're worried about a bunch of things17:21
bknudsonI'm concerned about the performance and complicating the code which will lead to bugs and potential security vulnerabilities.17:21
*** navid_ has joined #openstack-keystone17:21
xeklbragstad, I also think that the DB would have to somehow know what code version is connecting to it17:21
dstanekxek: i'm more worried about the added complication that is bound to break things17:21
dstaneksince we are supposed to be one the things that prevents unauthorized cloud use, I think we need to be careful17:22
lbragstadxek dstanek new question17:22
lbragstaddo we really need a spec for adding a unit test?17:23
bknudsonwe don't need a spec to add a unit test17:23
dstaneklbragstad: this one maybe... since it will change the way we work17:23
*** btully has joined #openstack-keystone17:23
lbragstadI think we need a spec for online migrations, but I don't think this addresses online migrations17:23
dstanekthis isn't just verifying new behavior, but forcing a new development process17:23
lbragstadso, maybe a blueprint so that we can track it17:23
dstanekwe also need to add the docs for this too17:24
dstanekwhat do i need when i need to change the schema17:24
lbragstadessentially, the way in which we migrate schema versions wouldn't change17:24
lbragstadbut we would just add release notes for every migration17:25
dstaneklbragstad: but doesn't the test force you to add expection when making a "breaking" change?17:25
*** EinstCrazy has quit IRC17:26
*** belmoreira has quit IRC17:26
bknudsonthe docs need to be in the test case17:26
xekmaybe adding an exception would just require you to make a release note, that the upgrade requires downtime, with information on what data will be migrated (for the operator to know how much time it will take)17:27
xekI think there may be some future releases which wouldn't do braking changes on the DB, and then the operator will know, that he doesn't have to stop his services17:28
odyssey4mexek lbragstad it may be good to continue the thread here:
*** dims has joined #openstack-keystone17:29
dstanekbknudson: you don't think that should be part of the developer docs?17:30
bknudsondeveloper docs would probably have more detail, but if it was with the testcase then you'd see the docs in the gerrit review.17:31
bknudsonmaybe we need to have a review checklist in developer docs17:35
*** navid_ has quit IRC17:35
*** ryansb has joined #openstack-keystone17:36
lbragstadodyssey4me i can respond with the new direction?17:38
odyssey4melbragstad sure, do it17:39
lbragstadso, to summarize17:39
odyssey4meit might be a good idea to reach out to some of the working groups too - for example the large scale deployments and enterprise working groups17:39
*** timcline has quit IRC17:40
lbragstadwe will have a new unit test to detect certain migration that could incur downtime, which will require documentation/release notes17:40
*** lhcheng has quit IRC17:40
lbragstadthen do we target online-migrations as a spec for N?17:41
lbragstadonce we have more feedback?17:41
xeklbragstad, I think so, or maybe we could still discuss it at the midcycle?17:42
lbragstadxek yeah, that would work.17:42
xekalthough I can't promise I will be there17:42
lbragstadxek I don't think it will be hard to get the spec in or marked complete17:42
lbragstadxek the tricky part is allowing time for other things to implement what ever online-migration we go with17:42
lbragstadall migrations in Mitaka would have to adhere to it17:43
dstanekxek: i wonder if that guy that wrote about the different options actually did them... the option three that he described is deficient compared to what we had to do. it wasn't table at a time it was groups based on the relations17:44
lbragstaddstanek where are you reading that?17:45
dstanekit's about migrations with no downtime17:46
*** NM has joined #openstack-keystone17:47
xekok, thanks for all the feedback! I'm rolling myself home, see you tomorrow17:47
dstanekxek: have a good one17:47
lbragstadxek o/17:47
dstaneklbragstad: AGI would buy a company and want to migrate their systems into the existing ones. we had to do all sorts of crazy migrations to have both websites up during large migrations.17:48
lbragstadi've used pt-online-schema-change before17:49
dstaneklbragstad: think 10s or 100s of millions of customers and their data moving across data centers17:49
lbragstadit's a neat tool17:49
lbragstadand you guys did that with changes to the application17:50
lbragstadand duplicate schemas, etc...17:50
dstanekyes and yes17:50
dstanekeach was different because the source systems were different17:50
lbragstadagreed, every migration is going to be handled differently17:51
dstanekthis should be a lot easier since it's in the sane logical DB, but i worry about making MySQL based assumptions if people are using PG17:52
dstaneklbragstad: pt-online-schema-change looks interesting. i've never used it.17:55
dstaneklbragstad: the foreign key issues it mentions are why we replicated groups of tables17:55
*** sigmavirus24 is now known as sigmavirus24_awa17:57
*** jbell8 has quit IRC18:03
*** mhickey has joined #openstack-keystone18:05
*** btully has quit IRC18:06
*** jbell8 has joined #openstack-keystone18:09
notmorgandstanek: lets just drop support for PGSQL :P18:11
notmorgandstanek: much easier that way18:11
*** ninag has quit IRC18:11
notmorgani mean... does anyone *really* use it?18:11
* notmorgan goes back to the corner18:12
*** ninag has joined #openstack-keystone18:12
dstaneknotmorgan: we can join the cool kids in mongo land....18:12
notmorgandstanek: nope.18:12
notmorgandstanek: we can't cause nope nope nope nope nope18:12
*** harlowja has joined #openstack-keystone18:12
dstaneknotmorgan: thanks dad, i was just trying to have some fun :-)18:13
*** ninag has quit IRC18:13
*** e0ne has quit IRC18:13
notmorgandstanek: fun...?!18:13
*** ninag has joined #openstack-keystone18:13
dstaneknotmorgan: yes, just like smashing your hand with a hammer!18:14
bknudsonI thought hpe had their own mongo driver18:15
*** ninag has quit IRC18:17
*** ninag has joined #openstack-keystone18:17
gyeebknudson, yes we do18:17
gyeeand it works well18:18
gyeewhat's wrong with mongo?18:18
bknudsonI used to think it was a bad idea but then I realized we got rid of all the foreign keys so there's no point to sql anymore.18:18
bknudsongyee: does it make online upgrades easier?18:18
notmorganbknudson, gyee, the issue with mongo is two fold: 1) AGPL (lawyer'd!), 2) testing is ugly right now in OpenStack18:19
gyeebknudson, yes18:19
gyeenotmorgan, valid points,18:19
gyee1) we can't do much18:19
notmorgan3) we have sucked at SQL quite a bit, adding in something that most developers know less about wont help18:19
gyeemongo is easy!18:19
notmorgan"it's webscale"18:20
gyeeif you know how to use a dict, you know mongo already18:20
notmorgangyee: *cough* that doesn't mean people wont design around mongo things that work that (similar to SQL) are terrible ideas18:20
gyeeI am not saying Mongo is better than SQL in all aspects18:21
notmorganat least SQL we have some level of folks who can look at it and go "uhhh. oh god why!?" i think we lack those folks in openstack for mongo18:21
gyeefor rolling upgrade, yes18:21
*** ninag has quit IRC18:21
gyeebut yeah, licensing is a killer18:21
notmorganamong other things.18:21
*** ninag has joined #openstack-keystone18:22
notmorganlack of solid knowledge of mongo and what is good vs "hey it doesn't break"18:22
notmorganmakes it a harder sell18:22
notmorganesp. when *everything* else uses sql18:22
gyeetechnically, you don't need much to use mongo18:22
notmorganif everything went mongo, this argument would go away quickly as people would ramp up18:22
notmorgangyee: that falls into the "well it works" category18:22
gyeeyeah, folks will have to weight the pros and cons18:23
notmorganbut is that a good idea? is it going to fall over as we scale? does it make something in the backend of mongo unhappy in <case>18:23
notmorganso i don't think we can say "move to mongo" because of a few reasons, but the tech side is def an issue too18:23
gyeefor mid-scale, 2-3 site, mongo is fine18:23
gyeeanything large, you'll need to think federation18:24
notmorgangyee: you're countering my statements with "it just works"18:24
notmorgangyee: i disagree with that sentiment18:24
gyeewhat does it mean "it just works"? :)18:24
notmorganwe can't just throw it out there and tell people "yeah this is good" and wait for the fallout. we need slightly more reasoning going into the choices18:24
notmorgani'm saying we don't have the critical mass of knowledge of "good" vs "bad" practices developing against mongo18:25
notmorganyou're saying "yeah but it's fine in small deployments" - those are two different things!18:25
gyeewell, open source is about choice, nothing is certain, use cases are different18:25
gyeewe can only provide data18:25
notmorganand we're not growing that skillset in openstack really18:25
notmorganso, i think a move ot mongo is generally unwise.18:26
notmorganupstream supported mongo*18:26
gyeeI am not advocating anything18:26
notmorganunless folks want to grow that skillset :)18:26
gyeejust sharing out experience18:26
notmorganwhich case, i'm happy to see more options.18:26
notmorgan[but i can also just ignore the storage backends most of the time, yay!]18:26
bknudsonwe have customers complaining about wanting to do online upgrades and I don't think you can totally accomplish it using sql18:27
zaoIf mysql is a problem, go postgres :D18:27
bknudsonunless you use blobs18:27
* zao looks for a winking smiley18:27
*** rderose has quit IRC18:28
bknudsonI mean the extra blobs are essentially nosql already18:28
gyeeextra is a hybrid blob :)18:31
*** ninag has quit IRC18:35
*** ninag has joined #openstack-keystone18:35
*** ninag has quit IRC18:38
*** ninag has joined #openstack-keystone18:38
*** mhickey has quit IRC18:41
dstanekgyee: designing a schema in mongo is more complicated than just using a dict18:43
dstanekgyee:  you still have to design for usage patterns and you have the added complexity of the working set (and memory mapping)18:44
gyeeand sharding18:44
gyeedstanek, for the basic stuff, we don't need awhole lot18:46
dstanekas long as the schema is designed correctly, that is true18:47
*** ayoung has joined #openstack-keystone18:47
*** ChanServ sets mode: +v ayoung18:47
dstaneki'd worry if tokens were ever put in it, but with ephemeral tokens that's not a big issue18:49
*** fawadkhaliq has quit IRC18:49
*** fawadkhaliq has joined #openstack-keystone18:49
gyeedstanek, I am still depressed from Monday Night Football18:51
gyeesaddest game in recent memory18:51
dstanekgyee: yeah, i knew i should have left before the game started18:51
*** btully has joined #openstack-keystone18:52
* gyee is looking forward to the next NFL draft already18:53
*** gsilvis__ has quit IRC18:53
*** fangxu has joined #openstack-keystone18:53
*** aginwala has joined #openstack-keystone18:55
*** jbell8 has quit IRC19:03
*** lhcheng has joined #openstack-keystone19:04
*** ChanServ sets mode: +v lhcheng19:04
*** browne has joined #openstack-keystone19:04
*** rderose has joined #openstack-keystone19:05
*** btully has quit IRC19:07
*** sigmavirus24_awa is now known as sigmavirus2419:09
*** lhcheng has quit IRC19:09
*** btully has joined #openstack-keystone19:10
lbragstadbknudson dstanek for the release notes around schema changes - what all would you expect to be in those notes?19:14
lbragstad1.) what is changing 2.) why it's changing 3.) how to migrate data if any ...19:14
openstackgerritwerner mendizabal proposed openstack/keystone-specs: Multifactor Authentication
*** aix has quit IRC19:20
*** spandhe has joined #openstack-keystone19:21
*** btully has quit IRC19:22
*** pwp has joined #openstack-keystone19:23
*** jasonsb has quit IRC19:24
*** btully has joined #openstack-keystone19:24
openstackgerritwerner mendizabal proposed openstack/keystone-specs: Multifactor Authentication
*** mancdaz has quit IRC19:25
*** mancdaz has joined #openstack-keystone19:26
*** fawadkhaliq has quit IRC19:28
*** fawadkhaliq has joined #openstack-keystone19:28
*** jbell8 has joined #openstack-keystone19:29
*** jbell8 has quit IRC19:30
*** rderose has quit IRC19:31
*** pwp has left #openstack-keystone19:32
*** jbell8 has joined #openstack-keystone19:32
*** briancurtin has quit IRC19:33
*** serverascode has quit IRC19:33
*** pwp has joined #openstack-keystone19:33
*** jraim has quit IRC19:33
*** zhiyan has quit IRC19:33
*** tyagiprince1 has joined #openstack-keystone19:34
*** tyagiprince has quit IRC19:35
*** tyagiprince1 is now known as tyagiprince19:35
*** btully has quit IRC19:37
*** fangxu has quit IRC19:38
*** RichardRaseley has joined #openstack-keystone19:40
*** btully has joined #openstack-keystone19:41
*** flwang has joined #openstack-keystone19:41
*** ayoung has quit IRC19:41
*** browne has quit IRC19:44
*** browne has joined #openstack-keystone19:45
*** aginwala has quit IRC19:45
*** pumaranikar has joined #openstack-keystone19:46
ryansbhi, I'm having a problem that seems to be coming from keystonemiddleware -- is this the right channel or is there a separate one for mw?19:47
*** aginwala has joined #openstack-keystone19:48
*** DuncanT has quit IRC19:54
*** woodster_ has quit IRC19:54
*** ramishra has quit IRC19:54
*** markvoelker has quit IRC19:55
bknudsonlbragstad: if the deployer wants to know how long their system is down then we should say that.19:56
*** gsilvis has joined #openstack-keystone19:57
lbragstadbknudson that would be dependent on how much data they have,19:58
lbragstadbknudson which we don't necessarily know19:58
bknudsonright, so tell them what it depends on19:58
*** markvoelker has joined #openstack-keystone19:59
lbragstadbknudson so, for example "in order to do this migration all entries from x table.row need to be converted to `something`".19:59
bknudsonlbragstad: we can just say that the downtime depends on the number of users or whatever it is.20:00
*** diazjf has joined #openstack-keystone20:01
flwangstevemar: ping20:02
stevemarflwang: pong20:02
flwangstevemar: could you please have a quick look for this
flwangwe're using osc-config to get the credentials but the auth_url is 'auth_url': ''20:03
flwangi suppose it should be like
*** jasonsb has joined #openstack-keystone20:04
bknudsonit's doing which doesn't exist, it needs to be
*** jraim has joined #openstack-keystone20:04
stevemarflwang: sort of, keystone should be able to discover the versions though20:04
flwangbknudson: yep, i see20:04
bknudsonfor some reason it's doing v2 auth.20:04
flwangstevemar: since you're also a osc guy, so it's the special question for you :D20:05
stevemarwhich is weird since it's specifying both user_domain_id and project_domain_id20:05
flwangstevemar: so, anything we should investigate continually?20:09
*** lhcheng has joined #openstack-keystone20:10
*** ChanServ sets mode: +v lhcheng20:10
stevemarflwang: not sure yet! :)20:10
*** jasonsb has quit IRC20:10
stevemarflwang: looking at how the tests are set up now20:10
*** aginwala has quit IRC20:11
*** mhickey has joined #openstack-keystone20:12
flwangstevemar: thanks for the hlep20:15
stevemarflwang: that makes it easier :)20:15
stevemaroh that's a nice idea, using the defined name 'devstack-admin'20:15
* stevemar will steal that idea for osc functional tests20:16
*** serverascode has joined #openstack-keystone20:16
flwangstevemar: is there any change in osc or the keystone related lib?20:16
flwangstevemar: you can't steal, mordred has a patent for that20:17
stevemarflwang: haha20:17
stevemarflwang: i don't think there are any changes that would have affected this20:17
stevemari saw another bug about someone sourcing the devstack rc file and it didn't have a version attached20:18
flwangstevemar: ok20:18
*** zhiyan has joined #openstack-keystone20:19
*** sigmavirus24 is now known as sigmavirus24_awa20:20
mordredstevemar: PLEASE DO use that20:20
mordredstevemar: there is even an oslotest base class which does the work20:20
stevemarmordred: i hear you'll send patent trolls to get me if i do20:21
*** lhcheng_ has joined #openstack-keystone20:21
mordredif anyone uses anything other than clouds.yaml for functional test auth, I swear I will kill them with knives20:21
lbragstadstevemar do we need a version bump of pycadf?20:21
stevemarlbragstad: we did, a new version was released about an hour ago20:22
stevemarfeel free to recheck everything20:22
stevemari just ran a recheck and it's passing gate again20:22
stevemari have a patch to requirements to bump the upper constraints on all keystone libraries, and to block pycadf 2.0.020:23
stevemarmordred: does occ use version discovery?20:24
*** lhcheng has quit IRC20:24
mordredstevemar: occ uses what's configured - I _believe_ that the version discovery URL is being written into the clouds.yaml by devstack20:26
flwangmordred: any comments for this
stevemarmordred: that's what i'm trying to check20:26
mordredstevemar: and I believe it points to v3 by default - we have an extra devstack test for shade that edits the clouds.yaml to force it to version v2 so that we can test that20:26
stevemarmordred: so flwang is sourcing devstack-admin for occ, but when he's running tests, it 404's (missing the v2.0 in auth_url20:27
*** briancurtin has joined #openstack-keystone20:27
mordredstevemar: yes. it should be missing the v2.0 in the auth_url20:28
mordredbecuase the auth_url in clouds.yaml is the version discovery url20:28
mordredbecuase that's what devstack defaults to20:28
*** btully has quit IRC20:28
stevemarmordred: so i guess OSC may not be handling auth_url properly?20:29
flwangmordred: but it's called 'auth_url'20:29
mordredauth_url is unversioned in devstack20:29
flwangmordred: +120:29
mordredbecause it' points to the version discovery url20:29
mordredstevemar: it should be - we use occ to test osc too20:29
*** rderose has joined #openstack-keystone20:30
mordredstevemar: I have to run - just know that what is put into clouds.yaml by devstack does work correctly for v2 and v3 of keystone (we test this with every shade patch)20:30
stevemarmordred: run run, i'll mull it over20:30
mordredif you don't have whatever is wrong with zaqar figured out before I get back, I'll dive in and help fix it20:30
stevemarcool :)20:31
*** btully has joined #openstack-keystone20:32
lbragstadstevemar fyi -
lbragstadstevemar that came in late20:33
flwangmordred: thanks20:33
stevemarlbragstad: yeah i saw movement on it this morning20:34
*** henrynash has joined #openstack-keystone20:34
*** ChanServ sets mode: +v henrynash20:34
stevemarnotmorgan: can you lift your -2 on ?20:34
*** aginwala has joined #openstack-keystone20:38
*** woodster_ has joined #openstack-keystone20:38
flwangstevemar: so should i expect this is a fix from osc-config?20:38
stevemarflwang: honestly, i'm not sure yet20:38
flwangstevemar: otherwise, i may have to hardcode the version for now since it's breaking our gate20:38
stevemarflwang: i think that is OK to do for now20:39
stevemarflwang: it's just the functional job right? it's non-voting i thought?20:39
flwangstevemar: ok, thanks. I will hardcode it for now, and when there is a fix, pls add me as a reviewer20:39
flwangstevemar: it's not voting for client, but it's voting for zaqar server :(20:39
flwangnow we're seeing differents errors, but i hope the server side issue can be fixed as well20:40
stevemarflwang: let me try to reproduce it locally20:42
flwangstevemar: ok, cool20:43
*** btully has quit IRC20:46
*** henrynash has quit IRC20:46
*** ramishra has joined #openstack-keystone20:48
*** DuncanT has joined #openstack-keystone20:48
samueldmq /q stevemar20:48
gordcstevemar: why does keystonemiddleware tell me auth-url is required now.20:49
*** fawadkhaliq has quit IRC20:49
stevemargordc: we're using keystoneauth1 now20:50
gordcstevemar: if i don't want to bother setting a auth-url, how can i do that? i'm not sure what exactly is pulling in keystoneauth1 right nwo20:51
gordcbut my tests are b*tching that i haven't setup auth-url20:51
*** alex_xu_ has quit IRC20:52
*** alex_xu has joined #openstack-keystone20:53
*** EinstCrazy has joined #openstack-keystone20:56
*** RichardRaseley has quit IRC20:57
*** raildo is now known as raildo-afk20:58
bknudsongordc: in a related note, ksm has a test fixture now.21:00
*** EinstCrazy has quit IRC21:00
bknudsonwe should probably revert the change since it's obviously not backwards-compatible.21:01
gordcbknudson: which change?21:03
*** sigmavirus24_awa is now known as sigmavirus2421:03
bknudsonunless that wasn't the commit that changed the behavior21:04
*** EinstCrazy has joined #openstack-keystone21:06
gordcbknudson: not sure. i'm not sure keystonemiddleware should be even loading in our tests so i think it just exposed a conf leak somewhere in our code21:07
*** henrynash has joined #openstack-keystone21:08
*** ChanServ sets mode: +v henrynash21:08
*** aginwala has quit IRC21:08
*** btully has joined #openstack-keystone21:09
*** aginwala has joined #openstack-keystone21:12
*** EinstCrazy has quit IRC21:12
*** lhcheng_ has quit IRC21:13
*** lhcheng has joined #openstack-keystone21:13
*** ChanServ sets mode: +v lhcheng21:13
*** pauloewerton has quit IRC21:14
*** diazjf1 has joined #openstack-keystone21:15
*** diazjf has quit IRC21:17
*** lhcheng has quit IRC21:18
*** btully has quit IRC21:22
lbragstadnotmorgan would you be able to lift your -2 here ? Or are we going to keep it since we have already voted on Mitaka specs?21:29
*** diazjf1 has quit IRC21:31
*** navid_ has joined #openstack-keystone21:31
*** tyagiprince has quit IRC21:32
*** btully has joined #openstack-keystone21:36
gordcbknudson: is keystonemiddleware 4.x suppose to work in liberty too?21:37
*** lhcheng has joined #openstack-keystone21:37
*** ChanServ sets mode: +v lhcheng21:37
bknudsongordc: yes!21:37
bknudsonI don't see why not.21:37
stevemargordc: yes....21:37
bknudsonthe only thing we were supposed to remove is py26 support21:37
stevemarwe just changed it from 3->4 cause of py2621:38
gordci see...21:38
gordci hate debugging broken gate when it runs fine locally21:38
*** dolphm is now known as dolphm_41821:38
*** fangxu has joined #openstack-keystone21:40
stevemaris there something better than for devstack?21:43
stevemari keep running into errors apparently21:43
stevemar(when stacking)21:43
stevemaris there a "nuke it" option21:43
gordc"rm -rf /"21:45
sigmavirus24stevemar: delete everything, make a new VM?21:45
*** ctina_ has joined #openstack-keystone21:45
stevemarsigmavirus24: i'm one attempt away from doing that21:46
*** btully has quit IRC21:48
*** ctina has quit IRC21:49
henrynashgyee: hi21:50
*** ctina_ has quit IRC21:50
henrynashnotmorgan: ping21:51
lbragstadstevemar what issues are you having?21:58
stevemarlbragstad: something about unavailable plugins when the command `openstack image create --os-config=devstack-admin`22:00
stevemarsomething like this:
lbragstadstevemar did you update your project repositories?22:01
stevemarlbragstad: i removed all cloned projects22:01
stevemarand ran clean.sh22:01
* lbragstad hands stevemar a bigger hammer22:02
*** NM has quit IRC22:02
*** mhickey has quit IRC22:03
stevemarlbragstad: thank you sir22:03
dstanekstevemar: i periodically run into issues that force me to create a new VM22:05
dstaneki do so many experiments that it's likely my fault22:05
*** petertr7 is now known as petertr7_away22:06
stevemardstanek: possiprobably22:06
*** thiagop has quit IRC22:06
*** NM has joined #openstack-keystone22:06
*** ninag has quit IRC22:07
*** aginwala has quit IRC22:08
*** ninag has joined #openstack-keystone22:08
*** ninag_ has joined #openstack-keystone22:12
*** ninag has quit IRC22:13
*** ninag_ has quit IRC22:16
gyeehenrynash, back22:17
gyeehenrynash, for implied roles, both prior and impled roles goes into the token22:17
*** henrynash has quit IRC22:26
*** aginwala has joined #openstack-keystone22:29
*** mancdaz has quit IRC22:30
*** mancdaz has joined #openstack-keystone22:31
*** richm has quit IRC22:32
*** navid_ has quit IRC22:32
*** navid_ has joined #openstack-keystone22:33
*** ninag has joined #openstack-keystone22:39
*** edmondsw has quit IRC22:41
*** adelia has quit IRC22:41
*** ninag has quit IRC22:44
*** btully has joined #openstack-keystone22:46
*** rderose has quit IRC22:51
*** rderose has joined #openstack-keystone22:51
*** rderose has quit IRC22:51
*** rderose has joined #openstack-keystone22:51
*** henrynash has joined #openstack-keystone22:52
*** ChanServ sets mode: +v henrynash22:52
henrynashgyee: hi22:52
*** navid_ has quit IRC22:55
*** rderose has quit IRC22:57
*** RichardRaseley has joined #openstack-keystone22:57
henrynashnotmorgan: ping22:57
gyeehenrynash, yes sir22:59
gyeehenrynash, your option #1 make a lot of sense22:59
henrynashgyee: hi….ok…so wanted to try and get us to some lind of agreement here….22:59
henrynashgyee: but you can’t jsut do 1), you have to do 2) and 3)  as well23:00
gyeehenrynash,  I am not worry too much about internal code23:00
henrynashgyee: that’s a lot of API work..and in manke case dulaiction of APIs23:00
gyeeits the API that we need to get it right, since they are contracts23:00
henrynashgyee: and ia just don’t se teh UX problem23:00
henrynashgyee: i simply do not see the confusion23:01
notmorganhenrynash: huh? sortof here23:01
notmorganhenrynash: but only very sortof. my brain is hurting cause i'm staring at neutron23:01
henrynashnotmorgan: ok, sorry….want to talk unicode, but currently depatingdomain specific role with gyee!23:01
notmorgani can do that23:01
henrynashgyee: a domain specific role (i.e. one that was created with a domain id) never goes in a token23:02
notmorganhere is my vote on domain specific roles: Something something, idea something something, mush, something something, as long as we don23:02
notmorgant blow out the token size, i don't care how roles end up in the token23:02
notmorganbut i worry that this is a "nice to have" vs a "super important feature"23:02
gyeenotmorgn, not that, its UX23:03
notmorganlike "oh we'd totally use it...." and then people don't23:03
henrynashnotmorgan: cwe haev real customer who want this23:03
notmorganUX can be worked through23:03
notmorganhenrynash: riht and i had real customers wanting it at metacloud too23:03
gyeeright now, the kinda weird to explain the difference to people23:03
henrynashgyee: a domain role never goes in a token.  Full stop23:03
henrynashgyee: a domain role is a handle to imply a set of roel that do go in a token23:04
gyeeimplied roles does23:04
notmorganwait implied role goes in the token?23:04
gyeehenrynash, have to tried to explain that to the doc people? :)23:04
notmorganno. afaik the implied role explodes out to the full list?23:04
henrynashgyee: implied roles always go in the token23:04
gyeenotmorgan, nope, both prior and implied roles goes into the token23:05
notmorgan[domain role "foo"] = ["role X"] = ["role y,z,q,r,t"] ?23:05
henrynashnotmorgan: in the “implied roles spec”, you have a prior role and those roels taht proor role implies23:05
*** btully has quit IRC23:05
notmorganand the last one is in the token23:05
gyeeall of them23:05
gyeehenrynash, see my last review comment23:05
*** fangxu has quit IRC23:05
gyeeI quoted that paragraph from the spec23:05
notmorganuh if "X" and "yzrqt" are in [in my example] i think it might be bad23:05
henrynashbut in a domain specifc role, then the prior role does NOT go in the token23:06
notmorganbut the "foo" role wouldn't be in.23:06
gyeehenrynash, yes, hence the confusion23:06
notmorganso, henrynash what is the customer asking for23:06
gyeethere's fundamental difference between the two23:06
notmorganlike functionality wise23:06
notmorganwhat is their intent for this?23:06
henrynashgyee: I don’t see the confusion at all…since the role definition states it is a domain specifc role…and they, by defiention, don’t go i tokens23:06
notmorgani think that will help us understand why it is needed23:06
henrynashchrist, the spec says it very clearly23:07
notmorganbecause right now it feels like abstraction for the sake of abstraction at the end user level23:07
gyeehenrynash, I am not worry about that part23:07
gyeeI am worry about overloading the assignment part23:07
henrynashvery very clearly23:07
gyeeassignment API23:07
notmorganhenrynash: i'm sorry i just jumped into the convo and haven't read the spec recently23:07
henrynashgyee; what’s teh worry there23:07
gyeenotmorgan, you are on the hook now :)23:07
gyeehenrynash, the worry is using the same API on two different set of resources result in different outcomes23:08
openstackgerritBrad Topol proposed openstack/keystone: Deprecate ldap Role
notmorganhenrynash: no, i'm sorry the spec does not spell out the reason a customer wants this or what they are trying to accomplish. it is spelling out just waht the role is.23:08
henrynashall we are doing is allowing a domain admin to create handle taht is meaningful to them that can imply a set of roels that were created by a cloud provider who has to define a set of generaic roels since tehy want to saisfy many types of customers23:08
notmorganis this another level of role grouping?23:09
notmorganis that really the intent?23:09
*** EinstCrazy has joined #openstack-keystone23:09
henrynashdomain specif roel groupoing, yes23:09
gyeenotmorgan, yes, they are essentially role groups23:09
gyeethey are not implied roles, at least by definition23:09
henrynashwhich is what I called hem in the first version of tehspec…and everyone barfed23:09
notmorgani feel this is a weird abstraction to add.23:09
notmorganbut not opposed to it23:09
henrynashgyee: that’s sematics…I call implied roels roel groups as well23:09
notmorganand i would move it back to domain-role-grouping23:10
gyeeimplied roles are not role groups23:10
notmorganor something [wont hold it up on naming though]23:10
henrynashgyee: why not23:10
gyeehenrynash, because they got expended into token response, as roles23:10
henrynashso do domain specifc role groups23:11
gyeeso in the token response, can you tell from the roles list whether the giving "role" is role group or implied role?23:11
notmorganhenrynash: i would describe them that way, regardless of the name :)23:11
notmorganhenrynash: like i said i wouldn't hold this up on a naming concern atm.23:11
henrynash(taht was to gyee)23:11
notmorganah ok23:11
gyeeif you can't tell, the they are roles23:11
henrynashsorry :-)23:11
henrynashtehy are all roles!23:11
gyeenope, not role groups23:11
gyeethey don't get to be part of the token23:12
gyeebig difference23:12
notmorganso for a public cloud perspective - i don't care about domain admins defining their own groups23:12
henrynashthe name of teh groiup diesn’t go in the token23:12
*** sigmavirus24 is now known as sigmavirus24_awa23:12
henrynashso in our public cloud d=custoemr absolutely wnat to do that23:12
notmorganfrom a private cloud perspective i would say it is *likely* someone could ask for a new implied-role-grouping that doesn't need to be domain specific23:12
gyeerole groups != roles23:12
gyeeimplied roles = roles23:13
*** topol has quit IRC23:13
gyeethere :)23:13
*** pumaranikar has quit IRC23:13
henrynashthe contest of both are roles23:13
henrynashcontents, sorry23:13
*** EinstCrazy has quit IRC23:13
henrynashok, so we’re not going to agree.23:13
henrynashI just don’t see the UX confusion at all23:14
gyeecall them roles will cause confusions23:14
*** gordc has quit IRC23:14
notmorganso henrynash, down at an implementation detail,23:15
notmorganthe grant in the DB ends up before for the domain-specific-role?23:15
notmorganor the underlying roles?23:15
notmorganjust so i can understand your view on how opaque this is23:15
notmorgan[i wont hold you to this answer if you change it down the line]23:15
*** btully has joined #openstack-keystone23:15
henrynashin the assignment table, the grant has a domain specific role ( i.e. teh ID of a role created with a domain_id as an attribute)23:16
notmorganand when you ask what roles the user has on project, do you get the domain-specific-role or the complete exploded list?23:16
notmorganor both items?23:17
henrynashso it works like groups and inheritance23:17
gyeesee, see, groups :)23:17
notmorgangyee: hold on23:17
henrynashif you use teh GRANT APIs, then you see teh physical assignemnts23:17
notmorganphysical assignment is the domain-specific-grant?23:18
notmorganor the exploded list?23:18
henrynashif you use list_assignments in effective mode, then like groups and inheriatnce (and implied roels) these get exapnded out so you see teheffective roles23:18
henrynashthis is how we do ingerited roels and group assinements today23:18
notmorgani worry this is going to make it harder to determine how a user has role X23:18
notmorganbut i am not able to specifically say it will23:19
henrynashteh list assigment respose, tells you the assignment that lef to a use haveing this role (e.g. inheritance, group, implied. etc.)23:19
notmorganyeah the inhertiance one is easy23:19
henrynashit provides a link to the url that was teh grant23:19
notmorganbut... the domain-specific one is a bit opaque23:20
notmorgandolphm_418: you are not a teapot23:20
gyeeone option would be to make domain-specific roles also implied roles23:20
henrynasherr, inheritance ain’t that eash (e.g. a user could have got a role on a project form a group grant on teh parent domain)…actualy teh common case in federation23:20
notmorganhenrynash: but it is a known role23:21
notmorganvs a role that confers some set of roles23:21
*** lhcheng has quit IRC23:21
notmorganbut again, i can't definitely say it will be an issue23:21
notmorganso, i don't see this as a huge value add.23:22
henrynashwell, there are all know roles….but yes, in erither of the solutions being proposed, the url that caused the user to get the actual role would refernce some other handke (a role-group or a domain-specific-role)23:22
notmorganbut as long as we can provide transparency on these new role constructs i am not opposed23:22
notmorgani also am not opposed to it being a top-level implied role, just owned at the domain level23:22
henrynashi thnnk we HAVE to be able to do that23:22
notmorganso the UX is identical23:23
*** pwp has quit IRC23:23
notmorganso here is my take: i think this is a very, very, very narrow use-case and not a value add23:24
henrynashmy proposal is really that…the top level role is domain specific, which implies a set of glocal roles (i.e. ones that turn up i policy files)23:24
notmorganit will be used in the minority of minority of deployments at best23:24
notmorgani think it makes things harder to follow unless we are very careful how we represent it23:24
notmorgani don't have a strong read on how important this is going to be for the environments that want it23:24
notmorgani'm not strongly opposed to it.23:25
notmorgani also am slightly more opposed than for it.23:25
henrynashok, apprecaite the candor23:25
henrynashwnat to swicth to unicode and url safe names?23:25
notmorgani would encourage this to be, if anything, a domain-specific reporesentation of the implied roles (with all the pitfalls that come with that)23:25
henrynashsoemthing dear to your heart, I know23:25
notmorganso the UX is the same if it is added23:25
henrynash(on doman roles….that is exactly teh proposal)23:26
notmorganthat way it's the same code-paths, same representations, etc, but allowing the domain to control it.23:26
notmorganand if the grouping at the global implied role appears in the token, the domain one would too23:26
notmorgani do mean identical ux23:26
gyeenotmorgan, don't think they are the same23:26
henrynashahh, hmm no we cant do that23:27
notmorgangyee: i'm saying if we do this, it should be the same23:27
notmorganhenrynash: force a prefix on them. OS_DOMAIN_ROLE:<rolename>23:27
gyeenotmorgan, how can that be? 1) domain-specific roles don't get included in the token, and 2) they can't be nested23:27
notmorganhenrynash: but i would say it should be *the* same mechanism23:27
henrynashso domain A has specific role called “developer” and so does domain B…but the imply differnet roles23:27
notmorganhenrynash: namespace it w/ the domain ;)23:28
openstackgerritMerged openstack/keystone: Add release notes for mitaka-1
gyeeif we are using the same API to infer roles, they will return different results23:28
notmorganuse the domain_id. *shrug*23:28
notmorgani'm done, i've added my $0.000000223:28
henrynashI was only pinging you to talk about unicode…not this!23:29
notmorganyou got more than you bargained for23:29
flaper87stevemar: hey, thanks for the review here:
flaper87if you have a chance, it'd be great to have your vote on the latest PS23:29
flaper87stevemar: ^23:29
henrynashso ho would a current customer you named their porjects in unicode handle urls today?23:29
notmorganflaper87: hey could use you to put pressure on nova to help make it so glance can live at a sub-url23:29
notmorganflaper87: if you don't mind ;)23:29
henrynashhow would they do : GET /projects?name=myrpojectname23:30
notmorganflaper87: the current answer is "make nova use the catalog", unfortunately that is not a small change. but also as an FYI just letting you know glance cannot live at like http://<api>/image/<glance url>23:30
notmorganflaper87: because nova explictly assumes /23:30
flaper87notmorgan: putting pressure? on nova? to benefit keystone and glance? I'm in23:31
notmorganflaper87: hehe23:31
notmorganflaper87: or at least helping nova fix this23:31
notmorganflaper87: here was my first pass but see sdague's response (which is fair)23:31
dolphm_418notmorgan: when i get all steamed up, i just shout23:31
flaper87notmorgan: jokes apart, what do you mean exactly ?23:31
notmorganflaper87: nova cannot use glance if glance is mounted as a sub-url of a host23:32
notmorganit must exist at /23:32
notmorgannova has no capability to handle /<string>/<glance_urls>23:32
henrynashnotmorgan: I do, honestly, want to understand how customer are using unicode nmed projects and domains oday23:32
notmorganflaper87: because they explictly configure HOST and PORT or do url_parse and drop the path23:32
notmorganhenrynash: easily, projects only ever get references in APIs via id23:33
flaper87notmorgan: ah-ha! gotcha. Lemme look at it. FWIW, we're doing quite some work on the nova.images code to allow it for using glance v2. Probably, making this change in Mitaka is going to be hard but let me take a look and chime in if needed23:33
notmorganhenrynash: names are really only used for "viewing"23:33
henrynashnotmorgan: so how would they do: GET /projects?name=myrpojectname23:33
stevemarflaper87: oh that one, i'll get back to that once mitaka-1 closes up23:33
notmorganflaper87: make it use the service catalog to lookup glance instead ;)23:33
notmorganflaper87: that would be even better.23:33
notmorganflaper87: but seriously, url-paths are important so glance can be sub-url mounted23:34
stevemarflaper87: i won't be reviewing stuff until monday :(23:34
flaper87notmorgan: yep, making it use the service catalog is right after this v1 -> v2 work23:34
notmorganflaper87: it's important for a PoC i'm doing to try and solve some auth-woes23:34
stevemarbha, lemme take a quick look23:34
stevemari'm so easily distracted23:34
flaper87stevemar: wuuuut? WHY? :(23:34
flaper87stevemar: still like you, don't worry23:34
stevemarflaper87: vacation tomorrow til sunday :D23:34
henrynashnotmorgan: so sorry to bug, but to my question, how would they do: GET /projects?name=mypojectname23:35
*** dulek has quit IRC23:36
notmorganhenrynash: unicode/utf-8 can be encoded in the URL23:37
notmorganand query string23:37
notmorgani am opposed to explicitly locking that out23:37
notmorganas was described23:37
*** aginwala has quit IRC23:37
flaper87stevemar: oh, in that case, do not load gerrit! I repeat, do not load gerrit23:37
henrynashok, so origionally the spec said “you cane’t have the reserved characters” in the name23:37
henrynashbut peopel wanted more clarity23:38
henrynashso I assume you would be OK with it as it was?23:38
*** btully has quit IRC23:38
*** dulek has joined #openstack-keystone23:39
*** aginwala has joined #openstack-keystone23:40
*** fangxu has joined #openstack-keystone23:41
henrynashnotmorgan: so (i think) you would be Ok with the wording I had in the previous version of the proposal that said “the name cannot contain any of the reserved charcters as described in section 2.2 of ” ?23:42
notmorganhenrynash: yes.23:43
notmorganhenrynash: unless they are encoded appropriately if possible23:43
notmorganand we need to decode appropriately23:43
notmorganthat was my dissent in the spec23:43
flaper87jamielennox: you also reviewed this once. It'd be awesome to get your feedback as well before we merge it:
henrynashnotmorgan: ok, let me circuel back to whoever didn’t like that stateemtn before!23:44
henrynashnotmorgan: always aiming for less dissnet!23:44
henrynashdissent, even23:44
*** henrynash has quit IRC23:45
*** slberger has left #openstack-keystone23:47
*** csoukup has quit IRC23:54
*** RichardRaseley has quit IRC23:56
*** dolphm_418 is now known as dolphm23:57

Generated by 2.14.0 by Marius Gedminas - find it at!