18:02:10 #startmeeting keystone 18:02:11 Meeting started Tue Sep 29 18:02:10 2015 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:14 traffice is bad today 18:02:14 The meeting name has been set to 'keystone' 18:02:21 gyee: for sure 18:02:27 #link https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting 18:02:32 agenda has been updated 18:02:36 i tossed a few items up 18:02:56 morgan: around? 18:03:00 stevemar: me too (3 mins ago) 18:03:16 yup, i see that 18:03:22 hail to stevemar! 18:03:25 * dstanek is here. better late than never 18:03:26 first agenda 18:03:34 #topic ptl change 18:03:41 well this is awkward :P 18:03:44 there's a new sheriff in town 18:03:49 stevemar: congrats! 18:03:52 ++ 18:03:55 morgan did an awesome job, let's all thank him first 18:03:57 +1 18:04:03 O/ and btw it said not me 18:04:07 thanks to everyone who ran. it was a tough choice 18:04:13 bknudson: ++ 18:04:20 stevemar, you prefer cash or check? 18:04:28 gyee: reviews 18:04:30 gyee: jewels 18:04:34 reviews too 18:04:42 yes 18:04:46 beer? 18:05:06 hope i don't disappoint :) but i feel like i have the world's largest safety net with such an awesome team 18:05:30 * morgan welcomes the new stevemar keystone overlord :P 18:05:40 stevemar: you won't, just keep rockin' :-) 18:05:43 you'll do great, just like our previous leaders 18:05:43 stevemar: we can always stage a coup if we have to :-) 18:05:52 \o 18:06:02 shaleh: i like your style 18:06:24 * morgan pours a drink and sits back. 18:06:27 not much else to say here, let's go on to a few lighter topics before we get into the weeds 18:06:28 start our own keystone project 18:06:37 #topic meeting time 18:06:57 shaleh, for the complete democracy we need an opposition! )) 18:07:14 we've had this time for a while, and it works, but i feel like we need to bring it up every release, just in case 18:07:28 jamielennox: amakarov marekd, looking at you guys 18:07:29 no issue with meeting time here 18:07:32 if this time didn't work for someone they're probably not here. 18:07:38 hehe 18:07:43 it really can't go any earlier for me 18:07:52 as is i'm about 50/50 18:07:55 I'm fine with current meeting time 18:08:03 but i'm ok with the current 18:08:07 i'm good with the current time 18:08:07 stevemar: we need to make the meeting at least 7hours later /s 18:08:23 morgan: just to really get jamielennox 18:08:30 morgan: deep night for me :( 18:08:39 it will be 4am for me and breton 18:08:41 Note the "/s" ;) 18:08:52 marekd: you're good with current? 18:08:59 stevemar: yeah 18:09:04 alright 18:09:19 as usual, no need to alternate meetings or have a second time, thats nice to hear 18:09:24 stevemar: i said a meeting 7hrs later would be a deep night for me 18:09:25 this time is great for us here in Brazil :) 18:09:27 and saves me from creating a ML post 18:09:44 neeeext topic 18:09:54 given geodistribution of team members i think that's the only reasonable team 18:10:08 #topic liberty-rc2 do we need one? 18:10:18 Yes 18:10:21 I assume one is needed for translations 18:10:23 For translations 18:10:31 alright, for translations 18:10:44 how do we know when translation work is complete? 18:11:00 stevemar: you just talked to dhellmann an hour ago about this... 18:11:04 :P 18:11:22 morgan: well yes, i wanted to use this time to see if there's anything else 18:11:45 i think this one is handy: https://review.openstack.org/#/c/215870/ 18:12:41 if an admin creates endpoints using v3, then v2's endpoint-list doesn't bring them up, that's my understanding anyway 18:12:53 stevemar: when would be liberty-rc2 cut? 18:13:05 marekd: next week iirc 18:13:13 marekd: soon, we still have a bit of time, maybe a week 18:13:15 morgan: ok. 18:13:27 we can backport https://review.openstack.org/#/c/215870/ if it doesn't make the rc 18:14:04 i was pretty close last time i looked 18:14:28 i would also take a look at this https://review.openstack.org/#/c/206561/ 18:14:28 i just wanted to verify the json coming back still looks correct and then i'd be happy with it 18:14:29 dstanek: agreed, i pulled it down and tried it out, i'd be happy to vouch for it 18:14:43 i can do that today and throw up a new vote 18:14:55 dstanek: cool 18:14:56 nevertheless we will need to backport it to kilo. 18:14:59 so we need to map 1 endpoint ID to multiple? 18:15:12 sorry I mean multiple to 1 18:15:17 i'd agree with having the feature, code seems strange to me - but i don't think the brain has really kicked in yet 18:15:18 marekd: hold up a sec, we can get into the other one in a minute 18:15:37 gyee: yes 18:15:48 jamielennox: drink moar coffee 18:16:04 would that be problematic as we need to maintain multiple IDs? 18:16:12 * lbragstad hands jamielennox more coffee 18:16:16 stevemar: yes sir! 18:16:17 jamielennox: it's mostly strange because of how we changed v2 and v3 endpoints 18:16:31 gyee: we don't in v2 18:16:41 morgan: so umm, how's this work, hypothetically if 215870 is merged, and we want it in rc2, we need to merge it to master - then cherry-pick it to liberty? and cut rc2 from stable/liberty? 18:16:49 yea, just looking as to why we build a v3_endpoints dict and iter that again, but ok 18:16:51 Yep 18:16:59 morgan: i am learnding 18:17:09 Back port to stable/Liberty which is where rc2 will be tagged from 18:17:15 \o/ 18:17:29 we used to have a -proposed branch 18:17:40 i like this way more, it's easier 18:17:40 I think https://review.openstack.org/#/c/217373/ should be backported to liberty and kilo 18:18:17 edmondsw: that's would be re-releasing a minor version of keystonemiddleware 18:18:23 yes 18:18:38 oh it's approved, doh 18:19:00 dstanek, ahh, its using the public endpoint ID as the legacy endpoint ID? 18:19:16 but what about the backport? that will require a new release of something, right? 18:19:21 I don't think we can do a minor version change in stable/liberty, it'll be the patch #. 18:19:37 morgan: for edmondsw's question, does the same logic still hold for keystonemiddleware? or do we just tag a new release from master? 18:19:45 http://semver.org/ 18:19:49 We can only do patch level .Z changes 18:19:58 For stable branches 18:20:03 do we do release of kilo for kmw? or is it just a normal new release from master? 18:20:22 dstanek: they have stable branches 18:20:24 Middleware has stable branches 18:20:47 If this is a real bug fix, it is a .z patch level 18:20:59 If it is a .y patch level it isn't a bug fix 18:21:03 the change should be released in master before it's released in stable releases. 18:21:05 the only hitch is updating the requirements in the stable branch too right? 18:21:10 It is a feature/behavior change 18:21:11 it's a real bug fix 18:21:40 stevemar: adding new requirements to stable is a no-go in most cases 18:21:45 edmondsw, yeah...that is a bug fix 18:21:58 So work around requirements changes. 18:22:27 If you do a .z release requirements won't change 18:22:31 In stable 18:22:38 #todo stevemar to bug dhellmann about ksm refresh and pulling in edmondsw's change 18:22:42 Upper bounds will (perhaps) 18:22:51 i'll dig into this 18:23:00 It will be a back port for that change. Simple 18:23:06 morgan: is there an rc2 in lp? 18:23:26 stevemar: for keystone or middleware? Middleware doesn't do rc 18:23:32 My 18:23:35 keystone proper 18:23:53 For keystone that is pending rel management to add the milestone 18:24:08 Talk to dhellmann and/or ttx 18:24:09 morgan: so then hows it work for targeting bugs? 18:24:24 how's rc2 work? We release right away, or on some schedule? 18:24:28 is there an rc2-potential for bugs? 18:24:34 ayoung: right away 18:24:36 Mark the bug as rc-potential and get the milestone added 18:24:40 Target the bug 18:24:41 bknudson: that would be a good idea 18:24:45 Merge the fix 18:24:46 cool 18:24:51 alrighty 18:24:53 Tag rc2 when needed 18:24:57 i think i already tagged it 18:25:16 :) 18:25:25 okay, marekd your go, you wanted to talk about another potential rc2 bug? 18:26:02 stevemar: yeah 18:26:29 https://review.openstack.org/#/c/206561/ this may actually blow up db_sync keystone 18:26:32 including kilo 18:26:50 i can work on it tomorrow and address the comments. 18:27:20 actually bug desc is better to read: https://bugs.launchpad.net/keystone/+bug/1478961 18:27:20 Launchpad bug 1478961 in Keystone "db sync on federation failed if there is existing data" [High,In progress] - Assigned to Marek Denis (marek-denis) 18:27:22 marekd: merging that change would blow up db_sync? or things will blow up without it? 18:27:27 so that won't require a backport migration will it? 18:27:45 i think it's just a liberty fix lbragstad 18:27:53 lbragstad: if a migration is back ported it has to be idempotent. 18:27:54 stevemar: cool, so no backport 18:28:00 dstanek: without that change things *might* blow up db_sync in some cases 18:28:15 stevemar: lbragstad this actually should be backported to kilo too. 18:28:21 This is another case where we need to collapse the separate migrate repos into the main one 18:28:23 marekd: is it possible to get a test to generate data to get into that situation? 18:28:39 marekd: that is likely un backportable to kill 18:28:42 Kilo 18:28:57 morgan: why? 18:29:10 If 006 doesn't exist in kilo you can't back port it 18:29:34 You can't have gaps and you can't reuse numbers 18:29:54 If it 18:30:00 for M we need to consider if we want the placeholder migrations 18:30:00 morgan: uh, let me check 18:30:02 no placeholders for the federation migrations? 18:30:22 dstanek: no placeholders for any non-main repo 18:30:34 dstanek: no placeholders for any extensions unfortunately 18:30:37 morgan: didn't know that and i've never bothered to look 18:30:39 We need to collapse all of these into the main repo 18:30:57 As part of mitaka to fix this silly 18:31:02 State 18:31:20 morgan: https://github.com/openstack/keystone/tree/stable/kilo/keystone/contrib/federation/migrate_repo/versions correct me if i am wrong but we do have 006 in kilo ? 18:31:24 The split repos was (in hindsight) a bad idea 18:32:04 Ok... Hm.. Maybe you can backport it 18:32:12 halfway through 18:32:17 Be wary. Back porting migrations is scary 18:32:17 and I really want to talk roles... 18:32:26 006 is already doing something else. 18:32:46 morgan: the only reason would be gaps in migrations numbering or something more 'political' ? 18:32:49 Anyway.. Just be warned there be dragons here 18:32:50 morgan, the split repos was not a bad idea. Merging all of the migrations into one repo is the bad idea, but lets let it go.... 18:32:56 morgan: ok 18:33:03 i will work on the fix and get back to you. 18:33:09 ayoung: yeah, we're still good for time, its just this and a heads up about release notes, then we're on to roles 18:33:14 marekd: and a test? 18:33:17 ayoung: the split repos was a terrible idea. It sounded good at the time 18:33:18 dstanek: and a test. 18:33:21 this is changing an existing migration to clean it up. should be fine. 18:33:30 marekd: thx! 18:33:32 bknudson: ++ 18:33:43 morgan: stevemar dstanek : i think jose revealed this bug while doing test upgrade to kilo on his dev env.... 18:33:48 the migration would have failed before so don't have to worry about it breaking anything. 18:33:52 * morgan was for split repos at the time. Anyway 18:33:57 bknudson: lol 18:33:57 If we are pulling info from an extension in, have the migration check for the table before making any changes 18:34:22 bknudson: for the test i'm more worried about ensuring we are fixing something 18:34:31 it won't be reverseable, but we've droppped that requirement anyway 18:34:34 marekd: let's proceed with fixing it 18:34:34 Anyway I'll take a bug to collapse the extra repos this cycle. 18:35:03 stevemar: OK, i will address the commments. 18:35:04 Let's backport that and get a test added 18:35:04 we can potentially backport it, i can't imagine too many ppl have stuff in their federation tables, you guys were the early adopters 18:35:19 morgan, we going to do all of the extensions at once? 18:35:26 Just be super careful backporting 18:35:37 and as bknudson said, it can't get worse 18:35:43 ayoung: I'll collapse each as an idempotent migration 18:35:48 morgan, ++ 18:35:51 yup 18:35:55 ++ 18:36:00 stevemar: i think lots of people are on very old releases, so they might get to the point we (cern) are today - let's then save them from the troubles. 18:36:16 esp the fix is in the end not super hard. 18:36:18 :-) 18:36:25 #todo stevemar to look into getting those 2 bugs backported and a possible ksm refresh 18:36:35 stevemar, #action for those 18:36:42 marekd: yup 18:36:45 stevemar: thanks! 18:36:49 next! 18:36:56 #topic release notes 18:37:18 we do that on etherpad first? 18:37:28 #link https://etherpad.openstack.org/p/keystone-liberty-release-notes 18:37:32 i think we already have on started 18:37:33 ^^ 18:37:43 the proposed release notes are at the bottom 18:38:10 #link https://etherpad.openstack.org/p/keystone-liberty-release-notes 18:38:11 we have an etherpad 18:38:11 the link to the wiki is there 18:38:11 and now here! 18:38:11 #link https://wiki.openstack.org/wiki/ReleaseNotes/Liberty#OpenStack_Identity_.28Keystone.29 18:38:11 please review and add as you see fit, i trust you all 18:38:23 We are claiming Fernet token as a hardened feature. We willing to take it out of experimental? 18:38:46 line 62 18:38:50 ayoung: yes, but it looks like it was in a good enough state to motivate someone to move it over (dolphm?) 18:38:54 it's hardened but I don't think it's moved out of experimental 18:38:59 maybe lbragstad knows 18:39:08 we never advertised fernet as experimental as far as I know 18:39:09 bknudson: had a patch up to document that it's experimental 18:39:31 lets be explicit one way ot the other with it, please 18:39:42 #link https://review.openstack.org/#/c/224888/ 18:39:44 so we have 3 levels now, experimental, harden, and stable? 18:40:05 gyee: no, i think the point was to just say that it was hardened 18:40:25 gyee: no, hardened isn't a thing, just yeah - what lbragstad says ^ 18:40:25 ^^ which of the state is equivalent to "may be used in prod" ? 18:40:29 stable and experimental 18:40:38 marekd: stable 18:40:52 experimental means we are subject to change things 18:41:00 yes 18:41:13 break API contracts / endpoints / blah 18:41:18 OK...review the rest...roels? 18:41:22 but would that be confusing to the operators? 18:41:34 its harden, but still experimental 18:41:34 I don't think we're planning to break the fernet API contract 18:41:37 stevemar: AFAIR fernet format doesn't have any 'contract' 18:41:38 20 minutes left... and no henrynash 18:42:00 fernet isn't working with tempest currently 18:42:06 bknudson: yes, 18:42:08 gyee: i'll clean up the wording :) 18:42:25 bknudson has some patches for fernet & federation i also have some patches and not actually sure what to do with them... 18:42:30 stevemar, k, thanks, I was having my operator hat on 18:42:33 I haven't looked, but I'll bet it still needs an external token format identifier, too. 18:42:35 bknudson: once we have it passing all the tempest tests (which we're close) I feel comfortable marking it as stable 18:42:53 y, we should change one of the gates to use it. 18:42:54 marekd: patches for what? 18:42:56 but, that will require some work around the subsecond problem that dolphm and i have been working on 18:43:03 which will require an API change 18:43:04 also, we need to change the gate that uses eventlet to stop doing that 18:43:08 If fernet can't pass tempest we can't mark it stable 18:43:09 stevemar: fernet + federation 18:43:11 FYI. 18:43:23 all the token formats should use the same resolution for timestamps 18:43:34 bknudson: ++ 18:43:37 Oooh ooooooh stevemar I'm gonna kill eventlet today for mitaka!! 18:43:39 deployers already think fernet is stable. 18:43:40 https://review.openstack.org/#/c/213742/ 18:43:41 marekd: link me later 18:43:47 morgan, ++ 18:43:50 yes, and correcting that will be an api change 18:44:02 morgan: i just got feedback from someone saying to *not* kill it 18:44:10 but, later 18:44:11 stevemar: why? 18:44:22 stevemar: in -keystone post meeting 18:44:29 morgan: meh, later, let's give ayoung time to chat about d20s 18:44:33 #topic roles 18:44:35 Rolllllllllles 18:44:36 ayoung: ^ 18:44:42 OK...so wish henry was here 18:44:59 oh right, henry can't make it today, my bad, was supposed to say that earlier :) 18:45:07 As you might have learned, right now is the most valuable time for ghetting newe features definied; we have rc out, and people can start thinking about the next release 18:45:16 * morgan roles 2d8: 4 and 7 18:45:18 we've got a few things that might finally make some progress 18:45:30 first.. bug 968696 issue 18:45:30 bug 968696 in OpenStack Compute (nova) ""admin"-ness not properly scoped" [High,Confirmed] https://launchpad.net/bugs/968696 18:45:38 morgan, rolls? 18:46:08 there was an issue if a user deleted a project but the resources were not deleted, there was no way to clean them up 18:46:19 dolphm, suggested that it should be a service specific task; 18:46:27 nova should clean vms, neutron networks and so on 18:46:29 so... 18:46:39 first spec for that is 18:46:40 ayoung: create an event and nova listens for it? 18:46:52 mistrial? 18:46:53 stevemar, not if it missed it in the past 18:46:57 we already have an event, that's why we closed the bug. 18:47:00 https://review.openstack.org/#/c/228477/ 18:47:11 taskflow? 18:47:17 RAFT :P 18:47:17 or not 18:47:43 it means we need to catch every exceptional case up front, with not way to fix things if they are broken 18:47:50 operators need more support than that 18:48:01 bknudson, ++ the only problem is that the services doesn't consume this event 18:48:29 keystone sends the notifcation to cielometer only today...the distribution of events is not something that is well handled 18:48:43 ayoung: ++ :( 18:48:46 and if there is additional workflow? 18:48:58 and i think it's not default for ceilometer to consume and store those events 18:49:04 so...lets state explicitly what this is: scope admin to an endpoint 18:49:58 instead of trying to shoehord this into the HMT approach, we make a new value for "scope" in a token...."endpoint"....we could do all of the catalog items, but I think that makes policy too hard to write correctly 18:49:58 customer can create their own queue right? I am talking Rabbit 18:50:22 gyee, its trickier than that...it should be a different listener on the same topic 18:50:44 and if a service goes down, we don't have guaranteed delivery etc 18:50:48 lets not count on events yet 18:51:03 ayoung: if i have a role on an endpoint, what can i do? any action on that endpoint? 18:51:20 stevemar, still depends on the role, just that the assignment is scoped to the endpoint 18:51:53 it would mean that to perform an admin operation, instead of admin on any project (what we have today) or some admin project (suggested) they would have it on the endpoint... 18:52:06 with endpoint filtering, assignment is restrict to endpoint 18:52:18 how do you assign a role for a user on an endpoint? 18:52:21 and we do an HMT style assignment for the normal case: assign the user the role on the whole catalog, requies a token scoped to the endpoint 18:52:44 bknudson, you assign a role to a user for a project and given set of endpoints 18:52:48 bknudson, it becomes a new value for assignmnet_type in the assignemnt table 18:53:00 gyee, not filtering... 18:53:14 this is scope of the assignment itself, not endpoints filtered by project 18:53:20 ok, a new assignment type 18:53:22 there would be no project in the token 18:53:23 the dots still aren't connecting for me 18:53:24 assigning role to a endpoint makes no sense 18:53:25 bknudson, yes 18:53:25 and scope 18:53:27 does it have anytging to do with dynamic policies or it's a completely separated topic? 18:53:53 marekd: separate-ish 18:54:00 gyee, assigning a role to a user on an endpoint; essentially, the role name only means something if the role is scoped to endpoint, not a project on the endpoint 18:54:13 stevemar: how are they connected, then? 18:54:17 marekd, similar, but separate from dynamic policy 18:54:17 unless we support endpoint scoping 18:54:26 which still doesn't make much sense without the project 18:54:51 gyee, to peform an admin action, instead of requesting a proejct scoped token, user requests and endpoint scoped token 18:54:51 ayoung: filling same bug, but in a different way, ok 18:55:01 marekd, yep 18:55:06 ayoung, yes, for admin action, it make a lot of sense 18:55:13 dolphm, does this pass your sanity checker? 18:55:35 OK...so that is the first item... 18:55:38 dolphm might be afk 18:55:40 what's the service catalog look like? 18:55:51 ~5 minutes left 18:55:53 bknudson, probably just the one endpoint 18:55:55 keystone won't be able to fill in the %(tenant_id)s 18:56:09 ok...so, we can cont that later 18:56:13 on to... 18:56:25 sounds like a good topic for Tokyo 18:56:38 bknudson, true, but anne gentle is trying to kill that anyway... 18:56:51 ayoung: i suggest you and henry really hash this out before tokyo 18:56:54 gyee, agreed, but want to have consensus before Tokyo or we will miss 'M' 18:57:04 stevemar, want everyone to know what we are talking about. 18:57:07 ayoung: yes, service catalog: this topic https://review.openstack.org/#/c/181393/13/specs/service-catalog.rst 18:57:12 ayoung: understood 18:57:13 Henrynash's proposal is really three things 18:57:13 ayoung, I am all for endpoint-scoping 18:57:19 1. namespacing of roles. 18:57:23 2. Role inference 18:57:29 don't wait for the meeting to talk about it. put together a workgroup or something. 18:57:35 3. virtual roles 18:57:49 I was just targetting the 2 18:57:52 ayoung: those mean nothing to us right now :) 18:58:01 I have working code for that. 18:58:16 stevemar, right...which is why I am typing as fast as I can 18:58:20 role infrecne.... 18:58:22 s'all good 18:58:32 an implied role means if I get "admin" I also get "member" 18:58:37 so policy rule can be simplified 18:58:46 put it up on a wiki and try to timeline it out and we can chat about it in -keystone 18:58:53 we're not gonna get it in 90 seconds 18:58:53 instead of saying "you need to have either admin or memeber" you can just say " you need to have member" 18:59:10 s/wiki/etherpad/g 18:59:38 stevemar, I put it up in a spec and I am working on example code... 18:59:44 everyone - please please read: https://review.openstack.org/#/c/181393/13/specs/service-catalog.rst 18:59:53 the thing Henry and I need to hash out is where his and mine aline 18:59:55 align 18:59:58 for sure 19:00:19 lets not interfere with infras time 19:00:23 #endmeeting