| *** r-daneel has quit IRC | 00:28 | |
| *** r-daneel has joined #openstack-keystone | 00:28 | |
| *** gyee has quit IRC | 00:44 | |
| *** imacdonn has quit IRC | 00:50 | |
| *** imacdonn has joined #openstack-keystone | 00:50 | |
| *** ChanServ has quit IRC | 01:00 | |
| *** ChanServ has joined #openstack-keystone | 01:03 | |
| *** barjavel.freenode.net sets mode: +o ChanServ | 01:03 | |
| *** r-daneel has quit IRC | 01:05 | |
| *** itlinux has quit IRC | 01:07 | |
| *** itlinux has joined #openstack-keystone | 01:07 | |
| *** felipemonteiro has quit IRC | 01:12 | |
| *** felipemonteiro has joined #openstack-keystone | 01:18 | |
| *** Nel1x has joined #openstack-keystone | 01:28 | |
| wxy-xiyuan | lbragstad: cmurphy : cool, thanks for the patch for placeholder. | 01:35 |
|---|---|---|
| *** felipemonteiro has quit IRC | 01:37 | |
| openstackgerrit | Merged openstack/keystone master: Add safety to the inferred target extraction during enforcement https://review.openstack.org/591203 | 01:38 |
| openstackgerrit | Merged openstack/keystone master: Convert role_assignments API to flask native dispatching https://review.openstack.org/590518 | 01:38 |
| openstackgerrit | Merged openstack/keystone master: Convert system (role) api to flask native dispatching https://review.openstack.org/590588 | 01:40 |
| *** raginbajin has quit IRC | 01:46 | |
| *** raginbajin has joined #openstack-keystone | 01:49 | |
| *** dansmith has quit IRC | 01:49 | |
| *** dansmith has joined #openstack-keystone | 01:51 | |
| *** threestrands has quit IRC | 02:08 | |
| openstackgerrit | Monty Taylor proposed openstack/keystoneauth master: Add support for ironic single-version responses https://review.openstack.org/595980 | 02:16 |
| *** Nel1x has quit IRC | 02:28 | |
| *** imacdonn has quit IRC | 02:28 | |
| *** nicolasbock has quit IRC | 02:28 | |
| *** wlmbasson has quit IRC | 02:28 | |
| *** jlvillal has quit IRC | 02:28 | |
| *** dims has quit IRC | 02:28 | |
| *** mnaser has quit IRC | 02:28 | |
| *** TheJulia has quit IRC | 02:28 | |
| *** rledisez has quit IRC | 02:28 | |
| *** adriant has quit IRC | 02:28 | |
| *** ildikov has quit IRC | 02:28 | |
| *** mordred has quit IRC | 02:28 | |
| *** Nel1x has joined #openstack-keystone | 02:30 | |
| *** imacdonn has joined #openstack-keystone | 02:30 | |
| *** nicolasbock has joined #openstack-keystone | 02:30 | |
| *** wlmbasson has joined #openstack-keystone | 02:30 | |
| *** dims has joined #openstack-keystone | 02:30 | |
| *** jlvillal has joined #openstack-keystone | 02:30 | |
| *** mnaser has joined #openstack-keystone | 02:30 | |
| *** TheJulia has joined #openstack-keystone | 02:30 | |
| *** rledisez has joined #openstack-keystone | 02:30 | |
| *** ildikov has joined #openstack-keystone | 02:30 | |
| *** adriant has joined #openstack-keystone | 02:30 | |
| *** mordred has joined #openstack-keystone | 02:30 | |
| *** openstack has joined #openstack-keystone | 13:19 | |
| *** ChanServ sets mode: +o openstack | 13:19 | |
| *** openstackstatus has joined #openstack-keystone | 13:22 | |
| *** ChanServ sets mode: +v openstackstatus | 13:22 | |
| *** openstackstatus has quit IRC | 13:22 | |
| *** openstackstatus has joined #openstack-keystone | 13:24 | |
| *** ChanServ sets mode: +v openstackstatus | 13:24 | |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Convert policy API to flask https://review.openstack.org/589950 | 13:29 |
| *** felipemonteiro has joined #openstack-keystone | 13:29 | |
| *** itlinux-away has quit IRC | 13:53 | |
| *** jaosorior has joined #openstack-keystone | 14:02 | |
| *** r-daneel has joined #openstack-keystone | 14:04 | |
| *** felipemonteiro has quit IRC | 14:16 | |
| mnaser | hi keystone team | 14:31 |
| mnaser | we're seeing this weird issue under centos | 14:31 |
| mnaser | in openstack ansible | 14:32 |
| mnaser | http://logs.openstack.org/96/595796/2/check/openstack-ansible-functional-centos-7/4fe8937/logs/ara-report/result/1c0f7c0b-6848-49ea-8189-6da35d4e348a/ | 14:32 |
| mnaser | any ideas? | 14:32 |
| cmurphy | mnaser: pysaml2 isn't installed? | 14:32 |
| *** jaosorior has quit IRC | 14:33 | |
| mnaser | cmurphy: shouldn't that just be installed when we add keystone? | 14:34 |
| mnaser | as in, isn't that in requirements.txt, i didn't check honestly | 14:34 |
| *** itlinux has joined #openstack-keystone | 14:35 | |
| cmurphy | mnaser: it is in requirements.txt | 14:35 |
| mnaser | cmurphy: ok, strange, let me see why it didn't get installed then | 14:35 |
| mnaser | http://logs.openstack.org/96/595796/2/check/openstack-ansible-functional-centos-7/4fe8937/logs/ara-report/result/d70c28b3-833e-4439-a84d-8973a96b2828/ | 14:36 |
| mnaser | "Running setup.py (path:/tmp/pip-build-dBwTWA/pysaml2/setup.py) egg_info for package pysaml2 produced metadata for project name unknown. Fix your #egg=pysaml2 fragments." | 14:36 |
| mnaser | interesting | 14:36 |
| elbragstad | is this master? | 14:37 |
| elbragstad | looks like it https://review.openstack.org/#/c/595796/ | 14:37 |
| *** viks__ has quit IRC | 14:38 | |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Add test case for expanding implied roles in system tokens https://review.openstack.org/596356 | 14:48 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Expand implied roles in system-scoped tokens https://review.openstack.org/596357 | 14:48 |
| *** jlvillal is now known as jlviva-viva | 14:52 | |
| *** itlinux has quit IRC | 14:55 | |
| *** pcaruana has quit IRC | 14:59 | |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Expand implied roles in system-scoped tokens https://review.openstack.org/596357 | 15:00 |
| *** r-daneel has quit IRC | 15:04 | |
| *** r-daneel has joined #openstack-keystone | 15:06 | |
| *** itlinux has joined #openstack-keystone | 15:09 | |
| *** r-daneel_ has joined #openstack-keystone | 15:18 | |
| *** r-daneel has quit IRC | 15:19 | |
| *** r-daneel_ is now known as r-daneel | 15:19 | |
| *** nicolasbock has joined #openstack-keystone | 15:21 | |
| *** imacdonn has joined #openstack-keystone | 15:22 | |
| *** raildo has quit IRC | 15:26 | |
| *** raildo has joined #openstack-keystone | 15:54 | |
| *** itlinux has quit IRC | 16:02 | |
| *** itlinux has joined #openstack-keystone | 16:06 | |
| *** itlinux is now known as itlinux-away | 16:16 | |
| *** itlinux-away is now known as itlinux | 16:16 | |
| *** r-daneel_ has joined #openstack-keystone | 16:18 | |
| *** gyee has joined #openstack-keystone | 16:19 | |
| *** r-daneel has quit IRC | 16:20 | |
| *** r-daneel_ is now known as r-daneel | 16:20 | |
| knikolla | o/ | 16:35 |
| openstackgerrit | Merged openstack/oslo.policy master: add lib-forward-testing-python3 test job https://review.openstack.org/591189 | 16:51 |
| mnaser | heyso | 16:53 |
| mnaser | say i know someone who runs a public cloud thats launching a new region | 16:54 |
| mnaser | how is the state of rocky right now in terms of rc's | 16:54 |
| elbragstad | 0.0 | 16:54 |
| mnaser | is deploying stable/rocky crazy at this state? | 16:54 |
| elbragstad | are you asking what i think you're asking? | 16:54 |
| cmurphy | mnaser: that could be anyone | 16:54 |
| cmurphy | :P | 16:55 |
| mnaser | because our gates in OSA seem to deploy it ok | 16:55 |
| mnaser | but i'm wondering if there aren't things that are still being flushed out | 16:55 |
| elbragstad | that sounds oddly like a "well, it worked in devstack" statement ;) | 16:55 |
| mnaser | or if it looks like the release will go out in the current state :p | 16:55 |
| elbragstad | to be honest, i *think* rc2 is going to be fine | 16:56 |
| elbragstad | we had little new development this release, but we did have a couple very large refactors | 16:56 |
| mnaser | i guess the flask stuff too | 16:56 |
| elbragstad | yeah - mainly flask | 16:56 |
| elbragstad | we also shuffled around a lot of stuff with the token provider interfaces, but i'm pretty confident in that with the testing we have | 16:57 |
| mnaser | cool | 16:58 |
| mnaser | the only thing i'm worried about is a bit of latency | 16:58 |
| elbragstad | because of keystone? | 16:58 |
| mnaser | 73ms | 16:58 |
| mnaser | for a rtt between where keystone is located and the region | 16:59 |
| elbragstad | oh - you're talking network latency with respect to that region? | 16:59 |
| elbragstad | aha - i see what you mean | 16:59 |
| mnaser | yeah, because we want to keep a single keystone (i will refuse to have multiple keystone instances, neverrr) | 16:59 |
| elbragstad | and you don't want to standup a multi-site master deployment? | 17:00 |
| mnaser | multi-site master as is multi-site galera? | 17:00 |
| elbragstad | right | 17:00 |
| mnaser | i dont think you can run a per-db only galera instance | 17:00 |
| mnaser | which means it would replicate a lot more stuff than it needs | 17:00 |
| mnaser | i guess the other alternative is to build a read-only slave in the region | 17:01 |
| mnaser | with normal boring replication | 17:01 |
| mnaser | given the state of users/projects/etc doesn't actually *change* all that much | 17:01 |
| elbragstad | if you have a cluster of api servers in region A backed to galera in region A | 17:01 |
| elbragstad | you couldn't do a cluster of api servers in region B backed to galera in region B? | 17:02 |
| mnaser | i could, but how do i make sure the data from region A makes it to region B | 17:02 |
| elbragstad | but both sets of galera nodes replicate to each other | 17:02 |
| mnaser | afaik galera cant do 'per db replication' | 17:02 |
| mnaser | its either the entire cluster or nothing | 17:02 |
| mnaser | and that would bring in all sorts of other dbs (our dbs host all services) | 17:03 |
| elbragstad | oh... | 17:03 |
| elbragstad | keystone isn't your only database i suppose | 17:03 |
| mnaser | yeah | 17:03 |
| elbragstad | got it | 17:03 |
| mnaser | i forgot, can a fernet token be validated locally? | 17:04 |
| elbragstad | dolph and i did a bunch of testing around multi-site multi-write support with galera deployed locally | 17:04 |
| mnaser | like is db access involved in fernet token vlaidation | 17:04 |
| elbragstad | mnaser: define locally? | 17:04 |
| mnaser | so i think i remember reading that fernet tokens used the /etc/fernet-keys to validate them | 17:04 |
| elbragstad | validation requires database reads in order to reconstruct the token response | 17:04 |
| mnaser | ah i see | 17:04 |
| elbragstad | but it's all reads | 17:05 |
| mnaser | so really all i'm adding is a 70ms rtt overhead | 17:05 |
| elbragstad | i believe so? | 17:05 |
| elbragstad | you could deploy keystone with a readonly database | 17:06 |
| elbragstad | and just point keystone_authtoken for each service to use the local keystone within the new region | 17:06 |
| elbragstad | (that might help cut down the network latency) | 17:06 |
| mnaser | looks like per db replication is just a really weird beast | 17:07 |
| elbragstad | so galera does support it? | 17:07 |
| mnaser | nope, mysql does but not galera from what it looks | 17:07 |
| mnaser | http://galeracluster.com/2015/07/geo-distributed-database-clusters-with-galera/ | 17:08 |
| mnaser | interesting | 17:08 |
| mnaser | but it also means all sorts of stuff will be there that dont have to be there | 17:09 |
| * elbragstad just found that same link | 17:09 | |
| elbragstad | is that going to be a non-starter from a security perspective? | 17:09 |
| mnaser | i guess i can use `replicate-do-db` | 17:09 |
| mnaser | not really. | 17:10 |
| mnaser | i mean my thoughts was going to be | 17:10 |
| mnaser | start with that latency and see what happens. | 17:10 |
| elbragstad | sure | 17:10 |
| mnaser | i dont think it would be *that* bad | 17:10 |
| elbragstad | if it does end up being terrible, it looks like there are paths for optimizations | 17:11 |
| mnaser | yeah to minimize the number of transactions/etc | 17:11 |
| mnaser | we can profile this stuff | 17:11 |
| mnaser | to help minimize round trips | 17:11 |
| mnaser | memcache doesnt really help, does it? | 17:11 |
| mnaser | we deploy it anyways but im not sure how much of help it would be | 17:11 |
| elbragstad | it would help on subsequent requests i think? | 17:12 |
| elbragstad | but you might be paying the 70 ms latency cost at least once | 17:12 |
| mnaser | yeah the at least once i think is a given | 17:12 |
| kmalloc | mnaser: we could work to help pre-seed things into memcache, but subsequent requests (even within the same transaction) would help | 17:12 |
| mnaser | kmalloc: i think i'd be okay with at least once hitting 70ms.. i'd certainly be more than happy to work to provide some profiling stuff to see what is taking up the time | 17:14 |
| mnaser | but i guess this work would be client-side stuff | 17:14 |
| elbragstad | ++ | 17:14 |
| elbragstad | profiling would be awesome | 17:14 |
| mnaser | yeah actually i think we wouldn't have multiple 70ms hits because | 17:14 |
| * elbragstad hasn't done any real profiling since 2015 :( | 17:14 | |
| mnaser | if i keep our api the same place, it will simply make 1 request | 17:14 |
| mnaser | so rather than hosting local apis which do multiple db requests at 70ms each | 17:15 |
| mnaser | contacting the remote api would instead be a single request at 70ms but inside that request, they'd all execute locally in that region | 17:15 |
| elbragstad | you mean each service would use the local region for token validation? | 17:15 |
| elbragstad | and the 70 ms penalty would only be at the edge of the request (e.g. hitting nova) | 17:16 |
| mnaser | elbragstad: nope, each service would talk to the same keystone which is at the current region | 17:17 |
| elbragstad | oh | 17:17 |
| mnaser | so <service @ region B> => <keystone @ region A> => <local db access> | 17:17 |
| mnaser | vs <service @ region B> => <keystone @ region B> => <remote db access> | 17:17 |
| elbragstad | local db is going to be in region B? | 17:18 |
| mnaser | sorry no, in region A in both cases | 17:18 |
| mnaser | i think that it would be quicker to talk to region A directly because it means that it's a single HTTP round trip | 17:18 |
| mnaser | rather than single HTTP round trip inside the region, but multiple MySQL query round trips to the other region hosting the data | 17:18 |
| elbragstad | oh i think i see what you mean | 17:18 |
| elbragstad | just make the request to region A from region B off the get go | 17:19 |
| kmalloc | mnaser: yeah don't do remote DB, imo | 17:19 |
| mnaser | yeah it's a logistical pain too | 17:19 |
| kmalloc | you may want to crank up the TTL in the token validation to the token lifespan | 17:20 |
| kmalloc | also with the "allow expired" you might even want to add in a caching-proxy local to region B for keystone A with some semblance of "sane" | 17:20 |
| kmalloc | that way ?allow_expired would only be a 70ms hit once as well | 17:20 |
| mnaser | hmm. that might be a fun thing on top of it to reduce hits | 17:20 |
| mnaser | it'll be a fun exercise.. | 17:21 |
| mnaser | lol | 17:21 |
| kmalloc | ++, just keep in mind that it expands exposure for token life/revocation | 17:21 |
| elbragstad | results from this would be good for our comprehensive caching guide (which currently doesn't exist) | 17:21 |
| kmalloc | up until recent weeks now, i would have considered asking if REDIS as a cache (configured for repl) might work to help. | 17:22 |
| kmalloc | since the pre-seed of token values into the memcache on issue could be replicated locally. | 17:22 |
| kmalloc | elbragstad: we should consider pre-seeding validated tokens with a known set of prefixes [independent of keystone's form] to the cache that the services would use. | 17:23 |
| elbragstad | i think we do that within keystone directly | 17:23 |
| kmalloc | aka, pull in the KSM logic and allow pre-warming the cache for a given token on issuance, we only warm the cache for keystone's validation | 17:23 |
| kmalloc | keystone's validation != what services store | 17:23 |
| elbragstad | right | 17:23 |
| elbragstad | i see what you mean | 17:23 |
| kmalloc | so we could do both | 17:23 |
| kmalloc | we could also (if we wanted to async it) allow pre-seed to N cache regions, it would be relatively easy to write a looper that reads off a queue for that | 17:24 |
| kmalloc | or at least have a way to send that data out so mnaser could write the loop/push thing. | 17:25 |
| *** itlinux is now known as itlinux-away | 17:27 | |
| *** itlinux-away is now known as itlinux | 17:29 | |
| *** itlinux is now known as itlinux-away | 17:29 | |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Add test case for expanding implied roles in system tokens https://review.openstack.org/596356 | 17:29 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Expand implied roles in system-scoped tokens https://review.openstack.org/596357 | 17:29 |
| *** itlinux-away is now known as itlinux | 17:30 | |
| *** itlinux is now known as itlinux-away | 17:30 | |
| *** itlinux-away has quit IRC | 17:35 | |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: WIP: Implement scope_type checking for credentials https://review.openstack.org/594547 | 18:15 |
| *** itlinux has joined #openstack-keystone | 20:13 | |
| *** raildo has quit IRC | 20:53 | |
| openstackgerrit | Gage Hugo proposed openstack/keystone master: Add details and clarify examples on casing https://review.openstack.org/590477 | 20:56 |
| *** itlinux has quit IRC | 20:58 | |
| elbragstad | alright... | 21:03 |
| elbragstad | incoming patch partially fixing bug 968696 | 21:03 |
| openstack | bug 968696 in OpenStack Identity (keystone) ""admin"-ness not properly scoped" [High,In progress] https://launchpad.net/bugs/968696 - Assigned to Adam Young (ayoung) | 21:03 |
| *** marvin_mhg has quit IRC | 21:04 | |
| elbragstad | it's still wip... but it _should_ pass tests... | 21:04 |
| elbragstad | looking to see if anyone has some bright ideas for testing that might result in less duplication | 21:04 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Add test case for expanding implied roles in system tokens https://review.openstack.org/596356 | 21:12 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: Expand implied roles in system-scoped tokens https://review.openstack.org/596357 | 21:12 |
| openstackgerrit | Lance Bragstad proposed openstack/keystone master: WIP: Implement scope_type checking for credentials https://review.openstack.org/594547 | 21:12 |
| elbragstad | cc kmalloc ^ | 21:40 |
| *** itlinux has joined #openstack-keystone | 22:19 | |
| kmalloc | elbragstad: nice | 22:22 |
| *** gyee has quit IRC | 22:52 | |
| *** itlinux has quit IRC | 22:57 | |
| *** r-daneel has quit IRC | 23:24 | |
| *** tristanC has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!