Wednesday, 2019-09-18

*** lbragstad has quit IRC00:19
*** lbragstad has joined #openstack-keystone00:19
*** jamesmcarthur has joined #openstack-keystone00:42
*** gyee has quit IRC00:50
*** jamesmcarthur has quit IRC01:07
*** markvoelker has quit IRC01:08
*** markvoelker has joined #openstack-keystone01:20
*** Ben78 has quit IRC02:02
*** jamesmcarthur has joined #openstack-keystone02:22
*** jamesmcarthur has quit IRC02:38
*** jamesmcarthur has joined #openstack-keystone02:47
*** markvoelker has quit IRC03:09
*** jamesmcarthur has quit IRC03:24
*** jamesmcarthur has joined #openstack-keystone03:36
openstackgerritColleen Murphy proposed openstack/oslo.policy master: (WIP) Modernize policy checker  https://review.opendev.org/68278303:38
*** jamesmcarthur has quit IRC03:50
openstackgerritColleen Murphy proposed openstack/keystone master: Allow system/domain scope for assignment tree list  https://review.opendev.org/68276204:03
*** etp has joined #openstack-keystone04:14
openstackgerritMerged openstack/keystone master: Split protection unit tests into its own job  https://review.opendev.org/68078804:22
*** etp has quit IRC04:28
*** pcaruana has joined #openstack-keystone04:46
*** jaosorior has quit IRC04:57
*** jaosorior has joined #openstack-keystone04:57
*** Luzi has joined #openstack-keystone04:59
*** markvoelker has joined #openstack-keystone05:10
*** markvoelker has quit IRC05:14
openstackgerritColleen Murphy proposed openstack/keystone master: (WIP) Consolidate policy deprecation warnings  https://review.opendev.org/68279806:06
openstackgerritColleen Murphy proposed openstack/keystone master: Fix validation of role assignment subtree list  https://review.opendev.org/68275006:14
openstackgerritColleen Murphy proposed openstack/keystone master: Allow system/domain scope for assignment tree list  https://review.opendev.org/68276206:14
*** xek_ has joined #openstack-keystone06:22
*** xek_ has quit IRC06:30
openstackgerritQitao proposed openstack/keystonemiddleware master: Update invalid link for README  https://review.opendev.org/68280406:38
*** jawad_axd has joined #openstack-keystone06:47
openstackgerritQitao proposed openstack/keystonemiddleware master: Update invalid link for README  https://review.opendev.org/68280406:47
*** rcernin has quit IRC06:57
*** trident has quit IRC07:08
*** trident has joined #openstack-keystone07:19
*** ivve has joined #openstack-keystone07:32
vishakhalbragstad: I cant see any policy endpoint API left  in https://review.opendev.org/#/c/682266/3/etc/policy.v3cloudsample.json.07:58
vishakha*project endpoint07:59
*** tkajinam has quit IRC08:04
openstackgerritHiroyuki Jo proposed openstack/oslo.policy master: Fix fault when deleting policy_file  https://review.opendev.org/68283808:56
*** pcaruana has quit IRC08:57
*** pcaruana has joined #openstack-keystone09:01
*** tesseract has joined #openstack-keystone09:44
openstackgerritRalf Haferkamp proposed openstack/keystone master: Fix PostgreSQL specifc issue with credentials encoding  https://review.opendev.org/68173609:46
*** markvoelker has joined #openstack-keystone09:59
*** markvoelker has quit IRC10:04
*** openstackgerrit has quit IRC10:06
*** markvoelker has joined #openstack-keystone10:16
*** markvoelker has quit IRC10:20
*** openstackgerrit has joined #openstack-keystone10:22
openstackgerritMerged openstack/keystone master: Make system tokens work with domain-specific drivers  https://review.opendev.org/68183310:22
openstackgerritMerged openstack/keystone master: Use correct repo for initial version check  https://review.opendev.org/68244710:46
openstackgerritRajat Dhasmana proposed openstack/keystone master: Fix test case in policy associations  https://review.opendev.org/68117310:48
*** jdwidari has joined #openstack-keystone11:00
*** pcaruana has quit IRC11:19
*** dave-mccowan has joined #openstack-keystone11:22
*** pcaruana has joined #openstack-keystone11:28
*** raildo has joined #openstack-keystone11:34
*** Luzi has quit IRC11:35
openstackgerritChason Chan proposed openstack/keystone master: Fix the 404 page  https://review.opendev.org/68287111:48
*** Luzi has joined #openstack-keystone11:49
lbragstadvishakha these are the ones we need to update12:03
lbragstadhttps://opendev.org/openstack/keystone/src/branch/master/keystone/common/policies/project_endpoint.py#L19-L6712:03
lbragstadinstead of using base.RULE_ADMIN_REQUIRED, we should update them to use base.SYSTEM_ADMIN and base.SYSTEM_READER12:04
vishakhalbragstad: thanx. I will work over it of nobody is12:05
*** markvoelker has joined #openstack-keystone12:05
lbragstadvishakha i haven't started on it, yet12:06
lbragstadand i'm not sure anyone else has either12:06
lbragstadvishakha thanks :)12:06
vishakhalbragstad: ok . Thanks for the update12:06
lbragstadvishakha also - if it makes it easier for you, just put everything in a single patch12:06
vishakhalbragstad: ok12:07
lbragstadlike this - https://review.opendev.org/#/c/682503/12:07
*** jamesmcarthur has joined #openstack-keystone12:09
*** jamesmcarthur has quit IRC12:16
*** awalende has joined #openstack-keystone12:19
*** jamesmcarthur has joined #openstack-keystone12:22
*** redrobot has joined #openstack-keystone12:26
*** jamesmcarthur has quit IRC12:31
*** Ben78 has joined #openstack-keystone12:40
*** jamesmcarthur has joined #openstack-keystone12:51
*** awalende has quit IRC13:20
*** Luzi has quit IRC13:49
openstackgerritLance Bragstad proposed openstack/keystone master: Add default roles and scope checking to project tags  https://review.opendev.org/68250314:03
lbragstad^ that one got quite a bit bigger with the additional test cases14:04
*** xek_ has joined #openstack-keystone14:06
*** efried has joined #openstack-keystone14:09
efriedcmurphy, lbragstad: I seem to remember we only need "password", not "v2password" or "v3password", when building conf opts for genconfig e.g. https://review.opendev.org/#/c/682565/6/cyborg/conf/nova.py -- can you please confirm?14:10
efriedkmalloc: ^14:10
lbragstadlooks like it's common to supply all three? http://codesearch.openstack.org/?q=ks_loading.get_auth_plugin_conf_options&i=nope&files=&repos=14:13
lbragstadcmurphy do you mind if i update the topic for all remaining policy patches?14:17
openstackgerritLance Bragstad proposed openstack/keystone master: Add default roles and scope checking to project tags  https://review.opendev.org/68250314:20
efriedlbragstad: common, yes, but iirc someone (maybe you, even :P) once told me it wasn't necessary.14:20
lbragstadefried lemme go dig14:20
*** openstackgerrit has quit IRC14:21
lbragstadhuh - i'm only seeing Password in keystoneauth1/identity/v3/password.py14:26
lbragstadi'm not sure if i'm missing something - but i can't seem to find where passwordv2 or passwordv3 would hook in?14:26
lbragstadcc mordred ^14:26
cmurphylbragstad: i don't mind14:29
cmurphyefried: password should be fine, it should figure out the right thing based on the auth_url and/or version discovery14:30
efriedcmurphy: this is for list_opts14:32
efriedso like, what shows up in the config reference, I think.14:32
efriedbut are you saying that the options from 'password' ought to be sufficient to a) list and b) configure because ksa can dtrt with just those?14:32
*** jawad_axd has quit IRC14:46
*** jawad_axd has joined #openstack-keystone14:46
*** jawad_axd has quit IRC14:46
lbragstadcmurphy tracking them with https://review.opendev.org/#/q/topic:train-rc1-policy-fixes+(status:open+OR+status:merged)14:52
lbragstadi think the only one that isn't tagged is the one for project-endpoint associations14:53
*** jaosorior has quit IRC14:57
cmurphyefried: i'm saying (b), i'm not totally sure about (a), v2password and v3password are valid plugins in keystoneauth so i guess it's valid to list them?15:00
cmurphylbragstad: sweet thank you15:00
cmurphyi think vishakha is on it15:00
lbragstadmhm15:01
lbragstadsounds like she is15:01
efriedsigh, okay, I guess there's no harm leaving it alone for now15:01
lbragstadproject tags and role assignments look good15:01
cmurphylbragstad: bnemec kmalloc i started looking at https://bugs.launchpad.net/keystone/+bug/1836568 and trying to address it the way we concluded in http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-08-13-16.00.log.html#l-6415:03
openstackLaunchpad bug 1836568 in oslo.policy "Logs filled with unnecessary policy deprecation warnings" [High,In progress] - Assigned to Ben Nemec (bnemec)15:03
cmurphywhich was basically http://eavesdrop.openstack.org/meetings/keystone/2019/keystone.2019-08-13-16.00.log.html#l-11415:03
cmurphyoslopolicy-checker is really in no shape to help with deprecated rules15:04
cmurphyit doesn't really even work right now https://review.opendev.org/68278315:04
cmurphybut the policy generator does work15:05
bnemecI noticed that.15:05
bnemecI was waiting until after feature freeze to pester you about it. :-)15:05
cmurphyheh15:05
lbragstadnice15:06
cmurphythis is my attempt at making the warnings go away in keystone https://review.opendev.org/68279815:06
cmurphywe'd need something like that to address the issue without needing a new oslo.policy release15:06
cmurphybut it uses the suppress warnings attribute which was only ever supposed to be for tests15:06
bnemecYeah, so does https://review.opendev.org/#/c/682117/ so I'm not in any position to throw stones. ;-)15:07
cmurphyha15:07
cmurphywe're our own worst enemies15:08
*** gyee has joined #openstack-keystone15:08
bnemecAlso, this could be a temporary thing just for this release and we could move the warning consolidation into oslo.policy next cycle.15:09
bnemecIt's a thing every project is going to need as they move to scope-aware policies.15:09
cmurphythat's true15:09
cmurphyso it's not an entirely terrible idea?15:09
bnemecAt first glance it seems reasonable to me.15:10
bnemecIt adds another step to figure out which policies are deprecated, but I suspect operators will prefer that to thousands of lines of deprecation warnings in their logs.15:11
bnemecDo we need https://review.opendev.org/#/c/682783/ for this to be viable though? Right now it tells them to run a broken tool. :-/15:12
*** xek__ has joined #openstack-keystone15:12
cmurphyno, it's only oslopolicy-checker that is broken, oslopolicy-policy-generate and oslopolicy-policy-upgrade work fine and actually hook into keystone's registered defaults instead of reading from a policy file15:13
cmurphymy other idea is just to resuscitate https://review.opendev.org/674940 but then oslopolicy-policy-generator will use the same short warnings15:13
*** xek_ has quit IRC15:14
*** ivve has quit IRC15:18
bnemecAh, okay. I actually think the reasoning behind the shorter deprecation warnings applies anyway, so I would be okay with doing both.15:18
cmurphyfair enough15:19
bnemecOf course, that's easy for me to say. I don't have to propose or review it. :-)15:19
*** openstackgerrit has joined #openstack-keystone15:22
openstackgerritColleen Murphy proposed openstack/keystone master: Allow system/domain scope for assignment tree list  https://review.opendev.org/68276215:22
*** markvoelker has quit IRC15:35
*** david-lyle has quit IRC15:35
*** dklyle has joined #openstack-keystone15:35
*** trident has quit IRC15:41
openstackgerritColleen Murphy proposed openstack/keystone master: DRY up credential policies  https://review.opendev.org/68248815:42
openstackgerritColleen Murphy proposed openstack/keystone master: Move remaining protection tests  https://review.opendev.org/68293115:42
*** jamesmcarthur has quit IRC15:45
*** jamesmcarthur has joined #openstack-keystone15:47
bnemecI opened a wishlist bug against oslo.policy to track moving the warning consolidation into the library: https://bugs.launchpad.net/oslo.policy/+bug/184455515:49
openstackLaunchpad bug 1844555 in oslo.policy "Provide a way to consolidate deprecation warnings in logs" [Wishlist,Confirmed]15:49
cmurphycool15:49
*** jamesmcarthur has quit IRC15:49
bnemecAlthough now that I think about it, I don't need to mention that here since it will show up in the weekly update email. :-)15:49
cmurphy:P15:50
*** jamesmcarthur has joined #openstack-keystone15:52
*** trident has joined #openstack-keystone15:53
*** markvoelker has joined #openstack-keystone15:53
*** jamesmcarthur has quit IRC15:57
*** jamesmcarthur has joined #openstack-keystone15:59
*** jawad_axd has joined #openstack-keystone16:07
*** jawad_axd has quit IRC16:11
*** xek__ has quit IRC16:11
*** efried is now known as efried_pto16:13
*** jawad_axd has joined #openstack-keystone16:18
*** jawad_axd has quit IRC16:22
*** gmann is now known as gmann_afk16:23
*** ayoung has joined #openstack-keystone16:24
*** ivve has joined #openstack-keystone16:32
cmurphystepping away for a bit16:35
*** jmlowe has quit IRC16:39
*** tesseract has quit IRC16:53
*** jamesmcarthur_ has joined #openstack-keystone17:09
*** jamesmcarthur has quit IRC17:12
*** AJaeger has joined #openstack-keystone17:22
openstackgerritAndreas Jaeger proposed openstack/keystone-specs master: Update docstheme options  https://review.opendev.org/68295217:25
AJaegerkeystone-specs cores, I'm updating the publish jobs for specs repositories and needed a test, hope the change above is acceptable for that and we can use that for testing ^17:26
kmalloc@AJaeger Thats a fine test case, let me know if you want / need us to actually merge it17:27
kmalloc@AJaeger i have no issues with any changes you are making, especially if it makes anything better (really, let us know what needs to land, we'll land it when you're ready)17:28
AJaegerkmalloc: I need you to merge it - to test that *publishing* works ;)17:28
kmallocwfm. will push that through once zuul passes check17:28
kmallocit's minimal17:28
AJaegerkmalloc: thanks. It's ready to land if Zuul is happy ;)17:28
kmalloc@AJaeger in all honesty, it might make sense to give you spec-core capability so you don't need to go through us for these types of changes. but i'd obviously need to confirm with other cores and cmurphy17:29
AJaegerkmalloc: this is a one time fun - thanks17:30
kmalloc:)17:30
kmallocuntil next time17:30
kmallocthat is17:30
AJaegerhope that's not too soon ;)17:30
AJaegerkmalloc: the change passed tests17:35
kmallocAJaeger: +A17:36
AJaegerMille Grazie, kmalloc !17:36
kmallocanytime!17:37
*** jmlowe has joined #openstack-keystone17:44
openstackgerritMerged openstack/keystone-specs master: Update docstheme options  https://review.opendev.org/68295217:47
AJaegeryeah, http://specs.openstack.org/openstack/keystone-specs/ was updated ;). All fine17:49
* AJaeger is happy and disappears again17:50
*** jmlowe has quit IRC17:53
openstackgerritAbhishek Mahajan proposed openstack/keystone master: "Unauthorized" error message needs more hints  https://review.opendev.org/68295517:58
*** AJaeger has left #openstack-keystone17:58
*** openstackgerrit has quit IRC18:37
*** openstackgerrit has joined #openstack-keystone18:54
openstackgerritMerged openstack/keystone master: Specify keystone is OS user for fernet and credential setup  https://review.opendev.org/67472518:54
cmurphythe coverage job is still timing out even after splitting out the protection tests :(18:57
cmurphylooks like they slow way down during the opportunistic db tests https://zuul.opendev.org/t/openstack/build/7be4ade9f5694cccabc5be6c6c45c3fc/log/job-output.txt#1298-129918:57
cmurphybut still it's strange it's so bad18:57
*** jmlowe has joined #openstack-keystone19:03
lbragstadodd...19:37
lbragstaddid something change with the opportunistic tests recently?19:37
cmurphynot in keystone afaik19:38
*** jmlowe has quit IRC19:42
cmurphylooks like we've been ignoring line length in pep8 since https://review.opendev.org/618954 i don't think that was intentional :/19:59
*** hoonetorg has quit IRC20:01
lbragstadaha!20:09
lbragstadi was so confused why line length was getting so long...20:09
*** pcaruana has quit IRC20:10
cmurphynot sure if should fix20:11
*** jmlowe has joined #openstack-keystone20:12
*** jamesmcarthur_ has quit IRC20:12
*** jamesmcarthur has joined #openstack-keystone20:17
*** jamesmcarthur has quit IRC20:17
*** jamesmcarthur has joined #openstack-keystone20:18
*** ayoung has quit IRC20:21
*** ayoung has joined #openstack-keystone20:22
bnemecDidn't you find a problem recently-ish where the opportunistic tests were always being skipped? Maybe fixing that contributed to this problem20:50
bnemec12 minutes for a single test seems unreasonably long though.20:51
cmurphylol yes i guess actually running tests takes time20:51
cmurphybut still20:51
bnemecrm -rf keystone/tests20:52
bnemecProblem solved. :-)20:52
lbragstadheh - keep the tests and delete keystone20:52
cmurphyno more rechecks ever20:52
cmurphylbragstad: but then the tests would fail20:52
bnemecI know there have been issues with storage being too slow for etcd. Maybe something similar is happening for the databases.20:52
cmurphyhmm could be20:53
bnemecI think they worked around that by having etcd run entirely in memory. Maybe there's some tweak that could be done to mysql to do similar?20:54
kmallocooh20:54
lbragstadisn't that sqlite?20:54
kmallocthat sounds like an awful idea20:54
kmallocfor MySQL20:54
kmallocshort of moving to something like NDB.20:55
kmalloci also wonder if we do any mysql tuning, because remember mysql comes out of the box tuned... terribly..or if that tuning somehow changed20:55
bnemecIt would not surprise me if it's tuned poorly.20:57
cmurphyi don't think we do any tuning, if it's mysql itself that is making performance this bad that tuning can fix then we should document that20:57
cmurphyi am guessing bnemec is right20:57
cmurphybecause it's not consistent20:57
cmurphyso we just get unlucky on some nodes20:57
bnemec\o/ cloud! :-P20:57
bnemecI take it back, don't run mysql in memory: "no transaction safety, locking issues, etc"20:59
bnemechttps://stackoverflow.com/questions/10692398/how-do-i-make-a-mysql-database-run-completely-in-memory20:59
bnemecinnodb_flush_log_at_trx_commit = 0 might cut down on disk activity and since we don't really care about this data it doesn't matter if we lose some in a crash.21:08
*** raildo has quit IRC21:14
openstackgerritColleen Murphy proposed openstack/keystone master: Make policy deprecation reasons less verbose  https://review.opendev.org/67494021:39
*** jamesmcarthur has quit IRC22:06
*** jamesmcarthur has joined #openstack-keystone22:09
openstackgerritColleen Murphy proposed openstack/keystone master: Consolidate policy deprecation warnings  https://review.opendev.org/68279822:11
openstackgerritColleen Murphy proposed openstack/keystone master: Consolidate policy deprecation warnings  https://review.opendev.org/68279822:15
openstackgerritColleen Murphy proposed openstack/keystone master: Allow system/domain scope for assignment tree list  https://review.opendev.org/68276222:19
*** markvoelker has quit IRC22:24
cmurphyrc1 patches in flight to review https://etherpad.openstack.org/p/keystone-train-rc1-todo22:28
*** jamesmcarthur has quit IRC22:33
*** jamesmcarthur has joined #openstack-keystone22:50
kmallocbnemec: oh i could have told you how bad of an idea mysql in memory is.22:57
kmallocbnemec, cmurphy: i'll take a quick stare at mysql, if this is a "why would you ever run mysql this way" issue, we can improve that22:58
*** tkajinam has joined #openstack-keystone23:02
*** jamesmcarthur has quit IRC23:09
*** rcernin has joined #openstack-keystone23:16
*** jamesmcarthur has joined #openstack-keystone23:34
*** efried_pto is now known as mriedem123:47
*** mriedem1 is now known as efried_pto23:47
*** ivve has quit IRC23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!