Monday, 2018-11-05

*** Dinesh_Bhor has joined #openstack-keystone00:31
*** Dinesh_Bhor has quit IRC00:31
*** david-outreachy has quit IRC00:59
*** Dinesh_Bhor has joined #openstack-keystone01:37
*** Dinesh_Bhor has quit IRC01:37
*** imus has quit IRC01:37
*** imus has joined #openstack-keystone01:38
*** Dinesh_Bhor has joined #openstack-keystone01:59
*** Dinesh_Bhor has quit IRC02:31
*** Dinesh_Bhor has joined #openstack-keystone02:40
openstackgerritVishakha Agarwal proposed openstack/keystone master: [WIP] Implement scope_type checking for ec2 credentials  https://review.openstack.org/60782002:41
*** crisloma has joined #openstack-keystone02:49
openstackgerritwangxiyuan proposed openstack/oslo.policy master: Add policy-upgrade tool  https://review.openstack.org/61390603:36
*** Dinesh_Bhor has quit IRC03:47
*** Dinesh_Bhor has joined #openstack-keystone03:51
*** dave-mccowan has quit IRC04:00
*** crisloma has quit IRC04:01
*** aojea has joined #openstack-keystone04:35
adriantkmalloc, cmurphy: Auth receipts have landed! Thanks for the review feedback! Will try and get a patch for the docs up this week between summit prep. Then what's my deadline for getting code supporting them merged into keystoneauth1 for Stien?04:36
adriantmy task list for them now is: 1. API docs, and MFA rules docs.  2. Keystoneauth support  3. Horizon support (multi-step login)  4. Openstackcli support (it attempts password, fails, then uses auth receipt to ask for missing auth values).04:38
*** aojea has quit IRC04:39
*** Nel1x has quit IRC04:58
*** pcaruana has joined #openstack-keystone05:23
*** Dinesh_Bhor has quit IRC05:24
*** Dinesh_Bhor has joined #openstack-keystone05:30
*** pcaruana has quit IRC05:32
*** zul has quit IRC05:49
*** felipemonteiro has joined #openstack-keystone06:49
*** jaosorior has joined #openstack-keystone07:13
*** jaosorior has quit IRC07:24
*** jaosorior has joined #openstack-keystone07:27
*** Dinesh_Bhor has quit IRC07:54
*** felipemonteiro has quit IRC08:00
*** pcaruana has joined #openstack-keystone08:06
*** ykarel has joined #openstack-keystone08:27
adriantkmalloc: ykarel seems to be getting some weird 401 errors in tests since merging the auth receipts patch. Weird thing, it is happening on random tests, as if the tokens are expiring. They are also doing fernet_rotate with a cron every 10 mins (sort of) with: "*/10 * * * *"08:29
adriantlbragstad, cmurphy: ^08:29
ykarelyup is doing cron that frequent is correct, what's the ideal config that should be done in deployment, and doc , any reference?08:30
ykarelkmalloc, lbragstad cmurphy ^^08:30
adriantI'm not sure what the auth receipts patch could have done to make the problem suddenly appear. I'm going to put up a patch for him to test with a lot of extra debug logging because the current keystone.log isn't showing me anything useful08:30
adriantI was worried that maybe my changes caused keystone_rotate to double rotate, but... I don't think so08:31
adriant*fernet_rotate08:31
*** amoralej has joined #openstack-keystone08:37
ykareladriant, is this difference expected after that keystone patch:08:39
ykarelpassing job before that patch: https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario004/3372/weirdo-project/logs/etc/keystone/fernet-keys/08:39
ykarelfailing job after that patch:- https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario004/3374/weirdo-project/logs/etc/keystone/fernet-keys/08:39
ykarelthe only difference in these two jobs is the keystone patch: https://github.com/openstack/keystone/commit/c785729efee0daf472301719e0b41e9ff2b7c64d and one nova patch(which seems unrelated)08:40
ykareljfi nova patch was: https://github.com/openstack/nova/commit/781c22818ff0e0aff839b1b47e2db1fba756c87508:41
openstackgerritAdrian Turjak proposed openstack/keystone master: [DO NOT MERGE] Extra debug for puppet-openstack-integration failures  https://review.openstack.org/61550208:47
adriantykarel: can you please try with the above patch ^ ?08:47
adriantnot sure if you can, but those extra debug statements would help me when looking at the keystone log during your tests08:47
ykareladriant, ack, amoralej can we test keystone ^^ patch in poi? i am not sure we build packages with Depends-On ?08:51
amoralejno, we don't08:51
amoralejwe need to test manually08:51
cmurphyo/08:51
ykarelamoralej, okk will test that locally08:52
adriantcmurphy: hello! Not really sure what could be happening here :(08:52
adriantit almost looks like a fernet over rotation, but their config appears to be 5 keys, and rotate every 10th minute i the hour08:53
adriantso unless their tests take over 50mins... that shouldn't be causing 401s08:53
cmurphythe logs sure make fernet look suspicious https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6726/weirdo-project/logs/keystone/keystone.txt.gz?level=WARNING#_2018-11-04_21_42_52_72308:55
cmurphyadriant: it looks like it's double-rotating every time https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6726/weirdo-project/logs/keystone/keystone.txt.gz?level=INFO#_2018-11-04_21_30_03_25008:56
cmurphyso this is not doing the right thing i guess http://git.openstack.org/cgit/openstack/keystone/tree/keystone/cmd/cli.py#n47108:57
adriantyeah, which is weird :/08:58
adrianthttps://github.com/openstack/keystone/blob/master/keystone/conf/fernet_tokens.py#L2008:59
adrianthttps://github.com/openstack/keystone/blob/master/keystone/conf/fernet_receipts.py#L2208:59
adriantunless....08:59
adriantOH08:59
adriantsec, lemme check something08:59
cmurphyhttps://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario004/3372/weirdo-project/logs/etc/keystone/keystone.conf.txt.gz08:59
cmurphythey set [fernet_tokens]/key_repository to /etc/keystone/fernet-keys without the /09:00
adriantyeah09:00
adriantthat's what I was thinking09:00
adrianthmmm09:00
adriantso sort of not entirely my fault... but also a weird one09:00
*** Dinesh_Bhor has joined #openstack-keystone09:01
cmurphyyeah weird corner case, we'll have to make it rstrip the / before the comparison or something09:01
adriantmaybe i can do some real filepath comparison?09:02
cmurphy++09:02
adriantwe can have it build a real OS path for both and then compare09:02
openstackgerritMerged openstack/keystone master: Use port 5000, keystone-wsgi-public and --http-socket  https://review.openstack.org/61473409:02
adriantykarel, amoralej: you two see the above?09:03
adriantit's an easy fix to your conf, but I will get it fixed with a patch soon09:03
amoralejykarel, ^ are you building a reproducer?09:04
* ykarel reading back09:04
adriantcmurphy: if only we could be python3 only right now I could use pathlib :(09:05
ykarelamoralej, yes i am building09:05
ykareladriant, so should i try ur patch or just that config fix, key_repository?09:05
adriantykarel: my patch will tell you the same thing09:05
adriantyou're rotating twice each time09:05
adriantjust change your config09:06
adriantor just use the default09:06
ykareladriant, okk so just trying the config change09:06
adriantmeanwhile I'll get a patch up to compare real paths rather than stirngs09:06
adriantso this problem goes away09:06
ykarelamoralej, ^^ i will try that in ci itself09:07
ykarelwith current repo09:07
amoralejack09:07
adriantcmurphy: any objections to me adding pathlib2 to keystone requirements? with a note to switch to py3 pathlib when py2 is dropped?09:15
adriantit's already in global requirements: https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L28209:15
adriantyes I could use other means, but pathlib is just so much cleaner, and come py3 we'll use it anyway09:15
cmurphyadriant: seems sort of heavy handed to add a new dependency when the same thing can be accomplished with just rstrip or os.path09:18
adriantfair, but I'll leave a note to switch to pathlib when we drop py209:19
cmurphyokay09:19
adriantcmurphy: it's just so nice: http://paste.openstack.org/show/734131/09:20
cmurphyadriant: i mean http://paste.openstack.org/show/734132/09:22
* adriant bows to cmurphy09:23
adriantdone, adding the code now09:23
cmurphy;)09:23
adriantI think some part of my just loves the idea of a path as an object with its own magical ability to compare against another of its kind rather than a string, but it is very much an irrational one :P09:26
openstackgerritwangxiyuan proposed openstack/oslo.limit master: Add limit check func  https://review.openstack.org/59652009:31
ykareladriant, amoralej testing https://review.openstack.org/#/c/614988/09:31
*** pcaruana has quit IRC09:31
openstackgerritAdrian Turjak proposed openstack/keystone master: Fix an issue with double fernet key rotation  https://review.openstack.org/61551609:32
amoralejykarel, iiuc ^ that's the fix09:32
*** pcaruana has joined #openstack-keystone09:32
ykarelamoralej, yes adriant is fixing it in keystone https://review.openstack.org/61551609:33
adriantoh wait, missed one09:33
adriantsec09:33
cmurphybah i don't think we have unit tests for the fernet_rotate cli09:33
openstackgerritAdrian Turjak proposed openstack/keystone master: Fix an issue with double fernet key rotation  https://review.openstack.org/61551609:34
ykarelamoralej, and in poi https://review.openstack.org/#/c/615513/ if keystone patch takes time09:34
adriantthere we go09:34
adriantI didn't change both fernet_setup and fernet_rotate ... :/09:34
amoralejok, ok, i see it now09:34
amoralejykarel,  Syntax error at 'fernet_max_active_keys' (file: /etc/puppetlabs/code/modules/openstack_integration/manifests/keystone.pp09:35
ykareli think i missed comma09:35
amoralejyeap09:35
ykarelFixed09:36
* adriant is happy this wasn't a huge break09:36
adriantapart from this no one should hit anything with auth receipts until they actually add rules, which considering there aren't any docs on how, is not something I expect to happen yet09:37
*** ykarel is now known as ykarel|lunch09:41
*** ykarel|lunch is now known as ykarel10:05
openstackgerritColleen Murphy proposed openstack/keystone master: Fix uwsgi --http flag  https://review.openstack.org/61552210:15
openstackgerritVishakha Agarwal proposed openstack/keystone master: Remove deprecated "bind" in token  https://review.openstack.org/61389110:18
openstackgerritVishakha Agarwal proposed openstack/keystone master: Remove deprecated "bind" in token  https://review.openstack.org/61389110:19
*** nkinder has quit IRC10:32
*** Dinesh_Bhor has quit IRC10:34
ykareladriant, amoralej cmurphy job passed with trailing slash in [fernet_token]/key_repository: http://logs.openstack.org/88/614988/4/check/puppet-openstack-integration-5-scenario002-tempest-centos-7/372d448/logs/testr_results.html.gz11:43
amoralejgood11:44
*** beekneemech has quit IRC11:53
*** bnemec has joined #openstack-keystone11:57
*** dave-mccowan has joined #openstack-keystone12:04
*** raildo has joined #openstack-keystone12:10
openstackgerritJuan Antonio Osorio Robles proposed openstack/oslo.policy master: WIP: Create OPA check  https://review.openstack.org/61422412:21
*** jroll has quit IRC12:32
*** jroll has joined #openstack-keystone12:34
*** amoralej is now known as amoralej|lunch13:06
*** imus has quit IRC13:10
*** zul has joined #openstack-keystone13:15
*** jistr is now known as jistr|call13:32
*** aojea has joined #openstack-keystone13:33
*** aojea has quit IRC13:40
*** aojea has joined #openstack-keystone13:40
*** aojea has quit IRC13:45
*** felipemonteiro has joined #openstack-keystone13:53
*** amoralej|lunch is now known as amoralej14:00
*** aojea has joined #openstack-keystone14:05
*** jistr|call is now known as jistr14:08
*** david-outreachy has joined #openstack-keystone14:08
*** nelsnelson has joined #openstack-keystone14:12
*** felipemonteiro has quit IRC14:21
amoralejcould we get reviews on https://review.openstack.org/#/c/615516/2 ?14:22
amoralejit's blocking promotions in RDO14:22
*** SteelyDan is now known as dansmith14:37
*** aojea has quit IRC14:40
*** aojea has joined #openstack-keystone14:41
*** felipemonteiro has joined #openstack-keystone14:42
*** felipemonteiro has quit IRC14:45
*** aojea has quit IRC14:45
*** mvkr has quit IRC14:47
*** ekikoh has joined #openstack-keystone14:48
cmurphykmalloc: good morning https://review.openstack.org/61544614:52
*** imus has joined #openstack-keystone14:56
*** munimeha1 has joined #openstack-keystone15:02
*** jistr is now known as jistr|call15:04
gagehugoo/15:10
*** ekikoh has quit IRC15:10
*** jistr|call is now known as jistr15:10
openstackgerritDavid.O proposed openstack/keystonemiddleware master: Documentation Fix - auth_url Port Number  https://review.openstack.org/61558215:13
*** mvkr has joined #openstack-keystone15:15
*** d34dh0r53 has quit IRC15:22
*** cloudnull has quit IRC15:22
*** eglute has quit IRC15:22
*** d34dh0r53 has joined #openstack-keystone15:23
*** eglute has joined #openstack-keystone15:23
*** chudler has left #openstack-keystone15:33
*** dklyle has joined #openstack-keystone16:02
*** pcaruana has quit IRC16:06
*** nels has joined #openstack-keystone16:13
*** nelsnelson has quit IRC16:15
*** imacdonn has quit IRC16:16
*** imacdonn has joined #openstack-keystone16:17
openstackgerritMerged openstack/keystone master: Fix an issue with double fernet key rotation  https://review.openstack.org/61551616:22
kmalloccmurphy: +2/+A there has to be a better way to do that16:32
kmallocbut... that fixes the immediate problem16:33
kmalloccmurphy: does that need backporting to rocky too?16:33
cmurphykmalloc: oh probably, i didn't check16:34
*** nkinder has joined #openstack-keystone16:34
cmurphyyeah we don't test that dev env thing really16:34
kmallocyup16:37
kmallocif it isn't tested, it's broken imo16:37
cmurphyagreed but it's documented so we should strive for it being unbroken16:37
kmallocyep16:38
cmurphykmalloc: https://review.openstack.org/61559816:41
openstackgerritMerged openstack/keystone master: Fix developer config dir flask aftermath  https://review.openstack.org/61544616:41
kmalloccmurphy: +2/+A16:43
*** nkinder has quit IRC16:47
*** openstackgerrit has quit IRC16:48
*** cwright has quit IRC16:48
*** cwright has joined #openstack-keystone16:49
*** gyee has joined #openstack-keystone17:02
*** openstackgerrit has joined #openstack-keystone17:09
openstackgerritMerged openstack/oslo.policy master: Add ability to pass in target data for the oslopolicy-checker  https://review.openstack.org/61331317:09
*** nels has quit IRC17:18
*** nelsnelson has joined #openstack-keystone17:21
*** ykarel has quit IRC17:23
*** mriedem has joined #openstack-keystone17:37
mriedemlbragstad: are you ok with me removing the lazy translation stuff from keystone? https://github.com/openstack/keystone/blob/e3c1633ea871cac1af5c9515752c3632edfcb476/keystone/server/flask/core.py#L2617:37
lbragstadmriedem that's causing issues with the upgrade checker goal, right?17:37
mriedemyeah. bnemec has a workaround for that for now https://review.openstack.org/#/c/615610/17:38
mriedembut i just didn't realize other projects still had that lazy translation stuff i nthem17:38
mriedempretty sure no one uses it17:38
*** stewie925 has joined #openstack-keystone17:38
mriedemso at best it's just dead code, at worst it's going to break thigns17:38
mriedemi can hit up the ops list as well17:40
lbragstadthe justification in https://bugs.launchpad.net/oslo.i18n/+bug/1801761 makes sense17:40
openstackLaunchpad bug 1801761 in oslo.upgradecheck "enable_lazy should be deprecated/removed" [High,In progress] - Assigned to Ben Nemec (bnemec)17:40
lbragstadi don't have any reason to hold on to it, but confirmation on the ML would be nice17:41
kmallocmriedem: happy to remove the lazy translate stuff if it's not needed17:48
*** dave-mccowan has quit IRC17:49
kmalloclbragstad: looks like i'm going to need to write a new memcache driver for oslo.cache becasue apparantly we broke redis when we fixed the URL processing in the arguemnt.url thing17:58
kmallocunrelated... someone should really tell ceiliometer not to use redis ever.17:58
*** erus has joined #openstack-keystone18:00
gagehugokmalloc: I was looking at https://bugs.launchpad.net/oslo.cache/+bug/1578466, could we add that effort in as well?18:06
openstackLaunchpad bug 1578466 in oslo.cache "cache should offer encryption in a similar manner to keystonemiddleware cache does" [Wishlist,Confirmed]18:06
kmallocgagehugo: that is just a proxy, it's a lot easier to develop18:07
kmallocgagehugo: it's been on a long long long long list of would like to do18:07
kmallocthat said, i worry about the CPU cost of both serializing and encrypt/signing the data18:07
* kmalloc would honestly rather deprecate that functionality in KSM18:08
*** aojea has joined #openstack-keystone18:08
gagehugohmm18:09
* kmalloc goes and gets pymemcache out and starts writing a driver.18:24
kmallocif it works i'll override the default in oslo.cache18:24
* kmalloc also has some oslomiddleware code to drop soon.18:24
*** aojea has quit IRC18:29
*** aojea has joined #openstack-keystone18:29
*** jmlowe has quit IRC18:30
*** aojea has quit IRC18:40
*** david-outreachy has quit IRC18:48
*** david-outreachy has joined #openstack-keystone18:48
*** aojea has joined #openstack-keystone18:51
*** aojea has quit IRC18:51
*** mriedem has left #openstack-keystone18:51
*** aojea has joined #openstack-keystone18:52
*** jmlowe has joined #openstack-keystone18:54
*** mvkr has quit IRC18:55
*** aojea has quit IRC18:56
*** david-outreachy has quit IRC19:02
*** david-outreachy has joined #openstack-keystone19:03
openstackgerritMerged openstack/keystonemiddleware master: Documentation Fix - auth_url Port Number  https://review.openstack.org/61558219:07
*** david-outreachy has quit IRC19:07
*** xek_ has joined #openstack-keystone19:09
*** xek has quit IRC19:12
*** zul has quit IRC19:24
*** amoralej is now known as amoralej|of19:41
*** amoralej|of is now known as amoralej|off19:41
*** jmlowe has quit IRC20:19
openstackgerritLance Bragstad proposed openstack/oslo.limit master: WIP: Expose enforcement API outside of ctx manager  https://review.openstack.org/61564320:27
*** jmlowe has joined #openstack-keystone20:39
*** david-outreachy has joined #openstack-keystone20:49
*** erus has quit IRC20:54
*** aojea has joined #openstack-keystone21:07
*** raildo has quit IRC21:15
*** itlinux has joined #openstack-keystone21:23
*** david-ou_ has joined #openstack-keystone21:37
*** david-ou_ has quit IRC21:38
*** david-outreachy has quit IRC21:40
*** imus has quit IRC21:48
*** felipemonteiro has joined #openstack-keystone22:28
*** munimeha1 has quit IRC22:42
*** itlinux has quit IRC22:45
*** mvkr has joined #openstack-keystone22:51
*** felipemonteiro has quit IRC23:07
*** lbragstad has quit IRC23:09
*** lbragstad has joined #openstack-keystone23:10
*** ChanServ sets mode: +o lbragstad23:10
*** xek__ has joined #openstack-keystone23:21
*** xek_ has quit IRC23:24
*** aojea has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!