*** gyee has quit IRC | 00:04 | |
*** dnguyen has quit IRC | 00:21 | |
*** Dinesh_Bhor has joined #openstack-keystone | 01:12 | |
*** charz has joined #openstack-keystone | 01:26 | |
*** Dinesh_Bhor has quit IRC | 02:01 | |
*** Dinesh_Bhor has joined #openstack-keystone | 02:02 | |
*** crislomabolivia has quit IRC | 02:05 | |
*** erus has quit IRC | 02:36 | |
openstackgerrit | wangxiyuan proposed openstack/oslo.limit master: Add limit check func https://review.openstack.org/596520 | 03:21 |
---|---|---|
*** Dinesh_Bhor has quit IRC | 03:38 | |
*** Dinesh_Bhor has joined #openstack-keystone | 03:41 | |
*** erus has joined #openstack-keystone | 03:47 | |
openstackgerrit | wangxiyuan proposed openstack/oslo.policy master: Avoid empty raise https://review.openstack.org/613712 | 03:48 |
*** dave-mccowan has quit IRC | 04:19 | |
*** Dinesh_Bhor has quit IRC | 04:21 | |
*** sapd1 has quit IRC | 04:38 | |
*** Dinesh_Bhor has joined #openstack-keystone | 04:49 | |
*** sapd1 has joined #openstack-keystone | 04:59 | |
*** Dinesh_Bhor has quit IRC | 05:00 | |
*** Dinesh_Bhor has joined #openstack-keystone | 05:15 | |
*** jaosorior has quit IRC | 05:32 | |
*** jaosorior has joined #openstack-keystone | 05:32 | |
*** pcaruana has joined #openstack-keystone | 05:36 | |
*** pcaruana has quit IRC | 05:47 | |
*** shyamb has joined #openstack-keystone | 05:54 | |
*** shyamb has quit IRC | 05:58 | |
*** shyamb has joined #openstack-keystone | 05:58 | |
*** shyamb has quit IRC | 06:07 | |
*** wxy-xiyuan has quit IRC | 06:16 | |
*** mnaser has quit IRC | 06:16 | |
*** wxy-xiyuan has joined #openstack-keystone | 06:16 | |
*** mnaser has joined #openstack-keystone | 06:17 | |
*** masayukig[m]1 has quit IRC | 06:18 | |
*** shyamb has joined #openstack-keystone | 06:28 | |
*** hoonetorg has quit IRC | 06:29 | |
*** shyamb has quit IRC | 06:32 | |
*** shyamb has joined #openstack-keystone | 06:32 | |
openstackgerrit | wangxiyuan proposed openstack/oslo.limit master: Add limit check func https://review.openstack.org/596520 | 06:37 |
*** hoonetorg has joined #openstack-keystone | 06:42 | |
*** tonyb has quit IRC | 06:50 | |
*** xek has joined #openstack-keystone | 07:01 | |
*** lbudai has joined #openstack-keystone | 07:03 | |
lbudai | kmalloc: I've missed your answers yesterday regarding the token authorization failure. Thank you for them. | 07:05 |
lbudai | The environment is openstack kilo | 07:05 |
*** shyamb has quit IRC | 07:10 | |
lbudai | kmalloc: I've tested the usernames/passwords used in the services config file, and i can use them in the "service" tenant. So that should be OK. | 07:10 |
vishakha | cmurphy: Thanks for the reply. I have one node with devstack installed and the files keystone that are available in /etc/apache2/sites-available are keystone-wsgi-admin.conf and keystone-wsgi-public.conf. No file that you mentioned wsgi-keystone.conf. And also when created one keystone.conf, after enabling it, apache service failed. | 07:29 |
*** shyamb has joined #openstack-keystone | 07:45 | |
*** shyamb has quit IRC | 07:49 | |
*** pcaruana has joined #openstack-keystone | 07:57 | |
*** pcaruana is now known as pcaruana|elisa| | 07:59 | |
*** sapd1 has quit IRC | 08:10 | |
cmurphy | vishakha: just use the keystone-wsgi-public.conf one | 08:11 |
vishakha | cmurphy: ok. I will update the public one. Thanks | 08:12 |
*** sapd1 has joined #openstack-keystone | 08:12 | |
*** Dinesh_Bhor has quit IRC | 08:16 | |
openstackgerrit | wangxiyuan proposed openstack/oslo.limit master: Add limit check func https://review.openstack.org/596520 | 08:23 |
*** Dinesh_Bhor has joined #openstack-keystone | 08:30 | |
*** shyamb has joined #openstack-keystone | 08:48 | |
openstackgerrit | inspurericzhang proposed openstack/keystone-tempest-plugin master: [Trivial Fix] update home-page url https://review.openstack.org/614147 | 08:49 |
openstackgerrit | Merged openstack/keystone master: Implement scope_type checking for credentials https://review.openstack.org/594547 | 08:58 |
*** shyamb has quit IRC | 09:36 | |
*** Dinesh_Bhor has quit IRC | 09:55 | |
*** Dinesh_Bhor has joined #openstack-keystone | 10:15 | |
vishakha | kmalloc: https://review.openstack.org/#/c/610479/ , For this Document change is merged. Pl review | 10:20 |
openstackgerrit | Colleen Murphy proposed openstack/keystone master: Delete "Preparing your environment" section https://review.openstack.org/614172 | 10:33 |
openstackgerrit | wangqiang-bj proposed openstack/keystoneauth master: fix wrong spelling of "unnecessary" https://review.openstack.org/614178 | 10:40 |
*** lbudai has quit IRC | 10:45 | |
*** shyamb has joined #openstack-keystone | 11:03 | |
*** dave-mccowan has joined #openstack-keystone | 11:11 | |
*** Dinesh_Bhor has quit IRC | 11:19 | |
*** mvkr has quit IRC | 11:30 | |
openstackgerrit | Lance Bragstad proposed openstack/oslo.policy master: Make upgrades more robust with policy overrides https://review.openstack.org/614195 | 11:48 |
*** erus has quit IRC | 11:59 | |
lbragstad | kmalloc ^ oslo.policy changes we were discussing late last week | 11:59 |
*** mvkr has joined #openstack-keystone | 12:02 | |
*** raildo has joined #openstack-keystone | 12:16 | |
openstackgerrit | Corey Bryant proposed openstack/keystone master: PY3: switch to using unicode text values https://review.openstack.org/611190 | 12:19 |
lbragstad | cmurphy do you have thoughts on https://review.openstack.org/#/c/605169/7 ? | 12:29 |
cmurphy | lbragstad: i'm wondering what happens for upgrades, would a federated user end up with two entries in the table? | 12:38 |
cmurphy | one from logging in before the change and one if they try to log in again afterward | 12:39 |
lbragstad | that's kinda what i was asking in the review, but more along the lines of the id format... | 12:40 |
lbragstad | but yeah, that's a good question | 12:40 |
cmurphy | oh no I think it's fine, the unique_id comes from http://git.openstack.org/cgit/openstack/keystone/tree/keystone/auth/plugins/mapped.py#n328 | 12:46 |
cmurphy | so it will be either the id of the local user or the name from REMOTE_USER | 12:47 |
cmurphy | i agree with your nits about the release note though | 12:48 |
*** lbudai has joined #openstack-keystone | 12:58 | |
*** shyamb has quit IRC | 13:01 | |
openstackgerrit | Lance Bragstad proposed openstack/keystone master: Pass context objects to policy enforcement https://review.openstack.org/605539 | 13:03 |
openstackgerrit | Lance Bragstad proposed openstack/keystone master: Remove obsolete credential policies https://review.openstack.org/597187 | 13:25 |
*** dklyle has quit IRC | 13:37 | |
*** dklyle has joined #openstack-keystone | 13:37 | |
lbragstad | vishakha https://review.openstack.org/#/c/589378/14/keystone/trust/backends/sql.py@192 we'll need to get a follow-up proposed for that | 13:52 |
*** dklyle has quit IRC | 14:02 | |
*** david-lyle has joined #openstack-keystone | 14:02 | |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/oslo.policy master: Add ability to pass in target data for the oslopolicy-checker https://review.openstack.org/613313 | 14:15 |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/oslo.policy master: Pass in policy name as part of the oslopolicy-check check call https://review.openstack.org/614223 | 14:15 |
openstackgerrit | Juan Antonio Osorio Robles proposed openstack/oslo.policy master: WIP: Create OPA check https://review.openstack.org/614224 | 14:15 |
*** itlinux has quit IRC | 14:20 | |
*** wxy| has joined #openstack-keystone | 14:23 | |
openstackgerrit | wangxiyuan proposed openstack/keystone master: Deprecate eventlet related configuration https://review.openstack.org/568764 | 14:28 |
*** pcaruana|elisa| has quit IRC | 14:33 | |
*** jmlowe has quit IRC | 14:41 | |
*** raildo has quit IRC | 14:43 | |
*** pcaruana|elisa| has joined #openstack-keystone | 14:45 | |
*** raildo has joined #openstack-keystone | 14:50 | |
*** jmlowe has joined #openstack-keystone | 15:02 | |
*** dave-mccowan has quit IRC | 15:03 | |
* lbragstad takes an early lunch to get a run in | 15:03 | |
*** itlinux has joined #openstack-keystone | 15:11 | |
*** gyee has joined #openstack-keystone | 15:29 | |
*** pcaruana|elisa| has quit IRC | 15:35 | |
*** pcaruana|elisa| has joined #openstack-keystone | 15:50 | |
kmalloc | vishakha: thanks | 15:54 |
kmalloc | lbragstad: ack | 15:54 |
kmalloc | lbragstad: hm. | 15:56 |
kmalloc | lbragstad: i think we need to do that both ways | 15:56 |
kmalloc | if old is overridden, use that check_str | 15:56 |
kmalloc | if new is overidden, use that check_str | 15:56 |
lbragstad | i think i added test cases for both | 15:56 |
kmalloc | ah | 15:56 |
kmalloc | also, it's failing pyXX :) | 15:56 |
* lbragstad thinks he did, at least | 15:56 | |
kmalloc | let me 2x check | 15:56 |
kmalloc | i only read the commit | 15:56 |
*** aojea_ has joined #openstack-keystone | 15:57 | |
kmalloc | lbragstad: let me drink more coffee | 15:58 |
kmalloc | but at first glance this looks to only handle the case of the overidden old policy | 15:58 |
kmalloc | i'll confirm in a bit, but the order i'm aiming for is: New Str (if operator supplied), old str (if opperator applied), (new_default OR old_default) | 15:59 |
kmalloc | in order. | 15:59 |
kmalloc | comma in that list indicates a STOP (do not process more rules) | 15:59 |
openstackgerrit | Lance Bragstad proposed openstack/oslo.policy master: Make upgrades more robust with policy overrides https://review.openstack.org/614195 | 16:00 |
*** pcaruana|elisa| has quit IRC | 16:00 | |
lbragstad | fixed the failing tests, it was based on the patch that requires a new version of oslo.context | 16:00 |
kmalloc | ah | 16:01 |
* kmalloc is here and mostly awake for meeting. | 16:01 | |
lbragstad | https://review.openstack.org/#/c/614195/2/oslo_policy/tests/test_policy.py@1240 | 16:02 |
*** dave-mccowan has joined #openstack-keystone | 16:23 | |
*** dnguyen has joined #openstack-keystone | 16:30 | |
openstackgerrit | Gage Hugo proposed openstack/keystone master: Move to password validation schema https://review.openstack.org/614294 | 16:32 |
*** shyamb has joined #openstack-keystone | 16:38 | |
*** ayoung has joined #openstack-keystone | 16:40 | |
*** shyamb has quit IRC | 16:44 | |
*** jrist has quit IRC | 16:51 | |
*** aojea_ has quit IRC | 16:53 | |
*** jrist has joined #openstack-keystone | 16:54 | |
*** mvkr has quit IRC | 16:55 | |
*** itlinux has quit IRC | 16:57 | |
cmurphy | kmalloc: i was thinking of having somebody work on part of this https://bugs.launchpad.net/keystonemiddleware/+bug/1736985 specifically looking for where devstack is still using auth_host/auth_port/auth_protocol junk and fixing those, what do you think? | 17:01 |
openstack | Launchpad bug 1736985 in keystonemiddleware "many things that were deprecated long ago were never removed" [Low,In progress] - Assigned to Morgan Fainberg (mdrnstm) | 17:01 |
cmurphy | kmalloc: meaning can you hold off on fixing those bits for now if you were planning on it ;) | 17:03 |
*** wxy| has quit IRC | 17:04 | |
ildikov | lbragstad: sorry, I didn't get a response from James and I also missed the meeting thanks to the switch to winter time here already... :/ | 17:05 |
*** itlinux has joined #openstack-keystone | 17:07 | |
lbragstad | ildikov no worries, we'll try again next week | 17:13 |
ildikov | lbragstad: we can do the edge call next week as usual | 17:14 |
kmalloc | cmurphy: yeah i just was hitting the bigger issue ones | 17:14 |
ildikov | lbragstad: and bring it up on the Keystone meeting if we feel it reached that stage that it makes sense to discuss with the broader Keystone team? | 17:15 |
kmalloc | cmurphy: the pkiz/rev list ones were a bit more involved, the rest of those were planned to remain open | 17:15 |
kmalloc | cmurphy: also because i need to rework some of the workflow of ksm to drop the webob stuff from keystone when we build authcontext | 17:16 |
kmalloc | i tagged the pkiz/rev list work to that bug as the bug was "open ended" on the cleanup bits | 17:17 |
*** aojea_ has joined #openstack-keystone | 17:19 | |
*** erus has joined #openstack-keystone | 17:22 | |
*** aojea_ has quit IRC | 17:31 | |
*** aojea_ has joined #openstack-keystone | 17:32 | |
lbragstad | ildikov sounds good | 17:35 |
knikolla | cmurphy: i'm forwarding you an email from another outreachy applicant. When do you have time in the following days to sync up? | 17:39 |
*** aojea_ has quit IRC | 17:39 | |
knikolla | Today I' | 17:40 |
*** aojea_ has joined #openstack-keystone | 17:40 | |
knikolla | I'm swamped as well, but starting tomorrow I should have time. | 17:40 |
*** aojea_ has quit IRC | 17:44 | |
*** ayoung has quit IRC | 17:46 | |
*** lbudai has quit IRC | 17:48 | |
openstackgerrit | Gage Hugo proposed openstack/keystone master: Remove check for disabled v3 https://review.openstack.org/613402 | 17:51 |
openstackgerrit | Gage Hugo proposed openstack/keystone master: Refactor flask domain config resources https://review.openstack.org/613182 | 18:10 |
cmurphy | knikolla: weird, that person already reached out to me but about the other project, I hadn't responded to them yet | 18:20 |
cmurphy | knikolla: I have time tomorrow | 18:21 |
kmalloc | i'll need to duck out for a few hours today | 18:24 |
kmalloc | doggo needs to go to the vet (he's hurting =/ and is sad) | 18:24 |
kmalloc | cmurphy, lbragstad: how useful is json home *really*? | 18:25 |
kmalloc | knikolla: ^ | 18:25 |
kmalloc | i get the feeling it's not really useful at all | 18:25 |
lbragstad | i like it because it advertises API status | 18:26 |
kmalloc | for the individual API endpoints | 18:26 |
kmalloc | that is a fair assertion to the benefit | 18:26 |
lbragstad | and we don't really have a replacement at the moment short of micro-versions =/ | 18:26 |
kmalloc | i was less worried about the replacement bits | 18:26 |
kmalloc | i was more concerned with the "how useful is it" | 18:27 |
kmalloc | because it feels mostly useless, barring the api status (stable, experimental, deprecated) | 18:27 |
lbragstad | imo - that's the important part... but i also don't consume it | 18:28 |
kmalloc | i don't think anyone really consumes it | 18:28 |
cmurphy | i think having a way to advertise api status is really useful since we don't have microversions | 18:28 |
kmalloc | cmurphy: ++ i agree | 18:28 |
cmurphy | but we've never socialized or documented this so i don't think anyone uses it | 18:28 |
kmalloc | and even with microversions... | 18:28 |
kmalloc | i'm mulling in my head the move to split auth to /auth | 18:29 |
*** itlinux has quit IRC | 18:29 | |
kmalloc | and what that really means | 18:29 |
kmalloc | still wanting to divorce crud api version from auth | 18:29 |
kmalloc | and what that means for auth discoverability (it could mean auth could be a separate service in it's own right) | 18:30 |
* lbragstad thinks he knows where this is going | 18:31 | |
kmalloc | especially for the well-defined auth endpoints for web-sso | 18:31 |
lbragstad | ;) | 18:31 |
kmalloc | mostly for the web-sso enhancements in the short term. | 18:31 |
kmalloc | i don't want to encode all the well-defined points in /v3/auth if that makes sense | 18:32 |
kmalloc | lbragstad: you know my long term goals, but i'm looking at the 1-2 cycle bits right now | 18:32 |
*** cabledude has joined #openstack-keystone | 18:38 | |
cabledude | good day experts... I'm struggling with a Mitaka to Ocata upgrade. nova_api will not start and keystone.log states this is due to MFA. How would I disable MFA for nova user? | 18:41 |
cabledude | found this "user["options"]["multi_factor_auth_enabled"] = False" in the documentation but am unsure how to implement. curl? | 18:42 |
cabledude | also seeing "Discovering versions from the identity service failed when creating the password plugin. Attempting to determine version from URL." when running openstack commands | 18:44 |
cabledude | they do complete successfully though | 18:44 |
lbragstad | cabledude yeah - we don't really have good documentation around user options unfortunately | 18:47 |
lbragstad | but you could use curl to set user options directly | 18:47 |
lbragstad | for example https://docs.openstack.org/keystone/latest/admin/identity-security-compliance.html#setting-an-account-lockout-threshold | 18:48 |
lbragstad | doing a PATCH /v3/users/{user_id} with a payload like http://paste.openstack.org/show/733663/ would set that specific user option | 18:49 |
*** cabledude has quit IRC | 18:51 | |
kmalloc | lbragstad: i'm trying to figure out how cabledude got that all enabled? | 18:52 |
lbragstad | same here- that should default to false? | 18:53 |
*** cabledude has joined #openstack-keystone | 18:53 | |
cabledude | Would anyone be able to help me understand how to define user MFA rules? | 18:54 |
cabledude | day 4 of head banging on this. any advice would be greatly appreciated! | 18:56 |
kmalloc | lbragstad: yeah | 18:56 |
kmalloc | cabledude: so. lets start with what have you done and what are you trying to accomplish | 18:57 |
kmalloc | cabledude: i wrote a huge chunk of that code, so let me see what i can do to help you | 18:57 |
kmalloc | it's thankfully not crazy complex | 18:57 |
kmalloc | ;) | 18:57 |
kmalloc | just undocumented | 18:57 |
lbragstad | and no client support i don't think | 18:58 |
lbragstad | :( | 18:58 |
kmalloc | yeah | 18:58 |
cabledude | thx! i have updated keystone via ubuntu upgrade docs. i can run openstack commands successfully | 18:58 |
kmalloc | we'll add that to SDK here | 18:58 |
kmalloc | lbragstad: ^ | 18:58 |
lbragstad | oh | 18:59 |
kmalloc | lbragstad: i wont even bother trying to add it to ksc. | 18:59 |
lbragstad | nice | 18:59 |
kmalloc | lbragstad: i want to make ksc officially "bit rot" by the end of stien | 18:59 |
cabledude | the nova_api service will not start and complains about not having a service_token | 18:59 |
lbragstad | at least we have an alternative to using cURL | 18:59 |
kmalloc | lbragstad: that is the plan | 18:59 |
kmalloc | hmmm. | 18:59 |
kmalloc | this is sounding like that bug where the dbsync somehow mis-matched the schema | 18:59 |
kmalloc | and the mfa table doesn't exist | 18:59 |
kmalloc | well user_options | 19:00 |
cabledude | keystone.log say MFA rules not satisfied. | 19:00 |
kmalloc | cabledude: you didn't try and enable MFA rules for the nova user right? | 19:00 |
cabledude | let me check | 19:00 |
kmalloc | just a plain upgrade | 19:00 |
kmalloc | cabledude: if you tried to enable rules i'd need to see what your user_options look like for the nova user | 19:00 |
cabledude | i didn not "try" to enable rules for any user. Just showed up after the upgrade. | 19:01 |
kmalloc | but the long/short is that the MFA rules are a series of logical ORs in lists, so if you want to force a user to use PASSWORD and TOTP you can say [['password', 'totp']] | 19:01 |
cabledude | mitaka to ocata | 19:01 |
kmalloc | ok can you check the keystone log for exceptions? | 19:01 |
kmalloc | and 2, can you confirm the values in the nova user's 'user_options' field when you do a get_user | 19:02 |
kmalloc | i can tell you where to look in the DB if you can't do a user get on nova | 19:02 |
cabledude | i have no MFA table :( | 19:02 |
kmalloc | there isn't an MFA table | 19:02 |
kmalloc | it's going to be user_options table | 19:03 |
lbragstad | user options are a generic tool that we use for some aspects of MFA | 19:03 |
cabledude | ah | 19:03 |
lbragstad | among other things | 19:03 |
kmalloc | there are PCI-DSS specific mechanisms added into it, so you can exempt users from being locked out | 19:03 |
cabledude | my user options table is empty | 19:03 |
kmalloc | ok | 19:03 |
kmalloc | that is a good start, it should mean that the MFA rules are not being processed | 19:04 |
cabledude | ok | 19:04 |
kmalloc | can you login (not in keystonemiddleware) directly to keystone with the nova user's credentials? | 19:04 |
kmalloc | and if not, what is the keystone log saying when you try | 19:05 |
kmalloc | and/or what error are you getting back (401) | 19:05 |
kmalloc | ? | 19:05 |
*** aojea_ has joined #openstack-keystone | 19:06 | |
cabledude | if i source nova credentials and do a "nova list" for example I get Unable to establish connection to http://controller:8774/v2.1 | 19:06 |
cabledude | because the nova_api service is not listening | 19:06 |
*** jmlowe has quit IRC | 19:07 | |
cabledude | keystone.log shows "MFA Rules not processed for user `95b6745259944d1bb0f540874700f899`. Rule list: `[]` (Enabled: `True`). check_auth_methods_against_rules /usr/lib/python2.7/dist-packages/keystone/auth/core.py:388" | 19:07 |
openstackgerrit | Morgan Fainberg proposed openstack/keystonemiddleware master: Remove PKI/PKIZ support https://review.openstack.org/613675 | 19:07 |
openstackgerrit | Morgan Fainberg proposed openstack/keystonemiddleware master: Fix revocation list/PKI[z] removal nits. https://review.openstack.org/613966 | 19:08 |
kmalloc | cabledude: that is 100% ok | 19:08 |
kmalloc | cabledude: that has no impact, there are no MFA rules so there is nothing being processed | 19:08 |
kmalloc | see the Rule List: [] | 19:09 |
kmalloc | that tells me we're doing the right thing | 19:09 |
kmalloc | :) | 19:09 |
kmalloc | FWIW, debug logs are not always super useful in production | 19:09 |
kmalloc | yeah so you are going to want to use the nova users information from your nova.conf (authtoken section) and try and login to keystone with it | 19:10 |
cabledude | ok so might you have any ideas why nova_api complains with " ERROR nova ImportError: cannot import name service_token" | 19:10 |
kmalloc | aaaaah | 19:10 |
kmalloc | that is a very different issue. | 19:10 |
cabledude | oh! | 19:10 |
kmalloc | that makes me think something isn't installed | 19:10 |
kmalloc | that is telling you that the module service_token isn't importable | 19:10 |
cabledude | still a keystone config issue? or nova? | 19:11 |
kmalloc | if you look in the Nova log, you should see a more in depth exception | 19:11 |
kmalloc | no this isn't likely to be a keystone issue | 19:11 |
kmalloc | or it is a keystonemiddleware one | 19:11 |
kmalloc | but not a keystone server one | 19:11 |
kmalloc | this is something wrong trying to start nova-api at the python level | 19:11 |
kmalloc | like, a package isn't installed | 19:12 |
kmalloc | how was your nova installed? distribution? (ubuntu/rdo/etc)? with git? docker? | 19:12 |
kmalloc | something else? | 19:12 |
cabledude | ubuntu | 19:12 |
cabledude | initially mitaka then followed the keystone upgrade doc to ocata | 19:13 |
kmalloc | hmm | 19:13 |
cabledude | found here https://docs.openstack.org/releasenotes/keystone/ocata.html | 19:13 |
kmalloc | you might have an out of date version of dependencies | 19:13 |
cabledude | sweet... | 19:13 |
cabledude | 2018-10-30 13:09:46.027 30194 CRITICAL nova [-] ImportError: cannot import name service_token 2018-10-30 13:09:46.027 30194 ERROR nova Traceback (most recent call last): 2018-10-30 13:09:46.027 30194 ERROR nova File "/usr/bin/nova-api", line 10, in <module> 2018-10-30 13:09:46.027 30194 ERROR nova sys.exit(main()) 2018-10-30 13:09:46.027 30194 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/cmd/api.py", li | 19:14 |
kmalloc | sec | 19:14 |
cabledude | from nova-api.log | 19:14 |
lbragstad | cabledude can you throw that into a paste? http://paste.openstack.org/ ? | 19:15 |
cabledude | there's more to it if you need the rest | 19:15 |
cabledude | sure hang on | 19:15 |
kmalloc | yeah you don't have keystoneauth1 or an out dated version of keystoneauth installed | 19:16 |
kmalloc | lbragstad: ^ that is from keystoneauth1.loading import service_token | 19:16 |
kmalloc | my guess is it is about when folks moved from ksc to ksa | 19:16 |
cabledude | Paste #733664 | 19:17 |
kmalloc | yeah so you'll need the correct version of keystoneauth installed on the system/venv/whatever that nova is running | 19:18 |
kmalloc | you're missing that dependency or an out dated version of keystoneauth1 | 19:19 |
kmalloc | it might also be missing other dependencies | 19:19 |
cabledude | I have python-keystoneauth1 2.18.0-0ubuntu2~cloud0 and python3-keystoneauth1 2.18.0-0ubuntu2~cloud0 | 19:19 |
cabledude | any idea what versions I would need? | 19:20 |
kmalloc | looking | 19:20 |
kmalloc | at least 2.16 | 19:21 |
kmalloc | so 2.18 should do it | 19:21 |
kmalloc | but not sure why you can't load for nova | 19:21 |
cabledude | do I need tokenless_auth configured? | 19:22 |
kmalloc | no | 19:22 |
kmalloc | the issue is that the nova API python code isn't able to find keystoneauth1.service_token module | 19:23 |
cabledude | do I need both python and python3 keystoneauth1 modules? | 19:23 |
kmalloc | are you running nova under python3 (i didn't think that was going to work) | 19:23 |
kmalloc | also, i have *no* idea how ubuntu has packaged this | 19:24 |
cabledude | Ha! | 19:24 |
kmalloc | i am not involved in the packaging of oepnstack (for any distro) | 19:24 |
kmalloc | this looks like py2.7 | 19:24 |
kmalloc | from the traceback | 19:24 |
kmalloc | so i wouldn't worry about python3 | 19:25 |
kmalloc | on the same node you have nova installed | 19:25 |
kmalloc | do "python" | 19:25 |
kmalloc | and then type "from keystoneauth1 import service_token" | 19:25 |
kmalloc | see if it loads | 19:25 |
kmalloc | if not, check to make sure the correct version of keystoneauth1 is in-fact installed where nova is looking for it | 19:25 |
cabledude | >>> from keystoneauth1 import service_token Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name service_token >>> | 19:26 |
kmalloc | hrm. ok | 19:26 |
lbragstad | computer says "no" | 19:26 |
*** aojea_ has quit IRC | 19:26 | |
cabledude | again... :( | 19:26 |
kmalloc | you might want to do import keystoneauth1 | 19:26 |
kmalloc | and then keystoneauth1.__version__ | 19:27 |
kmalloc | and see what version it thinks you have | 19:27 |
*** aojea_ has joined #openstack-keystone | 19:27 | |
kmalloc | i am pretty certain that the version you have is 2.15 or earlier | 19:27 |
cabledude | >>> keystoneauth1.__version__ '2.6.0' >>> | 19:28 |
kmalloc | yeah | 19:28 |
kmalloc | check to make sure you don't have keystoneauth1 installed via pip | 19:28 |
kmalloc | but that is the issue | 19:28 |
kmalloc | 2.6 is WAY older than what you need | 19:28 |
kmalloc | 2.16 or later (And chances are 2.18 or later) | 19:28 |
*** nelsnelson has joined #openstack-keystone | 19:29 | |
cabledude | 2.6 is older than 2.16? | 19:29 |
kmalloc | yes | 19:29 |
kmalloc | 2.6 is ~10 releases older | 19:30 |
kmalloc | that isn't 2.1.6 | 19:30 |
kmalloc | it's 2.16.0 | 19:30 |
cabledude | ah | 19:30 |
*** aojea_ has quit IRC | 19:30 | |
cabledude | ok hang tight a sec while I mess thing up more... | 19:31 |
kmalloc | semver is <major release>.<minor release>.<patch level> | 19:31 |
*** aojea_ has joined #openstack-keystone | 19:31 | |
kmalloc | major release is going to break lots of things between them | 19:31 |
kmalloc | minor releases tend to be additive and not majorly breaking/minor changes | 19:31 |
kmalloc | and patches are "OMG this was so broken, we fixed a bug" | 19:31 |
kmalloc | or "there was a typo" | 19:31 |
kmalloc | etc. | 19:31 |
kmalloc | patch level is non-breaking by definition | 19:32 |
kmalloc | (barring serious security flaws) | 19:32 |
openstackgerrit | Morgan Fainberg proposed openstack/keystonemiddleware master: Stop supporting revocation list https://review.openstack.org/613651 | 19:36 |
openstackgerrit | Morgan Fainberg proposed openstack/keystonemiddleware master: Remove PKI/PKIZ support https://review.openstack.org/613675 | 19:36 |
openstackgerrit | Morgan Fainberg proposed openstack/keystonemiddleware master: Fix revocation list/PKI[z] removal nits. https://review.openstack.org/613966 | 19:37 |
kmalloc | lbragstad, ayound, edmondsw: ^ fixed the concerns. some things are in the followup nits because I kept having weird rebases re-introduce typos | 19:39 |
kmalloc | ayoung* ^ | 19:39 |
lbragstad | thanks | 19:40 |
cabledude | kmalloc thank you! 2.6 was installed via pip. uninstalled and version is now 2.18 and nova starts! | 19:45 |
kmalloc | cabledude: happy to help! | 19:45 |
kmalloc | cabledude: you can always use pip --freeze to check that in the future | 19:45 |
cabledude | i never use pip. but guess I must have at some point 2 years ago when I set up the stack | 19:46 |
*** jmlowe has joined #openstack-keystone | 19:48 | |
kmalloc | i recommend immutable infrastructure | 19:49 |
kmalloc | for cases like this | 19:49 |
kmalloc | obviously, db servers need to be upgraded (schema) | 19:49 |
kmalloc | but typically i recommend re-deploying the control plane if you can, clean VM, container, something | 19:49 |
kmalloc | that way you aren't carrying forward awkward things like 2-year-old pip installs | 19:49 |
*** ayoung has joined #openstack-keystone | 19:50 | |
kmalloc | cabledude: glad we could help you solve the case of the weirdly out of date keystoneauth1 package | 19:50 |
*** aojea_ has quit IRC | 19:52 | |
*** nelsnelson has quit IRC | 19:56 | |
*** nelsnelson has joined #openstack-keystone | 19:58 | |
*** pcaruana|elisa| has joined #openstack-keystone | 20:12 | |
*** pcaruana|elisa| has quit IRC | 20:32 | |
*** imacdonn has quit IRC | 20:33 | |
*** imacdonn has joined #openstack-keystone | 20:34 | |
*** mvkr has joined #openstack-keystone | 20:37 | |
*** erus has quit IRC | 20:54 | |
*** aojea_ has joined #openstack-keystone | 20:57 | |
*** imus has quit IRC | 20:58 | |
*** raildo has quit IRC | 21:13 | |
*** nels has joined #openstack-keystone | 21:14 | |
openstackgerrit | ayoung proposed openstack/oslo.policy master: Unit test for CLI https://review.openstack.org/614356 | 21:14 |
*** nelsnelson has quit IRC | 21:15 | |
openstackgerrit | Merged openstack/keystone master: Set Default and resource limit as defined schema https://review.openstack.org/610479 | 21:24 |
openstackgerrit | Merged openstack/keystone master: Delete "Preparing your environment" section https://review.openstack.org/614172 | 21:25 |
ayoung | cmurphy, lbragstad knikolla https://review.openstack.org/#/c/607346/ Bump. Lets get that through | 21:32 |
lbragstad | I'll take a look either tonight or tomorrow | 21:34 |
lbragstad | thanks ayoung | 21:34 |
ayoung | thanks | 21:34 |
openstackgerrit | ayoung proposed openstack/keystone-specs master: Tokens with subsets of roles https://review.openstack.org/186979 | 21:43 |
openstackgerrit | ayoung proposed openstack/keystone-specs master: Allow a remote service to Validate Federation Mapping https://review.openstack.org/245588 | 21:47 |
*** david-lyle has quit IRC | 22:08 | |
*** aojea_ has quit IRC | 22:24 | |
openstackgerrit | Gage Hugo proposed openstack/keystone master: Remove compatability shim https://review.openstack.org/614361 | 22:27 |
*** dklyle has joined #openstack-keystone | 22:28 | |
*** xek has quit IRC | 22:31 | |
*** dnguyen has quit IRC | 23:01 | |
*** dnguyen has joined #openstack-keystone | 23:02 | |
*** prometheanfire has joined #openstack-keystone | 23:12 | |
prometheanfire | new verions of future seem to break keystone https://review.openstack.org/614314 | 23:13 |
kmalloc | oh that isn't fun | 23:22 |
mordred | prometheanfire: TIL about future | 23:22 |
kmalloc | prometheanfire: looks like it' | 23:23 |
kmalloc | s realted to pysaml | 23:23 |
kmalloc | prometheanfire: this is not going to be a quick turnaround/fix | 23:24 |
kmalloc | but it's odd that changing future broke that | 23:25 |
kmalloc | prometheanfire: lets just drop py27 support on keystone today :P | 23:25 |
kmalloc | easy fix! :P | 23:26 |
mordred | ++ | 23:27 |
mordred | dropping py27 support in zuul was the most glorious day | 23:27 |
kmalloc | honestly, if i thought i could get away with dropping py27 today I would | 23:27 |
mordred | we're getting closer ... | 23:27 |
kmalloc | and then propose a patch that rips out everysingle use of six | 23:27 |
kmalloc | i know... "closer" | 23:28 |
kmalloc | tbh, keystone could probably get away with it today. | 23:28 |
kmalloc | since it would be trivial to deploy it in a container | 23:28 |
kmalloc | for the OSes that don't have 3x | 23:28 |
kmalloc | but..... | 23:28 |
kmalloc | hmmmm | 23:28 |
kmalloc | cmurphy: it might not be doable to test the 404 thing easily | 23:34 |
kmalloc | cmurphy: in unit tests that is | 23:34 |
kmalloc | might need to be functional/tempest | 23:34 |
clarkb | fwiw the TC does have a statement of distro support | 23:36 |
clarkb | and currently that requires python2 | 23:36 |
prometheanfire | wooo | 23:41 |
prometheanfire | kill py27, good for me :P | 23:41 |
prometheanfire | clarkb: I think we know :P | 23:41 |
kmalloc | clarkb: my comment is mostly just snark | 23:52 |
kmalloc | clarkb: because i don't want to chase a fix in pysaml for a py27 issue | 23:52 |
clarkb | fair enough. I mostly point it out because there is/was a ton of confusion over switching test base distros and how that affects python versions | 23:53 |
clarkb | we've had a policy n place for a long long time and I guess many are unaware of it? | 23:53 |
kmalloc | the only concern i had was ambiguity (on the py35 front) based upon the resolutions and wording that was to the effect of "should not drop py35" | 23:54 |
kmalloc | py27 i figured would die sometime in the next cycle or 3 | 23:54 |
kmalloc | due to EOL timeframes | 23:54 |
*** dnguyen has quit IRC | 23:54 | |
kmalloc | and distro support | 23:54 |
kmalloc | honestly, i would love to see openstack py37+ only | 23:55 |
kmalloc | for lots and lots of reasons | 23:55 |
kmalloc | but that is not realistic | 23:55 |
kmalloc | hm. there should be no reason future change impacts this | 23:57 |
kmalloc | prometheanfire: ^ afaict | 23:57 |
kmalloc | i'm looking into it now, but.... it seems like this is something else going on | 23:57 |
clarkb | oh actually I may know this one | 23:58 |
clarkb | it looks like the same problem twine had | 23:58 |
kmalloc | that future is impacting the import of html? | 23:58 |
clarkb | no, it was some other dep had dropped html? | 23:59 |
clarkb | fungi: ^ if you are still around I think you helped fix this? | 23:59 |
kmalloc | hmmmm | 23:59 |
kmalloc | yeah something lost html somewhere | 23:59 |
clarkb | html was a dep of a thing and that thing dropped html then all the things broke | 23:59 |
kmalloc | *grump* | 23:59 |
kmalloc | ok you know what, i think i jsut want to add html to keystone's requirements then | 23:59 |
clarkb | that should do it | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!