| *** tosky has quit IRC | 00:14 | |
| *** tosky has joined #openstack-barbican | 08:13 | |
| *** JohnnyRainbow has joined #openstack-barbican | 10:15 | |
| *** noonedeadpunk has quit IRC | 10:32 | |
| *** noonedeadpunk has joined #openstack-barbican | 10:32 | |
| *** noonedeadpunk has quit IRC | 11:21 | |
| *** noonedeadpunk has joined #openstack-barbican | 11:25 | |
| *** raildo has joined #openstack-barbican | 12:23 | |
| *** moguimar has quit IRC | 14:17 | |
| *** moguimar has joined #openstack-barbican | 14:18 | |
| JohnnyRainbow | Is a somewhere a special role which needs to be added between octavia and barbican to make it operational? I created SNI container and default tls container from certs without password, but octaviaclient can't get certificates somewhere, which is strange, especially that I'm able to get it from my project. I added even ACL for octavia user, but it is the same all the time. Here is | 14:32 |
|---|---|---|
| JohnnyRainbow | "full" listing: https://paste.ofcode.org/PRdnQNacjmpuWyMjWfXnrG I know that we were trying to solve similar issue already, but it was with certs with pass, which was pretty OK that it was failing, but in that story it shouldn't take a place... :) | 14:33 |
| *** moguimar has quit IRC | 14:56 | |
| *** moguimar has joined #openstack-barbican | 14:58 | |
| openstackgerrit | Merged openstack/barbican stable/train: Use serial number or label for PKCS#11 tokens https://review.opendev.org/760154 | 16:37 |
| johnsom | JohnnyRainbow What version of the python-barbicanclient do you have installed there? | 16:51 |
| johnsom | It would be on the API controllers BTW | 17:00 |
| JohnnyRainbow | @johnsom do you mean barbican api controllers or octavia? | 17:30 |
| johnsom | JohnnyRainbow the Octavia API | 17:31 |
| johnsom | python-barbicanclient will be installed there, what I am looking for is the version there | 17:31 |
| JohnnyRainbow | just to be sure about my openstack architecture, I have separate VM for every single component, like barbican, octavia, cinder etc | 17:31 |
| johnsom | That is ok | 17:32 |
| JohnnyRainbow | Do I need to have python-barbicanclient installed on my octavia VM, isn't enough openstackclient? | 17:32 |
| JohnnyRainbow | the question is, because I see in api logs that all commands are going via curl in api...but I can be wrong :) | 17:33 |
| johnsom | Yes, it has to be there as python-barbicanclient is the python language binding for barbican. Octavia will use the libraries in python-barbicanclient to access barbican resouces | 17:33 |
| JohnnyRainbow | I have version 4.6.0-0ubuntu1.1 | 17:34 |
| JohnnyRainbow | of python-barbicanclient | 17:34 |
| johnsom | Ok, that is the problem. This patch is missing: https://github.com/openstack/python-barbicanclient/commit/4eec7121b39de3849b469c56d85b95520aab7bad | 17:35 |
| johnsom | I would try updating that package to a newer version, restarting the Octavia processes and trying again. | 17:36 |
| JohnnyRainbow | wow, you're fast...which version it should be ok? | 17:36 |
| JohnnyRainbow | I have barbican 8.0.1-0ubuntu1 installed on my barbican VMs | 17:37 |
| johnsom | What version of Octavia are you running? | 17:37 |
| JohnnyRainbow | 4.1.0 | 17:37 |
| johnsom | According to this: https://releases.openstack.org/stein/index.html#python-barbicanclient | 17:38 |
| johnsom | You want 4.8.1 | 17:38 |
| johnsom | of python-barbicanclient | 17:38 |
| johnsom | That will include the fix | 17:39 |
| johnsom | It's a bummer the ubuntu package hasn't been updated.... | 17:41 |
| JohnnyRainbow | hmm...should I compile by myself? | 17:41 |
| johnsom | Hmm, actually, as I look at the ubuntu changelog, they came the backported it.... | 17:42 |
| johnsom | http://changelogs.ubuntu.com/changelogs/pool/main/p/python-barbicanclient/python-barbicanclient_4.6.0-0ubuntu1.1/changelog | 17:42 |
| JohnnyRainbow | * d/p/0001-Allow-fetching-by-UUID-and-respect-interface.patch, | 17:43 |
| JohnnyRainbow | is it that one? | 17:43 |
| johnsom | d/p/0002-Secret-payload-should-also-be-fetched-by-UUID.patch | 17:43 |
| johnsom | So it must be some other issue. | 17:44 |
| JohnnyRainbow | can I somehow verify in the code of that python? | 17:44 |
| JohnnyRainbow | if that is included? | 17:44 |
| johnsom | Can you look in the Octavia API process logs and paste me the messages it logs when you attempt to use the certificate? | 17:44 |
| johnsom | Yes, you could look for the secrets.py file and check for the change included here: https://github.com/openstack/python-barbicanclient/commit/4eec7121b39de3849b469c56d85b95520aab7bad | 17:45 |
| johnsom | But I trust the packaging, I suspect it is some other issue. | 17:45 |
| JohnnyRainbow | ok, let me check secrets.py first and if it's there I will collect octavia api logs | 17:45 |
| johnsom | It is this change: https://github.com/openstack/python-barbicanclient/commit/4eec7121b39de3849b469c56d85b95520aab7bad#diff-317e364199973933c481d6a92b785455c02740e4b2d305d3c3f2acff13e0c8f7 | 17:46 |
| JohnnyRainbow | ok, fix is there as I see | 17:51 |
| JohnnyRainbow | that part of code can be found :) | 17:52 |
| JohnnyRainbow | but when I execute my command for create listener I do not see new prompts in /var/log/octavia for such listener creation...but I do not have debug flag set to true for octavia...should I set it to true? | 17:55 |
| johnsom | Hmm, well, with an error occurring I would expect there to be a message even without debug enabled, but turning debug on for the API would not hurt. | 17:57 |
| JohnnyRainbow | OK, here is a print from api.log: https://paste.ofcode.org/qcZ3PZFDxSqTT4magYLRJj | 18:08 |
| JohnnyRainbow | I see format_exception again | 18:09 |
| johnsom | Yep | 18:09 |
| JohnnyRainbow | hmm...I'm using p12 files and certificates are self generated by openssl, but without any passwords | 18:09 |
| johnsom | I need to open a bug to raise that from debug to info | 18:09 |
| JohnnyRainbow | is there any additional requirement? I was following some cookbooks available in openstack docs and also redhat docs and suse docs :) | 18:11 |
| johnsom | These are the best docs: https://docs.openstack.org/octavia/latest/user/guides/basic-cookbook.html#deploy-a-tls-terminated-https-load-balancer | 18:12 |
| JohnnyRainbow | yes, that is what I was using...my certs are generally created that way, so I don't know why it is format issue :) | 18:13 |
| johnsom | Yeah, me either. Give me a minute to look at a few things and think on this | 18:14 |
| JohnnyRainbow | thanks a lot! | 18:15 |
| openstackgerrit | Merged openstack/barbican stable/train: Use barbican.conf in barbican-manage https://review.opendev.org/760631 | 18:19 |
| johnsom | Ok, how do you feel about adding some logging code to the Octavia API? | 18:26 |
| JohnnyRainbow | I can do it, but my env is managed by puppet, so I need to turn off it for a while :) | 18:28 |
| JohnnyRainbow | but no problem | 18:28 |
| johnsom | https://www.irccloud.com/pastebin/uGx8OVjc/ | 18:28 |
| JohnnyRainbow | do I need to restart octavia? | 18:34 |
| JohnnyRainbow | after change? | 18:34 |
| johnsom | Just the API process | 18:34 |
| JohnnyRainbow | I do not see something more in api.log | 18:36 |
| JohnnyRainbow | should I paste it to you? | 18:36 |
| johnsom | Sure | 18:36 |
| JohnnyRainbow | https://paste.ofcode.org/mGKdv5cLsaPpkpEyp369pA | 18:37 |
| johnsom | Ok, I see one change that isn't present in stein. Are your endpoints HTTPS? | 18:37 |
| JohnnyRainbow | which one? Most of them are http | 18:38 |
| johnsom | Yeah, the new message is in there. It's a 404 though. | 18:38 |
| johnsom | Can you paste the output of "openstack endpoint list"? | 18:40 |
| JohnnyRainbow | sure, just a second | 18:40 |
| johnsom | https://controller.tc.eco.atman.pl:35357/v2.0/tokens Seems to be the URL returning the 404 | 18:40 |
| JohnnyRainbow | https://paste.ofcode.org/VWTpvrcbBpnnW8T626UFqs | 18:41 |
| johnsom | In the octavia.conf file for the Octavia API, in the keystone_authtoken section, is auth_url set ? | 18:47 |
| JohnnyRainbow | it is | 18:48 |
| JohnnyRainbow | to https://controller.tc.eco.atman.pl:35357 | 18:48 |
| johnsom | Ok, this might be the problem. It seems to be trying to connect to "https://controller.tc.eco.atman.pl:35357/v2.0/tokens" but I suspect the v2.0/tokens path doesn't exist | 18:50 |
| JohnnyRainbow | but auth_version is set to 3 in keystone_authtoken | 18:50 |
| johnsom | Yeah, I don't know why it would be trying v2.0 | 18:51 |
| JohnnyRainbow | hmm...but why only for barbican? :) | 18:52 |
| johnsom | Can you check the service_auth section is also version 3 and doesn't have the v2.0? | 18:52 |
| JohnnyRainbow | in octavia.conf? | 18:52 |
| johnsom | Yes | 18:52 |
| johnsom | I doubt it is wrong as you are able to create a load balancer. | 18:53 |
| JohnnyRainbow | it is 3 as well | 18:53 |
| johnsom | Ok, I have a long shot thing we can try | 18:54 |
| JohnnyRainbow | sure | 18:54 |
| johnsom | https://www.irccloud.com/pastebin/Bs0jrVOV/ | 18:55 |
| johnsom | Restart API , try again. I see this change in master, but not stein. Maybe somehow a CA mismatch is being reported as a 404 by keystone client. | 18:56 |
| JohnnyRainbow | https://paste.ofcode.org/UCzDYGb5mz7Y9b3VEvCuF | 19:03 |
| JohnnyRainbow | hmm...similar error | 19:03 |
| johnsom | Yeah, that was a longshot | 19:03 |
| johnsom | That doesn't seem like the right conf path really. Do you have "cafile" under the keystone_auth section in the octavia.conf? | 19:09 |
| JohnnyRainbow | do you mean under keystone_authtoken? | 19:10 |
| johnsom | yes | 19:10 |
| JohnnyRainbow | no, it's not there | 19:10 |
| JohnnyRainbow | is that a fault? | 19:12 |
| johnsom | I'm not sure really. I am still trying to figure out the v2.0 thing. | 19:14 |
| JohnnyRainbow | no worries, at least I see direction of a fault, but it is strange to me either :) | 19:15 |
| JohnnyRainbow | I'll be AFK for 1h | 19:19 |
| johnsom | Ok, I'm pretty sure the issue is with the "[service_auth]" section of the octavia.conf file. Specifically I think the "auth_url" in that section may be set incorrectly | 19:20 |
| johnsom | Or maybe the "interface" setting | 19:22 |
| JohnnyRainbow | in service auth I have cafile provided, and auth_url is the same, I mean: https://controller.tc.eco.atman.pl:35357 | 19:22 |
| johnsom | Is the "interface" setting admin? | 19:23 |
| JohnnyRainbow | which setting is that? | 19:25 |
| johnsom | [service_auth] interface = admin | 19:25 |
| JohnnyRainbow | I do not have this in my conf | 19:25 |
| johnsom | Hmm, ok, one thing to check. You can compare your [keystone_authtoken] and [service_auth] sections with those in the neutron.conf. They should be the same | 19:26 |
| JohnnyRainbow | hmm...service_auth is not in neutron.conf :) | 19:30 |
| JohnnyRainbow | but generally config is the same | 19:30 |
| johnsom | Yeah, I think that got removed | 19:30 |
| JohnnyRainbow | but other things are same, except www_authenticate_uri which is https://controller.tc.eco.atman.pl:35357/v3 for octavia and https://controller.tc.eco.atman.pl:35357 for neutron | 19:32 |
| JohnnyRainbow | but I believe it's not a case | 19:32 |
| johnsom | Well, I'm running low on ideas and you need to step away. I'm pretty sure the issue is a configuration issue in the keystone_authtoken and/or service_auth sections. | 19:32 |
| JohnnyRainbow | ok, anyway, thanks a lot for your help, I need to take some break, think and most probably find a solution which is just a step ahead :) | 19:33 |
| JohnnyRainbow | thanks a lot again | 19:33 |
| johnsom | You could try setting the auth_url to https://controller.tc.eco.atman.pl:35357/v3 to see if it stops trying v2.0 | 19:33 |
| johnsom | Ok, good luck. Let us know what you find | 19:34 |
| JohnnyRainbow | definitely! | 19:34 |
| JohnnyRainbow | thanks | 19:34 |
| *** dwilde has quit IRC | 20:46 | |
| *** d34dh0r53 has joined #openstack-barbican | 20:46 | |
| *** JohnnyRainbow has quit IRC | 23:54 | |
| *** tosky has quit IRC | 23:55 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!