*** __ministry has joined #senlin | 04:33 | |
*** sapd1 has joined #senlin | 06:04 | |
*** sapd1 has quit IRC | 07:14 | |
rm_work | yeah, it wasn't... | 13:44 |
---|---|---|
rm_work | so i am unclear as to WHY exactly it happened | 13:44 |
rm_work | but we managed to get it to apply | 13:44 |
rm_work | anyway, different subject -- updating to use the new services! | 13:46 |
rm_work | one service went away, and two new ones were created? or JUST two new ones I guess? | 13:46 |
*** jmlowe has joined #senlin | 16:00 | |
rm_work | AHHHH and FYI, it was a downgrade -- we tried to run an older version (migration 13) on a DB that was at migration 15. that's where the error message came from | 16:10 |
rm_work | and someone solved it by just manually updating the number in the table to 13... which i just discovered when i actually tried to upgrade correctly and it exploded because the migrations wouldn't apply T_T | 16:10 |
rm_work | ... | 16:28 |
rm_work | ok so have you seen this one? | 16:28 |
rm_work | RESP BODY: {"code": 500, "error": {"code": 500, "message": "wrap_socket() got an unexpected keyword argument '_context'", "type": "TypeError"}, "explanation": "The server has either erred or is incapable of performing the requested operation.", "title": "Internal Server Error"} | 16:28 |
rm_work | that's from `openstack cluster list --debug` | 16:28 |
rm_work | is that from me having too old of a client or something? | 16:35 |
rm_work | 2019-12-10 16:37:30,778 DEBUG [senlin.api.common.serializers] /opt/openstack/venv/senlin/lib/python3.7/site-packages/senlin/api/common/serializers.py:to_json:87 JSON response : {"code": 500, "error": {"code": 500, "message": "wrap_socket() got an unexpected keyword argument '_context'", "type": "TypeError"}, "explanation": "The server has either erred or is incapable of performing the requested operation.", "title": "Internal Server | 16:38 |
rm_work | Error"} | 16:38 |
rm_work | hmmm no | 16:44 |
rm_work | this may be an error on the senlin side? just doing a basic curl | 16:44 |
rm_work | eandersson / dtruong ^^ | 16:45 |
rm_work | curl "http://mycloud/v1/clusters" -H "X-Auth-Token: <mytoken>" | 16:46 |
rm_work | that's all I'm doing to get that 500 | 16:46 |
*** jmlowe has quit IRC | 16:50 | |
eandersson | never seen that one rm_work | 16:53 |
rm_work | hmm | 16:53 |
eandersson | what version is this rm_work? | 16:54 |
rm_work | master | 16:55 |
rm_work | as of an hour ago or so | 16:55 |
eandersson | So we no longer test regular api endpoint, we use the wsgi one for CI | 16:55 |
rm_work | hmmm | 16:55 |
rm_work | ffff | 16:55 |
rm_work | k | 16:55 |
rm_work | could be related | 16:55 |
eandersson | but I don't see how wrap_socket could be passed _context | 16:56 |
eandersson | https://github.com/openstack/senlin/blob/71d9a66abe93049a91d433f805c045abe135303a/senlin/api/common/wsgi.py#L284 | 16:56 |
eandersson | I have a PR to rip some of that out and replace it with oslo.service wsgi impl | 16:57 |
rm_work | yeah literally no idea | 16:58 |
rm_work | by keyword no less? | 16:58 |
eandersson | Honestly never tried enabling ssl without using wsgi | 16:59 |
eandersson | so could be an old bug | 16:59 |
rm_work | err actually | 16:59 |
rm_work | i shouldn't be running in SSL? | 16:59 |
eandersson | It’s the only reason that path would be called right? | 17:00 |
rm_work | the api-runner sits behind Apache | 17:00 |
rm_work | yeah i am thinking it must be in a different piece | 17:00 |
rm_work | maybe external | 17:01 |
rm_work | like, calling out to keystone to verify a token or something | 17:01 |
rm_work | ? | 17:01 |
eandersson | Yea maybe | 17:01 |
rm_work | wish i had more stacktrace | 17:01 |
rm_work | but this serializer kinda eats it | 17:01 |
rm_work | https://github.com/eventlet/eventlet/issues/526 | 17:16 |
rm_work | hmm | 17:16 |
rm_work | ok so | 17:16 |
rm_work | we need py3.6 | 17:16 |
rm_work | not py3.7 | 17:16 |
rm_work | senlin is not yet compatible with py37? | 17:16 |
rm_work | though it was fixed in late 2018 so not sure why it'd still be an issue now | 17:17 |
rm_work | you guys use eventlet I guess? :/ | 17:18 |
rm_work | eventlet is poison :( | 17:18 |
rm_work | https://review.opendev.org/#/c/454873/ | 17:19 |
rm_work | we even have https://review.opendev.org/#/c/462334/2/octavia/hacking/checks.py@257 | 17:19 |
rm_work | eandersson: ^^ | 17:19 |
eandersson | I thought we had a python 3.7 test | 17:23 |
eandersson | You using the wsgi endpoint right? Or senlin-api? | 17:23 |
rm_work | senlin-api i think | 17:24 |
rm_work | but eventlet is pulled in multiple places it looks like... and we may run into issues from that too | 17:24 |
dtruong | The fix for that issue is in eventlet >= 0.25.0 | 17:25 |
dtruong | So double check you are using that version or higher | 17:25 |
rm_work | eventlet==0.25.1 | 17:28 |
rm_work | but | 17:28 |
rm_work | i followed the advice in the last post of that thread and installed pyopenssl==19.1.0 | 17:28 |
rm_work | and it seems to work :D | 17:28 |
eandersson | Interesting | 17:41 |
rm_work | it appears that is the highest version of pyopenssl anyway | 17:42 |
rm_work | so i just unpinned | 17:42 |
rm_work | why ya'll gotta use eventlet at all? :D | 17:49 |
eandersson | Someone made that decision a long time ago :p[ | 18:19 |
rm_work | let's see ... how bored am I | 18:23 |
rm_work | one of my favorite pastimes *is* finding things I hate and burning them with righteous fire | 18:23 |
rm_work | eandersson: where's the patch you were talking about? | 18:43 |
eandersson | Sorry I haven't uploaded it yet. I am just removing the custom wsgi implementation and replacing it with oslo.service wsgi | 18:44 |
eandersson | Similar to what I did for Designate | 18:44 |
rm_work | ah yeah i saw your designate patch, that was just a devstack thing? | 18:48 |
rm_work | or was there another one | 18:48 |
rm_work | https://review.opendev.org/#/c/454873/9/octavia/cmd/octavia_worker.py | 18:49 |
rm_work | please avoid oslo.service <_< | 18:49 |
rm_work | we killed off oslo.service almost 3 years ago :D | 18:50 |
rm_work | look at cotyledon | 18:50 |
rm_work | and then remove any references to eventlet :D | 18:51 |
rm_work | eandersson: ^^ pretty please | 18:51 |
rm_work | ah hmm, i guess for the wsgi endpoint, MAYBE that's ok? as long as it doesn't import eventlet anywhere | 18:53 |
rm_work | we used `from wsgiref import simple_server` | 18:53 |
rm_work | for our wsgi definition | 18:53 |
rm_work | https://github.com/openstack/octavia/blob/master/octavia/cmd/api.py | 18:54 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!