Friday, 2018-03-09

*** sapd_ has quit IRC00:12
*** beagles|biab is now known as beagles00:18
openstackgerritCarlos Goncalves proposed openstack/octavia master: Rename q- to neutron- services
rm_workso we're not catching something correctly there00:30
rm_workand i don't think that's an error with your cert00:31
imacdonnI don't see anything obviously wrong with it00:31
rm_workif self.cryptography is an X509 object...00:31
rm_worki think to_cryptography should work00:31
rm_worki wonder what version of stuff you have00:31
imacdonnwas just pondering that too00:32
rm_workit might be too old in RDO / cent / RHEL / whatever00:32
imacdonnpython2-pyOpenSSL-16.2.0-3.el7.noarch ?00:33
rm_workshould be part of cryptography00:35
rm_worknot pyopenssl00:35
rm_worki think00:36
imacdonnignore that link .. bad Google!00:36
rm_workoh but do this00:37
rm_work python -c 'import OpenSSL; print(OpenSSL.__version__)'00:37
rm_workthat's not the same thing00:37
imacdonnas above ;)00:38
rm_workyeah ok00:38
rm_worksometimes it does weird mismatches00:38
rm_workof course i'm on 17.5.000:38
imacdonnfancy :P00:39
openstackgerritCarlos Goncalves proposed openstack/octavia master: Prevent awk matching itself when stopping Octavia
rm_workand cryptography?00:40
rm_workTHAT may be too old00:40
rm_workcan you uninstall that package and install cryptography with pip?00:40
* cgoncalves senses package bump request coming toward his direction00:41
imacdonnI'll try the fedora rawhide RPM ... it's 2.1.400:42
rm_workthat actually DOESN'T meet our requirements.txt00:42
cgoncalvesin our defense (downstream speaking) we follow what's in requirements.txt; upstream should take the responsability of bumping it in global-requirements00:42
rm_workcryptography!=2.0,>=1.9 # BSD/Apache-2.000:42
rm_work1.7.2 is not >= 1.900:43
rm_workmaybe it was that old in that release? though i doubt it00:46
cgoncalves"FIXME: system version is stuck to 1.7.2 for cryptography"00:46
rm_workthis is why we run things in virtualenvs or containers :P00:47
cgoncalvesso whoever updated from 1.6 knew octavia requires 1.900:47
cgoncalvesFWIW tripleo overcloud is containerized now in queens. undercloud containerized coming in rocky00:47
rm_workimacdonn: interested in results when you update :P00:48
rm_workbut brb getting food00:48
imacdonnok... multitasking a bit00:48
imacdonnlooks like it needs openssl 1.1 ... which is probably why they didn't want to update it00:53
rm_workyes, update openssl and pyopenssl and cryptography >_>00:58
rm_worksystem openssl upgrade may be a pain tho00:58
rm_worki wonder if there's guides to set it up side-by-side00:58
rm_worknot replace the whole system, just compile against a newer one00:58
openstackgerritMin Sun proposed openstack/neutron-lbaas-dashboard master: Cannot update ssl certificate when update listener
*** annp has joined #openstack-lbaas01:09
cgoncalvesimacdonn: I'm opening a bug report. could you please provide more info of your env and how you got the exception?01:12
imacdonncgoncalves: here? or on the bug ?01:12
rm_workcan we validate that the problem is resolved on an upgrade?01:13
cgoncalvesimacdonn: here since I need that info to open the bug, unless you want to open it yourself :)01:13
imacdonncgoncalves: Env is rdo-queens RPMs installed on Oracle Linux 7 ... trigger for the issue was attempting to create a listener, specifying a "container" that's actually a secret containing a PKCS#12 blob, like:01:14
imacdonnopenstack loadbalancer listener create --protocol-port 443 --protocol TERMINATED_HTTPS --name listener2 --default-tls-container= lb201:14
imacdonnwhich produces:
cgoncalvesimacdonn: barbican?01:15
cgoncalveshmm I think we don't support barbican yet01:16
imacdonnwell ..... it's there (RPMs) .. and it seems to work01:16
cgoncalvesright. octavia has been there since liberty or so and is only getting supported now :)01:17
cgoncalvesanyway it's definitely worth reporting and fixing01:18
imacdonn says it's there too .. but I don't know what you mean by "supported"01:18
cgoncalvesimacdonn: supported in OSP01:18
imacdonnoh, that :P01:19
cgoncalvesi'm not saying it's not supported. i just don't know01:19
imacdonn there may be an answer there, but I can't read it :) :P01:20
imacdonnif it's not supported, you can't support Octavia with TLS01:21
cgoncalvesimacdonn: what's the octavia NVR installed?01:22
cgoncalvesrpm -qa *octavia*01:23
*** Swami has quit IRC01:23
cgoncalvesimacdonn: "Barbican is expected to be shipped with Red Hat OpenStack Platform 13 . Barbican (Please note: only command-line client is included as Tech preview) is available as a Tech preview feature in RHOS10."01:24
cgoncalvesupdated september 7, 201701:24
rm_workpart of me is really enjoying seeing someone from Oracle struggle with Redhat content paywalls01:29
cgoncalvesimacdonn: bug 1553520 in openstack-octavia "Cannot create listener with TLS termination" [Urgent,New] - Assigned to amuller01:29
rm_workthough I do appreciate having someone here from oracle who's trying to actually participate in the community :)01:30
openstackgerritCarlos Goncalves proposed openstack/octavia master: [WIP] Add grenade support
*** imacdonn has quit IRC02:03
*** imacdonn has joined #openstack-lbaas02:03
*** sapd has joined #openstack-lbaas02:14
*** harlowja has quit IRC02:21
*** fnaval has joined #openstack-lbaas03:41
*** fnaval has quit IRC03:41
*** mburrows has joined #openstack-lbaas03:45
*** sapd has quit IRC04:18
*** sapd has joined #openstack-lbaas04:33
*** yamamoto_ has joined #openstack-lbaas04:52
*** ivve has joined #openstack-lbaas04:55
*** yamamoto has quit IRC04:56
*** mburrows has quit IRC05:20
*** harlowja has joined #openstack-lbaas05:25
*** kobis has joined #openstack-lbaas05:45
*** kobis has quit IRC05:49
*** harlowja has quit IRC06:00
*** gcheresh_ has joined #openstack-lbaas06:02
openstackgerritOpenStack Proposal Bot proposed openstack/neutron-lbaas master: Imported Translations from Zanata
*** kobis has joined #openstack-lbaas06:14
*** harlowja has joined #openstack-lbaas06:19
*** gcheresh_ has quit IRC06:22
*** kobis has quit IRC06:22
*** mburrows has joined #openstack-lbaas06:23
*** kobis has joined #openstack-lbaas07:23
*** pcaruana has joined #openstack-lbaas07:27
*** bonky has quit IRC07:45
*** rcernin has quit IRC08:00
*** b_bezak has joined #openstack-lbaas08:01
*** harlowja has quit IRC08:08
rm_workROFL that's a new one:
rm_workdeadlock on quotas08:25
rm_worki wonder if it's basically the same thing i found and helped fix in octavia (though I think johnsom was the one that really fixed it)08:26
rm_workand all of these "sys:1: ResourceWarning: unclosed file <_io.FileIO name=1 mode='wb' closefd=True>" are interesting too08:28
rm_worki wonder where that's coming from08:28
rm_workwish it showed where the fd was created08:28
*** tesseract has joined #openstack-lbaas08:38
*** threestrands_ has quit IRC08:48
*** yamamoto_ has quit IRC08:51
*** yamamoto has joined #openstack-lbaas08:53
dayourm_work, wanna me to fix the merge conflict for: :P08:54
rm_worki'm doing it right now08:54
dayouGreat work08:54
rm_workand done08:55
openstackgerritAdam Harwell proposed openstack/octavia master: Add element and flag to disable DHCP on amp images
dayouCool, how to do a thum up in irc?08:55
rm_workeh, no way I'm aware, but I read :thumsup: like it was parsed to emoji in my head :P08:55
rm_worki'm just going through my patches and bumping the ones I think need some attention08:56
rm_worki've lost track of a few almost08:56
rm_worktoo many pending patches...08:56
dayouLet's wait for johnsom's back08:57
dayouI'll push more in the pipe also08:57
rm_workwe will have a lot for him to look at when he's back ^_^08:58
dayoushock shock shock08:59
rm_workI want to knock out one of the API patches to add backup or timeouts09:00
rm_workor error-redirect09:00
rm_workmaybe on the plane if i have wifi09:01
dayousafe trip, man09:01
dayouI heard we just opened wifi acess for airplanes, but I haven't tried that09:02
dayouI mean in my country09:02
rm_worksometimes it is good but often it is disabled on the routes i take09:02
rm_workit's hard to run tox, if it needs to rebuild :P09:02
rm_workgotta rebuild all my envs now09:02
dayouI got them rebuilt last week, now I am stucked in Queens, due the neutron bug on master that would cause a lot of noise on my fan09:04
*** numans has quit IRC10:00
*** numans has joined #openstack-lbaas10:02
*** salmankhan has joined #openstack-lbaas10:09
*** salmankhan has quit IRC10:12
*** salmankhan has joined #openstack-lbaas10:13
*** salmankhan has quit IRC10:19
*** kobis has quit IRC10:20
*** kobis has joined #openstack-lbaas10:20
*** kobis has quit IRC10:21
*** kobis has joined #openstack-lbaas10:21
*** kobis has quit IRC10:22
*** kobis has joined #openstack-lbaas10:22
*** kobis has quit IRC10:22
*** kobis has joined #openstack-lbaas10:23
*** kobis has quit IRC10:23
*** kobis has joined #openstack-lbaas10:23
*** kobis has quit IRC10:24
openstackgerritMerged openstack/octavia master: Updated from global requirements
*** annp has quit IRC10:30
openstackgerritCarlos Goncalves proposed openstack/octavia master: [WIP] Add grenade support
*** mburrows has quit IRC11:04
*** ispp has quit IRC11:05
*** salmankhan has joined #openstack-lbaas11:06
*** yamamoto has quit IRC11:17
openstackgerritCarlos Goncalves proposed openstack/octavia master: [WIP] Add grenade support
*** yamamoto has joined #openstack-lbaas12:18
*** yamamoto has quit IRC12:25
*** gcheresh_ has joined #openstack-lbaas12:46
*** yamamoto has joined #openstack-lbaas13:02
*** yamamoto has quit IRC13:07
*** salmankhan has quit IRC13:37
*** yamamoto has joined #openstack-lbaas13:48
*** yamamoto has quit IRC13:53
*** AlexeyAbashkin has joined #openstack-lbaas13:56
*** AlexeyAbashkin has quit IRC14:01
*** AlexeyAbashkin has joined #openstack-lbaas14:03
*** AlexeyAbashkin has quit IRC14:07
*** gcheresh_ has quit IRC14:09
*** salmankhan has joined #openstack-lbaas14:15
*** salmankhan has quit IRC14:27
*** salmankhan has joined #openstack-lbaas14:32
*** yamamoto has joined #openstack-lbaas14:33
*** yamamoto has quit IRC14:37
*** AlexeyAbashkin has joined #openstack-lbaas14:43
*** AlexeyAbashkin has quit IRC14:47
*** AlexeyAbashkin has joined #openstack-lbaas14:51
xgerman_rm_work: safe flights14:53
openstackgerritHengqing Hu proposed openstack/octavia-dashboard master: List children pools on LB details page
*** yamamoto has joined #openstack-lbaas14:58
*** yamamoto has quit IRC14:58
*** fnaval has joined #openstack-lbaas15:10
*** bonky has joined #openstack-lbaas15:21
bonkyHi guys, quick question. Is it possible to create a clustered loadbalancer through neutron/octavia ?15:22
bonkyI'm just curious if its supported or not :)15:23
bonkyNo worries guys, found parameter in config.15:29
bonkyThanks anyway :p15:29
bonkyAnother question though, since I've struggled with my deployment of octavia for a couple of days now, I have like 200 loadbalancers in state "error" / "pending_delete".. It seems like they cant be deleted since they were never created.15:31
bonkyCan I force a removal of all these loadbalancers somehow ?15:32
bonkyIts an annoyingly long list at the moment =)15:32
*** AlexeyAbashkin has quit IRC15:45
*** fnaval has quit IRC15:45
*** fnaval has joined #openstack-lbaas15:46
*** yamamoto has joined #openstack-lbaas15:58
*** pcaruana has quit IRC16:02
*** yamamoto has quit IRC16:08
xgerman_mmh, octavia should let you delete them… if not there is always the DB16:16
cgoncalvesbonky: can you try deleting one of those zombie LBs and check what logs output?16:20
openstackgerritMerged openstack/octavia master: Fix kvm-centos.7 gate
bonkycgoncalves: 2018-03-09 16:28:02.858 1 INFO octavia.api.v1.controllers.load_balancer [req-0b01f598-94e7-4079-8af5-453a6ca6a1a9 5fc177cdfe7340399332ece2c09cd11c aff54c4fc2024c2c938def4effbba20e - default default] Load Balancer debc5396-7f4a-4175-bfdc-4c618cafa494 is immutable. 2018-03-09 16:28:02.873 1 DEBUG wsme.api [req-0b01f598-94e7-4079-8af5-453a6ca6a1a9 5fc177cdfe7340399332ece2c09cd11c aff54c4fc2024c2c938def4effbba20e - d16:28
bonkycgoncalves: 2018-03-09 16:28:29.949 1 DEBUG wsme.api [req-68be6258-0a30-4baa-81d9-dee05d9fcf3c 5fc177cdfe7340399332ece2c09cd11c aff54c4fc2024c2c938def4effbba20e - default default] Client-side error: Load Balancer ebf4c628-f480-4140-8028-dec204d7ced4 not found. format_exception /usr/lib/python2.7/site-packages/wsme/
bonkythats from the api-log16:28
bonkyClient-side error: Load Balancer f90495a1-245f-4008-bf7d-5ba81cb74af1 not found. format_exception /usr/lib/python2.7/site-packages/wsme/ <- this seems to be happening when I actually try to delete the loadbalancer16:30
cgoncalvesoctavia reports LB exists and same time it does not, funny16:31
cgoncalvesshould be fixed somehow, definitely. in the mean time, and since you're in a stage env I hope, just delete from DB as xgerman_ suggested16:32
bonkyCan I ask how to do that :D ?16:33
bonkyNever done anything in the db before16:33
xgerman_mmh, then it’s better to leave the DB alone ;-)16:33
bonkyxgerman_: Well if you provide an example, I'm sure I can take it from there ;)16:34
cgoncalvessomething like: $ mysql octavia -e 'delete from loadbalancer where status="error" and status="pending_delete"'16:35
xgerman_and then follow to constrain violations to the other tables16:35
cgoncalvesoops, provisioning_status="error" and operating_status="pending_delete"16:35
bonkyERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`octavia`.`vip`, CONSTRAINT `fk_vip_load_balancer_id` FOREIGN KEY (`load_balancer_id`) REFERENCES `load_balancer` (`id`))16:39
bonkyok, so how do I follow that16:40
xgerman_now you delete the row for that LB in octavia.vip16:45
bonkyhm ok, I'll figure it out, thanks!16:47
*** bonky has quit IRC16:53
openstackgerritCarlos Goncalves proposed openstack/octavia master: [DNM] Add experimental ovsfw-scenario job
*** harlowja has joined #openstack-lbaas17:00
*** ivve has quit IRC17:14
openstackgerritCarlos Goncalves proposed openstack/octavia master: [DNM] Add experimental ovsfw-scenario job
openstackgerritCarlos Goncalves proposed openstack/octavia master: Add experimental ovsfw-scenario job
*** harlowja has quit IRC17:54
*** harlowja has joined #openstack-lbaas17:56
*** harlowja has quit IRC18:00
*** dmellado has quit IRC18:16
xgerman_cgoncalves: Is Ovtavia Tech preview or fully supported in OSP?18:17
*** dmellado has joined #openstack-lbaas18:42
*** AJaeger has joined #openstack-lbaas18:44
AJaegerneutron-lbaas cores, please review and
AJaegerjohnsom: ^18:46
*** harlowja has joined #openstack-lbaas19:00
AJaegerthanks, xgerman_19:02
*** tesseract has quit IRC19:05
*** salmankhan has quit IRC19:06
*** bonky has joined #openstack-lbaas19:21
bonkyIs there any difference between (in neutron.conf, under DEFAULT-section), 'lbaasv2' and '' ? I have 'lbaasv2' currently19:24
*** blake has joined #openstack-lbaas19:27
xgerman_lbaasv2 is the alias19:28
imacdonnneutron_lbaas-12.0.0-py2.7.egg-info/entry_points.txt:lbaasv2 =
imacdonnif you don't actually need neutron-lbaas, you might consider skipping it and just doing straight octavia19:29
bonkyOh, I see, so this is where I get a bit confused. My thought process as of now is to do only octavia. neutron-lbaas is the agent based on right ?19:31
imacdonnneutron-lbaas can drive the old agent-based haproxy-in-a-network-namespace model, or it can (I guess) be a front-end to octavia19:32
bonky <- on this page, under the topic 'Configuring LBaaS v2 with Octavia', it says that I should have '' present in my neutron.conf.19:32
bonkyimacdonn: yes I see19:32
imacdonnor you can just avoid all of that and have octavia-api be the entry-point19:33
imacdonn(at least with the recent releases)19:33
imacdonn(as I understand it)19:33
blakeOnly if you want to use the Octavia driver. If you need the HAProxy driver, or a third party driver, you must use neutron-lbaas for the time being19:33
bonkyHm, I'm on Pike (OSP 12, rhel)19:34
*** AJaeger has left #openstack-lbaas19:34
imacdonnblake: right.. good clarification19:34
bonkyFrom a laymens perspective it makes sense right now to only make use of Octavia. I'm not sure why we would need anything else.19:35
imacdonnI guess Octavia may not be "supported" on OSP 12 ... it's not really clear (to me, yet) if it's supported in OSP 13, but it might be19:35
bonkyStill very confused about the whole ecosystem and the terms.19:35
bonkyimacdonn: Well I have octavia working in OSP 12. And it should be supported in OSP13 as I understand it.19:36
imacdonnthat sounds about right19:36
imacdonnbe aware that there's an issue with openssl, if you want to be able to create TLS LBs, though19:37
imacdonnnot sure where OSP 13 is, release-wise .. I'm using RDO19:37
bonkyok ok19:37
bonkyIs there a collection of "pros" vs "cons" using the 'agent-based haproxy-in-a-network-namespace'-model vs the 'octavia create vm's with a haproxy instance'-model ?19:45
xgerman_agent-based only has cons ;-)19:45
xgerman_Octavia is operator grade with HA, tenant separation, scalability, etc.19:46
bonkytenant separation and scalability I understand. What do you mean by 'operator grade with HA' ?19:47
xgerman_HA means we have sub-second failover19:47
bonkyBut dont you have failover with the 'agent-based one' ?19:48
bonkyyou have multiple controllers running the ha-proxy no ?19:48
xgerman_it’s not sub second… they will failover if they detect an agent as down19:49
bonkyhehe forgive me for my lack of knowledge, but what is "sub second" ?19:50
xgerman_but they don;t share haproxy state there so you will have to renegotiate SSL sessions and such19:50
xgerman_Octavia run haproxy in cluster mode so they share all that19:50
imacdonn"sub second" -> "happens in less than one second"19:50
bonkyThanks for your patience :)19:51
bonkyBut aren't talking about running ha-proxy in ACTIVE/ACTIVE mode now ? Which is not possible yet ? Or do I miss something ?19:52
bonkyclustering via keepalived ?19:53
xgerman_there are two thing: we use keepalived to do the sub second failover for Active-Passive19:53
bonkyOh ok, so they share state even though they are active/passive you mean =19:53
xgerman_but we also run haproxy in a way that the ACTIVE and PASSIVE one share “stick tables”19:53
bonkyoh ok, cool19:54
bonkythen I think I got it19:54
xgerman_if you need to save resources you can overcommit the Octavia VM flavor19:54
xgerman_I had an operator consider having some overcommitted tier to offer for free and some more real one to ask for money19:55
bonkyOk, I see19:56
*** b_bezak has quit IRC19:58
cgoncalvesxgerman_: full support in OSP13 (Queens release), at least that's what we're aiming at :)20:21
*** threestrands has joined #openstack-lbaas20:22
*** threestrands has quit IRC20:22
*** threestrands has joined #openstack-lbaas20:22
cgoncalvesimacdonn: Octavia is not supported on OSP12 (packages are available though); plan is to have it supported from OSP13. neutron-lbaas is still going to be supported but is in deprecation phase (both upstream and downstream)20:25
cgoncalvesimacdonn, bonky: please let me know if you have any further queries about OSP and OSP+Octavia :)20:25
bonkycgoncalves: cool! After struggling a bit I got octavia working in OSP 12 ;)20:26
cgoncalvesbonky: glad to know it's working well for you! :)20:33
imacdonnhe didn't actually say "well" .. lol20:41
*** gcheresh_ has joined #openstack-lbaas20:42
bonkyhaha, well, havent tried that much yet. trying to understand how it works and tidying up the configs a bit.20:43
* imacdonn realises that he just assumed bonky is a "he" .. so much for Internalation Womens' Day :)20:43
imacdonner International*20:44
bonkyOne thing I seem to have some trouble is the octavia user. The octavia user is located in the "defualt" domain. But all our projects and networks are only in our "company" domain. I'm not sure how that is suppose to work20:44
bonkyBut it may be a non issue20:44
xgerman_well, Octavia needs some resources — so with default quotas I ran against a wall with 10 LBs. Not sure if OSP is the same20:45
bonkyyea, same here20:45
bonkysince we are still testing i just bumped the default quotas20:46
imacdonn10 LBs, or 10 instances (amphorae) ?20:46
openstackgerritGerman Eichberger proposed openstack/octavia master: [WIP] Switch amphora agent to use privsep
xgerman_10 LBs aka 20 VMs (and FIPs and…)20:47
bonkyEhm, I actually hit a limit on the security groups. I'm not sure why though ?20:47
xgerman_we use those, too20:48
xgerman_each LB gets it’s own SG20:48
bonkyAnd the quotas given in 'neutron_lbaas.conf' seems to be the "one" that counts (as far as lbaas-related), same parameters in octavia.conf I cant get to work.. Does that make sense?20:49
bonkyok, well then I understand why I hit that limit.20:49
imacdonn[imacdonn@home ~]$ os limits show --absolute --project service | grep -i instances20:49
imacdonn| maxTotalInstances        |    10 |20:49
imacdonn| totalInstancesUsed       |     2 |20:49
imacdonnI guess mine defaults to 10 nova instances :P20:49
xgerman_My current quotas:20:50
xgerman_I think that’s public…20:51
imacdonnI think 10 is the default quota for a new project20:51
*** gcheresh_ has quit IRC20:52
imacdonnhmm, I installed cryptography 2.1.4 with pip, but still get this...21:00
imacdonn2018-03-09 20:59:42.707 21557 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/octavia/certificates/common/", line 35, in get_certificate21:00
imacdonn2018-03-09 20:59:42.707 21557 ERROR oslo_messaging.rpc.server     return self.certificate.to_cryptography().public_bytes(21:00
imacdonn2018-03-09 20:59:42.707 21557 ERROR oslo_messaging.rpc.server AttributeError: 'X509' object has no attribute 'to_cryptography'21:00
imacdonnI think it's pyOpenSSL, but rm_work seemed adamant otherwise21:01
cgoncalvesimacdonn: try pip install newer version of pyOpenSSL? :)21:02
imacdonnof course my LB is immutable now .. sigh21:03
imacdonnpyOpenSSL didn't fix it either ... assuming I'm getting the pip stuff right21:11
imacdonnwill have to look at it more later21:12
imacdonn[root@slc10jtj ~]# pip show cryptography | grep ^Version:21:14
imacdonnVersion: 2.1.421:14
imacdonn[root@slc10jtj ~]# pip show pyOpenSSL | grep ^Version:21:14
imacdonnVersion: 17.5.021:14
cgoncalvesimacdonn: can you pip show requests?21:16
imacdonnVersion: 2.14.221:16
imacdonnnot sure how requests comes into play here ?21:17
imacdonn    Export as a cryptography certificate.21:17
imacdonn    Return type:cryptography.x509.Certificate21:17
imacdonn    New in version
cgoncalvesreading similar reports here
imacdonnyeah, I found that link yesterday21:18
*** sshank has joined #openstack-lbaas21:19
imacdonnwait a sec.. I think I haz the dumb21:19
imacdonnI may not have restarted octavia services after updating pyOpenSSL21:20
*** threestrands has quit IRC21:22
cgoncalvesimacdonn: you could also try: python -c "import requests; requests.get('')"21:24
xgerman_harlowja: msgpack (0.5.3) has the same error21:24
harlowjacan u go farther back to <0.521:24
harlowjai'm assuming it once worked :-P21:25
xgerman_yes, I added a sed to massage OpenStack’s upper-constraints — but I cant be the only one using privsep?!21:25
imacdonncgoncalves: I'd have to figure out how to make that go through a proxy ... really don't think it has anything to do with requests ... will get back to it later ... meeting coming up in 5 min21:25
cgoncalvesimacdonn: ok21:26
harlowjaxgerman_ one would hope...21:26
xgerman_that is interesting:
harlowjaya, thus why < 0.521:27
harlowja0.5 started on jan 621:27
harlowjaso i'd hope 0.4.x may have worked at one point, lol21:27
harlowja(likely what gus used?)21:27
xgerman_wonder what msgpack-python is?21:29
xgerman_so they renamed with 0.5.0 msgpack-python to msgpack…21:38
harlowjaya, i think the github repo is the same21:38
xgerman_which means going back is not so easy21:38
harlowjanot sure why they did it, lol21:38
xgerman_so to force pre 0.5.0 on the system I would need to change privsep’s requirements (back?) toe the old name21:39
harlowjaseems that's a yes21:40
xgerman_guess oslo went to sh*t after you stepped down from PTL21:40
harlowjahard to tell21:41
harlowjajust a lot of softwae21:41
harlowjawith dependencies21:41
harlowja424 is a lot21:41
xgerman_and that ocata — meanwhile in Pike21:41
xgerman_or Queens21:41
harlowjaya, wrong branch21:41
harlowja~460 now21:41
harlowjaall the libraries21:41
harlowjawe are using all of pypi21:42
xgerman_ok, I will submit a patch to privsep to roll back the name change since it’s released under the old name anyway for now21:43
xgerman_(+ the new one)21:43
imacdonncgoncalves: (multitasking while on meeting) seems that the new pyOpenSSL did fix it21:44
imacdonnpreviously had python2-pyOpenSSL-16.2.0-3.el7.noarch .. per paste above, we need >= 17.1.021:45
imacdonnoctavia requirements.txt has >=16.2.0, queens upper-constraints has ==17.5.021:48
*** mburrows has joined #openstack-lbaas21:52
*** sshank has quit IRC21:57
cgoncalvesimacdonn: sh*t, again mismatching of octavia requirements.txt and u-c...22:02
imacdonncgoncalves: yup22:02
cgoncalvesimacdonn: could you please reinstall python-cryptography rpm and retry? so that we can narrow it down to only pyOpenSSL needing update22:03
imacdonncgoncalves: seems to be failing in a different way now....2018-03-09 22:12:07.885 22700 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-0d11cd1f-0a1b-445c-8677-01843a6a959c - ab1732cdc2dc45c98506436f2ef29b07 - - -] Could not connect to instance. Retrying.: SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)22:12
cgoncalves"it's just a warning" (tm)22:13
imacdonnyeah, but it retried ~13 times, then moved to:22:15
imacdonn2018-03-09 22:13:23.035 22700 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-0d11cd1f-0a1b-445c-8677-01843a6a959c - ab1732cdc2dc45c98506436f2ef29b07 - - -] Could not connect to instance. Retrying.: ConnectTimeout: HTTPSConnectionPool(host='', port=9443): Max retries exceeded with url: /0.5/listeners/0e22ee94-d17c-4e30-819d-8a535e42002d/certificates/ (Caused by ConnectTime22:15
imacdonnoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7efd4b7b8450>, 'Connection to timed out. (connect timeout=10.0)'))22:15
cgoncalvesimacdonn: ok, so we need pyOpenSSL>=17.1.0 *and* python-cryptography>=1.9?22:15
imacdonnI tested with cryptography 2.1.422:16
imacdonni.e. that plus pyOpenSSL 17.5.0 is is only combination that's worked so-far22:16
cgoncalvesimacdonn: abusing of your goodwill, could you pip install both with minimum versions set in u-c queens?22:31
imacdonncgoncalves: will try .. it's a pain to have to untangle everything when it fails :/22:33
cgoncalvespyopenssl 17.1.0 is clear to me checking changelog
imacdonnit's also spelled out at
imacdonnso the U-C for Queens says cryptography 2.1.4, and pyOpenSSL 17.5.022:35
cgoncalvesyes. I was just narrowing it down to what octavia queens really needs minimum22:36
cgoncalvesI'll need to open yet another set of rhbz, this time for pyopenssl...22:37
*** jdavis has joined #openstack-lbaas22:38
imacdonncgoncalves: so is there some other combo you wanted to test?22:41
cgoncalvesimacdonn: no, I think that's it. I just wanted to assert that the minimum requirements work22:47
cgoncalvesimacdonn: thank you!22:47
imacdonncgoncalves: I guess I would say that I've verified that the *recommended* versions work ... they're currently different from the minimum versions :/22:47
*** bonky has quit IRC22:48
cgoncalvesimacdonn: indeed! I owe you a few beers :)22:50
imacdonncgoncalves: heh .. no problem22:50
*** jdavis has quit IRC23:12
*** fnaval has quit IRC23:49
*** blake has quit IRC23:52
*** yamamoto has joined #openstack-lbaas23:54

Generated by 2.15.3 by Marius Gedminas - find it at!