| *** elopez has joined #openstack-astara | 00:55 | |
| *** xiayu has joined #openstack-astara | 00:56 | |
| *** elopez has quit IRC | 01:03 | |
| *** Guest8151 has quit IRC | 01:22 | |
| *** x58 has joined #openstack-astara | 01:22 | |
| openstackgerrit | xiayu proposed openstack/astara-appliance: Astara appliance oslo.rootwrap https://review.openstack.org/281034 | 02:15 |
|---|---|---|
| *** elopez has joined #openstack-astara | 02:42 | |
| *** shashank_hegde has joined #openstack-astara | 03:07 | |
| *** shashank_hegde has quit IRC | 03:28 | |
| *** elopez has quit IRC | 03:32 | |
| *** elopez has joined #openstack-astara | 03:33 | |
| *** shashank_hegde has joined #openstack-astara | 03:35 | |
| *** shashank_hegde has quit IRC | 03:42 | |
| *** shashank_hegde has joined #openstack-astara | 03:45 | |
| *** shashank_hegde has quit IRC | 03:47 | |
| *** elopez has quit IRC | 03:52 | |
| *** elopez has joined #openstack-astara | 03:53 | |
| *** elopez has quit IRC | 04:05 | |
| *** elopez has joined #openstack-astara | 04:05 | |
| *** shashank_hegde has joined #openstack-astara | 04:06 | |
| *** shashank_hegde has quit IRC | 04:30 | |
| openstackgerrit | xiayu proposed openstack/astara-appliance: Astara appliance oslo.rootwrap https://review.openstack.org/281034 | 06:02 |
| *** elopez_ has joined #openstack-astara | 07:27 | |
| *** elopez has quit IRC | 07:30 | |
| openstackgerrit | xiayu proposed openstack/astara-appliance: Astara appliance oslo.rootwrap https://review.openstack.org/281034 | 08:04 |
| *** pcaruana has joined #openstack-astara | 08:05 | |
| *** shashank_hegde has joined #openstack-astara | 08:16 | |
| openstackgerrit | xiayu proposed openstack/astara-appliance: Astara appliance oslo.rootwrap https://review.openstack.org/281034 | 08:55 |
| *** elopez_ has quit IRC | 09:01 | |
| *** shashank_hegde has quit IRC | 09:34 | |
| *** openstackgerrit has quit IRC | 10:02 | |
| *** openstackgerrit has joined #openstack-astara | 10:03 | |
| *** Liuqing has joined #openstack-astara | 13:38 | |
| *** justinlund1 has quit IRC | 13:58 | |
| *** justinlund has joined #openstack-astara | 14:00 | |
| *** Liuqing has quit IRC | 14:02 | |
| *** j_king has joined #openstack-astara | 15:50 | |
| *** pcaruana has quit IRC | 16:15 | |
| *** prithiv has joined #openstack-astara | 16:32 | |
| *** elopez has joined #openstack-astara | 16:37 | |
| *** nolan has quit IRC | 16:43 | |
| *** nolan has joined #openstack-astara | 16:44 | |
| ryanpetrello | adam_g you around? | 17:00 |
| *** elopez has quit IRC | 17:50 | |
| *** elopez has joined #openstack-astara | 17:50 | |
| *** elopez has quit IRC | 17:52 | |
| *** elopez has joined #openstack-astara | 17:52 | |
| openstackgerrit | Ryan Petrello proposed openstack/astara-appliance: Work around a potential bug in dnsmasq's config parser. https://review.openstack.org/281986 | 17:54 |
| ryanpetrello | adam_g and markmcclain ^ | 18:02 |
| ryanpetrello | testing this extensively today in production | 18:03 |
| ryanpetrello | but seem to work pretty well | 18:03 |
| ryanpetrello | will report back if I see anything weird | 18:03 |
| *** jordantardif has joined #openstack-astara | 18:17 | |
| *** prithiv has quit IRC | 18:22 | |
| *** prithiv has joined #openstack-astara | 18:35 | |
| *** shashank_hegde has joined #openstack-astara | 18:46 | |
| *** justinlund has quit IRC | 18:56 | |
| *** elo has joined #openstack-astara | 19:01 | |
| *** prithiv has quit IRC | 19:24 | |
| *** justinlund has joined #openstack-astara | 19:27 | |
| *** eric_lopez has joined #openstack-astara | 20:07 | |
| *** Guest79104 has quit IRC | 20:12 | |
| *** elo has quit IRC | 20:12 | |
| *** rods_ has joined #openstack-astara | 20:13 | |
| ryanpetrello | markmcclain: | 20:37 |
| ryanpetrello | https://github.com/openstack/astara/blob/stable/kilo/akanda/rug/openstack/common/rpc/amqp.py#L110 | 20:38 |
| ryanpetrello | what if we just totally replaced this with some alternative type of lock? | 20:38 |
| *** rods_ is now known as rods | 20:39 | |
| *** rods is now known as Guest65788 | 20:39 | |
| ryanpetrello | rebuilding my router | 20:45 |
| ryanpetrello | this seems to *work* | 20:46 |
| ryanpetrello | if we just replace the eventlet.semaphore.Semaphore w/ a Python threading.Semaphore | 20:46 |
| openstackgerrit | Ryan Petrello proposed openstack/astara: Use Python's threading.semaphore to avoid eventlet deadlocks. https://review.openstack.org/282054 | 20:55 |
| ryanpetrello | adam_g markmcclain ^ this is what we're currently testing | 20:55 |
| adam_g | ryanpetrello, what was it you tested last week? that didnt work? | 20:56 |
| ryanpetrello | no | 20:56 |
| ryanpetrello | testing this now | 20:56 |
| ryanpetrello | what we tried last week w/ the timeout argument to the eventlet semaphore doesn't work | 20:56 |
| ryanpetrello | I think the eventlet semaphore implementation is just buggy :/ | 20:57 |
| ryanpetrello | every hang we've seen hangs in the code path on line 115 of ^^ review | 20:57 |
| ryanpetrello | inside of the context manager that attempts to acquire the eventlet semaphore | 20:57 |
| ryanpetrello | so we're trying a threading.semaphore instead, hoping its implementation doesn't suffer the same issue | 20:57 |
| ryanpetrello | currently running this in iad2 and watching | 20:57 |
| ryanpetrello | I'm at a point where I've tried enough that I'd rather take eventlet out of the equation entirely here | 20:58 |
| ryanpetrello | do you guys have an open bug for this somewhere? | 20:58 |
| * drwahl couldn't find one... | 20:58 | |
| ryanpetrello | adam_g I'd like to let this bake a few days on our end | 21:01 |
| ryanpetrello | and see how it goes | 21:01 |
| j_king | i don't think the eventlet semaphore has the correct strong fairness properties required to be correct, causing hangs on acquire() | 21:25 |
| *** elopez has quit IRC | 22:27 | |
| *** justinlund has quit IRC | 23:03 | |
| *** eric_lopez has quit IRC | 23:12 | |
| *** openstackgerrit has quit IRC | 23:47 | |
| *** openstackgerrit_ is now known as openstackgerrit | 23:47 | |
| *** openstackgerrit_ has joined #openstack-astara | 23:47 | |
| *** openstackgerrit_ is now known as openstackgerrit | 23:48 | |
| *** openstackgerrit_ has joined #openstack-astara | 23:49 | |
| *** openstackgerrit_ has quit IRC | 23:55 | |
| *** openstackgerrit_ has joined #openstack-astara | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!