| mnaser | has anyone done some relative scale testing on octavia? | 13:04 |
|---|---|---|
| mnaser | on wallaby, i'm seeing things start to be pretty unhappy at ~50 lb ops at the same time | 13:04 |
| gthiemonge | mnaser: I've seen deployments with more that 2k LBs... main issue is the configuration of the health-manager service, it might require more workers to process incoming heartbeat messages from the amphorae | 13:07 |
| mnaser | gthiemonge: i think in my case the issues that surfaces were concurrent operations | 13:07 |
| mnaser | in my case a large number of concurrent creates/updates/deletes | 13:08 |
| mnaser | resulting in a whole bunch of PENDING_{CREATE,DELETE} stufff | 13:08 |
| gthiemonge | hm I'm not aware of issues there | 13:08 |
| mnaser | gthiemonge: for example right now i've got 36 lbs in PENDING_* states | 13:13 |
| gthiemonge | mnaser: do you know what they are doing? (normal operations - adding a pool, adding a member, or failovers?) | 13:18 |
| mnaser | gthiemonge: in this case, it's an openstack deployment provisioning a whole bunch of new loadbalancers (i.e. k8s LoadBalancer services) | 13:19 |
| gthiemonge | yeah there are a lot of configuration changes when using k8s, if members are updated every 5sec, the LB will spend a lot of time in a PENDING_UPDATE state | 13:24 |
| mnaser | gthiemonge: yeah.. and i think right now i'm seeing the octavia-worker just stalling out completetly and not responding, so doing some more.. checking | 13:45 |
| johnsom | mnaser Please tell me you are running the Octavia API as a wsgi app and not using the simple server command. | 15:14 |
| johnsom | Under wsgi we stress tested the API to around 7,000 concurrent API calls per second with no issue. | 15:14 |
| johnsom | Otherwise, almost all of our provisioning time is waiting on nova or neutron. | 15:16 |
| johnsom | At 7k we ran out of cores for the test tool | 15:20 |
| -opendevstatus- NOTICE: The Gerrit service on review.opendev.org has been restarted for a patch version upgrade, resulting in a brief outage | 21:42 | |
| *** yoctozepto4 is now known as yoctozepto | 23:37 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!