| tobiash[m] | corvus: how urgent is https://review.opendev.org/c/zuul/zuul/+/797182 to you? I'm asking because we'd need to check if we can make async reporting compatible to it. Otherwise we'd be blocked since this would have a performance impact on the event processing and we're already operating at its limits. | 13:22 |
|---|---|---|
| fungi | tobiash[m]: wouldn't it reduce the reporting load since the pipeline manager is taking care of writing to the db? | 13:48 |
| fungi | tobiash[m]: also you've blocked https://review.opendev.org/691253 as a wait-and-see for sos | 13:50 |
| tobiash[m] | fungi: most of this is running in the mainloop which is our bottleneck atm. We're running with 691253 since without it our system would break down | 14:16 |
| fungi | ahh, okay | 14:16 |
| fungi | so adding more database interactions in the pipeline manager would slow the main loop even more, i guess | 14:16 |
| fungi | are you thinking we'd need a database manager thread to collect the writes from a queue and performs the actual db interactions? | 14:19 |
| fungi | also if that queue is in zk, are writing those things to zk possibly just as costly as writing to the rdbms? | 14:20 |
| corvus | fungi: fwiw, i don't expect us te merge 691253 (async reporting) -- sos should solve the same problem without adding the complexity of async reporting | 14:33 |
| corvus | i mention that just so we're all proceeding from the same assumptions :) | 14:33 |
| corvus | fungi, tobiash: the intermediate reporting shouldn't have a significant impact on performance (a db write like that should be trivial) -- and if it isn't, then we should make optimizations so that it is. i understand "should" and "does" can be different, so i certainly understand wanting to check before committing to it. | 14:37 |
| corvus | tobiash: it's not urgent -- what kind of time frame are you thinking about? | 14:38 |
| tobiash[m] | corvus: I'm mainly concerned about the extra network roundtrips in the main loop (which don't matter if you are not at the limit but they do matter when you are). So I'd just want to make sure that we can add those reports to the async reporting as well. I think a few days would be sufficient to check that. | 15:37 |
| corvus | tobiash: sounds great, i'll wip it for now and we can check back in a week or so | 15:38 |
| tobiash[m] | from first sight I think it should work since those are like the start reporting mainly fire and forget | 15:39 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!