| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 829310: Release tenant read lock early on pending reconfig https://review.opendev.org/c/zuul/zuul/+/829310 | 06:26 | |
| @westphahl:matrix.org | Clark: thanks, I've change the log message ^ | 06:27 |
|---|---|---|
| -@gerrit:opendev.org- Albin Vass proposed: [zuul/zuul] 830333: Add default-parent pragma https://review.opendev.org/c/zuul/zuul/+/830333 | 09:24 | |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/nodepool] 830345: Update to DIB 3.19.0 https://review.opendev.org/c/zuul/nodepool/+/830345 | 10:21 | |
| @iwienand:matrix.org | Clark: the arm64, rocky and gentoo fixes all made it in so i quickly cut 3.19.0. if you'd like to shepherd ^ through should help all those builds | 10:21 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 827935: Populate missing change cache entries https://review.opendev.org/c/zuul/zuul/+/827935 | 12:28 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 830409: Add max retries to refresh of pipeline change list https://review.opendev.org/c/zuul/zuul/+/830409 | 12:50 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Felix Edel: | 14:08 | |
| - [zuul/zuul] 827299: Fix hanging MySQL database tests https://review.opendev.org/c/zuul/zuul/+/827299 | ||
| - [zuul/zuul] 827215: Report NODE_FAILURES caused by node request failures to SQL database https://review.opendev.org/c/zuul/zuul/+/827215 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 829491: Don't submit empty node requests to Zookeeper https://review.opendev.org/c/zuul/zuul/+/829491 | 14:08 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Benjamin Schanzel: [zuul/zuul] 829339: Unpin github3.py<3.0.0 requirement https://review.opendev.org/c/zuul/zuul/+/829339 | 14:08 | |
| @clarkb:matrix.org | corvus: Is https://review.opendev.org/c/zuul/zuul/+/829310 a change you'd like to review before I approve it? | 15:44 |
| @jim:acmegating.com | Clark: yes, was awaiting the reply | 15:45 |
| @jim:acmegating.com | and i think it's just one zk request to get the child list for the lock contenders | 15:47 |
| @clarkb:matrix.org | ya I suspect the zk cost isn't great there. I mentioend it beacuse we've tried to be careful about overdoing the zk requests | 15:50 |
| @jim:acmegating.com | i think the main concern would be starving the tenant due to multiple reconfigurations | 15:54 |
| @jim:acmegating.com | imagine a situation like that caused by https://review.opendev.org/829829 | 15:55 |
| @jim:acmegating.com | * imagine a situation like that fixed by https://review.opendev.org/829829 | 15:55 |
| @jim:acmegating.com | but this seems worth trying and observing | 15:57 |
| -@gerrit:opendev.org- Zuul merged on behalf of Ian Wienand: [zuul/nodepool] 830345: Update to DIB 3.19.0 https://review.opendev.org/c/zuul/nodepool/+/830345 | 16:46 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Simon Westphahl: [zuul/zuul] 829310: Release tenant read lock early on pending reconfig https://review.opendev.org/c/zuul/zuul/+/829310 | 17:20 | |
| @jpew:matrix.org | I'm getting ssh timeouts on OpenStack nodes when using Zuul, and I'm wondering what the best way to debug this is? | 18:33 |
| @jim:acmegating.com | jpew: i'd check the executor logs, and then use the ssh key on the executor to try connecting manually | 18:34 |
| @jpew:matrix.org | Zuul seems to be automatically restarting the builds.... can I autohold in this case or will it still retry and discard the node? | 18:35 |
| @jim:acmegating.com | it should hold them at retry_limit | 18:48 |
| @jim:acmegating.com | (it won't hold the initial retries, just the last one) | 18:48 |
| @jim:acmegating.com | set the retry limit lower to make it happen faster | 18:49 |
| @jpew:matrix.org | Is the autohold job a regex by chance? I have about 1 of 50 jobs that might fail | 18:50 |
| @clarkb:matrix.org | One thing that can cause this and be confusing to debug is if you lose zk conenctivity which causes zuul to unlock the nodes then nodepool deletes them and the process starts over. I've learned to check that quickly when there are test node connection issues just to rule it in or out quickly | 19:02 |
| @jpew:matrix.org | How do you check for that? | 21:31 |
| @clarkb:matrix.org | jpew: look for zookeeper disconnection messages in your zuul scheduler logs | 21:33 |
| @clarkb:matrix.org | I think zookeeper will also log them in its log | 21:33 |
| @clarkb:matrix.org | but it is a little less explicit since it reports only the IP addresses iirc | 21:33 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 22:26 | |
| - [zuul/nodepool] 821481: Allow disabling host-key-checking in statemachine https://review.opendev.org/c/zuul/nodepool/+/821481 | ||
| - [zuul/nodepool] 830524: Add provider image config to statemachine image upload https://review.opendev.org/c/zuul/nodepool/+/830524 | ||
| - [zuul/nodepool] 830525: Update AWS driver to use statemachine framework https://review.opendev.org/c/zuul/nodepool/+/830525 | ||
| - [zuul/nodepool] 830526: Refactor AWS driver tests https://review.opendev.org/c/zuul/nodepool/+/830526 | ||
| - [zuul/nodepool] 830527: Add additional tests to the aws driver https://review.opendev.org/c/zuul/nodepool/+/830527 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 821711: Add IBM Cloud VPC driver https://review.opendev.org/c/zuul/nodepool/+/821711 | 22:27 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!