| gmann | elodilles: +1 on your patch. I have proposed the generic template change which can be merged once we have stable/2023.1 https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/873051 | 01:53 |
|---|---|---|
| *** yadnesh|away is now known as yadnesh | 04:02 | |
| opendevreview | Ghanshyam proposed openstack/tempest master: Separate the extra tests to run in a separate job https://review.opendev.org/c/openstack/tempest/+/873055 | 05:29 |
| opendevreview | Ghanshyam proposed openstack/tempest master: Separate the extra tests to run in a separate job https://review.opendev.org/c/openstack/tempest/+/873055 | 05:41 |
| *** jpena|off is now known as jpena | 08:29 | |
| elodilles | gmann: thanks \o/ looks good! Note that we need to wait until *every* project has branched with stable/2023.1 | 08:50 |
| opendevreview | yatin proposed openstack/devstack stable/train: [Train Only] Run ensure-rust role https://review.opendev.org/c/openstack/devstack/+/872902 | 10:28 |
| opendevreview | yatin proposed openstack/grenade stable/train: [train-only] Add ensure-rust to grenade jobs https://review.opendev.org/c/openstack/grenade/+/872999 | 10:28 |
| *** yadnesh is now known as yadnesh|away | 13:24 | |
| *** Roamer`_ is now known as Roamer` | 14:00 | |
| *** tosky_ is now known as tosky | 14:19 | |
| gmann | elodilles: yeah | 15:55 |
| gmann | kopecmartin: can you check this too (ensure-rust in grenade jobs) https://review.opendev.org/c/openstack/grenade/+/872969 | 15:55 |
| opendevreview | Ghanshyam proposed openstack/tempest master: Separate the extra tests to run in a separate job https://review.opendev.org/c/openstack/tempest/+/873055 | 16:53 |
| gmann | dansmith: kopecmartin: ^^ on splitting the tests in job. let me know what you think. and in tempest-extra-tests-list.txt we can add other tests also in future if we think is kind of redundant and not needed to run as part of common full job | 16:55 |
| dansmith | gmann: yeah, it is unfortunate that we have to make things more parallel to avoid the timeout, just because of how much resource is consumed by a single tempest patch (which expands to the other projects, but less so) although it seems like that's probably a good initial step | 17:17 |
| dansmith | doesn't tempest have its own timeout timer? | 17:17 |
| dansmith | if so, maybe we could up the regular job timeout to 3h and run multiple batches of tempest in a single job, with their own shorter timers? | 17:17 |
| gmann | dansmith: yes we can up the job timeout for regular one. I was thinking the same in our earlier discussion too. let me push that on top of it | 17:22 |
| gmann | i feel full-parallel is also passing at good rate https://zuul.openstack.org/builds?job_name=tempest-full-parallel&skip=0 | 17:22 |
| gmann | that can be another option to try - run scenario tests in parallel | 17:23 |
| dansmith | gmann: to be clear, I'm not saying we should just up the job timeout in isolation, | 17:26 |
| dansmith | I'm saying _if_ we could run two sets of tempest tests, each with their own short timer, then we could let the job timer for the whole thing be longer | 17:27 |
| dansmith | i.e. "you have 1h to complete the non-admin tempest tests".. "pass, okay now you have 1h to complete the admin tests".. total job timeout=3h | 17:27 |
| gmann | dansmith: ohk. I misunderstood then. so breaking tests timeout in single job is not possible in tempest side. its 'single test timeout' and 'complete job timeout' | 17:29 |
| dansmith | okay | 17:30 |
| dansmith | presumably we could do that by just running `timeout 1h tox -e ...` in the job or `timeout 1h stestr ...` in tox.ini | 17:32 |
| *** jpena is now known as jpena|off | 17:32 | |
| gmann | but what additional it will help if we need to up the overall job time | 17:38 |
| gmann | to know which set of tests is running long and timeout ? | 17:38 |
| dansmith | no, it means we will still fail quickly if a test gets stuck, but will allow longer runtime if we're truly making progress | 17:41 |
| dansmith | I guess if tempest already has a per-test timeout, then it's the same as just upping the job timer | 17:41 |
| dansmith | I hadn't thought of that until you mentioned it | 17:42 |
| dansmith | splitting tests into separate jobs uses a lot more resource total (setup of two devstacks and two nodes while running) | 17:42 |
| dansmith | so just thinking if it's really better to do that vs. increasing the job timeout | 17:42 |
| dansmith | what is the per-test timer set to? 10m or something? | 17:43 |
| gmann | 20 min by default | 17:44 |
| dansmith | wow, okay | 17:45 |
| opendevreview | Ghanshyam proposed openstack/tempest master: Move tempest-full-py3-ipv6 job to periodic https://review.opendev.org/c/openstack/tempest/+/873163 | 17:47 |
| dansmith | gmann: finally got a clean run of this glance retry thing: https://review.opendev.org/c/openstack/tempest/+/872982 | 19:37 |
| gmann | dansmith: ack | 19:42 |
| dansmith | trying to make an opensearch query to quantify how often we hit that | 19:43 |
| dansmith | at least 5 yesterday, but it manifests a few different ways | 19:43 |
| opendevreview | Cyril Roelandt proposed openstack/devstack master: Glance: Properly copy rootwrap.d/* when USE_VENV is True https://review.opendev.org/c/openstack/devstack/+/866951 | 20:34 |
| opendevreview | Ghanshyam proposed openstack/tempest master: Move a few jobs to periodic https://review.opendev.org/c/openstack/tempest/+/873163 | 20:59 |
| gmann | dansmith: kopecmartin ^^ this is moving some jobs to periodic pipeline, i think they do not need to run on every change. this can help to save some resource | 21:01 |
| dansmith | gmann: ++ | 21:06 |
| opendevreview | Ghanshyam proposed openstack/tempest master: Fix creation of requested creds within the same project https://review.opendev.org/c/openstack/tempest/+/871018 | 23:02 |
| opendevreview | Ghanshyam proposed openstack/tempest master: DNM: for test https://review.opendev.org/c/openstack/tempest/+/871019 | 23:03 |
| opendevreview | Ghanshyam proposed openstack/tempest master: Use project_reader in server test https://review.opendev.org/c/openstack/tempest/+/871210 | 23:04 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!