| *** eharney has quit IRC | 00:06 | |
| *** eharney has joined #openstack-infra | 00:18 | |
| *** ijw_ has joined #openstack-infra | 00:20 | |
| *** gyee has quit IRC | 00:22 | |
| *** ijw has quit IRC | 00:23 | |
| *** ijw_ has quit IRC | 00:24 | |
| *** markvoelker has joined #openstack-infra | 00:26 | |
| *** markvoelker has quit IRC | 00:30 | |
| *** larainema has joined #openstack-infra | 01:01 | |
| *** slaweq has joined #openstack-infra | 01:11 | |
| *** slaweq has quit IRC | 01:15 | |
| *** dychen has joined #openstack-infra | 01:17 | |
| *** jamesmcarthur has joined #openstack-infra | 01:17 | |
| *** michael-beaver has quit IRC | 01:19 | |
| *** dchen has quit IRC | 01:20 | |
| *** markvoelker has joined #openstack-infra | 01:25 | |
| *** jcoufal has joined #openstack-infra | 01:28 | |
| *** markvoelker has quit IRC | 01:30 | |
| *** jcoufal has quit IRC | 01:31 | |
| *** dpawlik has joined #openstack-infra | 01:39 | |
| *** dpawlik has quit IRC | 01:43 | |
| *** ricolin has joined #openstack-infra | 01:48 | |
| *** jamesmcarthur has quit IRC | 01:48 | |
| *** jamesmcarthur has joined #openstack-infra | 01:49 | |
| *** jamesmcarthur has quit IRC | 01:49 | |
| *** jamesmcarthur has joined #openstack-infra | 01:49 | |
| openstackgerrit | Merged openstack/project-config master: Add uwsgi role to gerritbot https://review.opendev.org/677738 | 02:00 |
|---|---|---|
| openstackgerrit | Merged openstack/project-config master: Stop gating for networking-generic-switch-tempest-plugin https://review.opendev.org/678868 | 02:00 |
| openstackgerrit | Merged openstack/project-config master: Remove Ranger testing control from project-config https://review.opendev.org/677027 | 02:01 |
| *** ricolin has quit IRC | 02:04 | |
| *** ricolin has joined #openstack-infra | 02:05 | |
| *** jamesmcarthur has quit IRC | 02:06 | |
| *** apetrich has quit IRC | 02:10 | |
| *** jamesmcarthur has joined #openstack-infra | 02:18 | |
| *** ricolin has quit IRC | 02:32 | |
| *** ricolin has joined #openstack-infra | 02:32 | |
| *** openstackgerrit has quit IRC | 02:37 | |
| *** jamesmcarthur has quit IRC | 02:37 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:43 | |
| *** jamesmcarthur has joined #openstack-infra | 02:49 | |
| *** jamesmcarthur has quit IRC | 02:59 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 03:02 | |
| *** bhavikdbavishi has quit IRC | 03:03 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:03 | |
| *** rosmaita has quit IRC | 03:06 | |
| *** psachin has joined #openstack-infra | 03:10 | |
| *** rcernin has quit IRC | 03:13 | |
| *** rosmaita has joined #openstack-infra | 03:19 | |
| *** ricolin has quit IRC | 03:21 | |
| *** nicolasbock has quit IRC | 03:31 | |
| *** xinranwang has joined #openstack-infra | 03:38 | |
| *** dpawlik has joined #openstack-infra | 03:40 | |
| *** dpawlik has quit IRC | 03:44 | |
| *** udesale has joined #openstack-infra | 03:59 | |
| *** rcernin has joined #openstack-infra | 04:05 | |
| *** adriant has quit IRC | 04:05 | |
| *** adriant has joined #openstack-infra | 04:05 | |
| *** ramishra has joined #openstack-infra | 04:07 | |
| *** slaweq has joined #openstack-infra | 04:11 | |
| *** slaweq has quit IRC | 04:16 | |
| *** dave-mccowan has quit IRC | 04:41 | |
| AJaeger_ | thanks, clarkb | 05:00 |
| *** jamesmcarthur has joined #openstack-infra | 05:00 | |
| AJaeger_ | ianw: seeing clarkb's changes for fedora, did you want to update https://review.opendev.org/634626 and https://review.opendev.org/636082 ? | 05:00 |
| *** soniya29 has joined #openstack-infra | 05:02 | |
| *** jamesmcarthur has quit IRC | 05:04 | |
| *** raukadah is now known as chandankumar | 05:09 | |
| *** adriant has quit IRC | 05:10 | |
| *** adriant has joined #openstack-infra | 05:11 | |
| *** openstackgerrit has joined #openstack-infra | 05:15 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: executor: always setup an inventory, even for blank node list https://review.opendev.org/679184 | 05:15 |
| AJaeger_ | ianw: wow, that sounds like an interesting debug session. Thanks ^ | 05:24 |
| ianw | AJaeger_: all because i was watching the logs too closely :) | 05:30 |
| *** markvoelker has joined #openstack-infra | 05:31 | |
| *** markvoelker has quit IRC | 05:36 | |
| *** dpawlik has joined #openstack-infra | 05:41 | |
| *** dpawlik has quit IRC | 05:45 | |
| *** factor has joined #openstack-infra | 05:51 | |
| *** rcernin_ has joined #openstack-infra | 05:52 | |
| *** rcernin has quit IRC | 05:55 | |
| *** jaosorior has quit IRC | 05:56 | |
| AJaeger_ | ;) | 06:00 |
| *** dpawlik has joined #openstack-infra | 06:00 | |
| *** udesale has quit IRC | 06:04 | |
| *** udesale has joined #openstack-infra | 06:05 | |
| *** xinranwang has quit IRC | 06:08 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: executor: always setup an inventory, even for blank node list https://review.opendev.org/679184 | 06:10 |
| *** slaweq has joined #openstack-infra | 06:11 | |
| *** slaweq has quit IRC | 06:16 | |
| *** lpetrut has joined #openstack-infra | 06:20 | |
| *** markvoelker has joined #openstack-infra | 06:31 | |
| *** udesale has quit IRC | 06:31 | |
| *** udesale has joined #openstack-infra | 06:32 | |
| *** odicha has joined #openstack-infra | 06:34 | |
| *** udesale has quit IRC | 06:36 | |
| *** markvoelker has quit IRC | 06:36 | |
| *** udesale has joined #openstack-infra | 06:36 | |
| *** ricolin has joined #openstack-infra | 06:38 | |
| *** xenos76 has joined #openstack-infra | 06:41 | |
| *** gfidente has joined #openstack-infra | 06:45 | |
| *** pkopec has joined #openstack-infra | 06:47 | |
| *** xek has joined #openstack-infra | 06:55 | |
| *** slaweq has joined #openstack-infra | 06:58 | |
| *** stakeda has joined #openstack-infra | 07:05 | |
| *** tosky has joined #openstack-infra | 07:23 | |
| *** slaweq has quit IRC | 07:27 | |
| *** rcosnita has joined #openstack-infra | 07:28 | |
| *** jbadiapa has joined #openstack-infra | 07:28 | |
| *** slaweq has joined #openstack-infra | 07:31 | |
| *** lucasagomes has joined #openstack-infra | 07:37 | |
| *** apetrich has joined #openstack-infra | 07:37 | |
| *** rcernin_ has quit IRC | 07:40 | |
| *** jpena|off is now known as jpena | 07:40 | |
| *** trident has quit IRC | 07:40 | |
| *** jaosorior has joined #openstack-infra | 07:42 | |
| *** ginopc has joined #openstack-infra | 07:47 | |
| *** trident has joined #openstack-infra | 07:49 | |
| *** priteau has joined #openstack-infra | 07:54 | |
| priteau | Issues with Limestone yum mirrors? Saw this earlier today: http://mirror.regionone.limestone.openstack.org/centos/7/os/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 403 - Forbidden | 07:55 |
| priteau | https://79636ab1f0eaf6899dc0-686811860e75a35eabcecb5697905253.ssl.cf2.rackcdn.com/679129/1/check/kayobe-overcloud-upgrade-centos/b1fde9b/job-output.txt | 07:55 |
| *** dychen has quit IRC | 07:56 | |
| *** dychen has joined #openstack-infra | 07:57 | |
| *** rcernin_ has joined #openstack-infra | 07:57 | |
| *** jaosorior has quit IRC | 08:01 | |
| *** ralonsoh has joined #openstack-infra | 08:04 | |
| *** dychen has quit IRC | 08:05 | |
| *** lpetrut has quit IRC | 08:08 | |
| *** markvoelker has joined #openstack-infra | 08:10 | |
| *** e0ne has joined #openstack-infra | 08:11 | |
| *** markvoelker has quit IRC | 08:15 | |
| *** tkajinam has quit IRC | 08:17 | |
| *** e0ne has quit IRC | 08:20 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 08:20 | |
| *** e0ne has joined #openstack-infra | 08:34 | |
| *** jaosorior has joined #openstack-infra | 08:37 | |
| ianw | 'ReadTimeoutError("HTTPConnectionPool(host='mirror.regionone.limestone.openstack.org', port=80): | 08:38 |
| ianw | yeah just saw that | 08:38 |
| ianw | there's been a bunch of up/down messages for afs in the logs since [Thu Aug 29 06:19:07 2019] | 08:40 |
| ianw | it's now 08:39:36 | 08:40 |
| ianw | so possibly limestone external networking, causing us some afs issues. i don't think it's the afs servers as other mirrors don't seem to be showing similar | 08:40 |
| ianw | i think monitor for now | 08:41 |
| *** jaosorior has quit IRC | 08:49 | |
| *** rcosnita has quit IRC | 08:53 | |
| *** rcosnita has joined #openstack-infra | 08:54 | |
| *** gfidente has quit IRC | 08:57 | |
| *** rcosnita has quit IRC | 08:58 | |
| *** rcosnita_ has joined #openstack-infra | 08:58 | |
| *** xeno_os76_xyz has joined #openstack-infra | 08:58 | |
| *** e0ne has quit IRC | 08:59 | |
| *** xenos76 has quit IRC | 09:00 | |
| corvus | fungi: yeah, that threading behavior is not new; slightly annoying but hard to fix. thanks for testing | 09:15 |
| openstackgerrit | Merged ttygroup/gertty master: Fix a py3 error with exception formatting https://review.opendev.org/679050 | 09:16 |
| openstackgerrit | Merged ttygroup/gertty master: Add support for hashtags https://review.opendev.org/679051 | 09:16 |
| openstackgerrit | Merged ttygroup/gertty master: Use account_id to identify the user's own account https://review.opendev.org/679052 | 09:16 |
| openstackgerrit | Merged ttygroup/gertty master: Update googlesource example https://review.opendev.org/679055 | 09:16 |
| *** soniya29 has quit IRC | 09:17 | |
| *** ccamacho has joined #openstack-infra | 09:21 | |
| *** e0ne has joined #openstack-infra | 09:27 | |
| *** e0ne has quit IRC | 09:27 | |
| *** gfidente has joined #openstack-infra | 09:39 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 09:44 | |
| *** apetrich has quit IRC | 09:47 | |
| *** bhavikdbavishi has quit IRC | 10:02 | |
| *** markvoelker has joined #openstack-infra | 10:11 | |
| zbr | search code on gitea seems broken even for simple searches: https://opendev.org/openstack/tripleo-heat-templates/search?q=ContainerFluentdImage | 10:16 |
| zbr | github works fine https://github.com/openstack/tripleo-heat-templates/search?q=ContainerFluentdImage&unscoped_q=ContainerFluentdImage | 10:16 |
| *** dpawlik has quit IRC | 10:19 | |
| *** markvoelker has quit IRC | 10:20 | |
| *** derekh has joined #openstack-infra | 10:20 | |
| *** e0ne has joined #openstack-infra | 10:21 | |
| *** dtantsur|afk is now known as dtantsur | 10:24 | |
| *** pkopec has quit IRC | 10:26 | |
| *** pkopec has joined #openstack-infra | 10:26 | |
| *** e0ne has quit IRC | 10:27 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add reference pipelines file for Github driver https://review.opendev.org/672712 | 10:28 |
| *** dpawlik has joined #openstack-infra | 10:29 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 10:33 | |
| *** apetrich has joined #openstack-infra | 10:44 | |
| slaweq | frickler: hi | 10:51 |
| slaweq | frickler: can You help me with failing jobs in patch https://review.opendev.org/#/c/678024/ | 10:52 |
| slaweq | TBH I don't exactly understand where I should add this performa/os-faults project | 10:52 |
| slaweq | error is like https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_24/678024/10/check/tobiko-devstack-scenario-ubuntu-bionic/c6cb910/controller/logs/devstacklog.txt.gz | 10:53 |
| *** e0ne has joined #openstack-infra | 11:00 | |
| *** jaosorior has joined #openstack-infra | 11:02 | |
| *** tesseract has joined #openstack-infra | 11:15 | |
| *** dave-mccowan has joined #openstack-infra | 11:17 | |
| *** udesale has quit IRC | 11:18 | |
| *** Lucas_Gray has joined #openstack-infra | 11:19 | |
| *** e0ne has quit IRC | 11:20 | |
| *** e0ne has joined #openstack-infra | 11:23 | |
| *** lpetrut has joined #openstack-infra | 11:24 | |
| frickler | slaweq: iiuc you need to copy non-openstack projects manually into the /opt/stack workspace for devstack, see https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_24/678024/10/check/tobiko-devstack-scenario-ubuntu-bionic/c6cb910/ara-report/file/d31fd74a-c061-469c-ad4d-cb0cac3746e5/ | 11:26 |
| frickler | slaweq: it might be possible to change that in the devstack role, I'm not sure about the rules there, but maybe as a quick workaround you can do this "by hand" in your tobiko role | 11:27 |
| *** lpetrut has quit IRC | 11:30 | |
| *** e0ne has quit IRC | 11:32 | |
| *** rh-jelabarre has joined #openstack-infra | 11:34 | |
| *** jpena is now known as jpena|lunch | 11:35 | |
| dtantsur | hi folks! is it possible to change environment variables for a job without changing the job itself (e.g. devstack-tempest)? | 11:38 |
| dtantsur | I'm trying to find a way to override OS_TEST_TIMEOUT | 11:39 |
| *** e0ne has joined #openstack-infra | 11:40 | |
| *** Lucas_Gray has quit IRC | 11:45 | |
| *** jcoufal has joined #openstack-infra | 11:45 | |
| *** nicolasbock has joined #openstack-infra | 11:49 | |
| slaweq | frickler: thx a lot for help, I will try that | 11:56 |
| *** markvoelker has joined #openstack-infra | 12:10 | |
| *** e0ne has quit IRC | 12:11 | |
| AJaeger_ | dtantsur: in general, you can add variables to jobs in your project stanza. | 12:14 |
| dtantsur | AJaeger_: even environment variables? | 12:14 |
| AJaeger_ | so, you can add vars: tox_environment: OS_TEST_TIMEOUT=... | 12:14 |
| AJaeger_ | but that really depends on the job - so, check what devstack allows as passing in... | 12:15 |
| dtantsur | AJaeger_: this works for regular tox jobs, I guess, but I don't see tox_environment in run-tempest | 12:15 |
| AJaeger_ | and then figure out how to pass it | 12:15 |
| dtantsur | (also talking to frickler on #openstack-qa) | 12:15 |
| AJaeger_ | dtantsur: better discuss further with frickler, he konws more about devstack than I do... | 12:15 |
| *** markvoelker has quit IRC | 12:16 | |
| *** apetrich has quit IRC | 12:16 | |
| *** derekh has quit IRC | 12:23 | |
| *** jcoufal has quit IRC | 12:28 | |
| *** rfolco has joined #openstack-infra | 12:28 | |
| *** jcoufal has joined #openstack-infra | 12:30 | |
| *** larainema has quit IRC | 12:32 | |
| *** jcoufal has quit IRC | 12:33 | |
| *** rlandy has joined #openstack-infra | 12:34 | |
| *** markvoelker has joined #openstack-infra | 12:38 | |
| *** Lucas_Gray has joined #openstack-infra | 12:40 | |
| *** jpena|lunch is now known as jpena | 12:41 | |
| *** jaosorior has quit IRC | 12:41 | |
| *** jeliu_ has joined #openstack-infra | 12:42 | |
| *** markvoelker has quit IRC | 12:42 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: A Zuul reporter for Elasticsearch https://review.opendev.org/644927 | 12:42 |
| *** markvoelker has joined #openstack-infra | 12:44 | |
| *** markvoelker has quit IRC | 12:44 | |
| *** markvoelker has joined #openstack-infra | 12:45 | |
| *** lpetrut has joined #openstack-infra | 12:52 | |
| *** panda|rover|off is now known as panda|rover | 12:53 | |
| *** derekh has joined #openstack-infra | 12:58 | |
| *** udesale has joined #openstack-infra | 13:03 | |
| *** aaronsheffield has joined #openstack-infra | 13:04 | |
| *** jaosorior has joined #openstack-infra | 13:04 | |
| *** mriedem has joined #openstack-infra | 13:07 | |
| *** zbr has quit IRC | 13:08 | |
| *** rh-jelabarre has quit IRC | 13:10 | |
| *** zbr has joined #openstack-infra | 13:16 | |
| *** Goneri has joined #openstack-infra | 13:20 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add support for smart reconfigurations https://review.opendev.org/652114 | 13:23 |
| *** apetrich has joined #openstack-infra | 13:23 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add --check-config option to zuul scheduler https://review.opendev.org/542160 | 13:23 |
| *** gfidente has quit IRC | 13:27 | |
| *** gfidente has joined #openstack-infra | 13:27 | |
| *** kaisers has quit IRC | 13:29 | |
| *** odicha has quit IRC | 13:31 | |
| *** dklyle has quit IRC | 13:35 | |
| *** dklyle has joined #openstack-infra | 13:35 | |
| *** priteau has quit IRC | 13:35 | |
| *** e0ne has joined #openstack-infra | 13:40 | |
| *** rcernin_ has quit IRC | 13:41 | |
| *** e0ne has quit IRC | 13:43 | |
| *** efried is now known as efried_afk | 13:47 | |
| *** hongbin has joined #openstack-infra | 13:48 | |
| *** jaosorior has quit IRC | 13:51 | |
| *** lpetrut has quit IRC | 13:51 | |
| *** udesale has quit IRC | 13:51 | |
| *** gfidente has quit IRC | 13:52 | |
| *** jcoufal has joined #openstack-infra | 13:53 | |
| *** dave-mccowan has quit IRC | 13:54 | |
| *** signed8bit has quit IRC | 13:54 | |
| *** gfidente has joined #openstack-infra | 13:54 | |
| *** dave-mccowan has joined #openstack-infra | 13:55 | |
| *** rcernin_ has joined #openstack-infra | 13:55 | |
| *** jovial[m] has quit IRC | 13:55 | |
| *** noonedeadpunk has quit IRC | 13:56 | |
| *** fnordahl has quit IRC | 13:56 | |
| *** mordred has quit IRC | 13:57 | |
| *** d34dh0r53 has quit IRC | 13:57 | |
| *** fnordahl has joined #openstack-infra | 13:57 | |
| *** mordred has joined #openstack-infra | 13:59 | |
| *** d34dh0r53 has joined #openstack-infra | 14:01 | |
| *** noonedeadpunk has joined #openstack-infra | 14:03 | |
| *** ccamacho has quit IRC | 14:05 | |
| *** sgw has quit IRC | 14:12 | |
| *** rh-jelabarre has joined #openstack-infra | 14:17 | |
| *** e0ne has joined #openstack-infra | 14:22 | |
| *** dpawlik has quit IRC | 14:23 | |
| clarkb | corvus: https://review.opendev.org/#/c/679145/ was my spin on that and ya used the first 3 bytes for slight more containers (since it seemed part of mnasers problem was container size) | 14:31 |
| *** sgw has joined #openstack-infra | 14:31 | |
| clarkb | corvus: and kept the rest of the url contents as sometimes the project branch change info is useful when sharing links | 14:31 |
| *** Lucas_Gray has quit IRC | 14:32 | |
| *** Wryhder has joined #openstack-infra | 14:32 | |
| *** Wryhder is now known as Lucas_Gray | 14:33 | |
| *** Lucas_Gray has quit IRC | 14:35 | |
| *** jamesmcarthur has joined #openstack-infra | 14:42 | |
| *** rcosnita_ has quit IRC | 14:44 | |
| *** jbadiapa has quit IRC | 14:47 | |
| *** dpawlik has joined #openstack-infra | 14:52 | |
| *** dpawlik has quit IRC | 14:57 | |
| openstackgerrit | David Shrewsbury proposed zuul/zuul master: Add autohold delete/info commands to web API https://review.opendev.org/679057 | 14:59 |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Add release note for bug fix to test correct commit https://review.opendev.org/679285 | 15:00 |
| *** bhavikdbavishi has joined #openstack-infra | 15:00 | |
| *** jbadiapa has joined #openstack-infra | 15:00 | |
| *** diablo_rojo has joined #openstack-infra | 15:01 | |
| *** bhavikdbavishi has quit IRC | 15:04 | |
| *** jamesmcarthur has quit IRC | 15:07 | |
| *** jamesmcarthur has joined #openstack-infra | 15:07 | |
| *** bhavikdbavishi has joined #openstack-infra | 15:11 | |
| *** efried_afk is now known as efried | 15:13 | |
| *** e0ne has quit IRC | 15:14 | |
| *** udesale has joined #openstack-infra | 15:17 | |
| *** sthussey has joined #openstack-infra | 15:22 | |
| *** dpawlik has joined #openstack-infra | 15:23 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 15:26 | |
| *** dpawlik has quit IRC | 15:28 | |
| *** jbadiapa has quit IRC | 15:30 | |
| *** sgw has left #openstack-infra | 15:31 | |
| clarkb | zbr: yes this is why we haven't turned off codesearch.openstack.org | 15:31 |
| clarkb | zbr: we are hoping that the planned improvements to support elasticsearch for indexing rather than a local (non distributed) index will improve things in gitea, but I don't know that there is a planned date for that to happen | 15:32 |
| zbr | clarkb: if we can't fix it, maybe we can disable it? i fooled me few times. | 15:32 |
| clarkb | that may be possible by editing the html templates. We done that for other stuff already | 15:32 |
| clarkb | zbr: https://opendev.org/opendev/system-config/src/branch/master/docker/gitea/custom/templates | 15:33 |
| zbr | irony,... i cannot search for the search template :D | 15:34 |
| clarkb | slaweq: can you review https://review.opendev.org/#/c/679118/ to fix a different tobiko testing thing? | 15:35 |
| slaweq | clarkb: looking | 15:35 |
| slaweq | clarkb: approved, thx for this patch | 15:36 |
| clarkb | ty | 15:37 |
| *** chandankumar is now known as raukadah | 15:40 | |
| clarkb | mriedem: as a sanity check I think e-r, logstash, elasticsearch are all working again after the swift switch. Have you seen any other concerns with the behavior there that we might need to address? | 15:45 |
| clarkb | I'm out tomorrow and trying to ensure things I think I've fixed in the last week are indeed fixed | 15:45 |
| *** xeno_os76_xyz has quit IRC | 15:45 | |
| mriedem | clarkb: i haven't seen any issues either | 15:47 |
| mriedem | i've got https://review.opendev.org/#/q/status:open+project:opendev/elastic-recheck+branch:master+topic:cleanup-2019-08-26 to clean up some more old / invalid queries if yo'ure game | 15:47 |
| clarkb | done | 15:51 |
| clarkb | mriedem: brainstorm idea: I wonder if we could make a push post feature freeze for people to volunteer to identify bugs and help fix them | 15:52 |
| clarkb | rather than just say we are in a stabilization period do specific stabilization tasks | 15:52 |
| clarkb | eg get classification rate up to 80% or whatever | 15:53 |
| clarkb | and then start fixing identified bugs | 15:53 |
| clarkb | the biggest issue is the job timeouts which I think are related to swapping which is related to various inefficiency of memroy use | 15:53 |
| clarkb | well that and the osc slowness but bnemec is pushing the osc as a service change now which should help there | 15:53 |
| mriedem | yeah clearly this is far and away our biggest problem http://status.openstack.org/elastic-recheck/#1686542 | 15:54 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add reference pipelines file for Github driver https://review.opendev.org/672712 | 15:57 |
| *** hongbin has quit IRC | 15:58 | |
| *** lucasagomes has quit IRC | 15:59 | |
| *** gyee has joined #openstack-infra | 16:00 | |
| *** ykarel has joined #openstack-infra | 16:01 | |
| *** dpawlik has joined #openstack-infra | 16:01 | |
| *** jcoufal has quit IRC | 16:03 | |
| *** gfidente has quit IRC | 16:04 | |
| *** dpawlik has quit IRC | 16:06 | |
| *** jpena is now known as jpena|off | 16:11 | |
| *** markvoelker has quit IRC | 16:15 | |
| *** markvoelker has joined #openstack-infra | 16:15 | |
| *** dpawlik has joined #openstack-infra | 16:17 | |
| clarkb | fungi: https://review.opendev.org/#/c/644432/ | 16:17 |
| clarkb | that should fix the jeepyb gerrit acl problem | 16:17 |
| clarkb | However we probably want to manually run that without the puppet timeout the first time through | 16:17 |
| clarkb | and maybe test it on some repos too since we've not managed the retired acls in a while | 16:17 |
| fungi | yeah, we can run manage-projects for a single project | 16:19 |
| clarkb | we probably want to put gerrit in the emergency file, merge that, update jeepyb manually then run it against a single project, then against all projects with no timeout then remove gerrit from emergency file | 16:19 |
| clarkb | the puppet exec is likely to timeout otherwise | 16:19 |
| fungi | yup | 16:19 |
| *** dpawlik has quit IRC | 16:21 | |
| *** markvoelker has quit IRC | 16:27 | |
| AJaeger_ | dmsimard: could you review https://review.opendev.org/#/c/679115 , please? | 16:29 |
| openstackgerrit | Matt Riedemann proposed opendev/elastic-recheck master: Uncap SQLAlchemy in requirements.txt https://review.opendev.org/679302 | 16:32 |
| *** ykarel has quit IRC | 16:33 | |
| *** ykarel_ has joined #openstack-infra | 16:33 | |
| *** stewie925 has joined #openstack-infra | 16:36 | |
| *** sshnaidm|afk is now known as sshnaidm|off | 16:39 | |
| *** markvoelker has joined #openstack-infra | 16:40 | |
| *** nicolasbock has quit IRC | 16:41 | |
| *** nicolasbock has joined #openstack-infra | 16:41 | |
| *** igordc has joined #openstack-infra | 16:44 | |
| *** jamesmcarthur has quit IRC | 16:50 | |
| *** tesseract has quit IRC | 16:50 | |
| *** ykarel_ is now known as ykarel|away | 16:54 | |
| *** hosingh000 has joined #openstack-infra | 16:54 | |
| *** hosingh000 has quit IRC | 16:55 | |
| *** hosingh000 has joined #openstack-infra | 16:57 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed zuul/zuul-jobs master: WIP: intercept-job -- self-service SSH access https://review.opendev.org/679306 | 16:58 |
| *** derekh has quit IRC | 16:59 | |
| *** derekh has joined #openstack-infra | 16:59 | |
| *** e0ne has joined #openstack-infra | 17:02 | |
| *** ijw has joined #openstack-infra | 17:02 | |
| *** dtantsur is now known as dtantsur|afk | 17:03 | |
| *** e0ne has quit IRC | 17:04 | |
| *** e0ne has joined #openstack-infra | 17:04 | |
| *** e0ne has quit IRC | 17:04 | |
| *** xek has quit IRC | 17:05 | |
| *** jbadiapa has joined #openstack-infra | 17:07 | |
| *** derekh has quit IRC | 17:09 | |
| *** xenos76 has joined #openstack-infra | 17:10 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: web: add config page https://review.opendev.org/633667 | 17:11 |
| *** panda|rover is now known as panda|rover|off | 17:18 | |
| *** hosingh000 has quit IRC | 17:24 | |
| *** udesale has quit IRC | 17:34 | |
| *** e0ne has joined #openstack-infra | 17:35 | |
| *** e0ne has quit IRC | 17:35 | |
| *** jtomasek has quit IRC | 17:36 | |
| stewie925 | hi guys, how does an application instruct zuul to install something like libmysqlclient packages? per my understanding libmysqlclient is not part of requirements,txt? | 17:41 |
| *** ralonsoh has quit IRC | 17:43 | |
| clarkb | stewie925: the bindep config file is typically what we use for that | 17:43 |
| clarkb | many of the existing jobs like tox jobs will automatically install anything listed in the bindep.txt in the root of your project | 17:43 |
| clarkb | additionally most openstack projects use pymysql now which is a fully native python client for mysql that doesn't need libmysqlclient | 17:44 |
| stewie925 | thank you clarkb, is there a sample bindep file that I can look at? | 17:44 |
| *** psachin has quit IRC | 17:44 | |
| stewie925 | oh | 17:44 |
| clarkb | stewie925: https://docs.openstack.org/infra/bindep/readme.html#writing-requirements-files | 17:44 |
| clarkb | pymysql is marginally slower than the C bound libs when compared single threaded but performs as well or better in the greenthread with eventlet case because eventlet can properly juggle python routines | 17:45 |
| clarkb | it has also been more reliable in the eventlet case | 17:45 |
| *** jtomasek has joined #openstack-infra | 17:45 | |
| *** ricolin has quit IRC | 17:55 | |
| *** jtomasek has quit IRC | 18:08 | |
| *** dougwig has joined #openstack-infra | 18:12 | |
| *** jtomasek has joined #openstack-infra | 18:15 | |
| *** signed8bit has joined #openstack-infra | 18:15 | |
| *** dpawlik has joined #openstack-infra | 18:17 | |
| *** ijw has quit IRC | 18:19 | |
| *** ykarel|away has quit IRC | 18:19 | |
| *** ijw has joined #openstack-infra | 18:19 | |
| *** dpawlik has quit IRC | 18:22 | |
| *** ijw has joined #openstack-infra | 18:22 | |
| *** ijw has quit IRC | 18:23 | |
| *** ijw has joined #openstack-infra | 18:24 | |
| *** ijw has quit IRC | 18:26 | |
| *** ijw has joined #openstack-infra | 18:27 | |
| *** ijw_ has joined #openstack-infra | 18:28 | |
| fungi | and also it's pure python, so no need for external c libs | 18:29 |
| *** e0ne has joined #openstack-infra | 18:29 | |
| *** ijw_ has quit IRC | 18:29 | |
| fungi | which was especially important when it was also the only mysql client module which worked under python3 | 18:29 |
| fungi | (presumably some of the others have caught up by now) | 18:30 |
| *** ijw has quit IRC | 18:32 | |
| *** ijw_ has joined #openstack-infra | 18:32 | |
| *** e0ne has quit IRC | 18:32 | |
| stewie925 | thank you fungi for that added tidbit! | 18:34 |
| *** rlandy is now known as rlandy|biab | 18:34 | |
| *** ijw has joined #openstack-infra | 18:35 | |
| openstackgerrit | Merged opendev/elastic-recheck master: Uncap SQLAlchemy in requirements.txt https://review.opendev.org/679302 | 18:35 |
| *** ijw_ has joined #openstack-infra | 18:36 | |
| *** ijw_ has quit IRC | 18:37 | |
| *** ijw has quit IRC | 18:38 | |
| *** ijw has joined #openstack-infra | 18:39 | |
| *** ijw has quit IRC | 18:43 | |
| *** jtomasek has quit IRC | 18:54 | |
| *** e0ne has joined #openstack-infra | 18:55 | |
| *** hongbin has joined #openstack-infra | 18:58 | |
| *** e0ne has quit IRC | 19:01 | |
| *** ijw has joined #openstack-infra | 19:01 | |
| *** e0ne has joined #openstack-infra | 19:03 | |
| *** hongbin has quit IRC | 19:03 | |
| *** ijw has quit IRC | 19:04 | |
| *** slaweq has quit IRC | 19:04 | |
| *** e0ne has quit IRC | 19:04 | |
| *** e0ne has joined #openstack-infra | 19:05 | |
| *** ijw has joined #openstack-infra | 19:05 | |
| *** e0ne has quit IRC | 19:05 | |
| *** spsurya has quit IRC | 19:07 | |
| *** slaweq has joined #openstack-infra | 19:11 | |
| *** jeliu_ has quit IRC | 19:12 | |
| *** armstrong_ has joined #openstack-infra | 19:13 | |
| *** armstrong_ is now known as armstrong | 19:15 | |
| *** slaweq has quit IRC | 19:15 | |
| *** bhavikdbavishi has quit IRC | 19:21 | |
| *** diablo_rojo has quit IRC | 19:27 | |
| openstackgerrit | Merged zuul/zuul master: Add reference pipelines file for Gerrit driver https://review.opendev.org/672683 | 19:34 |
| *** rlandy|biab is now known as rlandy | 19:39 | |
| *** eharney has quit IRC | 19:50 | |
| *** slaweq has joined #openstack-infra | 19:59 | |
| *** slaweq has quit IRC | 20:04 | |
| *** slaweq has joined #openstack-infra | 20:11 | |
| openstackgerrit | Matt Riedemann proposed opendev/elastic-recheck master: Include "Invalid" bugs in cleanup CLI https://review.opendev.org/678608 | 20:15 |
| openstackgerrit | Matt Riedemann proposed opendev/elastic-recheck master: Delete old queries: 2019-08-26 https://review.opendev.org/678609 | 20:15 |
| *** slaweq has quit IRC | 20:16 | |
| *** eharney has joined #openstack-infra | 20:17 | |
| *** dpawlik has joined #openstack-infra | 20:18 | |
| clarkb | fungi: have time to review https://review.opendev.org/#/c/672847/ which is the other piece of adding more debugging to afs | 20:19 |
| fungi | sure, i need a break from hacking on election tooling anyway | 20:20 |
| *** ijw_ has joined #openstack-infra | 20:20 | |
| clarkb | also on the gerrit side of things we have https://review.opendev.org/#/c/678413/1 https://review.opendev.org/#/c/678487/ and https://review.opendev.org/678486 | 20:21 |
| clarkb | improvements for small problems we've had that mordred got info on at the summit I guess | 20:21 |
| *** dpawlik has quit IRC | 20:23 | |
| *** ijw has quit IRC | 20:24 | |
| *** Goneri has quit IRC | 20:35 | |
| *** jamesmcarthur has joined #openstack-infra | 20:36 | |
| *** jeliu_ has joined #openstack-infra | 20:40 | |
| *** factor has quit IRC | 20:42 | |
| openstackgerrit | Merged opendev/elastic-recheck master: Include "Invalid" bugs in cleanup CLI https://review.opendev.org/678608 | 20:42 |
| openstackgerrit | Merged opendev/elastic-recheck master: Delete old queries: 2019-08-26 https://review.opendev.org/678609 | 20:42 |
| *** factor has joined #openstack-infra | 20:42 | |
| *** lpetrut has joined #openstack-infra | 20:45 | |
| *** lpetrut has quit IRC | 20:51 | |
| clarkb | pabelanger: for https://review.opendev.org/#/c/676697/1 are there known issues with the 2.7 -> 2.8 upgrade we should be concerned about? | 20:52 |
| *** icarusfactor has joined #openstack-infra | 20:53 | |
| *** factor has quit IRC | 20:54 | |
| *** rh-jelabarre has quit IRC | 20:57 | |
| *** ijw_ has quit IRC | 21:02 | |
| openstackgerrit | Merged opendev/system-config master: AFS server restart and audit logging : helper script https://review.opendev.org/672847 | 21:03 |
| openstackgerrit | Merged openstack/project-config master: Manages build reqs in alexandria project https://review.opendev.org/679155 | 21:03 |
| *** ijw has joined #openstack-infra | 21:07 | |
| *** rfolco has quit IRC | 21:09 | |
| *** slaweq has joined #openstack-infra | 21:11 | |
| *** slaweq has quit IRC | 21:15 | |
| *** pkopec has quit IRC | 21:18 | |
| *** rcernin_ has quit IRC | 21:25 | |
| *** jamesmcarthur has quit IRC | 21:26 | |
| openstackgerrit | Donny Davis proposed openstack/project-config master: Lowering max-servers for FN to accomdate NUMA https://review.opendev.org/679350 | 21:27 |
| donnyd | I have to lower the max-servers # for normal jobs so the NUMA jobs will actually schedule | 21:28 |
| clarkb | I guess those jobs aren't done via our nodepool? (if they were the quotas should be sorted out by nodepool) | 21:30 |
| donnyd | yes they are, but I have the limit set higher than what FN will actually do with a hypervisor out of service | 21:32 |
| clarkb | ah | 21:32 |
| *** markvoelker has quit IRC | 21:32 | |
| donnyd | oh yea, i have a hypervisor out of service | 21:33 |
| donnyd | should have it back in service sometime in sep | 21:34 |
| *** jeliu_ has quit IRC | 21:35 | |
| donnyd | i have been using qutoa to control it thus far, but figured i should probably update the project to reflect something a little closer to accurate | 21:35 |
| clarkb | ya if it is expected to be a while then probably best to do it in nodepool's config | 21:36 |
| donnyd | at least another 3 weeks | 21:36 |
| donnyd | been a little while since i have been in here - hope all is well | 21:36 |
| donnyd | looks like the mirror is doing pretty good performance wise | 21:37 |
| donnyd | getting up into the 500m/s range on the regular | 21:37 |
| donnyd | At some point in the future I also need to replace my edge FW's nic with a much faster one | 21:38 |
| donnyd | maybe we can get that # up in the 3g/s range | 21:38 |
| *** jamesmcarthur has joined #openstack-infra | 21:38 | |
| *** rfolco has joined #openstack-infra | 21:42 | |
| clarkb | ya I think things have been working well | 21:43 |
| *** jamesmcarthur has quit IRC | 21:43 | |
| openstackgerrit | Merged opendev/system-config master: Split out bazel builder and gerrit base image https://review.opendev.org/673142 | 21:53 |
| openstackgerrit | Merged openstack/project-config master: Lowering max-servers for FN to accomdate NUMA https://review.opendev.org/679350 | 21:55 |
| *** jamesmcarthur has joined #openstack-infra | 21:59 | |
| paladox | mordred we figured out why our builds at wikimedia were failing with the gerrit upgrade. It's due to https://bugs.chromium.org/p/gerrit/issues/detail?id=10885 which https://gerrit-review.googlesource.com/c/gerrit/+/231396 shows a fix | 22:00 |
| *** jamesmcarthur has quit IRC | 22:04 | |
| *** jamesmcarthur has joined #openstack-infra | 22:10 | |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** eharney has quit IRC | 22:12 | |
| *** slaweq has quit IRC | 22:15 | |
| *** jamesmcarthur has quit IRC | 22:17 | |
| *** rlandy has quit IRC | 22:18 | |
| *** ijw has quit IRC | 22:18 | |
| *** dpawlik has joined #openstack-infra | 22:19 | |
| *** armstrong has quit IRC | 22:22 | |
| *** dpawlik has quit IRC | 22:24 | |
| *** tosky has quit IRC | 22:29 | |
| *** jamesmcarthur has joined #openstack-infra | 22:31 | |
| *** diablo_rojo has joined #openstack-infra | 22:32 | |
| *** ijw has joined #openstack-infra | 22:34 | |
| *** markvoelker has joined #openstack-infra | 22:35 | |
| *** jamesmcarthur has quit IRC | 22:35 | |
| *** ijw has quit IRC | 22:37 | |
| *** ijw has joined #openstack-infra | 22:38 | |
| *** markvoelker has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-infra | 22:43 | |
| *** tkajinam has joined #openstack-infra | 22:56 | |
| *** xenos76 has quit IRC | 22:56 | |
| *** markvoelker has joined #openstack-infra | 23:01 | |
| *** markvoelker has quit IRC | 23:11 | |
| *** aaronsheffield has quit IRC | 23:22 | |
| pabelanger | clarkb: no, I haven't see anything. But figured cap on 2.7 is the safest | 23:24 |
| pabelanger | if / when we bump 2.8 in zuul | 23:24 |
| clarkb | I think we should try to track zuul defaults as closely as possible | 23:27 |
| fungi | i concur | 23:28 |
| pabelanger | sure I'm fine with that, I just didn't test all of openstack for 2.8 compat | 23:29 |
| fungi | that will happen when we switch to 2.8... "eventually tested" | 23:32 |
| fungi | s/when/at varying points and for a long while after/ | 23:33 |
| clarkb | I think the "beta" testing opendev has done for zuul has been useful. Both for zuul and opendev. If we expect there to be necessary migration requirements zuul should document them and opendev can test them | 23:34 |
| clarkb | if we don't expect there to be anything to do we can move forward and if we learn that there were things that need to be done we feed that back to zuul | 23:35 |
| fungi | wfm | 23:35 |
| *** dchen has joined #openstack-infra | 23:36 | |
| clarkb | if however we think large installs like zuul should be wary of ansible updates then I'm not sure zuul should make those changes | 23:36 |
| clarkb | er like opendev | 23:36 |
| *** dchen has quit IRC | 23:36 | |
| *** dchen has joined #openstack-infra | 23:38 | |
| *** sthussey has quit IRC | 23:52 | |
| *** gyee has quit IRC | 23:55 | |
| ianw | does anyone know where we got to with the opensuse mirrors? | 23:56 |
| ianw | https://zuul.opendev.org/t/openstack/build/fa10296fc0344723a866f1da37a6a50c/log/nodepool/builds/test-image-0000000004.log#338 | 23:56 |
| ianw | [repo-update|http://mirror.iad.rax.openstack.org/opensuse/update/leap/15.0/oss/] Valid metadata not found at specified URL | 23:56 |
| ianw | i'm wondering if i need to pull it out of the dib gate for a bit? | 23:56 |
| ianw | on a related note | 23:58 |
| ianw | 2019-08-29 10:44:37 | Starting ForwardMulti from 536871007 to 536871007 on afs02.dfw.openstack.org (as of Thu Aug 29 00:44:37 2019). | 23:58 |
| ianw | 2019-08-29 18:23:34 | updating VLDB ... done | 23:58 |
| ianw | 2019-08-29 18:23:34 | Released volume mirror.fedora successfully | 23:58 |
| clarkb | ianw: last I remember wsa the OBS mirrors not mirroring beacuse their upstreams didn't mirror those repos anymore | 23:59 |
| *** hongbin has joined #openstack-infra | 23:59 | |
| clarkb | but fungi did a sync without those mirrors in place which should've gotten us a working mirror but maybe upstrema for that was still sad? | 23:59 |
| clarkb | we may want to stop mirroring the OBS stuff until it can be done reliable and just mirror the distro for now | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!