| clarkb | thats the same git fetch erorr | 00:00 |
|---|---|---|
| clarkb | Error: /Stage[main]/Project_config/Vcsrepo[/etc/project-config]: Could not evaluate: Execution of '/usr/bin/git fetch --tags origin' returned 128: fatal: '/dev/null' does not appear to be a git repository | 00:00 |
| clarkb | for some reason git is running against /dev/null? | 00:00 |
| mordred | clarkb: *weird* | 00:02 |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard-webclient master: Show Email Addresses when Searching https://review.opendev.org/589713 | 00:02 |
| clarkb | oh! | 00:03 |
| clarkb | could this be related to the git rpeo cache changes? | 00:03 |
| mordred | oh! | 00:03 |
| mordred | maybe so | 00:03 |
| clarkb | I think that is the thread to pull on for now | 00:03 |
| clarkb | since it coincided with that merging | 00:03 |
| *** beekneemech has joined #openstack-infra | 00:06 | |
| *** Lucas_Gray has quit IRC | 00:07 | |
| mordred | wow. we're still using zuul-cloner | 00:07 |
| *** Lucas_Gray has joined #openstack-infra | 00:08 | |
| mordred | clarkb: so - sudo -E /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git - should that change to sudo -E /usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git/opendev.org now? | 00:08 |
| *** bnemec has quit IRC | 00:08 | |
| clarkb | mordred: ya I think that was the main change | 00:09 |
| clarkb | (to namespace the git repos) | 00:09 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 00:10 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 00:10 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Update puppet-4 project-config installation https://review.opendev.org/689282 | 00:10 |
| mordred | clarkb: k. maybe that fixes | 00:10 |
| *** beekneemech has quit IRC | 00:12 | |
| *** bnemec has joined #openstack-infra | 00:14 | |
| *** weifan has quit IRC | 00:16 | |
| *** weifan has joined #openstack-infra | 00:16 | |
| *** weifan has quit IRC | 00:17 | |
| *** weifan has joined #openstack-infra | 00:17 | |
| *** weifan has quit IRC | 00:18 | |
| *** weifan has joined #openstack-infra | 00:18 | |
| *** weifan has quit IRC | 00:18 | |
| *** weifan has joined #openstack-infra | 00:18 | |
| *** weifan has quit IRC | 00:19 | |
| *** weifan has joined #openstack-infra | 00:19 | |
| *** Lucas_Gray has quit IRC | 00:19 | |
| *** weifan has quit IRC | 00:20 | |
| *** weifan has joined #openstack-infra | 00:20 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Replace use of zuul cloner with git clones https://review.opendev.org/689284 | 00:20 |
| mordred | clarkb: ^^ also - maybe let's just not use zuul-cloner | 00:20 |
| *** weifan has quit IRC | 00:21 | |
| *** weifan has joined #openstack-infra | 00:21 | |
| *** weifan has quit IRC | 00:22 | |
| *** weifan has joined #openstack-infra | 00:22 | |
| *** weifan has quit IRC | 00:22 | |
| *** weifan has joined #openstack-infra | 00:22 | |
| *** weifan has quit IRC | 00:23 | |
| *** weifan has joined #openstack-infra | 00:23 | |
| *** weifan has quit IRC | 00:24 | |
| *** weifan has joined #openstack-infra | 00:24 | |
| *** weifan has quit IRC | 00:25 | |
| *** weifan has joined #openstack-infra | 00:25 | |
| *** weifan has quit IRC | 00:25 | |
| *** weifan has joined #openstack-infra | 00:26 | |
| *** weifan has quit IRC | 00:26 | |
| *** weifan has joined #openstack-infra | 00:26 | |
| *** weifan has quit IRC | 00:27 | |
| *** weifan has joined #openstack-infra | 00:27 | |
| *** weifan has quit IRC | 00:28 | |
| *** weifan has joined #openstack-infra | 00:28 | |
| *** weifan has quit IRC | 00:29 | |
| *** weifan has joined #openstack-infra | 00:29 | |
| *** weifan has quit IRC | 00:29 | |
| *** weifan has joined #openstack-infra | 00:30 | |
| *** weifan has quit IRC | 00:30 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Replace use of zuul cloner with git clones https://review.opendev.org/689284 | 00:47 |
| *** zhurong has quit IRC | 00:48 | |
| *** markvoelker has joined #openstack-infra | 00:49 | |
| *** dpawlik has joined #openstack-infra | 00:53 | |
| *** dpawlik has quit IRC | 00:58 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jobs to build gerrit master branch https://review.opendev.org/688482 | 01:15 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add checks plugin to gerrit master build https://review.opendev.org/688483 | 01:15 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Replace use of zuul cloner with git clones https://review.opendev.org/689284 | 01:15 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 01:15 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 01:15 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 01:15 |
| openstackgerrit | Oliver Walsh proposed openstack/diskimage-builder master: Ensure nouveau is blacklisted in initramfs too https://review.opendev.org/679992 | 01:15 |
| *** igordc has quit IRC | 01:25 | |
| *** rlandy has quit IRC | 01:29 | |
| *** rh-jelabarre has quit IRC | 01:36 | |
| *** hwoarang has quit IRC | 01:43 | |
| *** hwoarang has joined #openstack-infra | 01:49 | |
| *** rosmaita has quit IRC | 01:52 | |
| *** weifan has joined #openstack-infra | 01:53 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Replace use of zuul cloner with git clones https://review.opendev.org/689284 | 01:57 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jobs to build gerrit master branch https://review.opendev.org/688482 | 01:57 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add checks plugin to gerrit master build https://review.opendev.org/688483 | 01:57 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 01:57 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 01:57 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 01:57 |
| *** weifan has quit IRC | 01:58 | |
| *** jamesmcarthur has quit IRC | 02:03 | |
| *** rosmaita has joined #openstack-infra | 02:04 | |
| *** harlowja has quit IRC | 02:07 | |
| *** Garyx_ has joined #openstack-infra | 02:17 | |
| *** Garyx has quit IRC | 02:19 | |
| *** harlowja has joined #openstack-infra | 02:23 | |
| *** zhurong has joined #openstack-infra | 02:39 | |
| *** jamesmcarthur has joined #openstack-infra | 02:43 | |
| *** jerryz has quit IRC | 02:45 | |
| *** jamesmcarthur has quit IRC | 02:51 | |
| *** ricolin_ has joined #openstack-infra | 02:51 | |
| *** dpawlik has joined #openstack-infra | 02:54 | |
| *** dpawlik has quit IRC | 02:58 | |
| *** jamesmcarthur has joined #openstack-infra | 03:09 | |
| *** uberjay has quit IRC | 03:09 | |
| *** dave-mccowan has quit IRC | 03:21 | |
| *** uberjay has joined #openstack-infra | 03:24 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jobs to build gerrit master branch https://review.opendev.org/688482 | 03:27 |
| *** jamesmcarthur has quit IRC | 03:28 | |
| mordred | clarkb, corvus: ^^ amusingly enough - the switch to prepare-workspace-git caused the /dev/null remote to be kept, where it wasn't there before. This in turn broke the submodule update command in the gerrit build playbook :) | 03:28 |
| *** jamesmcarthur has joined #openstack-infra | 03:29 | |
| *** whoami-rajat has joined #openstack-infra | 03:32 | |
| *** jamesmcarthur has quit IRC | 03:34 | |
| *** psachin has joined #openstack-infra | 03:36 | |
| *** psachin has quit IRC | 03:36 | |
| *** psachin has joined #openstack-infra | 03:37 | |
| *** goldyfruit has quit IRC | 03:46 | |
| *** jamesmcarthur has joined #openstack-infra | 03:59 | |
| *** ykarel|away has joined #openstack-infra | 04:00 | |
| *** ykarel|away is now known as ykarel | 04:00 | |
| *** jamesmcarthur has quit IRC | 04:06 | |
| *** diablo_rojo has quit IRC | 04:22 | |
| *** soniya29 has joined #openstack-infra | 04:28 | |
| *** igordc has joined #openstack-infra | 04:35 | |
| *** igordc has quit IRC | 04:35 | |
| *** igordc has joined #openstack-infra | 04:35 | |
| *** auristor has quit IRC | 04:45 | |
| *** auristor has joined #openstack-infra | 04:47 | |
| *** soniya29 has quit IRC | 04:52 | |
| *** dpawlik has joined #openstack-infra | 04:55 | |
| *** surpatil has joined #openstack-infra | 04:55 | |
| *** adriant has joined #openstack-infra | 04:57 | |
| *** dpawlik has quit IRC | 04:59 | |
| ykarel | any known issue with download.cirros-cloud.net? | 05:02 |
| *** jamesmcarthur has joined #openstack-infra | 05:03 | |
| ykarel | seems it's too slow currently | 05:03 |
| *** igordc has quit IRC | 05:05 | |
| *** jamesmcarthur has quit IRC | 05:07 | |
| *** dpawlik has joined #openstack-infra | 05:15 | |
| *** auristor has quit IRC | 05:20 | |
| *** auristor has joined #openstack-infra | 05:23 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add checks plugin to gerrit master build https://review.opendev.org/688483 | 05:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 05:23 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 05:23 |
| *** ykarel is now known as ykarel|afk | 05:23 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 05:23 |
| *** jtomasek has quit IRC | 05:35 | |
| *** jamesmcarthur has joined #openstack-infra | 05:42 | |
| *** pgaxatte has joined #openstack-infra | 05:46 | |
| *** jamesmcarthur has quit IRC | 05:47 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jobs to build gerrit master branch https://review.opendev.org/688482 | 05:50 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add checks plugin to gerrit master build https://review.opendev.org/688483 | 05:50 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 05:50 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 05:50 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 05:50 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove gerrit repo remote for submodules https://review.opendev.org/689314 | 05:50 |
| mordred | ianw: if you're around, feel like looking at https://review.opendev.org/#/c/689284/4 and https://review.opendev.org/#/c/689314/1 ? | 05:51 |
| *** pgaxatte has quit IRC | 05:51 | |
| *** pgaxatte has joined #openstack-infra | 05:52 | |
| openstackgerrit | Merged openstack/project-config master: Update neutron stable grafana dashboards https://review.opendev.org/689233 | 05:57 |
| *** ykarel|afk is now known as ykarel | 06:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/project-config master: Normalize projects.yaml https://review.opendev.org/689318 | 06:12 |
| *** dpawlik has quit IRC | 06:20 | |
| *** jtomasek has joined #openstack-infra | 06:22 | |
| *** dpawlik has joined #openstack-infra | 06:27 | |
| *** slaweq has joined #openstack-infra | 06:30 | |
| *** jtomasek_ has joined #openstack-infra | 06:30 | |
| *** ociuhandu has joined #openstack-infra | 06:31 | |
| openstackgerrit | Merged openstack/project-config master: Normalize projects.yaml https://review.opendev.org/689318 | 06:31 |
| *** jtomasek has quit IRC | 06:31 | |
| *** pgaxatte has quit IRC | 06:34 | |
| *** ociuhandu has quit IRC | 06:35 | |
| *** freefood_ has joined #openstack-infra | 06:35 | |
| openstackgerrit | Frode Nordahl proposed openstack/project-config master: Add gerritbot trigger for OpenDev/x charm projects https://review.opendev.org/689325 | 06:38 |
| *** jamesmcarthur has joined #openstack-infra | 06:43 | |
| *** jamesmcarthur has quit IRC | 06:48 | |
| *** trident has quit IRC | 07:00 | |
| *** ramishra has joined #openstack-infra | 07:01 | |
| *** gfidente has joined #openstack-infra | 07:04 | |
| *** trident has joined #openstack-infra | 07:04 | |
| *** iurygregory has joined #openstack-infra | 07:05 | |
| *** lpetrut has joined #openstack-infra | 07:05 | |
| *** udesale has joined #openstack-infra | 07:08 | |
| *** jamesmcarthur has joined #openstack-infra | 07:21 | |
| *** tesseract has joined #openstack-infra | 07:22 | |
| *** pgaxatte has joined #openstack-infra | 07:23 | |
| *** jamesmcarthur has quit IRC | 07:26 | |
| *** pcaruana has joined #openstack-infra | 07:26 | |
| *** pkopec has joined #openstack-infra | 07:29 | |
| *** ricolin_ is now known as ricolin | 07:30 | |
| chandankumar | /join #tripleo-ci | 07:32 |
| *** tosky has joined #openstack-infra | 07:33 | |
| *** apetrich has joined #openstack-infra | 07:33 | |
| *** jamesmcarthur has joined #openstack-infra | 07:37 | |
| *** kopecmartin|off is now known as kopecmartin | 07:39 | |
| *** jamesmcarthur has quit IRC | 07:41 | |
| *** jpena|off is now known as jpena | 07:42 | |
| *** ralonsoh has joined #openstack-infra | 07:51 | |
| *** rpittau|afk is now known as rpittau | 08:01 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 08:06 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 08:06 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 08:06 |
| *** xek__ has joined #openstack-infra | 08:07 | |
| *** lucasagomes has joined #openstack-infra | 08:09 | |
| *** jamesmcarthur has joined #openstack-infra | 08:09 | |
| *** dmsimard has quit IRC | 08:14 | |
| *** dmsimard1 has joined #openstack-infra | 08:14 | |
| *** e0ne has joined #openstack-infra | 08:30 | |
| *** priteau has joined #openstack-infra | 08:36 | |
| *** hamzy_ has joined #openstack-infra | 08:41 | |
| *** hamzy has quit IRC | 08:43 | |
| *** jamesmcarthur has quit IRC | 08:45 | |
| *** ykarel is now known as ykarel|lunch | 08:46 | |
| *** dansmith has quit IRC | 09:07 | |
| *** dansmith has joined #openstack-infra | 09:24 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add reference pipelines file for Github driver https://review.opendev.org/672712 | 09:26 |
| *** dtantsur|afk is now known as dtantsur | 09:31 | |
| *** ykarel|lunch is now known as ykarel | 09:39 | |
| *** jamesmcarthur has joined #openstack-infra | 09:41 | |
| *** jamesmcarthur has quit IRC | 09:46 | |
| *** dpawlik has quit IRC | 09:47 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - Support for branch creation/deletion https://review.opendev.org/685116 | 09:47 |
| *** ociuhandu has joined #openstack-infra | 09:47 | |
| *** hwoarang has quit IRC | 09:49 | |
| *** kjackal has quit IRC | 09:49 | |
| *** kjackal has joined #openstack-infra | 09:49 | |
| *** pgaxatte has quit IRC | 09:52 | |
| *** hwoarang has joined #openstack-infra | 09:53 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - add the enqueue_ref unit test https://review.opendev.org/687351 | 09:58 |
| *** ramishra has quit IRC | 09:59 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Keep tenant_last_reconfigured float precision (time.sleep removal) https://review.opendev.org/689394 | 10:03 |
| *** ykarel is now known as ykarel|afk | 10:07 | |
| *** xek__ has quit IRC | 10:08 | |
| *** dpawlik has joined #openstack-infra | 10:09 | |
| *** xek has joined #openstack-infra | 10:11 | |
| *** priteau has quit IRC | 10:13 | |
| *** bnemec has quit IRC | 10:13 | |
| *** e0ne has quit IRC | 10:14 | |
| *** bnemec has joined #openstack-infra | 10:15 | |
| *** xek_ has joined #openstack-infra | 10:15 | |
| *** xek has quit IRC | 10:16 | |
| *** fdegir has quit IRC | 10:16 | |
| *** georgk has quit IRC | 10:16 | |
| *** fdegir has joined #openstack-infra | 10:17 | |
| *** georgk has joined #openstack-infra | 10:17 | |
| *** e0ne has joined #openstack-infra | 10:25 | |
| *** ramishra has joined #openstack-infra | 10:27 | |
| *** xek_ has quit IRC | 10:38 | |
| *** xek has joined #openstack-infra | 10:40 | |
| *** e0ne has quit IRC | 10:42 | |
| *** ykarel|afk is now known as ykarel | 10:43 | |
| *** e0ne has joined #openstack-infra | 10:44 | |
| *** e0ne has quit IRC | 10:45 | |
| *** pgaxatte has joined #openstack-infra | 10:48 | |
| *** hwoarang has quit IRC | 10:48 | |
| *** hwoarang has joined #openstack-infra | 10:49 | |
| *** e0ne has joined #openstack-infra | 10:54 | |
| *** zigo has quit IRC | 10:54 | |
| *** e0ne has quit IRC | 10:59 | |
| *** e0ne_ has joined #openstack-infra | 10:59 | |
| *** jpena is now known as jpena|lunch | 11:03 | |
| *** e0ne_ has quit IRC | 11:03 | |
| *** zigo has joined #openstack-infra | 11:03 | |
| *** jamesmcarthur has joined #openstack-infra | 11:04 | |
| openstackgerrit | Oliver Walsh proposed openstack/diskimage-builder master: Ensure nouveau is blacklisted in initramfs too https://review.opendev.org/679992 | 11:08 |
| *** dpawlik has quit IRC | 11:09 | |
| *** tkajinam has quit IRC | 11:16 | |
| *** xek has quit IRC | 11:16 | |
| *** e0ne has joined #openstack-infra | 11:18 | |
| *** e0ne_ has joined #openstack-infra | 11:20 | |
| *** e0ne has quit IRC | 11:23 | |
| *** e0ne_ has quit IRC | 11:25 | |
| *** pkopec_ has joined #openstack-infra | 11:30 | |
| openstackgerrit | Oliver Walsh proposed openstack/diskimage-builder master: Fix syntax error in selinux-fixfiles-restore https://review.opendev.org/689430 | 11:32 |
| *** pkopec has quit IRC | 11:33 | |
| *** e0ne has joined #openstack-infra | 11:40 | |
| *** pkopec_ has quit IRC | 11:41 | |
| donnyd | clarkb: fungi 48 hours since the launcher restart and no more issues with nodepool seeing nodes stuck in deleting | 11:46 |
| *** e0ne has quit IRC | 11:49 | |
| *** e0ne has joined #openstack-infra | 11:50 | |
| *** e0ne has quit IRC | 11:51 | |
| *** e0ne has joined #openstack-infra | 11:52 | |
| *** rh-jelabarre has joined #openstack-infra | 11:53 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Keep tenant_last_reconfigured float precision (time.sleep removal) https://review.opendev.org/689394 | 11:53 |
| *** whoami-rajat has quit IRC | 11:54 | |
| *** dpawlik has joined #openstack-infra | 11:56 | |
| *** jpena|lunch is now known as jpena | 11:58 | |
| *** rfolco|ruck has joined #openstack-infra | 12:04 | |
| *** jamesmcarthur has quit IRC | 12:09 | |
| *** ramishra has quit IRC | 12:12 | |
| *** udesale has quit IRC | 12:13 | |
| *** udesale has joined #openstack-infra | 12:16 | |
| *** e0ne has quit IRC | 12:16 | |
| *** rlandy has joined #openstack-infra | 12:16 | |
| donnyd | ianw: I used to have several error node launches per day from ipv6 connectivity issues. Haven't had a single one in the last 48 hours | 12:20 |
| donnyd | I am thinking the ipv6 thing should be good to go | 12:20 |
| *** e0ne has joined #openstack-infra | 12:21 | |
| *** ociuhandu has quit IRC | 12:22 | |
| *** ociuhandu has joined #openstack-infra | 12:23 | |
| *** dtantsur is now known as dtantsur|brb | 12:24 | |
| *** larsks has quit IRC | 12:25 | |
| *** jamesmcarthur has joined #openstack-infra | 12:27 | |
| *** larsks has joined #openstack-infra | 12:32 | |
| *** jovial[m] has joined #openstack-infra | 12:34 | |
| jovial[m] | is it possible to install an additional ansible role on the zuul executor and then use that role with `include_role`? | 12:36 |
| *** e0ne has quit IRC | 12:37 | |
| *** e0ne has joined #openstack-infra | 12:37 | |
| *** jamesmcarthur has quit IRC | 12:40 | |
| *** e0ne has quit IRC | 12:42 | |
| *** eharney has joined #openstack-infra | 12:44 | |
| *** jamesmcarthur has joined #openstack-infra | 12:45 | |
| openstackgerrit | Merged opendev/storyboard-webclient master: Show Email Addresses when Searching https://review.opendev.org/589713 | 12:53 |
| *** pgaxatte has quit IRC | 12:53 | |
| *** pgaxatte has joined #openstack-infra | 12:55 | |
| *** larsks has quit IRC | 13:00 | |
| *** larsks has joined #openstack-infra | 13:01 | |
| clarkb | jovial[m]: https://zuul-ci.org/docs/zuul/user/config.html#attr-job.roles | 13:09 |
| *** surpatil has quit IRC | 13:10 | |
| *** surpatil has joined #openstack-infra | 13:10 | |
| jovial[m] | thank you | 13:11 |
| *** mriedem has joined #openstack-infra | 13:16 | |
| *** jamesmcarthur has quit IRC | 13:18 | |
| *** e0ne has joined #openstack-infra | 13:20 | |
| *** ociuhandu has quit IRC | 13:25 | |
| *** ociuhandu has joined #openstack-infra | 13:26 | |
| *** goldyfruit has joined #openstack-infra | 13:28 | |
| openstackgerrit | Merged zuul/nodepool master: Pull minikube log data https://review.opendev.org/688775 | 13:30 |
| openstackgerrit | Merged zuul/nodepool master: Remove unused k8s log config https://review.opendev.org/688961 | 13:30 |
| *** ociuhandu has quit IRC | 13:31 | |
| *** eharney has quit IRC | 13:33 | |
| *** e0ne has quit IRC | 13:33 | |
| *** e0ne has joined #openstack-infra | 13:34 | |
| *** e0ne has quit IRC | 13:34 | |
| *** e0ne has joined #openstack-infra | 13:35 | |
| *** ykarel is now known as ykarel|afk | 13:36 | |
| *** prometheanfire has quit IRC | 13:37 | |
| *** ykarel|afk is now known as ykarel | 13:37 | |
| *** gfidente has quit IRC | 13:37 | |
| *** prometheanfire has joined #openstack-infra | 13:38 | |
| *** rcernin has quit IRC | 13:40 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Remove unused nodepool-functional-py35-src https://review.opendev.org/689457 | 13:41 |
| *** e0ne has quit IRC | 13:42 | |
| *** e0ne has joined #openstack-infra | 13:42 | |
| *** ricolin has quit IRC | 13:43 | |
| *** larsks has quit IRC | 13:44 | |
| *** larsks has joined #openstack-infra | 13:46 | |
| *** rascasoft has quit IRC | 13:47 | |
| *** dave-mccowan has joined #openstack-infra | 13:47 | |
| *** surpatil has quit IRC | 13:49 | |
| *** rascasoft has joined #openstack-infra | 13:49 | |
| *** gfidente has joined #openstack-infra | 13:49 | |
| *** jamesmcarthur has joined #openstack-infra | 13:52 | |
| *** xek has joined #openstack-infra | 13:52 | |
| *** pcaruana has quit IRC | 13:53 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Remove unused functional testing playbooks https://review.opendev.org/689457 | 13:54 |
| *** psachin has quit IRC | 13:54 | |
| *** dave-mccowan has quit IRC | 13:56 | |
| *** jamesmcarthur has quit IRC | 13:56 | |
| *** Goneri has joined #openstack-infra | 13:57 | |
| *** e0ne has quit IRC | 13:59 | |
| *** e0ne has joined #openstack-infra | 14:03 | |
| *** yamamoto has joined #openstack-infra | 14:04 | |
| *** jproulx has joined #openstack-infra | 14:05 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: WIP: Take advantage of fetch-output https://review.opendev.org/689473 | 14:07 |
| openstackgerrit | Michal Pryc proposed zuul/nodepool master: Implement a Devnest nodepool driver https://review.opendev.org/689474 | 14:09 |
| Shrews | O.o | 14:09 |
| openstackgerrit | Michal Pryc proposed zuul/nodepool master: Implement a Devnest nodepool driver https://review.opendev.org/689474 | 14:11 |
| *** efried_afk is now known as efried | 14:11 | |
| Shrews | that's certainly an interesting driver | 14:11 |
| *** e0ne has quit IRC | 14:13 | |
| *** dpawlik has quit IRC | 14:18 | |
| *** e0ne has joined #openstack-infra | 14:18 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Cleanup openshift pre.yaml playbook https://review.opendev.org/689479 | 14:25 |
| *** sgw has quit IRC | 14:26 | |
| *** e0ne has quit IRC | 14:27 | |
| *** e0ne has joined #openstack-infra | 14:27 | |
| corvus | mordred: i've reviewed topic:gerrit-images -- comments on 688482 and 630406 | 14:29 |
| *** larsks has quit IRC | 14:29 | |
| *** larsks has joined #openstack-infra | 14:30 | |
| *** gfidente has quit IRC | 14:32 | |
| *** gfidente has joined #openstack-infra | 14:36 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Take advantage of fetch-output https://review.opendev.org/689473 | 14:43 |
| *** armax has joined #openstack-infra | 14:46 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Disable namespacing https://review.opendev.org/689219 | 14:46 |
| *** bnemec has quit IRC | 14:49 | |
| *** diablo_rojo has joined #openstack-infra | 14:51 | |
| *** jamesmcarthur has joined #openstack-infra | 14:53 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Use zuul-registry as buildset registry https://review.opendev.org/689238 | 14:53 |
| *** bnemec has joined #openstack-infra | 14:55 | |
| *** psachin has joined #openstack-infra | 14:57 | |
| *** dtantsur|brb is now known as dtantsur | 14:57 | |
| *** jamesmcarthur has quit IRC | 14:58 | |
| *** FlorianFa has quit IRC | 15:01 | |
| *** bnemec is now known as beekneemech | 15:02 | |
| * fungi heads out to grab lunch, bbiab | 15:06 | |
| *** gyee has joined #openstack-infra | 15:07 | |
| *** ociuhandu has joined #openstack-infra | 15:08 | |
| *** udesale has quit IRC | 15:10 | |
| *** udesale has joined #openstack-infra | 15:11 | |
| *** ociuhandu has quit IRC | 15:12 | |
| *** Goneri has quit IRC | 15:16 | |
| *** e0ne has quit IRC | 15:18 | |
| *** e0ne has joined #openstack-infra | 15:19 | |
| *** e0ne has quit IRC | 15:22 | |
| *** kopecmartin is now known as kopecmartin|off | 15:23 | |
| *** mattw4 has joined #openstack-infra | 15:25 | |
| *** e0ne has joined #openstack-infra | 15:26 | |
| *** jamesmcarthur has joined #openstack-infra | 15:27 | |
| *** e0ne has quit IRC | 15:29 | |
| *** mattw4 has quit IRC | 15:29 | |
| *** eharney has joined #openstack-infra | 15:30 | |
| *** Goneri has joined #openstack-infra | 15:31 | |
| *** igordc has joined #openstack-infra | 15:32 | |
| *** e0ne has joined #openstack-infra | 15:32 | |
| *** beekneemech has quit IRC | 15:34 | |
| *** e0ne has quit IRC | 15:37 | |
| *** eernst has joined #openstack-infra | 15:38 | |
| *** e0ne has joined #openstack-infra | 15:39 | |
| *** ociuhandu has joined #openstack-infra | 15:43 | |
| *** ociuhandu has quit IRC | 15:43 | |
| *** ociuhandu has joined #openstack-infra | 15:44 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Use zuul-registry as buildset registry https://review.opendev.org/689238 | 15:47 |
| *** yamamoto has quit IRC | 15:50 | |
| *** yamamoto has joined #openstack-infra | 15:51 | |
| *** yamamoto has quit IRC | 15:51 | |
| *** yamamoto has joined #openstack-infra | 15:52 | |
| *** ykarel is now known as ykarel|afk | 15:55 | |
| *** ykarel|afk is now known as ykarel | 15:55 | |
| *** e0ne_ has joined #openstack-infra | 15:56 | |
| *** yamamoto has quit IRC | 15:57 | |
| *** e0ne has quit IRC | 15:58 | |
| *** dtantsur is now known as dtantsur|afk | 16:01 | |
| *** panda has quit IRC | 16:01 | |
| *** panda has joined #openstack-infra | 16:03 | |
| *** rpittau is now known as rpittau|afk | 16:04 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Use zuul-registry as buildset registry https://review.opendev.org/689238 | 16:07 |
| *** lucasagomes has quit IRC | 16:08 | |
| *** ociuhandu has quit IRC | 16:11 | |
| *** e0ne_ has quit IRC | 16:16 | |
| *** e0ne has joined #openstack-infra | 16:17 | |
| *** pgaxatte has quit IRC | 16:19 | |
| *** ociuhandu has joined #openstack-infra | 16:19 | |
| *** rlandy is now known as rlandy|brb | 16:20 | |
| *** ociuhandu has quit IRC | 16:24 | |
| *** markvoelker has quit IRC | 16:26 | |
| *** markvoelker has joined #openstack-infra | 16:27 | |
| *** pkopec has joined #openstack-infra | 16:27 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Make the buildset registry port configurable https://review.opendev.org/689240 | 16:36 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Test buildset registry with k8s and docker https://review.opendev.org/689280 | 16:36 |
| *** e0ne has quit IRC | 16:39 | |
| *** pkopec has quit IRC | 16:39 | |
| *** pkopec has joined #openstack-infra | 16:39 | |
| *** igordc has quit IRC | 16:40 | |
| *** e0ne has joined #openstack-infra | 16:42 | |
| *** mattw4 has joined #openstack-infra | 16:42 | |
| *** jpena is now known as jpena|off | 16:45 | |
| *** e0ne has quit IRC | 16:45 | |
| *** michael-beaver has joined #openstack-infra | 16:49 | |
| *** e0ne has joined #openstack-infra | 16:50 | |
| *** e0ne has quit IRC | 16:53 | |
| *** whoami-rajat has joined #openstack-infra | 16:56 | |
| zbr | do we have nay problems with our mirrors because I see failure to find openssl-devel on redhat?! https://e319cf60a72a6d3e844d-a999ae1d5baedde86711d4f3bf719537.ssl.cf2.rackcdn.com/689527/1/check/openstack-tox-linters/863737f/job-output.txt | 17:04 |
| *** weifan has joined #openstack-infra | 17:05 | |
| fungi | #status log deleted erroneous puppet-mistral project from pypi.org per request from release team http://eavesdrop.openstack.org/irclogs/%23openstack-release/%23openstack-release.2019-10-18.log.html#t2019-10-18T13:00:50 | 17:07 |
| openstackstatus | fungi: finished logging | 17:07 |
| *** tosky has quit IRC | 17:08 | |
| zbr | os is me misreading bindep parsing error. | 17:08 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Use zuul-registry as buildset registry https://review.opendev.org/689238 | 17:08 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Make the buildset registry port configurable https://review.opendev.org/689240 | 17:08 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Test buildset registry with k8s and docker https://review.opendev.org/689280 | 17:08 |
| fungi | zbr: "Parse error at line 40, column 0: expected EOF. trail: [comment rule]: | 17:08 |
| fungi | yep | 17:08 |
| *** lpetrut has quit IRC | 17:09 | |
| fungi | just pulled it up to look | 17:09 |
| fungi | try running bindep against a copy of that bindep.txt locally | 17:09 |
| zbr | my guess: old version not knowing about redhat platform, or error because there was no newline at end of entry. | 17:10 |
| fungi | odds are the error is in the line before openssl-devel | 17:10 |
| zbr | anyway. none should raise a traceback | 17:10 |
| fungi | it says it expected to reach the end of the file at column 0 of line 40 but instead found text "openssl-devel..." | 17:11 |
| fungi | so my guess is the problem is on line 39 | 17:11 |
| *** e0ne has joined #openstack-infra | 17:11 | |
| fungi | ahh, nope, judging from https://review.opendev.org/#/c/689527/2/bindep.txt the openssl-devel line is 39 | 17:12 |
| fungi | so i agree, something happening there | 17:12 |
| *** weifan has quit IRC | 17:13 | |
| fungi | interestingly, `tox -e bindep` locally with that change checked out does not report any issues | 17:15 |
| fungi | oh, you pushed up a new patchset already | 17:16 |
| *** e0ne has quit IRC | 17:16 | |
| fungi | after checking out 689527,1 i can confirm, the problem is a track of final linefeed | 17:17 |
| fungi | er, lack of final linefeed | 17:17 |
| *** pkopec_ has joined #openstack-infra | 17:17 | |
| *** ociuhandu has joined #openstack-infra | 17:18 | |
| fungi | if i open bindep.txt in vi and then write it again (so that a trailing newline is appended to that last line of the file) then it works | 17:18 |
| *** pkopec has quit IRC | 17:18 | |
| *** tesseract has quit IRC | 17:18 | |
| fungi | zbr: if you want to propose a patch to make bindep happy with files that lack a final \n then that seems like a reasonable improvement to me from a postel's law/robustness principle perspective, even if i happen to think it's bad form to not end the last line of a file (try to cat that on a terminal and watch your shell prompt get appended to the last line of output, ick) | 17:20 |
| zbr | fungi: on most repos i benefit from a eol fixed in pre-commit, but this repo one does not have pre-commit on it. | 17:21 |
| zbr | anyway, I do agree, just that is too late now. if i remember on monday, i will make a CR. | 17:21 |
| *** ociuhandu has quit IRC | 17:22 | |
| zbr | the recent issue with ansible-lint kept me busy this week, and the joy is not over yet. | 17:22 |
| fungi | no worries, you're the first to ever point this out as far as i can recall, so i doubt it's urgent ;) | 17:22 |
| *** diablo_rojo has quit IRC | 17:25 | |
| *** rlandy|brb is now known as rlandy | 17:32 | |
| *** markvoelker has quit IRC | 17:34 | |
| *** weifan has joined #openstack-infra | 17:37 | |
| *** ykarel is now known as ykarel|away | 17:44 | |
| *** pkopec_ has quit IRC | 17:44 | |
| *** jamesmcarthur has quit IRC | 17:44 | |
| *** eharney has quit IRC | 17:45 | |
| *** jerryz has joined #openstack-infra | 17:46 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Use zuul-registry as buildset registry https://review.opendev.org/689238 | 17:52 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Make the buildset registry port configurable https://review.opendev.org/689240 | 17:53 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Test buildset registry with k8s and docker https://review.opendev.org/689280 | 17:54 |
| *** udesale has quit IRC | 17:56 | |
| *** ykarel|away has quit IRC | 17:59 | |
| *** tosky has joined #openstack-infra | 18:00 | |
| openstackgerrit | Mohammed Naser proposed zuul/zuul master: Zuul Web: add /api/user/authorizations endpoint https://review.opendev.org/641099 | 18:01 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul master: [WIP] admin REST API: zuul-web integration https://review.opendev.org/643536 | 18:01 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul master: Add API endpoint to get authenticators https://review.opendev.org/689560 | 18:01 |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Take advantage of fetch-output https://review.opendev.org/689473 | 18:08 |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Take advantage of fetch-output https://review.opendev.org/689473 | 18:10 |
| *** kjackal has quit IRC | 18:10 | |
| *** psachin has quit IRC | 18:15 | |
| *** kjackal has joined #openstack-infra | 18:15 | |
| *** igordc has joined #openstack-infra | 18:17 | |
| mordred | corvus: oh wow - I also don't know what that web_server.py is - fascinating | 18:17 |
| corvus | mordred: hahaha | 18:18 |
| corvus | mordred: i feel like we might be on to something with this crazy code review idea | 18:19 |
| mordred | corvus: bah. | 18:20 |
| *** weifan has quit IRC | 18:23 | |
| *** weifan has joined #openstack-infra | 18:25 | |
| *** e0ne has joined #openstack-infra | 18:26 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jobs to build gerrit master branch https://review.opendev.org/688482 | 18:28 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add checks plugin to gerrit master build https://review.opendev.org/688483 | 18:28 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 18:28 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 18:28 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 18:28 |
| *** weifan has quit IRC | 18:29 | |
| mordred | Shrews: config mismatch between two files in https://review.opendev.org/#/c/689473 | 18:30 |
| *** e0ne has quit IRC | 18:31 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Store a list of held node per held build in hold request https://review.opendev.org/682466 | 18:31 |
| Shrews | mordred: darn your eagle eyes | 18:32 |
| *** weifan has joined #openstack-infra | 18:32 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Take advantage of fetch-output https://review.opendev.org/689473 | 18:32 |
| *** weifan has quit IRC | 18:33 | |
| *** weifan has joined #openstack-infra | 18:35 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Store a list of held node per held build in hold request https://review.opendev.org/682466 | 18:37 |
| *** weifan has quit IRC | 18:37 | |
| mordred | fungi: feel like a +A on https://review.opendev.org/#/c/689284/ and https://review.opendev.org/#/c/689314 ? | 18:46 |
| fungi | mordred: we'll see if i am after i look at them. just a sec ;) | 18:48 |
| mordred | fungi: aww - but blind approvals are so much fun :) | 18:48 |
| *** iurygregory has quit IRC | 18:49 | |
| *** weifan has joined #openstack-infra | 18:51 | |
| AJaeger | corvus: can we remove the use-cached-repos role? I think https://review.opendev.org/#/c/680705/1 is ready and if you confirm, let's ask config-core for review | 18:53 |
| mordred | AJaeger: +2 from me - but left off +A pending response from corvus | 18:55 |
| AJaeger | corvus: want to abandon your test change https://review.opendev.org/682912 for prepare-workspace-git? I think that's all for that topic, isn't it? | 18:55 |
| AJaeger | thanks, mordred | 18:56 |
| *** trident has quit IRC | 18:57 | |
| *** eharney has joined #openstack-infra | 18:58 | |
| *** trident has joined #openstack-infra | 19:00 | |
| *** e0ne has joined #openstack-infra | 19:03 | |
| *** yamamoto has joined #openstack-infra | 19:05 | |
| mordred | thanks fungi | 19:06 |
| *** yamamoto has quit IRC | 19:11 | |
| paladox | Do you use h2cache for websessions? :) | 19:22 |
| corvus | AJaeger: approved -- i think that's just cleanup | 19:31 |
| AJaeger | great ;) that's what I expected but wanted to be safe | 19:32 |
| *** markvoelker has joined #openstack-infra | 19:36 | |
| *** markvoelker has quit IRC | 19:38 | |
| *** markvoelker has joined #openstack-infra | 19:38 | |
| *** gyee has quit IRC | 19:39 | |
| *** gyee has joined #openstack-infra | 19:39 | |
| openstackgerrit | Merged opendev/base-jobs master: Remove use-cached-repos role https://review.opendev.org/680705 | 19:40 |
| *** e0ne has quit IRC | 19:42 | |
| *** ociuhandu has joined #openstack-infra | 19:45 | |
| *** e0ne has joined #openstack-infra | 19:47 | |
| mordred | ooh good: "msg": "Failed to update apt cache: E:Failed to fetch https://download.docker.com/linux/ubuntu/dists/bionic/stable/binary-amd64/Packages.bz2 File has unexpected size (9417 != 7889). Mirror sync in progress? | 19:49 |
| mordred | corvus: install-docker role is having sads atm | 19:49 |
| *** ociuhandu has quit IRC | 19:50 | |
| *** igordc has quit IRC | 19:54 | |
| Shrews | mordred: ugh. that's... frustrating | 19:55 |
| *** jerryz has quit IRC | 19:58 | |
| mordred | I feel like maybe that's an apt repo we should mirror | 19:59 |
| mordred | oh - we DO mirror that | 20:00 |
| mordred | why is that role not using that mirror | 20:00 |
| corvus | mordred: we mirror the docker dpkg repository? | 20:05 |
| fungi | i didn't think we mirrored that, but we could i expect | 20:05 |
| corvus | and yeah, i hit that a while ago and rechecked, so i guess it's a somewhat persistent issue | 20:06 |
| corvus | we may want to think about doing that. also the projectatomic ppa | 20:07 |
| corvus | would be really sweet if we could just use distros for this | 20:08 |
| *** ralonsoh has quit IRC | 20:08 | |
| mordred | corvus: we have a proxy-pass configured for download.docker.com/linux in playbooks/roles/mirror/templates/mirror.vhost.j2 | 20:11 |
| corvus | mordred: ah, so we are caching, but we're not protecting against invalid mirrors | 20:11 |
| corvus | i mean, aside from the fact that we aren't using that | 20:11 |
| corvus | but if we we're, we wouldn't be :) | 20:12 |
| mordred | yeah. well - we ALSO seem to have a deb-docker repo in reprepro | 20:13 |
| mordred | corvus: http://mirror.dfw.rax.opendev.org/deb-docker | 20:13 |
| corvus | so i guess if we used that we might get jobs workin again | 20:18 |
| mordred | uhm. so ... the install-docker role is totally set up to use use-docker-mirror which is being run in the failing jobs | 20:19 |
| mordred | it's based off of zuul_site_mirror_fqdn | 20:20 |
| mordred | there's an override variable, docker_mirror | 20:21 |
| *** jerryz has joined #openstack-infra | 20:22 | |
| *** igordc has joined #openstack-infra | 20:23 | |
| *** xarses has quit IRC | 20:24 | |
| *** e0ne has quit IRC | 20:25 | |
| mordred | so I think we need to set docker_mirror_base_url to http://{{ zuul_site_mirror_fqdn }}/deb-docker | 20:29 |
| fungi | debian-stretch and debian-buster image builds are now failing on this: 2019-10-18 17:14:16.489 | cp: cannot stat '/opt/dib_tmp/dib_build.vehEjx10/mnt/opt/git/openstack/project-config/nodepool/scripts': No such file or directory | 20:30 |
| fungi | https://nb01.openstack.org/debian-stretch-0000049689.log | 20:30 |
| fungi | oh, so are the ubuntu-bionic images | 20:31 |
| fungi | i wonder if we recently changed something which is now breaking all/many of our image builds? | 20:31 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: DNM Use mirror for buildset-registry https://review.opendev.org/689578 | 20:31 |
| fungi | yep, fedora-29 as well | 20:32 |
| fungi | infra-root: ^ i think all image builds are currently broken | 20:32 |
| mordred | \o/ | 20:32 |
| mordred | fungi: that looks related to the git cache location change? | 20:33 |
| mordred | fungi: should be /opt/git/opendev.org in there | 20:33 |
| fungi | oh, yep | 20:33 |
| fungi | last successful builds were just over a day ago, so timing makes sense | 20:34 |
| mordred | fungi: fix coming | 20:34 |
| fungi | cool, i had only gotten as far as finding the source for the affected dib element | 20:35 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Update nodepool element for disk cache update https://review.opendev.org/689579 | 20:35 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Update nodepool element for disk cache update https://review.opendev.org/689579 | 20:35 |
| fungi | mordred: thanks for the quick fix! | 20:37 |
| mordred | corvus: ok. so - there's two different things here - install-docker needs to configure docker to use dockerhub mirrors | 20:41 |
| mordred | corvus: AND - there's a separate setting for installing docker itself from a mirror | 20:41 |
| mordred | the DNM patch above worked and installed docker | 20:41 |
| corvus | mordred: should we set that as a site var? | 20:43 |
| openstackgerrit | Monty Taylor proposed opendev/base-jobs master: Install docker from the reprepro mirror https://review.opendev.org/689581 | 20:44 |
| corvus | or base jobs :) | 20:45 |
| mordred | corvus: either way :) | 20:45 |
| mordred | oh - you know - that's probably not the right thing to do | 20:45 |
| corvus | why not? | 20:45 |
| mordred | because red hat | 20:45 |
| corvus | oh i guess that's why it's constructed like that... | 20:46 |
| mordred | yeah. it's just - the construction is still wrong for us | 20:46 |
| corvus | maybe we should shelve this and hold out for the new zuul-jobs mirror structure | 20:46 |
| corvus | (assuming that the docker repo gets fixed soon -- i guess if it doesn't then we won't have anything else to do other than get the new mirror roles done) | 20:47 |
| mordred | maybe we should ... although our docker jobs are gonna be busted until docker figures out that their repo is broken | 20:47 |
| mordred | this is a good point | 20:47 |
| mordred | I'm going to abandon that change for now | 20:47 |
| corvus | mordred: well, we got as far as this in the mirror stuff: https://zuul-ci.org/docs/zuul-jobs/mirror.html | 20:47 |
| corvus | basically landed some docs, but haven't landed a change that implements any of it yet | 20:48 |
| corvus | but it gives us something to hang a change to install-docker off of | 20:48 |
| mordred | yeah | 20:48 |
| *** bdodd has quit IRC | 20:51 | |
| *** e0ne has joined #openstack-infra | 20:53 | |
| mordred | corvus: ooh - what if I just set that variable in system-config for now | 20:54 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Use mirror for buildset-registry https://review.opendev.org/689578 | 20:57 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Replace use of zuul cloner with git clones https://review.opendev.org/689284 | 20:57 |
| mordred | let's see if that works | 20:57 |
| openstackgerrit | Merged openstack/project-config master: Update nodepool element for disk cache update https://review.opendev.org/689579 | 20:58 |
| corvus | mordred: yeah, that should be fine | 20:58 |
| * fungi has just discovered https://www.freedesktop.org/software/systemd/man/systemd.swap.html | 20:59 | |
| fungi | and so systemdos grows a little more | 20:59 |
| corvus | fungi: what failed? (i assume... i mean... that's the usual way someone discovers systemd has grown) | 21:00 |
| mordred | fungi: I can't unsee that | 21:01 |
| fungi | corvus: fielding a question from a friend on what the "right" way was to add a swap device these days | 21:05 |
| *** lpetrut has joined #openstack-infra | 21:06 | |
| fungi | systemd apparently wants to get rid of fstab | 21:06 |
| *** e0ne has quit IRC | 21:06 | |
| fungi | at least that's all i can figure | 21:06 |
| mordred | systemd wants to get rid of linux | 21:06 |
| corvus | mordred: seems to have worked | 21:07 |
| fungi | he was all, like, "should i use systemd.swap?" and then i was, like, "systemd.swap? oh no..." | 21:07 |
| *** goldyfruit has quit IRC | 21:10 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove gerrit repo remote for submodules https://review.opendev.org/689314 | 21:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add jobs to build gerrit master branch https://review.opendev.org/688482 | 21:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add checks plugin to gerrit master build https://review.opendev.org/688483 | 21:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Move bazelrc options to /etc/bazel.bazelrc https://review.opendev.org/689288 | 21:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Squash two ansible warnings https://review.opendev.org/689262 | 21:11 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 21:11 |
| mordred | there's the rest of the stack rebased on the docker mirror patch | 21:11 |
| mordred | fungi: https://review.opendev.org/#/c/689578/ if you get a sec | 21:11 |
| *** lpetrut has quit IRC | 21:13 | |
| *** markvoelker has quit IRC | 21:14 | |
| corvus | mordred: you've got +2s from me on all those | 21:26 |
| *** dklyle has quit IRC | 21:30 | |
| *** weifan has quit IRC | 21:32 | |
| *** kjackal has quit IRC | 21:33 | |
| mordred | corvus: woot, thanks! | 21:36 |
| *** weifan has joined #openstack-infra | 21:36 | |
| *** rfolco|ruck has quit IRC | 21:40 | |
| mordred | corvus: sigh. new can of worms - I've got a fix for one | 21:45 |
| mordred | corvus: for the other: https://bc47c5785ac3af9d2bfe-4a713343959510ed7518ba6548162809.ssl.cf2.rackcdn.com/689578/2/check/system-config-build-image-gitea-init/f125976/job-output.txt at 2019-10-18 21:10:37.006207 | 21:45 |
| mordred | corvus: any thoughts? | 21:45 |
| fungi | "Get https://registry-1.docker.io/v2/opendevorg/jinja-init/manifests/latest: unauthorized: incorrect username or password" | 21:47 |
| fungi | that i guess | 21:47 |
| *** weifan has quit IRC | 21:48 | |
| mordred | yeah - but we should be getting that through the passthrough buildset registry I would have thought | 21:49 |
| mordred | oh - unless the gitea-init job is not using it? | 21:49 |
| *** jbadiapa has quit IRC | 21:50 | |
| mordred | no - it's using the right base jobs | 21:50 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Several updates because the world is a dark place https://review.opendev.org/689578 | 21:50 |
| mordred | fungi: incidentally - I have discovered that python:slim is now based on buster ^^ as is shown in that patch | 21:50 |
| fungi | hah, nice | 21:50 |
| mordred | also - had to squash 2 previous patches because they each fix half of the problem | 21:50 |
| mordred | touching the image job with the variable caused a rebuild of all of the images, which caused us to try to rebuild python-base - this is such a fun rabbithole | 21:51 |
| fungi | oh, btw i confirmed that we did in fact get successful debian-stretch and debian-buster images built between when i added the debian-security mirror symlink hack (still need to work on transitioning that to a different mirror layout) and when the source dir problem cropped up | 21:51 |
| mordred | cool | 21:52 |
| corvus | mordred: do you have a link to the zuul build for that? | 21:55 |
| corvus | nm, i dug it up | 21:57 |
| corvus | http://zuul.opendev.org/t/openstack/build/f1259767fbfd413aa29ea2c74adcd6b9 | 21:57 |
| *** weifan has joined #openstack-infra | 21:58 | |
| mordred | corvus: yah - that's the one | 21:59 |
| corvus | mordred: https://zuul.opendev.org/t/openstack/build/77c89b61a6c44e04a71ea6bcef735620/log/docker/buildset_proxy.txt#45 | 21:59 |
| corvus | mordred: dockrhub barfed - 500 response code | 22:00 |
| fungi | eww | 22:00 |
| mordred | corvus: yay! | 22:00 |
| corvus | i think after the dust settles on zuul-registry, it will be easy for us to make sure we're using our docker mirror when we go upstream | 22:00 |
| mordred | ++ | 22:01 |
| fungi | does retrying there make sense (or is it even safe)? | 22:01 |
| corvus | (right now, it's possible but gnarly -- after zuul-registry it will be just standard) | 22:01 |
| corvus | fungi: it is retrying - | 22:01 |
| corvus | fungi: oh the playbook | 22:01 |
| corvus | hrm.... i think it'll be easier to just press ahead with zuul registry | 22:01 |
| mordred | ++ | 22:01 |
| corvus | if we change that now, i'm following a moving target | 22:01 |
| fungi | well, i meant the zuul-registry communication with dockerhub, but i likely misunderstood where the 500 response happened | 22:02 |
| mordred | so - that particular job fail should be fixed with the new build - so https://review.opendev.org/#/c/689578 should now have system-config green | 22:02 |
| corvus | fungi: oh, well that isn't using zuul-registry | 22:02 |
| fungi | got it | 22:02 |
| *** weifan has quit IRC | 22:02 | |
| corvus | (right now, that's the old system, which is "attempt to fetch from buildset registry; attempt to fetch from buildset proxy; attempt to fetch from dockerhub with wrong password") | 22:03 |
| fungi | yep, okay | 22:03 |
| corvus | (next system will be "attemp to fetch from buildset registry; attempt to fetch from dockerhub with no password") | 22:03 |
| fungi | i forgot we've only got zuul-registry in use in some places | 22:03 |
| corvus | yeah only the intermediate registry right now | 22:03 |
| corvus | (after that, we'll do "attempt to fetch from buildset registry; attempt to fetch from opendev mirror; attempt to fetch from dockerhub with no password") | 22:04 |
| fungi | that sounds extra robust | 22:04 |
| *** tosky has quit IRC | 22:04 | |
| corvus | then maybe we throw in retries if we're not happy | 22:04 |
| *** rlandy has quit IRC | 22:04 | |
| fungi | sure, but no point in prematurely optimizing that | 22:04 |
| mordred | corvus, fungi: since I had to squash 2 patches and add another change, I'm going to have to bug you for reviews on https://review.opendev.org/#/c/689578 :) | 22:07 |
| mordred | you've seen most of it already | 22:07 |
| corvus | wfm | 22:08 |
| mordred | yay gitea-init build passed this time :) | 22:08 |
| mordred | corvus: also - after this lands, zuul images will be buster based | 22:09 |
| mordred | since it's an update to python-base | 22:09 |
| fungi | so the libjemalloc1->2 bump was from the image moving to buster? | 22:09 |
| mordred | should be same python version though | 22:09 |
| mordred | yup | 22:09 |
| mordred | it was 1 in stretch which is what python:slim was until recently | 22:09 |
| mordred | we don't rebuild python-base very often :) | 22:10 |
| fungi | cool | 22:10 |
| *** dklyle has joined #openstack-infra | 22:10 | |
| fungi | i've been doing some performance experiments on storyboard, and noticed that query times are nearly halved immediately following a mysql restart... any wagers on what could cause that? i was thinking maybe cache memory pressure | 22:11 |
| *** weifan has joined #openstack-infra | 22:11 | |
| fungi | but could also just be cache traversal as the cache gets large, i suppose | 22:12 |
| *** mriedem has quit IRC | 22:13 | |
| mordred | fungi: the only thing that would be different post-restart would be thread-cache | 22:14 |
| mordred | in that there would not be anything in the thread-cache | 22:14 |
| fungi | ahh | 22:14 |
| mordred | perhaps it's worth turning thread-cache down | 22:14 |
| fungi | also something significantly spiked cache memory around 05:35-05:40z today | 22:15 |
| fungi | it was reasonably low after my restart experiments yesterday | 22:15 |
| *** dklyle has quit IRC | 22:15 | |
| fungi | but then suddenly doubled between snmp polls | 22:15 |
| *** david-lyle has joined #openstack-infra | 22:15 | |
| fungi | i was mostly trying to work out if we needed to bump up the instance flavor a notch, whether that would improve performance measurably | 22:16 |
| *** weifan has quit IRC | 22:16 | |
| fungi | can definitely try tuning thread-cache though | 22:18 |
| *** rh-jelabarre has quit IRC | 22:19 | |
| fungi | right now we seem to have thread_cache_size=8 | 22:19 |
| mordred | hrm. | 22:19 |
| mordred | that's the default - and really shouldn't be bad :( | 22:19 |
| fungi | and thread_stack=256K | 22:19 |
| mordred | what do we have max_connections set to? | 22:20 |
| fungi | 151 | 22:20 |
| fungi | i suppose that could be low? | 22:20 |
| fungi | presumably a restart closes all persistent connections | 22:20 |
| mordred | yah. I mean - I tend to argue that max_connections should be set very low on mysql | 22:21 |
| fungi | also query_cache_limit=1M and query_cache_size=16M | 22:21 |
| mordred | because the cost of creating a mysql connection is very low | 22:21 |
| fungi | no idea if those are reasonable, or even relevant | 22:21 |
| mordred | it's tough to say - there are folks who argue against the query cache because on a busy system you can spend more time doing qc invalidations than actually using it | 22:22 |
| fungi | that definitely makes sense | 22:22 |
| mordred | otoh - on systems that are more ready heavy, qc can be a significant win | 22:22 |
| fungi | likely depends on relative performance of the query cache, filesystem cache, and filesystem's underlying block device | 22:23 |
| mordred | since it's caching the entire query result keyed on the query - so it turns queries into memcached like performance | 22:23 |
| *** slaweq has quit IRC | 22:24 | |
| fungi | oh, and key_buffer_size=16M | 22:24 |
| mordred | key_buffer_size shouldn't matter - that's only for myisam index lookups | 22:25 |
| *** weifan has joined #openstack-infra | 22:25 | |
| *** weifan has quit IRC | 22:29 | |
| *** weifan has joined #openstack-infra | 22:42 | |
| *** mattw4 has quit IRC | 22:46 | |
| *** weifan has quit IRC | 22:46 | |
| mordred | ok. so - we're watching the spacewalk livestream on youtube - and it cut to commercial | 22:48 |
| mordred | because we're in japan - it was a japanese commercial | 22:48 |
| mordred | and was LUDICROUSLY ADORABLE cat food commercial starring a kitten | 22:49 |
| mordred | if all commercials were japanese kitten commercials, I'd be much more in favor of commercial television | 22:49 |
| *** weifan has joined #openstack-infra | 22:50 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Several updates because the world is a dark place https://review.opendev.org/689578 | 22:51 |
| mordred | corvus, fungi: AND - had to squash the following patch in as well (because the job change triggers all the container builds, we need the git remote change too) | 22:53 |
| mordred | if y'all are still around and want to review it - cool. if not - since it's literally a squash of 2 +A'd changes, I'm going to +A it if it's green | 22:53 |
| Shrews | mordred: wasn't the spacewalk like 12 hours ago? or is it still going on? | 22:55 |
| fungi | maybe japan exists in the past | 22:56 |
| Shrews | that, or mordred is really Scott Bakula and living an episode of Quantum Leap | 22:58 |
| openstackgerrit | Tim Burke proposed opendev/bindep master: Send `which` stderr to /dev/null, too https://review.opendev.org/689602 | 22:59 |
| fungi | oh boy | 23:00 |
| *** yamamoto has joined #openstack-infra | 23:08 | |
| *** michael-beaver has quit IRC | 23:09 | |
| *** slaweq has joined #openstack-infra | 23:11 | |
| mordred | fungi: thanks! | 23:12 |
| mordred | Shrews: I dunno man - maybe it was - but it at least *seems* to be streaming live currently - so I'll just pretend it's happening now | 23:13 |
| *** yamamoto has quit IRC | 23:13 | |
| Shrews | well, at least there are kittens | 23:14 |
| *** markvoelker has joined #openstack-infra | 23:15 | |
| *** slaweq has quit IRC | 23:15 | |
| fungi | https://vignette.wikia.nocookie.net/fallout/images/2/26/WL_Hero_Pebbles.png/revision/latest?cb=20190923114735 | 23:18 |
| * fungi has a poster of mr. pebbles in his lab | 23:18 | |
| *** markvoelker has quit IRC | 23:19 | |
| *** weifan has quit IRC | 23:45 | |
| *** weifan has joined #openstack-infra | 23:45 | |
| *** weifan has quit IRC | 23:46 | |
| *** weifan has joined #openstack-infra | 23:46 | |
| *** weifan has quit IRC | 23:47 | |
| *** weifan has joined #openstack-infra | 23:47 | |
| *** weifan has quit IRC | 23:47 | |
| *** weifan has joined #openstack-infra | 23:47 | |
| *** weifan has quit IRC | 23:48 | |
| *** weifan has joined #openstack-infra | 23:48 | |
| *** weifan has quit IRC | 23:49 | |
| *** weifan has joined #openstack-infra | 23:49 | |
| *** weifan has quit IRC | 23:50 | |
| *** weifan has joined #openstack-infra | 23:50 | |
| *** weifan has quit IRC | 23:51 | |
| *** weifan has joined #openstack-infra | 23:51 | |
| *** weifan has quit IRC | 23:51 | |
| *** weifan has joined #openstack-infra | 23:51 | |
| *** weifan has quit IRC | 23:52 | |
| *** weifan has joined #openstack-infra | 23:52 | |
| *** weifan has quit IRC | 23:53 | |
| *** weifan has joined #openstack-infra | 23:53 | |
| *** weifan has quit IRC | 23:54 | |
| *** weifan has joined #openstack-infra | 23:54 | |
| *** weifan has quit IRC | 23:55 | |
| *** weifan has joined #openstack-infra | 23:55 | |
| *** weifan has quit IRC | 23:55 | |
| *** weifan has joined #openstack-infra | 23:55 | |
| *** gyee has quit IRC | 23:56 | |
| *** weifan has quit IRC | 23:56 | |
| *** weifan has joined #openstack-infra | 23:56 | |
| *** weifan has quit IRC | 23:57 | |
| *** weifan has joined #openstack-infra | 23:57 | |
| *** weifan has quit IRC | 23:58 | |
| *** weifan has joined #openstack-infra | 23:58 | |
| *** weifan has quit IRC | 23:59 | |
| *** weifan has joined #openstack-infra | 23:59 | |
| *** weifan has quit IRC | 23:59 | |
| *** weifan has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
