| *** ociuhandu has joined #openstack-infra | 00:02 | |
| *** ociuhandu has quit IRC | 00:03 | |
| *** ociuhandu has joined #openstack-infra | 00:04 | |
| *** ociuhandu has quit IRC | 00:09 | |
| *** ijw has joined #openstack-infra | 00:10 | |
| *** ijw has quit IRC | 00:14 | |
| *** dave-mccowan has joined #openstack-infra | 00:25 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Add efi packages for ironic-agent https://review.opendev.org/707195 | 00:28 |
|---|---|---|
| *** ijw has joined #openstack-infra | 00:40 | |
| *** ijw has quit IRC | 00:45 | |
| *** dave-mccowan has quit IRC | 00:48 | |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: nodepool-elements: Use venv for utilities https://review.opendev.org/707750 | 00:53 |
| *** ijw has joined #openstack-infra | 01:12 | |
| *** ricolin has joined #openstack-infra | 01:16 | |
| *** ijw has quit IRC | 01:16 | |
| *** rlandy has joined #openstack-infra | 01:32 | |
| *** rlandy has quit IRC | 01:33 | |
| *** Xuchu has joined #openstack-infra | 01:33 | |
| *** armax has joined #openstack-infra | 01:42 | |
| *** ijw has joined #openstack-infra | 01:43 | |
| *** ijw has quit IRC | 01:48 | |
| *** xinranwang has joined #openstack-infra | 02:09 | |
| *** ijw has joined #openstack-infra | 02:14 | |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: nodepool-elements: Use venv for utilities https://review.opendev.org/707750 | 02:17 |
| *** ijw has quit IRC | 02:18 | |
| *** ociuhandu has joined #openstack-infra | 02:23 | |
| *** ociuhandu has quit IRC | 02:29 | |
| *** Lucas_Gray has quit IRC | 02:30 | |
| *** ijw has joined #openstack-infra | 02:45 | |
| *** ijw has quit IRC | 02:50 | |
| *** ijw has joined #openstack-infra | 03:16 | |
| *** ijw has quit IRC | 03:20 | |
| *** dchen has quit IRC | 03:26 | |
| *** dchen has joined #openstack-infra | 03:39 | |
| *** dchen has quit IRC | 03:39 | |
| *** dchen has joined #openstack-infra | 03:42 | |
| *** goldyfruit_ has quit IRC | 03:42 | |
| *** goldyfruit_ has joined #openstack-infra | 03:42 | |
| *** ijw has joined #openstack-infra | 03:47 | |
| *** ijw has quit IRC | 03:52 | |
| *** ykarel|away is now known as ykarel | 04:05 | |
| *** yamamoto has joined #openstack-infra | 04:09 | |
| *** udesale has joined #openstack-infra | 04:10 | |
| *** yamamoto has quit IRC | 04:14 | |
| *** yamamoto has joined #openstack-infra | 04:16 | |
| *** ijw has joined #openstack-infra | 04:18 | |
| *** ijw has quit IRC | 04:23 | |
| *** ijw has joined #openstack-infra | 04:30 | |
| *** smarcet has joined #openstack-infra | 04:31 | |
| *** smarcet has left #openstack-infra | 04:31 | |
| *** rkukura has quit IRC | 04:31 | |
| *** rkukura has joined #openstack-infra | 04:33 | |
| *** ramishra has joined #openstack-infra | 04:40 | |
| *** yamamoto has quit IRC | 04:41 | |
| *** yamamoto has joined #openstack-infra | 04:42 | |
| *** ijw has quit IRC | 04:43 | |
| *** imacdonn has quit IRC | 04:43 | |
| *** imacdonn has joined #openstack-infra | 04:44 | |
| *** jamesmcarthur has joined #openstack-infra | 04:44 | |
| *** jamesmcarthur has quit IRC | 04:46 | |
| *** jamesmcarthur has joined #openstack-infra | 04:46 | |
| *** ijw has joined #openstack-infra | 04:51 | |
| *** dpawlik has joined #openstack-infra | 05:21 | |
| *** yamamoto has quit IRC | 05:21 | |
| *** yamamoto has joined #openstack-infra | 05:27 | |
| *** ociuhandu has joined #openstack-infra | 05:30 | |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #openstack-infra | 05:34 | |
| *** ijw has quit IRC | 05:34 | |
| *** ociuhandu has quit IRC | 05:35 | |
| *** ijw has joined #openstack-infra | 05:49 | |
| *** ijw has quit IRC | 05:54 | |
| *** ramishra_ has joined #openstack-infra | 06:03 | |
| *** ramishra has quit IRC | 06:05 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Allow mirror-update.opendev.org to send stats https://review.opendev.org/708061 | 06:13 |
| *** lmiccini has joined #openstack-infra | 06:16 | |
| *** ijw has joined #openstack-infra | 06:21 | |
| *** jamesmcarthur has quit IRC | 06:22 | |
| *** ijw has quit IRC | 06:26 | |
| *** dpawlik has quit IRC | 06:31 | |
| *** dpawlik has joined #openstack-infra | 06:31 | |
| *** TristanSullivan has quit IRC | 06:34 | |
| *** dpawlik has quit IRC | 06:42 | |
| *** dpawlik has joined #openstack-infra | 06:48 | |
| *** ijw has joined #openstack-infra | 06:52 | |
| AJaeger | config-core, could you review https://review.opendev.org/707086 , https://review.opendev.org/707210 , https://review.opendev.org/706271 , please? | 06:56 |
| *** ijw has quit IRC | 06:56 | |
| *** AJaeger has quit IRC | 06:57 | |
| *** iurygregory has joined #openstack-infra | 06:57 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Allow mirror-update.opendev.org to send stats https://review.opendev.org/708061 | 06:59 |
| *** ociuhandu has joined #openstack-infra | 07:01 | |
| *** ykarel is now known as ykarel|afk | 07:01 | |
| *** AJaeger has joined #openstack-infra | 07:02 | |
| *** ociuhandu has quit IRC | 07:05 | |
| openstackgerrit | Merged openstack/project-config master: End gating for networking-calico, as retiring project from OpenStack https://review.opendev.org/707086 | 07:10 |
| openstackgerrit | Merged openstack/project-config master: retire coreos-image-builder https://review.opendev.org/707210 | 07:10 |
| openstackgerrit | Merged openstack/project-config master: Update Neutron Grafana dashboard https://review.opendev.org/706271 | 07:10 |
| *** pkopec has joined #openstack-infra | 07:11 | |
| *** ykarel|afk is now known as ykarel | 07:14 | |
| AJaeger | thanks, ianw ! | 07:16 |
| *** ijw has joined #openstack-infra | 07:23 | |
| *** ijw has quit IRC | 07:28 | |
| jrosser | this looks stuck in the check queue for ~40 hours https://review.opendev.org/#/c/707939/ | 07:34 |
| *** pgaxatte has joined #openstack-infra | 07:34 | |
| AJaeger | jrosser: looking at http://zuul.opendev.org/t/openstack/status - it's waiting for a node | 07:37 |
| jrosser | yes, I think it’s been like that all weekend | 07:38 |
| AJaeger | http://zuul.opendev.org/t/openstack/status/change/707939,1 | 07:38 |
| AJaeger | infra-root, any idea? | 07:38 |
| AJaeger | the job that is waiting is openstack-ansible-deploy-aio_proxy-ubuntu-bionic | 07:38 |
| *** Lucas_Gray has joined #openstack-infra | 07:40 | |
| AJaeger | jrosser: you could abandon/restore, rebase etc - or better wait to have an infra-root look up whether there's a problem that needs fixing... | 07:41 |
| *** Jeffrey4l has quit IRC | 07:41 | |
| *** Jeffrey4l has joined #openstack-infra | 07:41 | |
| *** apetrich has joined #openstack-infra | 07:46 | |
| *** Wryhder has joined #openstack-infra | 07:48 | |
| *** Lucas_Gray has quit IRC | 07:49 | |
| *** Wryhder is now known as Lucas_Gray | 07:49 | |
| *** ralonsoh has joined #openstack-infra | 07:52 | |
| *** ijw has joined #openstack-infra | 07:55 | |
| *** slaweq_ has joined #openstack-infra | 07:58 | |
| *** slaweq_ is now known as slaweq | 07:59 | |
| *** ijw has quit IRC | 07:59 | |
| *** ociuhandu has joined #openstack-infra | 08:01 | |
| *** ociuhandu has quit IRC | 08:03 | |
| *** ociuhandu has joined #openstack-infra | 08:04 | |
| *** rpittau|afk is now known as rpittau | 08:10 | |
| *** piotrowskim has joined #openstack-infra | 08:13 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add optional support for circular dependencies https://review.opendev.org/685354 | 08:15 |
| *** tkajinam has quit IRC | 08:15 | |
| *** ociuhandu has quit IRC | 08:16 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally allow zoned executors to process unzoned jobs https://review.opendev.org/673840 | 08:18 |
| *** apetrich has quit IRC | 08:19 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally allow zoned executors to process unzoned jobs https://review.opendev.org/673840 | 08:21 |
| *** lpetrut has joined #openstack-infra | 08:22 | |
| *** ykarel is now known as ykarel|lunch | 08:22 | |
| *** lpetrut has quit IRC | 08:22 | |
| *** lpetrut has joined #openstack-infra | 08:23 | |
| *** arif-ali has joined #openstack-infra | 08:24 | |
| *** ijw has joined #openstack-infra | 08:26 | |
| *** tesseract has joined #openstack-infra | 08:30 | |
| *** ijw has quit IRC | 08:31 | |
| *** ociuhandu has joined #openstack-infra | 08:35 | |
| *** amoralej|off is now known as amoralej | 08:37 | |
| *** ccamacho has joined #openstack-infra | 08:37 | |
| *** Lucas_Gray has quit IRC | 08:41 | |
| *** dtantsur|afk is now known as dtantsur | 08:41 | |
| *** apetrich has joined #openstack-infra | 08:45 | |
| *** ociuhandu has quit IRC | 08:46 | |
| *** ociuhandu has joined #openstack-infra | 08:47 | |
| *** tosky has joined #openstack-infra | 08:51 | |
| *** Lucas_Gray has joined #openstack-infra | 08:52 | |
| *** ociuhandu has quit IRC | 08:52 | |
| *** xek has joined #openstack-infra | 08:52 | |
| *** jpena|off is now known as jpena | 08:52 | |
| *** lucasagomes has joined #openstack-infra | 08:53 | |
| *** ijw has joined #openstack-infra | 08:58 | |
| *** lpetrut has quit IRC | 09:03 | |
| *** ijw has quit IRC | 09:03 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure: remove connectors burden and simplify code https://review.opendev.org/696134 | 09:05 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - Refresh token and recheck sign when token regenerated https://review.opendev.org/698208 | 09:05 |
| *** roman_g has joined #openstack-infra | 09:15 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: A Zuul reporter for Elasticsearch https://review.opendev.org/644927 | 09:15 |
| AJaeger | config-core, please review https://review.opendev.org/706070 https://review.opendev.org/707262 https://review.opendev.org/707682 https://review.opendev.org/706479 | 09:16 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add reference pipelines file for Github driver https://review.opendev.org/672712 | 09:17 |
| frickler | infra-root: that job jrosser mentioned seems to try to use a node that no longer exists http://paste.openstack.org/show/789631/ | 09:18 |
| *** FlorianFa has joined #openstack-infra | 09:18 | |
| frickler | jrosser: I don't think I can to anything to debug this further or resolve it, best do a rebase as AJaeger mentioned | 09:18 |
| jrosser | frickler: sure, i can do that. So it's a zuul side error rather than one in the job setup? | 09:19 |
| jrosser | rebase has kicked it off again | 09:21 |
| AJaeger | jrosser: Yes, Zuul side error - your change looks fine | 09:25 |
| *** ijw has joined #openstack-infra | 09:29 | |
| *** Lucas_Gray has quit IRC | 09:31 | |
| *** derekh has joined #openstack-infra | 09:33 | |
| *** Lucas_Gray has joined #openstack-infra | 09:34 | |
| *** ijw has quit IRC | 09:34 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: URLTrigger driver time based https://review.opendev.org/635567 | 09:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Fix evaluation of range file_comments https://review.opendev.org/708076 | 09:44 |
| *** ociuhandu has joined #openstack-infra | 09:45 | |
| *** ociuhandu has quit IRC | 09:49 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Clarify RequirementError message https://review.opendev.org/707620 | 09:49 |
| noonedeadpunk | fungi: Regarding autohold for https://review.opendev.org/#/c/705222/ I think I've rechecked - can you kindly check if node is ok and place my keys there? | 09:51 |
| noonedeadpunk | also folks, https://review.opendev.org/#/c/707939/ seems to be stuck. Can you somehow cancel it or re-schedule? | 09:53 |
| noonedeadpunk | ah, disregard ^ | 09:53 |
| *** gfidente has joined #openstack-infra | 09:54 | |
| *** udesale_ has joined #openstack-infra | 10:00 | |
| *** udesale_ has quit IRC | 10:01 | |
| *** udesale has quit IRC | 10:01 | |
| *** udesale has joined #openstack-infra | 10:01 | |
| *** ijw has joined #openstack-infra | 10:01 | |
| *** yamamoto has quit IRC | 10:03 | |
| *** ykarel|lunch is now known as ykarel | 10:05 | |
| *** ijw has quit IRC | 10:06 | |
| AJaeger | config-core, https://review.opendev.org/#/c/707360/ is ready to merge - repo is retired. Please review | 10:12 |
| *** ociuhandu has joined #openstack-infra | 10:22 | |
| *** yamamoto has joined #openstack-infra | 10:22 | |
| *** ociuhandu has quit IRC | 10:27 | |
| *** yamamoto has quit IRC | 10:27 | |
| *** ijw has joined #openstack-infra | 10:32 | |
| *** ijw has quit IRC | 10:38 | |
| *** udesale has quit IRC | 10:38 | |
| *** ociuhandu has joined #openstack-infra | 10:44 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack/project-config master: Stop running py2-based jobs for devstack-gate https://review.opendev.org/708092 | 11:00 |
| frickler | config-core: ^^ that is needed to unblock devstack-gate. I'm willing to discuss some of the alternative solutions if you think they should be preferred | 11:01 |
| *** lpetrut has joined #openstack-infra | 11:03 | |
| *** rpittau is now known as rpittau|bbl | 11:03 | |
| *** ijw has joined #openstack-infra | 11:03 | |
| AJaeger | frickler: want to move jobs in-repo? | 11:05 |
| *** roman_g has quit IRC | 11:05 | |
| AJaeger | frickler: now read the last paragraph of the commit message, sorry for asking | 11:05 |
| frickler | AJaeger: I commented on that in the commit message, I don't think it is worth the effort anymore | 11:05 |
| AJaeger | frickler: so, fine with me... | 11:05 |
| *** ijw has quit IRC | 11:08 | |
| openstackgerrit | Merged openstack/project-config master: Remove networking-calico from infrastructure systems https://review.opendev.org/707360 | 11:12 |
| *** ociuhandu has quit IRC | 11:20 | |
| *** Lucas_Gray has quit IRC | 11:24 | |
| *** Lucas_Gray has joined #openstack-infra | 11:31 | |
| *** ijw has joined #openstack-infra | 11:35 | |
| *** Lucas_Gray has quit IRC | 11:37 | |
| *** ijw has quit IRC | 11:40 | |
| *** Lucas_Gray has joined #openstack-infra | 11:43 | |
| *** gfidente has quit IRC | 11:47 | |
| *** ramishra_ has quit IRC | 11:52 | |
| *** ociuhandu has joined #openstack-infra | 11:54 | |
| *** ykarel is now known as ykarel|afk | 11:54 | |
| *** ramishra has joined #openstack-infra | 11:55 | |
| *** ociuhandu has quit IRC | 11:59 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Fix github app authentication to work with checks API endpoints (2) https://review.opendev.org/708109 | 11:59 |
| *** rfolco has joined #openstack-infra | 12:04 | |
| *** dychen has joined #openstack-infra | 12:06 | |
| *** dychen has quit IRC | 12:06 | |
| *** ijw has joined #openstack-infra | 12:07 | |
| *** ijw has quit IRC | 12:12 | |
| *** Lucas_Gray has quit IRC | 12:12 | |
| *** Lucas_Gray has joined #openstack-infra | 12:14 | |
| *** factor has joined #openstack-infra | 12:15 | |
| *** ianychoi has joined #openstack-infra | 12:22 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Report retried builds via sql reporter. https://review.opendev.org/633501 | 12:24 |
| *** udesale has joined #openstack-infra | 12:31 | |
| *** rosmaita has joined #openstack-infra | 12:32 | |
| *** xinranwang has quit IRC | 12:33 | |
| *** jpena is now known as jpena|lunch | 12:35 | |
| *** rpittau|bbl is now known as rpittau | 12:37 | |
| *** jaosorior has joined #openstack-infra | 12:37 | |
| *** ijw has joined #openstack-infra | 12:39 | |
| *** rh-jelabarre has joined #openstack-infra | 12:43 | |
| *** ijw has quit IRC | 12:43 | |
| *** nicolasbock has joined #openstack-infra | 12:47 | |
| *** Lucas_Gray has quit IRC | 12:52 | |
| *** Lucas_Gray has joined #openstack-infra | 12:55 | |
| *** rlandy has joined #openstack-infra | 12:55 | |
| *** ociuhandu has joined #openstack-infra | 13:07 | |
| *** Lucas_Gray has quit IRC | 13:07 | |
| *** ijw has joined #openstack-infra | 13:11 | |
| *** ociuhandu has quit IRC | 13:12 | |
| *** amoralej is now known as amoralej|lunch | 13:12 | |
| *** ijw has quit IRC | 13:15 | |
| *** adriancz has quit IRC | 13:20 | |
| *** gfidente has joined #openstack-infra | 13:25 | |
| *** jpena|lunch is now known as jpena | 13:34 | |
| *** Lucas_Gray has joined #openstack-infra | 13:36 | |
| *** ociuhandu has joined #openstack-infra | 13:37 | |
| *** dave-mccowan has joined #openstack-infra | 13:41 | |
| *** smarcet has joined #openstack-infra | 13:42 | |
| *** ijw has joined #openstack-infra | 13:42 | |
| *** ijw has quit IRC | 13:46 | |
| *** ykarel|afk is now known as ykarel | 13:46 | |
| kevinz | clarkb ianw: Hi there, I'm working on Nova CI on arm64, https://review.opendev.org/#/c/708102/, and define 1 jobs running on it. But it looks that the mirror has some issues there: http://paste.openstack.org/show/789644/ | 13:48 |
| kevinz | it looks some sources in mirror 404 | 13:50 |
| *** amoralej|lunch is now known as amoralej | 13:53 | |
| *** Goneri has joined #openstack-infra | 13:53 | |
| fungi | noonedeadpunk: ssh root@149.202.177.130 | 13:59 |
| noonedeadpunk | fungi: nice, thanks! | 13:59 |
| fungi | yw! | 14:00 |
| fungi | kevinz: your paste is truncated (the lodgeit service there has a limited content size and i assume the error you were trying to show is later in the log) | 14:01 |
| kevinz | fungi: yes, thanks for reminding. I create a new one:http://paste.openstack.org/show/789645/ | 14:03 |
| *** dave-mccowan has quit IRC | 14:06 | |
| fungi | kevinz: interestingly, i can retrieve the files is says are not found, for example http://mirror.london.linaro-london.openstack.org/ubuntu-ports/dists/bionic/main/binary-arm64/Packages | 14:07 |
| *** dave-mccowan has joined #openstack-infra | 14:09 | |
| *** ijw has joined #openstack-infra | 14:13 | |
| *** smarcet has quit IRC | 14:15 | |
| *** lbragstad__ is now known as lbragstad | 14:15 | |
| *** smarcet has joined #openstack-infra | 14:16 | |
| *** ijw has quit IRC | 14:18 | |
| *** lbragstad has quit IRC | 14:21 | |
| *** smarcet has quit IRC | 14:21 | |
| *** eharney has quit IRC | 14:22 | |
| ralonsoh | fungi, https://review.opendev.org/#/c/707715/2/global-requirements.txt | 14:23 |
| ralonsoh | maybe you can help me with this | 14:23 |
| ralonsoh | I don't think this is correct | 14:23 |
| ralonsoh | there is no definition for py3 | 14:23 |
| ralonsoh | only py2 | 14:23 |
| *** smarcet has joined #openstack-infra | 14:24 | |
| mordred | fungi, frickler: if you have a sec, quick/easy patch: https://review.opendev.org/#/c/708023/ | 14:24 |
| mordred | ralonsoh: that seems like a bug | 14:25 |
| ralonsoh | mordred, thanks for the confirmation | 14:25 |
| ralonsoh | I'll propose a patch | 14:25 |
| mordred | ralonsoh: cool | 14:26 |
| *** Lucas_Gray has quit IRC | 14:28 | |
| *** ijw has joined #openstack-infra | 14:31 | |
| *** Lucas_Gray has joined #openstack-infra | 14:32 | |
| AJaeger | config-core, please review https://review.opendev.org/706070 https://review.opendev.org/707262 https://review.opendev.org/707682 and https://review.opendev.org/706479 | 14:37 |
| mordred | config-core: and also https://review.opendev.org/#/c/708023/ please | 14:38 |
| AJaeger | mordred: frickler trades in https://review.opendev.org/708092 for yours ;) | 14:39 |
| mordred | AJaeger, frickler: good trade! | 14:40 |
| AJaeger | ;) | 14:41 |
| AJaeger | thanks | 14:41 |
| openstackgerrit | sebastian marcet proposed opendev/puppet-openstackid master: Updated deployment scripts https://review.opendev.org/708131 | 14:43 |
| kevinz | fungi: wow,,,let me check,,, | 14:45 |
| kevinz | thx | 14:45 |
| openstackgerrit | Merged openstack/project-config master: Add polymer-bridges project from gerrit https://review.opendev.org/708023 | 14:47 |
| openstackgerrit | Merged openstack/project-config master: Update branch checkout and u-c.txt for Octavia https://review.opendev.org/706070 | 14:52 |
| openstackgerrit | Merged openstack/project-config master: Stop running py2-based jobs for devstack-gate https://review.opendev.org/708092 | 14:52 |
| openstackgerrit | Merged openstack/project-config master: Disable Storyboard for airshipctl https://review.opendev.org/707262 | 14:52 |
| *** larainema_ has joined #openstack-infra | 14:54 | |
| *** larainema_ is now known as larainema | 14:55 | |
| *** jhesketh has quit IRC | 14:55 | |
| *** artom has joined #openstack-infra | 14:56 | |
| *** jhesketh has joined #openstack-infra | 14:56 | |
| AJaeger | config-core, next step for static.o.o is https://review.opendev.org/706733 - please review | 14:58 |
| AJaeger | infra-root, please review https://review.opendev.org/#/c/706319/ and https://review.opendev.org/#/c/706319/ - and create AFS volume for this. | 14:59 |
| *** lbragstad has joined #openstack-infra | 15:02 | |
| openstackgerrit | Merged openstack/project-config master: Disable direct releases in js-openstack-lib ACL https://review.opendev.org/707682 | 15:02 |
| openstackgerrit | Merged openstack/project-config master: Fix ACLs for deliverables no longer under RelMgt https://review.opendev.org/706479 | 15:03 |
| *** eharney has joined #openstack-infra | 15:05 | |
| *** gfidente has quit IRC | 15:06 | |
| frickler | AJaeger: that was twice the same review in your last request? | 15:06 |
| *** Lucas_Gray has quit IRC | 15:06 | |
| mordred | frickler: guess we have to review it twice? | 15:07 |
| *** Lucas_Gray has joined #openstack-infra | 15:08 | |
| openstackgerrit | Merged openstack/project-config master: Publish tarballs artifacts to AFS in parallel https://review.opendev.org/706733 | 15:11 |
| openstackgerrit | Merged openstack/project-config master: Add OpenStack ideas promote jobs https://review.opendev.org/706319 | 15:18 |
| *** gfidente has joined #openstack-infra | 15:19 | |
| *** smarcet has quit IRC | 15:20 | |
| *** ykarel is now known as ykarel|away | 15:20 | |
| *** Lucas_Gray has quit IRC | 15:21 | |
| *** Lucas_Gray has joined #openstack-infra | 15:22 | |
| *** smarcet has joined #openstack-infra | 15:22 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add support for polymer-bridges submodule https://review.opendev.org/708018 | 15:22 |
| mordred | corvus: ^^ that should fix the job - once we pulse zuul to pick up the new project | 15:24 |
| *** lpetrut has quit IRC | 15:29 | |
| *** smarcet has quit IRC | 15:29 | |
| *** smarcet has joined #openstack-infra | 15:32 | |
| *** Wryhder has joined #openstack-infra | 15:33 | |
| *** gagehugo has joined #openstack-infra | 15:33 | |
| *** Lucas_Gray has quit IRC | 15:33 | |
| *** Wryhder is now known as Lucas_Gray | 15:34 | |
| mordred | AJaeger, corvus: hrm http://zuul.opendev.org/t/opendev/config-errors | 15:36 |
| *** nicolasbock has quit IRC | 15:38 | |
| mordred | puppet-beaker-rspec-puppet-4-infra is defined in openstack-zuul-jobs - but I don't think we include that repo in the opendev tenant | 15:38 |
| clarkb | good morning. I'm feeling mostly human again. Will try to catch up on reviews and similar backlog today | 15:40 |
| fungi | welcome back! | 15:40 |
| mordred | clarkb: yay! | 15:41 |
| clarkb | I notice that the certchecker has mirror.ord.rax.opendev.org's cert expiring in a couple weeks. But checking it with my browser I'm told expiration is in may | 15:42 |
| clarkb | maybe it is looking at anotherport? or we haven't fully swapped out apache processes? | 15:42 |
| *** ijw has quit IRC | 15:45 | |
| *** ijw has joined #openstack-infra | 15:46 | |
| *** smarcet has quit IRC | 15:46 | |
| *** smarcet has joined #openstack-infra | 15:47 | |
| fungi | my browser says not after march 5 for https://mirror.ord.rax.opendev.org/ | 15:47 |
| fungi | nevermind, after force-refreshing my browser grabbed the new cert, it seems | 15:49 |
| fungi | it must have cached the may 3 expiration? strange if so | 15:49 |
| clarkb | fungi: I wonder if the apache reload action isn't forceful enough? could it be we've got a worker that is hanging around that has loaded the old cert? | 15:50 |
| fungi | you may be correct, following another force refresh it says march 5 again | 15:51 |
| fungi | i've done a `sudo systemctl reload apache2` in case we weren't actually doing it by the last time this cert was renewed | 15:52 |
| clarkb | my understanding was that once all old connections had gone away each worker would be restarted | 15:52 |
| fungi | will refresh a bunch more and see if it persists | 15:52 |
| clarkb | fungi: fwiw I did a reload manually a week ago too | 15:52 |
| clarkb | I'm just wondering if we've got a client out there that is managing to persist an old worker | 15:53 |
| fungi | hrm, the reload didn't help, i just got another march 5 after a few more reloads | 15:53 |
| fungi | and yeah, could be something has a worker tied up | 15:54 |
| * fungi checks process times | 15:54 | |
| fungi | parent apache2 process is 19351 from last year. there's a fork (26712) dated january 13 | 15:55 |
| fungi | the other firks are all from a few minutes ago | 15:56 |
| fungi | er, forks | 15:56 |
| fungi | so your theory may be correct | 15:56 |
| fungi | apache will let us set max worker lifetimes, that may be something to consider | 15:56 |
| fungi | i can also try sending a hangup to 26712 but first i want to strace it | 15:57 |
| fungi | looks like it's blocked on a read() call to fd 15 (whatever that fd is, could be a local file or a tcp socket i suppose) | 15:57 |
| fungi | it's definitely not doing anything, just spinning | 15:58 |
| *** ccamacho has quit IRC | 15:58 | |
| *** ccamacho has joined #openstack-infra | 15:58 | |
| clarkb | we can also just fall back on a restart if this happens infrequently (but I don't think we have enough data on that yet) | 15:59 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add support for polymer-bridges submodule https://review.opendev.org/708018 | 16:00 |
| fungi | i have a hunch it's stuck reading from afs after a burp | 16:00 |
| fungi | but will see if lsof can shed any light | 16:01 |
| *** smarcet has quit IRC | 16:02 | |
| fungi | oof, lsof says it's got a ton of reads on package lists (centos and ubuntu both) | 16:05 |
| *** ccamacho has quit IRC | 16:05 | |
| mordred | clarkb: if you have a sec in your brainhole - we have a config error in the opendev tenant that is really weird to me that maybe there's something I'm missing? | 16:06 |
| mordred | http://zuul.opendev.org/t/opendev/config-errors <-- did we remove openstack-zuul-jobs from opendev tenant recently maybe? | 16:06 |
| openstackgerrit | Merged opendev/puppet-openstackid master: Updated deployment scripts https://review.opendev.org/708131 | 16:08 |
| clarkb | or maybe the error had been there for a while? I think your hunch that it is because we don't include ozj is correct though | 16:08 |
| fungi | okay, the package list reads were ephemeral, they're not showing up now | 16:10 |
| AJaeger | frickler, mordred : Sorry! https://review.opendev.org/706240 was the other one | 16:11 |
| mordred | clarkb: yeah - I don't see how this could have ever worked :) | 16:12 |
| AJaeger | mordred: the config-errors comes from *adding* system-config to opendev AFAIR | 16:13 |
| AJaeger | mordred: we never had openstack-zuul-jobs there | 16:13 |
| mordred | nod. so they've been there all along | 16:13 |
| AJaeger | so, the error has been around for a couple of months | 16:14 |
| mordred | cool | 16:14 |
| mordred | oh - and the thing I was really worried about - why weren't the jobs I cared about running - is because I was looking at the wrong tenant | 16:15 |
| *** ysastri has joined #openstack-infra | 16:15 | |
| AJaeger | mordred: we have system-config configured for multiple tenants, AFAIK plan is to remove it from openstack... We might want to move jobs around. | 16:15 |
| AJaeger | I think ianw was working on that slowly | 16:16 |
| mordred | yeah. I think there's some juggling we have to do for that | 16:16 |
| *** smarcet has joined #openstack-infra | 16:24 | |
| *** Lucas_Gray has quit IRC | 16:26 | |
| *** Lucas_Gray has joined #openstack-infra | 16:28 | |
| *** roman_g has joined #openstack-infra | 16:31 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Add support for polymer-bridges submodule https://review.opendev.org/708018 | 16:39 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Bump bazel version to 2.1.0 https://review.opendev.org/708211 | 16:39 |
| openstackgerrit | Merged zuul/zuul-jobs master: install-docker: option to install docker-compose https://review.opendev.org/707902 | 16:42 |
| openstackgerrit | Merged zuul/zuul-jobs master: fetch-javascript-tarball: introduce zuul_use_fetch_output https://review.opendev.org/681603 | 16:42 |
| openstackgerrit | Merged zuul/zuul-jobs master: Skip test-setup.sh in pep8 jobs https://review.opendev.org/704414 | 16:42 |
| openstackgerrit | Merged zuul/zuul-jobs master: fetch-subunit-output: introduce zuul_use_fetch_output https://review.opendev.org/681882 | 16:42 |
| *** ysastri has quit IRC | 16:43 | |
| *** rpittau is now known as rpittau|afk | 16:54 | |
| *** jamesmcarthur has joined #openstack-infra | 16:57 | |
| *** pgaxatte has quit IRC | 17:02 | |
| *** lucasagomes has quit IRC | 17:02 | |
| *** dtantsur is now known as dtantsur|afk | 17:03 | |
| *** Lucas_Gray has quit IRC | 17:10 | |
| *** lmiccini has quit IRC | 17:15 | |
| *** udesale has quit IRC | 17:17 | |
| *** gfidente has quit IRC | 17:18 | |
| clarkb | roman_g: I'm catching up on things from last week (sorry I got sick). I wanted to know if you had made any progress on your new CI jobs | 17:18 |
| *** ociuhandu_ has joined #openstack-infra | 17:20 | |
| *** ccamacho has joined #openstack-infra | 17:20 | |
| *** jamesmcarthur has quit IRC | 17:21 | |
| *** dpawlik has quit IRC | 17:22 | |
| fungi | doing some snapshot comparisons of tcp sockets returned by lsof for that long-running apache2 fork, the only connections which have been active for over half an hour at the moment look like outbound requests from the mirror (one of which is in close_wait and the other is established) | 17:23 |
| mordred | clarkb: ianw wrote up some good thoughts on the pip/virtualenv stuff in comments on https://review.opendev.org/#/c/707499/ | 17:23 |
| mordred | clarkb: I think his plan there is a. good one | 17:23 |
| *** ociuhandu has quit IRC | 17:24 | |
| clarkb | mordred: thanks. catching up on that is on my todo list | 17:24 |
| *** ociuhandu_ has quit IRC | 17:25 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Make ara-report role to zuul_return an artifact https://review.opendev.org/704045 | 17:27 |
| *** dklyle has joined #openstack-infra | 17:29 | |
| *** factor has quit IRC | 17:30 | |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #openstack-infra | 17:34 | |
| clarkb | fungi: your hunch re afs is probably correct. I wonder if we can simply kill that one child process safely or if we should do a proper service restart | 17:35 |
| fungi | clarkb: well, i'm not so sure it's afs at this point because the files i saw listed didn't hang around | 17:37 |
| fungi | but there's a close_wait connection to instance-48.cloud.fedoraproject.org:https and an established connection to 167.99.233.229:https (something in digital ocean with no reverse dns but is serving a percona website) | 17:38 |
| *** yamamoto has joined #openstack-infra | 17:39 | |
| *** smarcet has quit IRC | 17:49 | |
| *** ociuhandu has joined #openstack-infra | 17:51 | |
| openstackgerrit | Merged opendev/system-config master: Introduce ideas repo redirect https://review.opendev.org/706240 | 17:53 |
| clarkb | fungi: I wonder if we should just kill that process and move on? | 17:54 |
| *** smarcet has joined #openstack-infra | 17:54 | |
| fungi | clarkb: yeah, i feel like it's probably hung trying to proxy some requests which got stuck on a network hiccup | 17:57 |
| fungi | those two connections are still present. my gut says at least one of them has been since january 13 when that fork was spawned | 17:58 |
| fungi | i'll send it a hangup and see what happens | 17:58 |
| clarkb | ok | 17:58 |
| fungi | sighup did nothing. trying sigterm next | 17:59 |
| fungi | also no luck. sigkill i guess. hope the parent process doesn't get confused | 18:00 |
| *** ociuhandu has quit IRC | 18:00 | |
| *** derekh has quit IRC | 18:00 | |
| *** tesseract has quit IRC | 18:02 | |
| fungi | that worked, apache is still serving, and repeated force-refreshes of the site now consistently return may 3 for the cert expiration | 18:02 |
| fungi | so good call on there being a stale worker | 18:02 |
| clarkb | Something to keep an eye on with our certcheck results I guess | 18:03 |
| fungi | looks like if a worker has an outbound proxy connection which never terminates (for whatever reason) then it will continue serving contemporary certs to new requests which get scheduled to it | 18:04 |
| *** SpamapS has quit IRC | 18:05 | |
| *** ociuhandu has joined #openstack-infra | 18:05 | |
| *** icey has quit IRC | 18:05 | |
| *** rcernin has quit IRC | 18:06 | |
| *** mordred has quit IRC | 18:06 | |
| *** icey has joined #openstack-infra | 18:06 | |
| clarkb | hrm we lost mordred /me waits for mordred to return to discuss the pip venv stuff | 18:06 |
| clarkb | fungi: ya because it requires a new process to see the new cert | 18:06 |
| clarkb | reload achives that by telling the old process to recycle themselves | 18:07 |
| *** mordred has joined #openstack-infra | 18:09 | |
| *** jpena is now known as jpena|off | 18:09 | |
| clarkb | mordred: welcome back | 18:09 |
| clarkb | mordred: I've reviewed the comments on that pip/venv change | 18:09 |
| mordred | clarkb: cool | 18:09 |
| clarkb | mordred: want to make sure I understand. Basically install utilities like bindep and os-testr using python3 -m venv based virtuaelnvs on all platforms including centos 7 | 18:09 |
| clarkb | (tox too possibly?) | 18:10 |
| clarkb | then for virtualenv and pip installations move that into the jobs thsemvesl | 18:10 |
| clarkb | *themselves | 18:10 |
| mordred | yeah - with python3 and python3-venv installed via distro packages | 18:10 |
| mordred | well - for tox ... | 18:10 |
| *** armax has quit IRC | 18:11 | |
| mordred | actually - that might work fine for tox too | 18:11 |
| clarkb | ya I think it would. Its just the top level pip that really suffers | 18:11 |
| mordred | the main gotcha is tox+python2 vs tox+python3 that we'll need to think about | 18:11 |
| clarkb | oh right | 18:11 |
| clarkb | because it defaults to what it was installed under | 18:11 |
| mordred | well - we can do the stuff I've got in that element in a base job | 18:11 |
| mordred | and if get-pip and friends are cached on the image- it shouldn't be that costly | 18:12 |
| clarkb | ya | 18:12 |
| mordred | but we could even then avoid doing that role in things like devstack and the osa jobs | 18:12 |
| mordred | maybe having tox installed in python3 is just a thing we need to eat and deal with any fallout | 18:13 |
| clarkb | maybe we can preinstall tox but not add it to $PATH | 18:13 |
| *** SpamapS has joined #openstack-infra | 18:13 | |
| clarkb | and then jobs can opt into using it from that path and avoid the runtime setup cost but accept python3 | 18:13 |
| mordred | clarkb: ++ that's a great idea | 18:13 |
| mordred | and we set a goal at some point of having all of openstack's tox usage be from python3 tox with any python2 tox envs listed explicitly | 18:14 |
| fungi | evrardjp: http://zuul.opendev.org/t/openstack/build/39ec474499674dccb8c700388d949a42 is the build that should have published the change to the https://governance.openstack.org/tc/ sidebar | 18:14 |
| clarkb | mordred: ya. | 18:14 |
| clarkb | mordred: basically point out to people "not being specific is a bug now, and should be fixed" | 18:14 |
| clarkb | mordred: and that can be fixed without us breaking the world first | 18:15 |
| mordred | yup | 18:15 |
| evrardjp | fungi: yes I read this log and didn't find an issue. It even shows that files have changed | 18:15 |
| evrardjp | when I checked the "check" job render, I see ideas correctly | 18:15 |
| mordred | clarkb: we could even make a branch-specific variant of openstack-unittests that does an ensure-tox to install a python2 tox - and have it match stable>=train | 18:16 |
| fungi | evrardjp: oh! i bet there's a delay for the vos release of the afs volume that's published into... checking | 18:16 |
| mordred | clarkb: becuase really for u and forward any tox.ini files should be python3 friendly | 18:16 |
| clarkb | mordred: there might be one or two exceptions like swift, but I imagine we can address those directly | 18:17 |
| mordred | makes the stable tox jobs slightly more expensive at runtime | 18:17 |
| mordred | yah | 18:17 |
| evrardjp | fungi: else, I can dig into the role | 18:17 |
| evrardjp | fungi: yes that was it! | 18:17 |
| evrardjp | Ctrl-F5 now, and you'll see it | 18:18 |
| openstackgerrit | Merged openstack/diskimage-builder master: Allow python3 to be used in Debian https://review.opendev.org/705920 | 18:18 |
| *** armax has joined #openstack-infra | 18:18 | |
| clarkb | mordred: ya I think I'm liking this plan. It should address the tox + python version issue and manages consistency on the utility installation side across platforms | 18:18 |
| evrardjp | sadly the url seems broken. | 18:18 |
| clarkb | mordred: also seems to simplify stuff a bit | 18:18 |
| evrardjp | we'll see the results when https://review.opendev.org/#/c/706320/ will merge | 18:18 |
| fungi | evrardjp: yeah, so promote job uploaded to afs at 18:09:55 but we didn't see the change go live until the next vos release completed around 18:15 | 18:18 |
| evrardjp | awesome. Thanks ! | 18:19 |
| *** rcernin has joined #openstack-infra | 18:20 | |
| mordred | clarkb: yah. that was what I was thinking | 18:23 |
| clarkb | the last thing that may get weird is platforms that have multiple python3s but the previous setup didn't address that either | 18:24 |
| clarkb | and as long as we use a sane default I don't think it matters too much | 18:24 |
| mordred | I agre | 18:24 |
| *** armax has quit IRC | 18:25 | |
| *** armax has joined #openstack-infra | 18:28 | |
| *** yamamoto has quit IRC | 18:30 | |
| *** larainema has quit IRC | 18:30 | |
| AJaeger | mordred: did you see https://review.opendev.org/#/c/707750/ and its dependency as well? | 18:32 |
| *** yamamoto has joined #openstack-infra | 18:33 | |
| *** jamesmcarthur has joined #openstack-infra | 18:37 | |
| *** ramishra has quit IRC | 18:38 | |
| *** ociuhandu has quit IRC | 18:40 | |
| *** ociuhandu has joined #openstack-infra | 18:41 | |
| *** yamamoto has quit IRC | 18:42 | |
| *** ahosam has joined #openstack-infra | 18:42 | |
| *** ociuhandu has quit IRC | 18:47 | |
| *** yamamoto has joined #openstack-infra | 18:51 | |
| *** amoralej is now known as amoralej|off | 18:52 | |
| mordred | I'd seen the depend - but looking at that now | 18:54 |
| *** ralonsoh has quit IRC | 18:55 | |
| *** sshnaidm is now known as sshnaidm|afk | 18:57 | |
| openstackgerrit | Merged openstack/project-config master: Add new airship citycloud region to grafana https://review.opendev.org/706958 | 18:59 |
| *** auristor has quit IRC | 19:00 | |
| *** auristor has joined #openstack-infra | 19:02 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add swift proxy to contrib https://review.opendev.org/708233 | 19:03 |
| *** Tengu has quit IRC | 19:07 | |
| *** Tengu has joined #openstack-infra | 19:09 | |
| *** Tengu has quit IRC | 19:14 | |
| *** Tengu has joined #openstack-infra | 19:15 | |
| *** Tengu has quit IRC | 19:21 | |
| *** Tengu has joined #openstack-infra | 19:22 | |
| *** yamamoto has quit IRC | 19:26 | |
| *** sreejithp has joined #openstack-infra | 19:32 | |
| zbr | hi! i added some question about log collection guidelines on https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting but if there is a quick answer to them, maybe we can avoid using the meeting? | 19:43 |
| *** Tengu has quit IRC | 19:44 | |
| *** Tengu has joined #openstack-infra | 19:46 | |
| *** openstackstatus has quit IRC | 19:49 | |
| *** Tengu has quit IRC | 19:51 | |
| *** Tengu has joined #openstack-infra | 19:55 | |
| fungi | zbr: i believe the intent of that entry about third-party ci reporting was that any systems leaving feedback on patches in our public code review system should use urls which anyone who has access to review the source code (i.e. the general public) should also have a reasonable expectation of viewing relevant logs in their browser | 19:58 |
| fungi | for logs which would never make sense to view in a browser, that's probably not expected anyway | 19:59 |
| fungi | but if the job produces some logs it should likely have at least one you can view in a browser with a reduced/summary output | 19:59 |
| fungi | this was written to counter some of the third-party ci systems which had been leaving links to inaccessible/private log systems, as well as public log servers which failed to provide typical content types (resulting in all logs getting downloaded, even if only small plaintext files) | 20:01 |
| *** jaosorior has quit IRC | 20:02 | |
| *** ociuhandu has joined #openstack-infra | 20:04 | |
| clarkb | ya the major concern was that many people were publishing links on rfc 1918 private networks or using 'localhost' and generally making things unbrowseable | 20:07 |
| clarkb | for the archive stuff, I don't know that we want to police that? | 20:07 |
| zbr | my concerns are of a bit different nature | 20:07 |
| clarkb | I feel like individual projects can set those expectations themselves | 20:07 |
| zbr | some people seem to support creating .tar.gz files with all/most logs from each host, so forcing user to download the archive locally in order to see what is inside | 20:08 |
| zbr | and i am not very happy about this practice | 20:08 |
| clarkb | zbr: a single archive for all logs you mean? | 20:09 |
| zbr | yeah | 20:09 |
| *** ociuhandu has quit IRC | 20:09 | |
| fungi | i still miss the old days when you could ask an ftp server for a tarball of any subtree | 20:09 |
| clarkb | yes, I can see how that would be less useful | 20:09 |
| zbr | on the other hand, I do understand that in certain cases logs could be huge, and keeping them uncompress can cause serious storage issues for them. | 20:09 |
| clarkb | zbr: yup and when you download them in a brwoser often not very useful anyway | 20:10 |
| zbr | can we document a recommented max size for which is would better to gz a log? | 20:10 |
| clarkb | for me I think that is the key thing | 20:10 |
| clarkb | if the log is generally expected to be of size reasonable for a browser to view then making it presentable in that manner is nice | 20:10 |
| clarkb | if it isn't (say the journld serialized logs in tempest jobs) then compressing an archive is fine | 20:11 |
| zbr | my take is: if file > 50MB, gz it. | 20:11 |
| zbr | or a size that we see appropiate | 20:11 |
| clarkb | ya I think that size is about where firefox starts to have a hard time | 20:11 |
| fungi | i don't want to mandate something like that, even for our own ci system, but publishing a recommendation seems like a fine idea | 20:12 |
| clarkb | right, I think requirements for third party ci that go beyond the basics of making logs available should be up to the projects consuming the logs | 20:12 |
| clarkb | but we can provide guidance on what a reasonable experience looks like | 20:12 |
| zbr | is not about mandating, but is about helping other make decent decisions | 20:13 |
| fungi | i'm cool with that, sounds good to me | 20:13 |
| zbr | so 50mb seems ok as a treshold? | 20:13 |
| fungi | i think it depends a lot on what you think a reasonably powerful browser client system can handle. i'd even say 10mb text is a lot to load in a page | 20:14 |
| zbr | I am pefect ok with 10MB too. | 20:14 |
| clarkb | fungi: I want to say that my ancient desktop cpu + firefox can handle about 50MB | 20:14 |
| clarkb | but definitely starts to be unhappy in the 100MB range | 20:14 |
| clarkb | this cpu is so old I went to disable HT and discovered it had no HT | 20:14 |
| clarkb | winning | 20:15 |
| fungi | and the odds that i'll go looking through even a 1mb text output in a web browser are slim, but a lot of that comes down to personal preference | 20:15 |
| zbr | especially as on rdo log servers we do not have compression at storage level, so there is a lot of pressure to gz stuff. | 20:15 |
| clarkb | (its 7 years old I think) | 20:15 |
| clarkb | zbr: fwiw before we switched to swift hosted logs, we tried to compress everything then used server settings to uncompress as necessary for clients | 20:17 |
| clarkb | the goal there was to have a balance between storage requirements and client useability | 20:17 |
| zbr | clarkb: fungi: thanks for the comment. i will propose some clarifications base on that tomorrow, now I am too tired. | 20:17 |
| fungi | get some rest, and have a good night! | 20:18 |
| fungi | and yeah, if i were to redesign that old compression system we used, i'd have sharded logs by date so that the find could operate over a small subset of the tree rather than having to go looking through the entire 12tb corpus of data on every pass | 20:19 |
| * clarkb finds lunch | 20:25 | |
| *** ijw has quit IRC | 20:41 | |
| *** ociuhandu has joined #openstack-infra | 20:42 | |
| *** eharney has quit IRC | 20:43 | |
| *** nicolasbock has joined #openstack-infra | 20:43 | |
| *** ociuhandu has quit IRC | 20:45 | |
| *** ociuhandu has joined #openstack-infra | 20:45 | |
| *** ociuhandu has quit IRC | 20:51 | |
| ianw | AJaeger: thanks for mentioning https://review.opendev.org/706733 ... i will rsync /srv/static over to the afs volume in a little bit, and then, if the publishing is going ok we can look at switching DNS entries for static.openstack.org | 20:59 |
| *** Goneri has quit IRC | 20:59 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Cleanup handlers defined in OS_LOG_DEFAULTS https://review.opendev.org/707942 | 21:05 |
| *** cdearborn has joined #openstack-infra | 21:18 | |
| *** artom has quit IRC | 21:20 | |
| openstackgerrit | Merged opendev/system-config master: Allow mirror-update.opendev.org to send stats https://review.opendev.org/708061 | 21:21 |
| clarkb | ianw: not sure if you saw but I think your approach for the pip/virtualenv stuff makes sense | 21:21 |
| clarkb | ianw: is that something we want to discuss in the meeting tomorrow? | 21:21 |
| * clarkb compiling agenda now | 21:22 | |
| clarkb | oh yup already there, thanks | 21:22 |
| ianw | clarkb: thanks, yeah just put some links in mostly | 21:23 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Avoid gear version 0.15.0 https://review.opendev.org/708259 | 21:33 |
| openstackgerrit | Tobias Henkel proposed opendev/gear master: Revert "Add BSD/Darwin support." https://review.opendev.org/708261 | 21:36 |
| *** eharney has joined #openstack-infra | 21:39 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Don't set OS_LOG_DEFAULTS if unset https://review.opendev.org/707783 | 21:41 |
| *** smarcet has quit IRC | 21:45 | |
| *** xek has quit IRC | 21:45 | |
| *** nicolasbock has quit IRC | 21:46 | |
| *** smarcet has joined #openstack-infra | 21:48 | |
| openstackgerrit | Merged opendev/gear master: Revert "Add BSD/Darwin support." https://review.opendev.org/708261 | 21:51 |
| *** sreejithp has quit IRC | 21:57 | |
| *** ociuhandu has joined #openstack-infra | 22:00 | |
| *** jamesmcarthur has quit IRC | 22:01 | |
| *** jamesmcarthur has joined #openstack-infra | 22:02 | |
| *** Goneri has joined #openstack-infra | 22:04 | |
| *** ociuhandu has quit IRC | 22:06 | |
| *** jamesmcarthur has quit IRC | 22:07 | |
| *** pkopec has quit IRC | 22:15 | |
| *** lennyb has quit IRC | 22:17 | |
| *** slaweq has quit IRC | 22:17 | |
| openstackgerrit | Clark Boylan proposed opendev/gear master: Revert "Revert "Add BSD/Darwin support."" https://review.opendev.org/708267 | 22:20 |
| *** lennyb has joined #openstack-infra | 22:20 | |
| *** smarcet has quit IRC | 22:21 | |
| *** ijw has joined #openstack-infra | 22:25 | |
| *** ijw has quit IRC | 22:29 | |
| *** ijw_ has joined #openstack-infra | 22:29 | |
| openstackgerrit | Clark Boylan proposed opendev/gear master: Revert "Revert "Add BSD/Darwin support."" https://review.opendev.org/708267 | 22:32 |
| clarkb | I'm going to pop out for a bike ride now. Back in a bit | 22:34 |
| *** rh-jelabarre has quit IRC | 22:44 | |
| ianw | i think there's something up with bridge runs, looking | 22:46 |
| ianw | 2001:4800:7817:104:be76:4eff:fe04:829b pings but no ssh ... | 22:46 |
| ianw | console is full of hung task messages | 22:49 |
| ianw | going to hard reboot it | 22:49 |
| ianw | it looks to me etherpad-dev is also dead | 22:54 |
| ianw | same thing on console | 22:55 |
| *** tkajinam has joined #openstack-infra | 22:59 | |
| *** tkajinam has quit IRC | 22:59 | |
| ianw | ok, running again | 22:59 |
| ianw | #status log rebooted logstash-worker15 and etherpad-dev due to hung task kernel errors on console | 23:00 |
| ianw | ... hows about i restart statusbot too :/ | 23:00 |
| *** tkajinam has joined #openstack-infra | 23:01 | |
| *** openstackstatus has joined #openstack-infra | 23:01 | |
| *** ChanServ sets mode: +v openstackstatus | 23:01 | |
| ianw | #status log rebooted logstash-worker15 and etherpad-dev due to hung task kernel errors on console | 23:02 |
| openstackstatus | ianw: finished logging | 23:02 |
| *** smarcet has joined #openstack-infra | 23:03 | |
| *** igordc has joined #openstack-infra | 23:03 | |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** slaweq has quit IRC | 23:16 | |
| openstackgerrit | Merged zuul/zuul master: Avoid gear version 0.15.0 https://review.opendev.org/708259 | 23:25 |
| *** ociuhandu has joined #openstack-infra | 23:30 | |
| *** dychen has joined #openstack-infra | 23:31 | |
| *** ociuhandu has quit IRC | 23:35 | |
| *** tosky has quit IRC | 23:49 | |
| *** goldyfruit_ has quit IRC | 23:51 | |
| *** iokiwi has joined #openstack-infra | 23:52 | |
| *** dychen has quit IRC | 23:53 | |
| *** dychen has joined #openstack-infra | 23:53 | |
| *** artom has joined #openstack-infra | 23:53 | |
| *** smarcet has quit IRC | 23:54 | |
| clarkb | ok back | 23:55 |
| clarkb | ianw: thank you for sorting out ansible | 23:57 |
| ianw | np, ... the actual thing i wanted which was stats flowing from the vos release process is now working :) | 23:57 |
| clarkb | is http://grafana.openstack.org/d/owUHOpwWz/nodepool-airship-citycloud?orgId=1 showing no data for others? I wonder if I got the cloud provider name wrong | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!