| *** goldyfruit_ has joined #openstack-infra | 00:02 | |
| *** Goneri has quit IRC | 00:12 | |
| *** Goneri has joined #openstack-infra | 00:14 | |
| *** jamesmcarthur has quit IRC | 00:24 | |
| *** jamesmcarthur has joined #openstack-infra | 00:25 | |
| *** jamesmcarthur has quit IRC | 00:30 | |
| *** jamesmcarthur has joined #openstack-infra | 00:55 | |
| *** rfolco|bbl has quit IRC | 00:59 | |
| *** jamesmcarthur has quit IRC | 01:02 | |
| *** diablo_rojo has quit IRC | 01:16 | |
| *** jamesmcarthur has joined #openstack-infra | 01:19 | |
| *** jamesmcarthur has quit IRC | 01:25 | |
| *** jamesmcarthur has joined #openstack-infra | 01:26 | |
| *** yamamoto has joined #openstack-infra | 01:33 | |
| *** markvoelker has joined #openstack-infra | 01:36 | |
| *** markvoelker has quit IRC | 01:40 | |
| *** jamesmcarthur has quit IRC | 01:50 | |
| *** jamesmcarthur has joined #openstack-infra | 01:51 | |
| *** jamesmcarthur has quit IRC | 02:00 | |
| *** jamesmcarthur has joined #openstack-infra | 02:00 | |
| *** auristor has quit IRC | 02:21 | |
| *** auristor has joined #openstack-infra | 02:33 | |
| *** jamesmcarthur has quit IRC | 02:33 | |
| *** setuid is now known as zZZZzzetuid | 02:35 | |
| *** jamesmcarthur has joined #openstack-infra | 02:50 | |
| *** ykarel has joined #openstack-infra | 02:56 | |
| *** jamesmcarthur has quit IRC | 03:06 | |
| *** jamesmcarthur has joined #openstack-infra | 03:06 | |
| *** e0ne has joined #openstack-infra | 03:15 | |
| *** jamesmcarthur has quit IRC | 03:17 | |
| *** ricolin_ has joined #openstack-infra | 03:18 | |
| *** e0ne has quit IRC | 03:19 | |
| *** psachin has joined #openstack-infra | 03:26 | |
| *** prometheanfire has quit IRC | 03:30 | |
| *** prometheanfire has joined #openstack-infra | 03:31 | |
| *** ykarel has quit IRC | 03:47 | |
| *** eernst has quit IRC | 04:01 | |
| *** ykarel has joined #openstack-infra | 04:07 | |
| *** udesale has joined #openstack-infra | 04:08 | |
| *** igordc has joined #openstack-infra | 04:19 | |
| *** ramishra has joined #openstack-infra | 04:41 | |
| *** odicha has joined #openstack-infra | 04:53 | |
| *** ramishra has quit IRC | 05:02 | |
| *** surpatil has joined #openstack-infra | 05:12 | |
| *** ramishra has joined #openstack-infra | 05:14 | |
| *** xenos76 has joined #openstack-infra | 05:21 | |
| *** igordc has quit IRC | 05:22 | |
| *** markvoelker has joined #openstack-infra | 05:38 | |
| *** markvoelker has quit IRC | 05:42 | |
| *** igordc has joined #openstack-infra | 05:51 | |
| *** AJaeger has quit IRC | 05:53 | |
| *** AJaeger has joined #openstack-infra | 05:59 | |
| *** whoami-rajat has joined #openstack-infra | 06:02 | |
| *** hwoarang has quit IRC | 06:03 | |
| *** hwoarang has joined #openstack-infra | 06:06 | |
| *** slaweq has joined #openstack-infra | 06:11 | |
| *** hrw has left #openstack-infra | 06:15 | |
| *** pgaxatte has joined #openstack-infra | 06:18 | |
| *** jbadiapa has joined #openstack-infra | 06:20 | |
| *** pcaruana has joined #openstack-infra | 06:22 | |
| *** yamamoto has quit IRC | 06:26 | |
| *** ramishra has quit IRC | 06:34 | |
| *** rakhmerov has joined #openstack-infra | 06:40 | |
| *** ramishra has joined #openstack-infra | 06:43 | |
| *** kopecmartin|off is now known as kopecmartin | 06:53 | |
| *** apetrich has joined #openstack-infra | 06:53 | |
| *** roman_g has joined #openstack-infra | 06:56 | |
| *** jaosorior has joined #openstack-infra | 07:01 | |
| *** xek has joined #openstack-infra | 07:01 | |
| *** ccamacho has joined #openstack-infra | 07:09 | |
| *** jtomasek has joined #openstack-infra | 07:10 | |
| *** tosky has joined #openstack-infra | 07:17 | |
| *** iurygregory has joined #openstack-infra | 07:24 | |
| *** rpittau|afk is now known as rpittau | 07:30 | |
| *** pkopec has joined #openstack-infra | 07:35 | |
| *** fresta has joined #openstack-infra | 07:41 | |
| *** ykarel is now known as ykarel|lunch | 07:44 | |
| *** jpena|off is now known as jpena | 07:45 | |
| *** igordc has quit IRC | 07:50 | |
| *** xenos76 has quit IRC | 08:04 | |
| *** ociuhandu has joined #openstack-infra | 08:10 | |
| *** pcaruana has quit IRC | 08:13 | |
| *** pcaruana has joined #openstack-infra | 08:18 | |
| *** tkajinam has quit IRC | 08:21 | |
| *** ociuhandu has quit IRC | 08:28 | |
| *** ociuhandu has joined #openstack-infra | 08:28 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Include session expired reason in API fetch error message. https://review.opendev.org/686976 | 08:29 |
|---|---|---|
| *** ykarel|lunch is now known as ykarel | 08:32 | |
| *** ociuhandu has quit IRC | 08:33 | |
| *** ociuhandu has joined #openstack-infra | 08:36 | |
| *** derekh has joined #openstack-infra | 08:40 | |
| *** ociuhandu has quit IRC | 08:41 | |
| *** e0ne has joined #openstack-infra | 08:55 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 08:58 |
| *** ihti has joined #openstack-infra | 08:58 | |
| *** zzehring has quit IRC | 09:07 | |
| *** jaosorior has quit IRC | 09:15 | |
| *** tobiash has quit IRC | 09:21 | |
| *** tobiash has joined #openstack-infra | 09:23 | |
| *** ricolin_ is now known as ricolin | 09:29 | |
| *** yamamoto has joined #openstack-infra | 09:30 | |
| *** yamamoto has quit IRC | 09:36 | |
| *** yamamoto has joined #openstack-infra | 09:39 | |
| *** yamamoto has quit IRC | 09:41 | |
| *** yamamoto has joined #openstack-infra | 09:41 | |
| *** whoami-rajat has quit IRC | 09:42 | |
| *** whoami-rajat has joined #openstack-infra | 09:42 | |
| *** yamamoto has quit IRC | 09:43 | |
| *** panda|off is now known as panda | 09:54 | |
| *** xenos76 has joined #openstack-infra | 09:55 | |
| *** rascasoft has quit IRC | 10:04 | |
| *** rascasoft has joined #openstack-infra | 10:05 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Include session expired reason in API fetch error message. https://review.opendev.org/686976 | 10:17 |
| *** yamamoto has joined #openstack-infra | 10:19 | |
| *** derekh has quit IRC | 10:25 | |
| *** gfidente has joined #openstack-infra | 10:27 | |
| *** udesale has quit IRC | 10:36 | |
| *** udesale has joined #openstack-infra | 10:36 | |
| *** yamamoto has quit IRC | 10:39 | |
| *** xarses_ has quit IRC | 10:40 | |
| *** xarses_ has joined #openstack-infra | 10:40 | |
| *** yamamoto has joined #openstack-infra | 10:41 | |
| *** yamamoto has quit IRC | 10:46 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: default to NetworkManager for CentOS and Fedora https://review.opendev.org/686319 | 10:47 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: Use wrappers to call pip for glean install https://review.opendev.org/685643 | 10:47 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Fix networking for CentOS 8 https://review.opendev.org/684513 | 10:47 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add NetworkManager and dhcp-client for CentOS 8 https://review.opendev.org/686090 | 10:47 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: CentOS 8 minimal testing and support https://review.opendev.org/684860 | 10:48 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] use mirrors for functional test builds https://review.opendev.org/685885 | 10:48 |
| *** yamamoto has joined #openstack-infra | 10:49 | |
| *** ihti has quit IRC | 10:52 | |
| *** yamamoto has quit IRC | 10:53 | |
| *** ihti has joined #openstack-infra | 10:53 | |
| *** xenos76 has quit IRC | 11:06 | |
| *** ykarel is now known as ykarel|afk | 11:10 | |
| *** pcaruana has quit IRC | 11:11 | |
| *** udesale has quit IRC | 11:11 | |
| *** derekh has joined #openstack-infra | 11:12 | |
| *** armax has joined #openstack-infra | 11:22 | |
| *** ykarel|afk is now known as ykarel | 11:29 | |
| *** armax has quit IRC | 11:32 | |
| *** jpena is now known as jpena|lunch | 11:33 | |
| *** yamamoto has joined #openstack-infra | 11:37 | |
| *** markvoelker has joined #openstack-infra | 11:40 | |
| *** Lucas_Gray has joined #openstack-infra | 11:41 | |
| *** xenos76 has joined #openstack-infra | 11:42 | |
| *** pcaruana has joined #openstack-infra | 11:42 | |
| *** markvoelker has quit IRC | 11:45 | |
| *** rh-jelabarre has joined #openstack-infra | 11:53 | |
| *** markvoelker has joined #openstack-infra | 11:54 | |
| *** Goneri has quit IRC | 11:58 | |
| *** yamamoto has quit IRC | 12:02 | |
| *** yamamoto has joined #openstack-infra | 12:03 | |
| *** yamamoto has quit IRC | 12:03 | |
| *** yamamoto has joined #openstack-infra | 12:04 | |
| *** rfolco|bbl has joined #openstack-infra | 12:07 | |
| *** Lucas_Gray has quit IRC | 12:07 | |
| *** weshay is now known as weshay|ruck | 12:08 | |
| *** Lucas_Gray has joined #openstack-infra | 12:09 | |
| *** yamamoto has quit IRC | 12:12 | |
| *** rfolco|bbl is now known as rfolco | 12:14 | |
| *** goldyfruit_ has quit IRC | 12:21 | |
| *** jpena|lunch is now known as jpena | 12:31 | |
| *** rlandy has joined #openstack-infra | 12:32 | |
| *** rosmaita has joined #openstack-infra | 12:34 | |
| *** jaosorior has joined #openstack-infra | 12:40 | |
| *** jamesmcarthur has joined #openstack-infra | 12:49 | |
| *** jamesmcarthur has quit IRC | 12:49 | |
| *** jamesmcarthur has joined #openstack-infra | 12:49 | |
| *** ramishra has quit IRC | 12:51 | |
| *** dklyle has quit IRC | 12:57 | |
| *** dklyle has joined #openstack-infra | 12:57 | |
| *** kjackal has quit IRC | 12:58 | |
| *** kjackal has joined #openstack-infra | 12:58 | |
| openstackgerrit | Jan Gutter proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 13:04 |
| *** dave-mccowan has joined #openstack-infra | 13:04 | |
| *** Goneri has joined #openstack-infra | 13:08 | |
| *** jcoufal has joined #openstack-infra | 13:10 | |
| *** jcoufal has quit IRC | 13:12 | |
| *** mriedem has joined #openstack-infra | 13:15 | |
| openstackgerrit | Jan Gutter proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 13:18 |
| *** xek has quit IRC | 13:19 | |
| *** derekh has quit IRC | 13:24 | |
| *** ociuhandu has joined #openstack-infra | 13:25 | |
| *** beekneemech is now known as bnemec | 13:29 | |
| *** ociuhandu has quit IRC | 13:31 | |
| *** goldyfruit_ has joined #openstack-infra | 13:35 | |
| openstackgerrit | Jan Gutter proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 13:38 |
| *** ociuhandu has joined #openstack-infra | 13:40 | |
| *** aaronsheffield has joined #openstack-infra | 13:44 | |
| *** zzehring has joined #openstack-infra | 13:45 | |
| *** ramishra has joined #openstack-infra | 13:47 | |
| *** factor has quit IRC | 13:48 | |
| *** ociuhandu has quit IRC | 13:53 | |
| openstackgerrit | Jan Gutter proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 13:54 |
| *** ociuhandu has joined #openstack-infra | 13:54 | |
| *** jamesdenton has quit IRC | 13:56 | |
| *** jamesdenton has joined #openstack-infra | 13:57 | |
| *** ociuhandu has quit IRC | 13:57 | |
| *** ociuhandu has joined #openstack-infra | 13:57 | |
| *** jamesmcarthur has quit IRC | 14:08 | |
| *** odicha has quit IRC | 14:12 | |
| *** jrosser has quit IRC | 14:14 | |
| *** jrosser has joined #openstack-infra | 14:15 | |
| *** pgaxatte has quit IRC | 14:28 | |
| *** jamesmcarthur has joined #openstack-infra | 14:28 | |
| *** Lucas_Gray has quit IRC | 14:33 | |
| *** hjensas has quit IRC | 14:34 | |
| *** jamesmcarthur has quit IRC | 14:34 | |
| *** jamesmcarthur has joined #openstack-infra | 14:38 | |
| openstackgerrit | Merged zuul/nodepool master: Do not overwrite image upload ZK data on delete https://review.opendev.org/681857 | 14:42 |
| *** jamesmcarthur has quit IRC | 14:43 | |
| *** jamesmcarthur has joined #openstack-infra | 14:43 | |
| *** prometheanfire has quit IRC | 14:43 | |
| *** ykarel is now known as ykarel|afk | 14:44 | |
| *** prometheanfire has joined #openstack-infra | 14:44 | |
| *** diablo_rojo has joined #openstack-infra | 14:54 | |
| *** yamamoto has joined #openstack-infra | 14:56 | |
| *** jamesmcarthur has quit IRC | 15:00 | |
| *** bdodd has joined #openstack-infra | 15:00 | |
| *** cmorpheus is now known as cmurphy | 15:01 | |
| *** yamamoto has quit IRC | 15:01 | |
| *** mriedem is now known as mriedem_afk | 15:03 | |
| openstackgerrit | Thierry Carrez proposed openstack/devstack-gate master: Update grenade settings for stable/train https://review.opendev.org/687072 | 15:03 |
| *** jamesmcarthur has joined #openstack-infra | 15:08 | |
| *** mattw4 has joined #openstack-infra | 15:27 | |
| *** ociuhandu has quit IRC | 15:27 | |
| openstackgerrit | Kerby proposed zuul/nodepool master: AWS driver: add ability to determine AMI id using filters https://review.opendev.org/683205 | 15:27 |
| *** ociuhandu has joined #openstack-infra | 15:27 | |
| *** ociuhandu has quit IRC | 15:31 | |
| *** ociuhandu has joined #openstack-infra | 15:34 | |
| *** factor has joined #openstack-infra | 15:35 | |
| *** ykarel|afk is now known as ykarel|away | 15:38 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 15:38 |
| Shrews | infra-root: fyi, might be a linaro mirror problem. debian-buster builds are continually failing because it cannot d/l packages. ex) https://nb03.openstack.org/debian-buster-arm64-0000051226.log | 15:40 |
| clarkb | Shrews: thats a buster mirror problem | 15:40 |
| clarkb | reprepro does not seem to be building a valid index that matches the packages in the repo | 15:41 |
| clarkb | fungi and I have poked at it a bit but have not yet determined why it happens (the whole reason we use reprepro is to avoid this problem) | 15:41 |
| Shrews | infra-root: any objections to restarting all 3 nodepool builders? I want to get the image leak fix in | 15:42 |
| mordred | infra-root: morning! I'm here for a couple of hours today - then am *mostly* out until friday because apparently it's important for me to go to a meeting at my employer and I get the feeling it's one of those where they'll be grumpy if I'm laptoping | 15:42 |
| fungi | mordred: i would say "enjoy" but that's probably optimistic | 15:43 |
| clarkb | Shrews: no objection from me | 15:43 |
| *** ykarel|away has quit IRC | 15:43 | |
| fungi | Shrews: wfm, thanks! | 15:43 |
| mordred | fungi: I think I'm going to define "enjoy" in this case as "continue existing" | 15:44 |
| *** surpatil has quit IRC | 15:45 | |
| *** markvoelker has quit IRC | 15:45 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 15:46 |
| fungi | mordred: enjoyment is likely predicated on continuing to exist, so that's reasonable | 15:47 |
| *** ociuhandu has quit IRC | 15:48 | |
| Shrews | #status log Restarted nodepool-builder on nb01, nb02, and nb03 to pick up image leak fix | 15:48 |
| openstackstatus | Shrews: finished logging | 15:48 |
| *** ociuhandu has joined #openstack-infra | 15:49 | |
| Shrews | mordred: you are suggesting that there will be no grumpiness if you *aren't* laptoping | 15:49 |
| mordred | fungi: I've heard that ego annihilation can also be a desired state, but something tells me I won't achieve that this week | 15:49 |
| mordred | Shrews: I'm not sure the contrapositive is valid in this case | 15:50 |
| *** ramishra has quit IRC | 15:52 | |
| *** ociuhandu has quit IRC | 15:53 | |
| corvus | i'm going to continue work on registry shadowing and connecting zuul's zuul to gerrit's gerrit. | 15:56 |
| *** chandankumar is now known as raukadah | 15:57 | |
| mordred | corvus: ++ | 15:58 |
| *** markvoelker has joined #openstack-infra | 16:00 | |
| *** rpittau is now known as rpittau|adk | 16:02 | |
| *** rpittau|adk is now known as rpittau|afk | 16:02 | |
| mordred | clarkb, fungi: I've got this up https://review.opendev.org/#/c/683125/ as part of the gerrit images stack - but I guess I have a question as to whether this is a thing we *want*? I can make arguments in my head both ways | 16:02 |
| *** ykarel has joined #openstack-infra | 16:03 | |
| clarkb | mordred: I've happily been using sudo on the gitea nodes | 16:04 |
| clarkb | I would say that having the sudo logs of actions is a nice thing | 16:04 |
| fungi | i've just about finished unwinding the automation we currently rely on for creating storyboard projects and project-groups from our projects.yaml content, and it's unfortunate... puppet exec runs a utility from the storyboard repo to directly manipulate database records. so the the bad news is that team management automation is going to require a lot more ground-up effort to do it properly through | 16:05 |
| fungi | the api/sdk, but i guess there's a silver lining that we could migrate project and project-group creation to it and deprecate the old tool and the related puppet exec | 16:05 |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 16:05 |
| fungi | my cocktail napkin design is to have ansible ingest the projects.yaml and then invoke an sdk-based script or maybe hit the rest api directly... is that sane? | 16:06 |
| mordred | clarkb: nod. kk. that was one of my internal arguments - I'll go ahead and abandon it and just keep with the sudo | 16:07 |
| mordred | fungi: yes - I think that's a very good design | 16:08 |
| *** goldyfruit___ has joined #openstack-infra | 16:09 | |
| corvus | fungi: boartty may have code you can steal | 16:09 |
| mordred | fungi: although - based on the gitea work - we might wind up having a custom ansible module for it so that we can do it all in one process - rather than a bazillion | 16:09 |
| fungi | corvus: oh! good point, i hadn't thought of that | 16:09 |
| fungi | mordred: so rather than have ansible parse the projects.yaml, just use our parser from jeepyb or something along those lines? | 16:10 |
| mordred | or - I suppose that's what you were getting at by sdk-based-script - you should be able to crib module boilerplate stuff from the gitea role | 16:10 |
| mordred | fungi: I think having ansible parse the yaml is a fine idea - you could have it pass the parsed yaml dict to the module as a parameter | 16:11 |
| *** goldyfruit_ has quit IRC | 16:11 | |
| *** apetrich has quit IRC | 16:12 | |
| fungi | ahh, okay | 16:13 |
| fungi | and yeah, thanks for the gitea module/role pointer | 16:13 |
| *** e0ne has quit IRC | 16:17 | |
| *** apetrich has joined #openstack-infra | 16:17 | |
| *** iurygregory has quit IRC | 16:17 | |
| *** ociuhandu has joined #openstack-infra | 16:18 | |
| *** ociuhandu_ has joined #openstack-infra | 16:20 | |
| openstackgerrit | Jan Gutter proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 16:21 |
| *** gyee has joined #openstack-infra | 16:22 | |
| *** gyee_ has joined #openstack-infra | 16:22 | |
| *** gyee_ has quit IRC | 16:23 | |
| *** gyee has quit IRC | 16:23 | |
| *** gyee has joined #openstack-infra | 16:23 | |
| *** ociuhandu has quit IRC | 16:23 | |
| *** ociuhandu_ has quit IRC | 16:25 | |
| *** ykarel is now known as ykarel|away | 16:27 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 16:30 |
| *** trident has quit IRC | 16:33 | |
| *** ociuhandu has joined #openstack-infra | 16:36 | |
| *** kjackal has quit IRC | 16:37 | |
| *** trident has joined #openstack-infra | 16:37 | |
| *** ociuhandu has quit IRC | 16:41 | |
| *** mriedem_afk is now known as mriedem | 16:47 | |
| *** ociuhandu has joined #openstack-infra | 16:54 | |
| *** mgoddard has quit IRC | 16:57 | |
| *** mgoddard has joined #openstack-infra | 17:02 | |
| *** jamesmcarthur has quit IRC | 17:06 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 17:06 |
| openstackgerrit | Ghanshyam Mann proposed openstack/devstack-gate master: Update grenade settings for stable/train https://review.opendev.org/686774 | 17:08 |
| *** jpena is now known as jpena|off | 17:11 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 17:16 |
| openstackgerrit | Gabor Lekeny proposed zuul/zuul master: Decode k8s ServiceAccount bearer token https://review.opendev.org/687107 | 17:17 |
| *** pcaruana has quit IRC | 17:21 | |
| *** psachin has quit IRC | 17:22 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: WIP implement shadow proxying https://review.opendev.org/687113 | 17:25 |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 17:26 |
| *** jamesmcarthur has joined #openstack-infra | 17:31 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add type annotations https://review.opendev.org/686249 | 17:36 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add hypothesis based test for the storage module https://review.opendev.org/687118 | 17:36 |
| *** bdodd has quit IRC | 17:39 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add hypothesis based test for the storage module https://review.opendev.org/687118 | 17:40 |
| openstackgerrit | Ghanshyam Mann proposed openstack/openstack-zuul-jobs master: Add stable/train to periodic-stable templates https://review.opendev.org/687122 | 17:44 |
| *** pcaruana has joined #openstack-infra | 18:00 | |
| *** ociuhandu has quit IRC | 18:01 | |
| *** ociuhandu has joined #openstack-infra | 18:02 | |
| *** goldyfruit___ has quit IRC | 18:03 | |
| *** goldyfruit has joined #openstack-infra | 18:06 | |
| *** ociuhandu has quit IRC | 18:06 | |
| *** woojay has left #openstack-infra | 18:14 | |
| *** ociuhandu has joined #openstack-infra | 18:16 | |
| corvus | fungi, clarkb, mordred, tristanC: from an ops pov, do you think we could switch out our use of docker-compose with something like podman-compose fairly easily? | 18:17 |
| clarkb | is podman easily installable on not rhel/fedora/centos? | 18:17 |
| clarkb | that is likely the biggest hurdle? | 18:17 |
| tristanC | corvus: i haven't tried podman-compose, can give it a try for the zuul-quickstart | 18:18 |
| corvus | tristanC: that would be great -- if it works there, it's probably not too hard to use it on our prod systems | 18:18 |
| corvus | clarkb: i think the projectatomic ppa may have it? | 18:18 |
| tristanC | clarkb: buildah/podman are fairly simple to install, it's just an userland tool afterall | 18:18 |
| clarkb | ya looks like you have to use the ppa which provides development versions | 18:19 |
| clarkb | a bunch of other distros provide a more stable podman install | 18:19 |
| clarkb | (at least according to their docs) | 18:20 |
| corvus | so the *good* news is it probably has that multiple-regisry mirror support :) | 18:20 |
| *** ociuhandu has quit IRC | 18:20 | |
| *** ricolin has quit IRC | 18:21 | |
| *** ociuhandu has joined #openstack-infra | 18:21 | |
| tristanC | corvus: oh yes, there is a convenient /etc/containers/registries.d/ directory where you can configure registries | 18:21 |
| clarkb | podman-compose like docker-compose is on pypi | 18:22 |
| clarkb | is podman itself relatively stable at this point? | 18:22 |
| clarkb | I know that tripleo had a lot of problems with it not too long ago. I suppose if it is what they shipped in rhel8 then it probably is stable | 18:23 |
| corvus | basically, what i'm wondering is, a) can opendev move off of docker and onto something based on the containers/ tools and, if so, b) is that enough of a demonstration to convince us that we can we refocus the zuul-registry work on supporting just that part of the ecosystem. | 18:23 |
| *** ociuhandu has quit IRC | 18:23 | |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 18:23 |
| clarkb | I think if your intent is to support this as a generic zuul tool then you should not drop support for docker | 18:24 |
| *** ociuhandu has joined #openstack-infra | 18:24 | |
| clarkb | for better or worse docker is the tool most people are going to start with when they spin up zuul in that space I think | 18:24 |
| clarkb | and many are going to have preexisting tooling around it | 18:24 |
| corvus | to be clear, i expect jobs to continue to support building and pushing images to dockerhub until docker runs out of money | 18:25 |
| corvus | but do the image build and test jobs need to support the docker runtime? | 18:25 |
| corvus | oh, a related question -- k8s still uses the docker runtime for image pulls, right? | 18:26 |
| clarkb | I think that there will be people out there that want to test image builds with docker and zuul should likely support that | 18:27 |
| tristanC | corvus: if you use containerd or cri-o through CRI, k8s doesn't use or need docker | 18:27 |
| *** e0ne has joined #openstack-infra | 18:28 | |
| corvus | tristanC: are you sure it doesn't use the same moby code to pull images? | 18:28 |
| clarkb | docker and podman are different code bases, if what you have in production is tied to docker you want to test that docker works for you with its set of bugs and behaviors | 18:28 |
| clarkb | opendev can probably switch without too much trouble but that isn't goign to be true of everyone, nor will people want their CI system to dictate what they can run in production | 18:29 |
| *** ociuhandu has quit IRC | 18:29 | |
| *** ociuhandu has joined #openstack-infra | 18:29 | |
| tristanC | corvus: i don't know where the pull code is coming from though, github.com/docker/docker is listed in cri-o go.mod | 18:30 |
| mordred | corvus: yes, I think we should take a stab at moving opendev to podman and friends | 18:31 |
| mordred | like, I think it's worth trying | 18:31 |
| mordred | clarkb: I agree - jobs should still be able to use docker and have it work - I think broadly speaking zuul's container support should be friendly to both docker and non-docker | 18:32 |
| clarkb | ya separately we can choose to use different tools because they make our lives easier, I just don't expect everyone to be able to make that decision | 18:33 |
| *** gyee has quit IRC | 18:33 | |
| mordred | corvus: I agree with tristanC - I do not know enough about the cri-o backend to know it's using the moby code or not - although I'd imagine with the folks working on cri-o that there's also a decent chance it might be using whatever skopeo is using | 18:33 |
| *** ociuhandu has quit IRC | 18:34 | |
| mordred | clarkb: that said - the zuul image jobs themselves have people point to a Dockerfile - so the argument could be made that we could just have those jobs use something other than docker to build the images with teh Dockerfile - if all someone wants is "I have this dockerfile and this context, please to give me an image" | 18:35 |
| clarkb | it is interesting that containers/ is happy to change behavior here when it comes to registries but not when it comes to ipv6 addresses :/ | 18:35 |
| clarkb | personally I'd be a whole lot more interested in podman if the line we got re ipv6 was "we have to maintain bug compat" | 18:35 |
| clarkb | *if the line wasn't | 18:35 |
| mordred | yeah, I agree - that response was pretty disappointing | 18:36 |
| mordred | but - you know - if buildah can do a better job of building images for us and the general zuul user has no idea that this happened - awesome | 18:37 |
| *** ociuhandu has joined #openstack-infra | 18:39 | |
| *** mriedem has quit IRC | 18:40 | |
| *** mriedem has joined #openstack-infra | 18:40 | |
| fungi | looks like the work to get podman into debian (from where it would likely trickle into a future ubuntu) is now only blocked on packaging conmon and some cni bits: https://wiki.debian.org/Podman | 18:43 |
| mordred | cool | 18:43 |
| fungi | https://bugs.debian.org/930440 | 18:44 |
| openstack | Debian bug 930440 in wnpp "ITP: podman -- Library and tool for running OCI-based containers in Pods" [Wishlist,Open] | 18:44 |
| fungi | and for buildah: https://bugs.debian.org/928083 | 18:47 |
| openstack | Debian bug 928083 in wnpp "ITP: golang-github-containers-buildah -- A tool that facilitates building OCI images" [Wishlist,Open] | 18:47 |
| fungi | seems there was some progress before the freeze for buster, so likely will be resuming now that the dust has settled | 18:49 |
| *** ociuhandu has quit IRC | 18:49 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: quick-start: ensure apt-get update is ran non-interactively https://review.opendev.org/687134 | 18:51 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Replace docker by podman for quick-start https://review.opendev.org/687135 | 18:51 |
| AJaeger | corvus: cri-o uses runC as Docker does | 18:56 |
| mordred | AJaeger: that's for running - do we know if it uses the same thing for pulling/fetching images? | 18:58 |
| AJaeger | mordred: I think for pulling/fetching it's different - but don't trust me ;) | 18:59 |
| openstackgerrit | Tim Burke proposed zuul/zuul-registry master: Rework the stream_blob/stream_object API https://review.opendev.org/686827 | 19:00 |
| AJaeger | config-core, anybody feeling strongly on https://review.opendev.org/683028 - it has one -1 by mnaser. It's the "Idenetify voting bots" gerrit ACL | 19:02 |
| clarkb | AJaeger: I think mnaser makes a good point | 19:05 |
| clarkb | its totally fine config and a project can opt to have it | 19:05 |
| clarkb | however, if the governing body of that project thinks it is a bad idea I'm not sure we should do it | 19:06 |
| openstackgerrit | Clark Boylan proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 19:07 |
| corvus | as an "experiment" i think we already have the results: adam and others already acknowledged mis-clicking the field themselves. | 19:08 |
| AJaeger | clarkb: the governing body gave one +1 and one -1 ;) | 19:10 |
| AJaeger | I suggested in the review changing the order, going from "backport" to "Stable-Backport" so that it's after "Review"... | 19:11 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add hypothesis based test for the storage module https://review.opendev.org/687118 | 19:12 |
| fungi | i thought part of the assertion was that these presumed bots were using whichever field appeared first, but maybe i'm misremembering | 19:12 |
| fungi | or are you suggesting that's a way to stop having to care about the bots mistakenly setting the backports vote and polluting its purpose? | 19:13 |
| AJaeger | the later | 19:13 |
| clarkb | fungi: ya I think that is what AJaeger is saying. If you move the backports after code review then you avoid pollution of that field | 19:14 |
| fungi | yeah, just realized i should read the comment he put on that review instead of asking hare-brained questions in here ;) | 19:14 |
| fungi | sorry | 19:14 |
| *** spartakos has joined #openstack-infra | 19:16 | |
| AJaeger | fungi: sorry for not beeing clear | 19:16 |
| * AJaeger updated the change and will ignore it now | 19:17 | |
| *** gyee has joined #openstack-infra | 19:18 | |
| *** pcaruana has quit IRC | 19:32 | |
| openstackgerrit | Clark Boylan proposed zuul/nodepool master: Use real uuids in fake cloud resource IDs https://review.opendev.org/687144 | 19:34 |
| *** ykarel|away has quit IRC | 19:34 | |
| openstackgerrit | Tim Burke proposed zuul/zuul-registry master: Rework the stream_blob/stream_object API https://review.opendev.org/686827 | 19:41 |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: WIP: experimenting with using ZK for fake driver https://review.opendev.org/687150 | 19:49 |
| *** spartakos has quit IRC | 19:50 | |
| *** zZZZzzetuid is now known as setuid | 19:58 | |
| *** xek has joined #openstack-infra | 19:58 | |
| *** spartakos has joined #openstack-infra | 19:59 | |
| *** spartakos has quit IRC | 20:02 | |
| *** otherwiseguy_ has quit IRC | 20:02 | |
| openstackgerrit | Merged zuul/zuul-registry master: Rework the stream_blob/stream_object API https://review.opendev.org/686827 | 20:09 |
| *** kjackal has joined #openstack-infra | 20:13 | |
| *** igordc has joined #openstack-infra | 20:13 | |
| *** jamesmcarthur has quit IRC | 20:19 | |
| *** kjackal has quit IRC | 20:25 | |
| *** xenos76 has quit IRC | 20:26 | |
| *** xenos76 has joined #openstack-infra | 20:31 | |
| *** adriant has quit IRC | 20:32 | |
| *** slaweq has quit IRC | 20:33 | |
| *** iokiwi has quit IRC | 20:33 | |
| clarkb | infra-root can we get https://review.opendev.org/#/c/686775/ in so that we don't have to remember to do that by hand in the future? | 20:38 |
| clarkb | fungi: ^ I think that may be you for reviews | 20:38 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add hypothesis rule based state machine test and fix path issue https://review.opendev.org/687157 | 20:40 |
| *** xenos76 has quit IRC | 20:51 | |
| *** tonyb has joined #openstack-infra | 20:52 | |
| *** anteaya has joined #openstack-infra | 20:53 | |
| *** Goneri has quit IRC | 20:55 | |
| *** diablo_rojo has quit IRC | 20:56 | |
| *** iokiwi has joined #openstack-infra | 20:57 | |
| *** adriant has joined #openstack-infra | 20:57 | |
| *** yamamoto has joined #openstack-infra | 20:59 | |
| *** yamamoto has quit IRC | 21:04 | |
| openstackgerrit | Merged opendev/system-config master: Update zuul ansible installs when restarting zuul https://review.opendev.org/686775 | 21:05 |
| *** goldyfruit_ has joined #openstack-infra | 21:05 | |
| *** goldyfruit has quit IRC | 21:08 | |
| *** xek has quit IRC | 21:18 | |
| *** goldyfruit_ has quit IRC | 21:22 | |
| *** markvoelker has quit IRC | 21:29 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add hypothesis based test for the storage module https://review.opendev.org/687118 | 21:43 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add hypothesis rule based state machine test and fix path issue https://review.opendev.org/687157 | 21:43 |
| *** jtomasek has quit IRC | 21:54 | |
| *** jtomasek has joined #openstack-infra | 21:55 | |
| *** jtomasek has quit IRC | 21:55 | |
| *** smarcet has joined #openstack-infra | 21:56 | |
| *** whoami-rajat has quit IRC | 22:02 | |
| *** trident has quit IRC | 22:03 | |
| openstackgerrit | Goutham Pacha Ravi proposed opendev/irc-meetings master: Add gouthamr as co-chair for manila meetings https://review.opendev.org/687171 | 22:03 |
| *** trident has joined #openstack-infra | 22:04 | |
| *** goldyfruit_ has joined #openstack-infra | 22:10 | |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** markvoelker has joined #openstack-infra | 22:11 | |
| clarkb | I'm catching up on meeting agenda items. Can someone help me understand the CNAME entry at https://etherpad.openstack.org/p/openstack-org-dns ? | 22:16 |
| *** slaweq has quit IRC | 22:16 | |
| clarkb | is the idea that we'll set up dns to be eg etherpad.openstack.org CNAME etherpad.opendev.org CNAME etherpad01.opendev.org A someIP ? | 22:17 |
| clarkb | then delegate acme authorization to acme.opendev.org for etherpad.openstack.org ? | 22:17 |
| openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: CentOS 8 minimal testing and support https://review.opendev.org/684860 | 22:18 |
| fungi | clarkb: no need to delegate, just cname that as well | 22:18 |
| clarkb | ya delegate was probably a bad term, but basically thats a cname similar to what we do for foo.openstack.org | 22:18 |
| clarkb | did I get the first bit right re CNAMEs? | 22:18 |
| fungi | acme-challenge.etherpad.openstack.org as a cname to acme.opendev.org or whatever | 22:18 |
| fungi | clarkb: yep | 22:18 |
| clarkb | and the benefit of there is we only have to create a single record per service? | 22:19 |
| fungi | the result would then allow us to obtain le certs for etherpad.openstack.org without our automation touching the openstack.org dns | 22:19 |
| clarkb | well you have to create two CNAMEs | 22:19 |
| clarkb | but I guess people are ok with that? | 22:19 |
| fungi | right, manually | 22:19 |
| clarkb | also why is it OSF doing that? | 22:20 |
| fungi | and odds are we won't be stuffing new opendev services into openstack.org so could just do them up front | 22:20 |
| clarkb | (I don't ask eg jimmy to edit dns for us for example) | 22:20 |
| fungi | clarkb: "osf" there was shorthand for "not the opendev sysadmins but maybe the openstack infra team" | 22:20 |
| fungi | whoever is administrating the openstack.org zone | 22:21 |
| fungi | our automation doesn't need to know/care about how that dns is managed | 22:22 |
| clarkb | ok so we haven't been asked to stop making edits to that zone | 22:22 |
| fungi | nope | 22:22 |
| *** markvoelker has quit IRC | 22:22 | |
| fungi | this came up while you'd gone fishin' | 22:22 |
| fungi | i reignited the static.o.o migration discussion during thursday tc office hour since wednesday office hour is kinda dead | 22:23 |
| clarkb | k. Just making sure I didn't miss any logistical changes that ended up being important | 22:23 |
| fungi | and the discussion worked its way around to the challenges (technically, logistically and ideologically) of maintaining ssl for stuff like docs.openstack.org and why it might be nice for that to become a redirect to docs.opendev.org/openstack or whatever | 22:25 |
| clarkb | tristanC: did you see my comment on https://review.opendev.org/#/c/686474/ ? | 22:25 |
| fungi | so the plan in that etherpad was a compromise to give us the ability to do certs for other domains opendev doesn't control but hosts sites/services for | 22:26 |
| fungi | independent of the system used to host those domains | 22:26 |
| *** e0ne has quit IRC | 22:26 | |
| openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: Add CentOS 8 support https://review.opendev.org/684308 | 22:27 |
| *** pkopec has quit IRC | 22:27 | |
| *** mriedem has quit IRC | 22:27 | |
| fungi | basically if whoever controls those domains can add appropriate cnames, then we can do our le certs without interacting with whatever dns hosting is involved on that side | 22:27 |
| clarkb | and if we also cname the service itself we can move it around too | 22:28 |
| fungi | yup | 22:28 |
| fungi | but of course reminding folks that we're also happy to host dns in our code-review-driven system if they're up for that | 22:28 |
| clarkb | tristanC: can you see my comment on https://review.opendev.org/#/c/686749/5 too? I think these are both very close if we can clean up these last few things and double check the changes we are making are correct | 22:30 |
| *** smarcet has quit IRC | 22:40 | |
| openstackgerrit | Clark Boylan proposed opendev/glean master: Import ipv6 configuration done by cloud-init for Red Hat distro https://review.opendev.org/686749 | 22:49 |
| clarkb | tristanC: ^ that should fix the CI issue | 22:49 |
| clarkb | (it was the args.distro vs distro variable name) | 22:49 |
| tristanC | clarkb: oh, i was about to submit that change.. thanks! | 22:50 |
| tristanC | clarkb: i also replied to your comments | 22:50 |
| clarkb | thanks (I don't want this to get forgotten as stable red hat distros on all the clouds would be nice to have) | 22:50 |
| clarkb | if this passes CI I think the next thing we should do is build a test image and boot it in FN and limestone and some non ipv6 only cloud and do some real world checks | 22:52 |
| clarkb | I can help with that | 22:52 |
| tristanC | clarkb: iiuc we use glean to avoid cloudinit deps, but if we stop doing pip/tox in dib, could we switch back to cloud-init? | 22:53 |
| tristanC | or i'm not sure what is the reason of glean in the first place | 22:54 |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 22:54 |
| donnyd | Anything I can help with on this? | 22:54 |
| clarkb | tristanC: the original reason was due to cloud init not supporting rackspace network configuration | 22:55 |
| clarkb | tristanC: If cloud init supports rackspace now then I think we can consider switching back, however the deps issue is one we'll likely need to do testing around | 22:56 |
| clarkb | tristanC: we would also want to ship a cloud init config that severely reduced what cloud init does by default | 22:56 |
| clarkb | (we dno't want different users configured on every different distro for example, we also don't want cloud init formating and mounting volumes, etc) | 22:56 |
| clarkb | donnyd: no we need to update how our images are configured to try and force network manager to manage an interface even if the kernel configures it first via an RA. that is what the change above hopes to do | 22:57 |
| clarkb | tristanC: also not doing pip/tox in dib doesn't help when tests try to install package globally and fail because requests can't be removed or whatever | 22:58 |
| donnyd | I left my comments the patch. I am curious if the case in there also covers ipv6_dhcpv6_stateless | 22:58 |
| donnyd | As that is the FN and limestone config if I am not mistaken | 22:58 |
| clarkb | donnyd: I think you get ipv6_slaac for that case from config drive | 22:59 |
| donnyd | but I am not sure how glean really works, so it may not be relevant | 22:59 |
| * clarkb attempts to confirm | 22:59 | |
| donnyd | Yea, stateless uses slaac for actual config.. but I am not sure if glean is asking openstack that question or not | 23:00 |
| donnyd | and I am also not sure how Openstack responds | 23:00 |
| clarkb | sort of, it is checking for the interface type from the config drive json | 23:01 |
| clarkb | "type": "ipv6_dhcpv6-stateless" seems to be what we get on limestone | 23:01 |
| clarkb | ya so thats buggy, thanks for the review | 23:02 |
| openstackgerrit | Clark Boylan proposed opendev/glean master: Import ipv6 configuration done by cloud-init for Red Hat distro https://review.opendev.org/686749 | 23:03 |
| clarkb | donnyd: tristanC ^ I think that is what we want there | 23:03 |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 23:04 |
| clarkb | tristanC: re cloud init I guess what I'm trying to say is it is doable but will require careful transitioning and isn't somethign we can do quickyl I don't think. | 23:05 |
| donnyd | I need to spin up a dhcpv6 stateful network and find out what we need to cover that case too. I am pretty sure it requires extra params in the ifcfg | 23:05 |
| clarkb | donnyd: ya it will require a different set of settings which are not yet supported there | 23:05 |
| *** harlowja has quit IRC | 23:05 | |
| clarkb | if you want to sort out what those settings are a new chagne would be great :) | 23:05 |
| donnyd | maybe for a future patch. I am thinking we probably don't need to wait on that for this patch | 23:06 |
| clarkb | correct | 23:06 |
| tristanC | clarkb: ok thanks for the detail. If it's possible then it seems like the right thing to do as it's included by default in upstream cloud-image, and those should be supported with regards to cloud network configuration | 23:07 |
| fungi | yes, the main reason we want glean these days, afaik, is so that we can avoid polluting our images with all of cloud-init's dependency tree since a lot of that conflicts with things our jobs want to install/test. we could of course switch to installing cloud-init into a venv or something and then set up the init to look for it there | 23:07 |
| donnyd | I can probably get something running. I would have to setup some extra things and it's all working so well I will have to be careful not to break it | 23:07 |
| *** rcernin has joined #openstack-infra | 23:08 | |
| clarkb | fungi: that and avoiding the myriad of default behavior it has that is just silly (imo) | 23:09 |
| fungi | tristanC: an example is when devstack tries to `pip install requests` (or any of the openstack projects which depend on it) and then pip wants to uninstall the system-provided python-requests package but can't, and we can't uninstall it beforehand without also finding and uninstalling all its reverse-dependencies too | 23:09 |
| clarkb | fungi: thankfully I believe cloud init is fully configurable and we can disable all of the sillyness, we just have to sort out what all the things we need are and supply our own config | 23:09 |
| fungi | yep, the configuration part isn't intractible | 23:09 |
| fungi | the system package pollution on the other hand is harder to deal with | 23:09 |
| fungi | i think we'd end up with it in a venv like we do with glean | 23:10 |
| fungi | the pip and distro answers to that problem are, of course, "don't sudo pip install" (which is a reasonable answer, but means devstack needs a major overhaul if so) | 23:11 |
| *** hwoarang has quit IRC | 23:12 | |
| donnyd | totally unrelated, but I have been tracking the swift logs thing pretty closely to see what it needs to operate. We are currently at 1.38TB | 23:12 |
| clarkb | donnyd: has it started to level off yet? we have 30 days of retention so in theory after about 30 solid days of use we should see it plateau | 23:12 |
| clarkb | and if we multiply that number by the number of clouds supplying swift to us I think that closely resembles our old fileserver disk usage | 23:13 |
| fungi | there were some extended outages for swift in fn though which i think happened within the past 30 days, so could slightly depress the current measurement | 23:13 |
| donnyd | https://grafana.fortnebula.com/d/9MMqh8HWk/openstack-utilization?orgId=2&refresh=30s&fullscreen&panelId=26&from=now-60d&to=now | 23:13 |
| fungi | but it's likely close | 23:14 |
| donnyd | I know, not great metrics.. I am not great at that | 23:14 |
| donnyd | fungi: If I am not mistaken the swift logs were only down the for week my generator was being installed | 23:14 |
| donnyd | but yea, that will probably skew them a little bit | 23:15 |
| fungi | right, that's what i meant | 23:15 |
| donnyd | I wish i was better with the grafana's. I have the data tracked, I just don't know how to display it better | 23:15 |
| timburke | donnyd, you scared me for a moment! i'm so used to seeing % _used_ instead of _free_... *whew* | 23:16 |
| fungi | pabelanger is a whiz with grafana dashboards | 23:16 |
| donnyd | timburke: LOL | 23:16 |
| donnyd | I gather what zabbix is capable of gathering and I have had gnocchi running... I just don't know how to do anything useful with it | 23:17 |
| donnyd | timburke: what that graph should do is exactly what you are saying. I should be able to take the percentage used and turn it into a percentage left or something better than this | 23:18 |
| donnyd | its pretty much garbage. I am trying to be as transparent as possible with the metrics so everyone else can see what the backend of this cloud looks like | 23:19 |
| timburke | which is fantastic! thank you for that, it'd been really fun seeing so much of how the zuul-logs transition has gone | 23:20 |
| *** goldyfruit_ has quit IRC | 23:20 | |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul-website master: Link to instructions on vulnerability reporting https://review.opendev.org/685799 | 23:21 |
| timburke | no criticism was intended ;-) | 23:21 |
| donnyd | Oh I was criticizing myself... Those swift metrics seriously need some help. Also I am curious if there is anything else anyone wants to know... like what other data would you want to see | 23:23 |
| donnyd | I am not showing hypervisor disk util right now, but surely could | 23:23 |
| donnyd | I just don't know what is useful as a consumer | 23:23 |
| *** hwoarang has joined #openstack-infra | 23:25 | |
| *** dchen has joined #openstack-infra | 23:27 | |
| *** dychen has joined #openstack-infra | 23:29 | |
| donnyd | https://grafana.fortnebula.com/d/9MMqh8HWk/openstack-utilization?orgId=2&fullscreen&panelId=28&from=now-60d&to=now&refresh=30s | 23:31 |
| donnyd | timburke: ^^^^^ | 23:31 |
| donnyd | is that better? | 23:31 |
| openstackgerrit | Luigi Toscano proposed zuul/zuul-jobs master: fetch-subunit-output: collect additional subunits (2nd try) https://review.opendev.org/674334 | 23:31 |
| timburke | donnyd, 👍 -- i'll have to watch for usage to come down as things calm down after the release and old logs expire | 23:35 |
| donnyd | oh we also wanted to track workload when that happens as well right | 23:35 |
| *** smarcet has joined #openstack-infra | 23:36 | |
| timburke | yeah -- and object-expirer cycle times will be of particular interest, i think | 23:36 |
| *** dchen has quit IRC | 23:39 | |
| *** dychen has quit IRC | 23:39 | |
| *** dchen has joined #openstack-infra | 23:40 | |
| *** tosky has quit IRC | 23:40 | |
| *** dchen has quit IRC | 23:43 | |
| donnyd | i totally killed the conversation tristanC and clarkb were having. Sry about that | 23:44 |
| clarkb | no worries, I think tristanC had to call it a day anyway | 23:45 |
| *** dchen has joined #openstack-infra | 23:46 | |
| donnyd | clarkb: can you think of any data that is interesting that would be helpful from your end in infra? | 23:52 |
| donnyd | maybe a stat that could show if something had a positive effect or negative one? | 23:53 |
| clarkb | I think for us it is mostly ensuring that the auto expiration is workin and keeping us at a sustainable disk usage level | 23:53 |
| clarkb | because we'll want to bump up the retention period if we have room | 23:53 |
| donnyd | Well I think the target utilization was set at 4TB right? | 23:54 |
| donnyd | did we have a specific amount of space we were targeting? | 23:55 |
| *** goldyfruit_ has joined #openstack-infra | 23:56 | |
| clarkb | not a specific amount of space as much as bump up the retention period as large as possible | 23:57 |
| clarkb | the previous numbers were guestimated based on what the old fileserver was storing | 23:57 |
| clarkb | but now we've got real numbers for in swift storage so we should double check the estimates | 23:57 |
| donnyd | oh ok. Well from my side it's a little less than 1.5tb, so if we bump to 60 days retention we should expect around 3tb | 23:58 |
| donnyd | which isn't a bunch really | 23:58 |
| donnyd | I need to find a way to get ```swift stat``` represented in the dashboard | 23:58 |
| *** mattw4 has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!