| clarkb | new grafana is the next job (currently running against gitea-lb) | 00:01 |
|---|---|---|
| opendevreview | Clark Boylan proposed opendev/zone-opendev.org master: Switch grafana cname to grafana02 https://review.opendev.org/c/opendev/zone-opendev.org/+/940668 | 00:07 |
| clarkb | infra-root ^ the new service is up and seems to work you can check it https://grafana02.opendev.org I think we can update the cname record now | 00:07 |
| clarkb | working on getting the meeting agenda out now | 00:08 |
| ianw | should i ask about why all the panels now say they require angular which is deprecated? | 00:09 |
| clarkb | yes! but I don't know the answer yet. I figured that would be a next problem to solve | 00:10 |
| clarkb | https://grafana.com/docs/grafana/latest/developers/angular_deprecation/ | 00:11 |
| ianw | heh i just felt a bit triggered that it's going to require changes in grafyaml | 00:11 |
| clarkb | ianw: my suspicion is that the graphite datasource relies on angular in grafana 10 which is where they deprecated it. Then I'm hoping once we get 11 going they will have updated the graphite source to not use angular anymore | 00:12 |
| clarkb | but I didn't want to dig into that until we started testing 11 | 00:12 |
| ianw | hrm it seems more like the panel type than the data source? | 00:12 |
| clarkb | that post says its both | 00:13 |
| clarkb | but ya could also be the panels have migrated too. We aren't using any custom plugins or addons so I'm hoping that 11 just does the swap for us | 00:13 |
| ianw | https://grafana.com/docs/grafana/latest/developers/angular_deprecation/angular-plugins/#automatic-migration-of-plugins ... that does make it feel like a grafyaml issue :/ | 00:13 |
| ianw | yeah i guess the problem is that grafana may rewrite your dashboard for you, but if we keep loading it with old panels from grafana that probably breaks | 00:14 |
| clarkb | hrm ya though I guess we just have to remap things? | 00:14 |
| clarkb | probably the easiest thing is to deploy with 11 see how it automatically updates things then update grafyaml to write that target in the first place | 00:15 |
| clarkb | and in theory that works with 10 and we can switchover before upgrading | 00:15 |
| ianw | ++ | 00:15 |
| ianw | we can also move to loading the pages from exported js directly - so make the updated panel and save the js, then load that, rather than go through grafana. it just seems like there's really not much community interest in declarative language for dashboards, you're just expected to edit via ui | 00:16 |
| ianw | s/through grafana/through grafyaml/ | 00:16 |
| clarkb | 10 still has support so my immediate goal was updating to latest 10 and the OS update then figuring it out from there | 00:18 |
| clarkb | heh and in the time I started looking at this there is a new 10.4 release | 00:19 |
| ianw | heh, another day, another js framework ... hi ho hi ho, it's off to refactoring we go ... | 00:20 |
| clarkb | annoyingly release-10.4.15 doesn't have changelog entries for that release | 00:23 |
| clarkb | now to figureout where github permalinking went | 00:23 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update to the even more recent grafana 10.4.15 https://review.opendev.org/c/opendev/system-config/+/940669 | 00:24 |
| clarkb | they don't seem to have a :10 tag on docker hub so we hvae to update to pinned versions like this for now | 00:24 |
| *** diablo_rojo_phone is now known as Guest7964 | 00:30 | |
| ianw | i remember we used to pull :latest and that broke things https://github.com/grafana/grafana/discussions/47177#discussioncomment-3362402 ... i'm not sure where the discussion on stable releases ever ended up | 01:34 |
| karolinku[m] | Hey, I'm planning to work on adding support for keyfiles in Glean project (https://issues.redhat.com/browse/RDO-443). It is related to support for Centos 10 release. Is that OK for you to add such feature? maybe TheJulia ianw will know | 09:38 |
| tonyb | karolinku[m]: please do! | 10:53 |
| fungi | karolinku[m]: to be clear, it would be glean writing the new networkmanager config format to files in /etc/NetworkManager/system-connections/ at boot, in situations where interface autoconfiguration (dhcp/slaac) is not sufficient? | 12:52 |
| fungi | that seems reasonable, as long as it can be generally applicable to other distros that may start using newer nm at some point | 12:53 |
| fungi | though i'm surprised centos isn't using systemd-networkd | 12:53 |
| frickler | I also wonder whether we really need to keep using glean for that, or whether cloud-init might not be mature enough by now. seems like a good idea to leave updating things to others | 12:54 |
| fungi | frickler: the cloud-init problem was never maturity, it was the fact that it pollutes te images with a ton of otherwise unnecessary python libraries | 12:56 |
| * fungi checks the situation there... | 12:57 | |
| fungi | i count 10 first-order python3-* libs in the dependencies for the current version in noble, not sure how large the transitive set is: https://packages.ubuntu.com/noble-updates/cloud-init | 12:58 |
| fungi | granted, if we can get devstack and openstack functional tests to always use venvs (without --system-site-packages) for everything, it my be less of a problem than it was previously | 13:00 |
| fungi | the "minimal" nature of glean is that it has 0 runtime python dependencies outside the stdlib | 13:01 |
| frickler | ah, so same issue that cirros has | 13:02 |
| karolinku[m] | in fact I'm not expert to decide whether Glean should be developed further or it should be replaced by other solution, so I started discussion:) as tonyb helped find out, is that Gean's lack of keyfiles support caused problem with booting C10. | 13:02 |
| opendevreview | Karolina Kula proposed openstack/diskimage-builder master: WIP: Add support for CentOS Stream 10 https://review.opendev.org/c/openstack/diskimage-builder/+/934045 | 13:11 |
| fungi | frickler: i don't know about cirros's problems to say whether this is similar. to be clear it's not a matter of image size, it's that images with cloud-init installed may then come with versions of various python libraries that conflict with other versions of the same libraries some jobs may want to install | 13:17 |
| fungi | out of curiosity, i did a dry-run installing cloud-init on one of my netbooks just now and it was going to pull in 21 different python libraries (at least several of which are in openstack's constraints list). this netbook is far from having a minimal package set installed, so i don't know how many more it might mean on test nodes | 13:21 |
| fungi | but like i say, if all our python jobs have improved to the point that they're thoroughly insulated from the system-wide module search paths, it might no longer be a real concern | 13:24 |
| frickler | for cirros the initial issue was having to start including python at all, which would already almost double the image size. anyway, let's not further delve into this then, at least I won't currently | 13:29 |
| frickler | the other thing to look at however would be whether centos 10 cloud images actually need to use networkmanager, or if systemd-networkd or similar would be options more suited for a cloud environment. nm to me is a desktop tool ... but I also might just be oblivious to modern developments | 13:30 |
| fungi | yeah, that's the position debian's taking: systemd-networkd for cloud guests and servers, network-manager for portable devices and desktops | 13:31 |
| fungi | at least that seemed to be the consensus the last time the discussion came up there | 13:33 |
| jrosser | fwiw one of the first things that osa jobs do on centos/rocky is get systemd_netword installed | 13:33 |
| jrosser | then everything is identical on all OS from that point on | 13:33 |
| jrosser | but it does come from EPEL, and is as such not a stock part of the distro | 13:34 |
| fungi | well that was exciting... i lost all contact with rackspace flex sjc3 for about 15 minutes | 14:03 |
| *** ykarel_ is now known as ykarel | 14:28 | |
| fungi | i've rechecked bindep change 940258 for new results, now that pbr 6.1.1 is on pypi the previously failing jobs should pass | 15:00 |
| fungi | worked!!! | 15:22 |
| fungi | i'll refresh that stack now to simplify the first change, and drop the dnm one for now | 15:22 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Use PBR's pyproject.toml build-backend support https://review.opendev.org/c/opendev/bindep/+/816741 | 15:22 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Evacuate most metadata out of setup.cfg https://review.opendev.org/c/opendev/bindep/+/938520 | 15:22 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Drop support for Python 3.6 https://review.opendev.org/c/opendev/bindep/+/938568 | 15:22 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Drop requirements.txt https://review.opendev.org/c/opendev/bindep/+/938570 | 15:22 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Drop tox.ini https://review.opendev.org/c/opendev/bindep/+/940219 | 15:22 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Drop auxiliary requirements files https://review.opendev.org/c/opendev/bindep/+/940711 | 15:32 |
| fungi | another likely contentious change ^ | 15:32 |
| corvus | fungi: i feel like that could use some more explanation -- because that's a lot of near-duplication, and i don't get from the commit message why we would want to do that (not that i object, i just don't understand, and therefore, can't attempt to make a suggestion to reduce duplication) | 15:34 |
| fungi | corvus: fair, also we could probably define a list in noxfile.py and reuse it throughout | 15:35 |
| corvus | yep, that's possible, and if there's like one thing that needs to be changed, then we can perform surgery on one item in the list, for example | 15:35 |
| corvus | (but right now, the change would look like test_requirements == open('test-requirements.txt').read() and i don't know why that's better yet) | 15:36 |
| fungi | i'm not sold on a number of changes in that stack, but insofar as a demonstration of what other projects outside our direct sphere are doing with python packaging also works with pbr, i'm putting together a menu of changes we could chose to do or leave out to form a modernized standard for our own packages | 15:36 |
| clarkb | ya centos is all in on network manager | 15:41 |
| clarkb | and they dropped the old sysconfig support in 10 apparently | 15:41 |
| clarkb | I agree adding support for the configuration that network manager does want is reasonable in glean | 15:41 |
| clarkb | we don't have any plans to replace glean on our systems at least. It is possible that cloud init would work on all clouds with all platforms we support today but then we'd still be open to it taking actions we don't want (like formatting volumes and mounting them based on metadata contents iirc) | 15:42 |
| opendevreview | Jeremy Stanley proposed opendev/bindep master: Drop auxiliary requirements files https://review.opendev.org/c/opendev/bindep/+/940711 | 15:44 |
| fungi | corvus: ^ there's the dry form | 15:44 |
| fungi | also makes it a little clearer that we can, e.g., not install coverage in normal test jobs and only in the coverage job | 15:45 |
| clarkb | re bindep I think the goal I have in mind is to illustrate how pbr consuming projects can look like modern python package projects too. | 15:45 |
| clarkb | this requirements and noxfile stuff seems a bit orthogonal (its another illustrative cleanup but not directly related to pbr, pyproject.toml and setuptools) | 15:46 |
| fungi | well, the requirements.txt removal shows putting the install_requires into pyproject.toml (the python packaging community seems to regard requirements.txt as an old, pip-only wart) | 15:47 |
| clarkb | https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/ it seems more nuanced than that | 15:48 |
| fungi | if we have pbr populate the install_requires from requirements.txt then we have "dynamic dependencies" from the perspective of python packaging people, which seems to be a huge pain point because they want dependencies to be a declarative thing that can be extracted and served as part of the pypi api, for example | 15:48 |
| clarkb | basically requirements files are specific and package install_requires are not | 15:48 |
| clarkb | but still having nox pip install a test-requirements.txt file is orthogonal to all this aiui | 15:49 |
| fungi | agreed, that was moer of a "if we're remocing requirements.txt, should we also remove test-requirements/txt for consistency?" | 15:49 |
| clarkb | and fit the description in the document above for building specific environments (in this case to execute some sort of test or other) | 15:49 |
| clarkb | fwiw I'm not opposed to the change I just wanted to decouple it from the pyproject efforst since I don't think they are super related 9other than I guess that dropping requirements.txt is part of the pacakge refactor) | 15:50 |
| fungi | yeah, i also tried to sort that stuff toward the end | 15:54 |
| fungi | arranged somewhat in order from most relevant and least contentious to least relevant and most contentious | 15:54 |
| clarkb | infra-root if we can land https://review.opendev.org/c/opendev/system-config/+/940669 then check that grafana02 is still happy then we can land https://review.opendev.org/c/opendev/zone-opendev.org/+/940668 and start grafana01 cleanups | 15:56 |
| clarkb | should be safe to proceed with the first change since grafana01 is still serving content until the second change to update dns lands | 15:56 |
| clarkb | I've approved the 10.4.15 bump and will approve the cname update once that deploys and looks happy | 16:44 |
| opendevreview | Merged opendev/system-config master: Update to the even more recent grafana 10.4.15 https://review.opendev.org/c/opendev/system-config/+/940669 | 17:19 |
| opendevreview | Merged opendev/zone-opendev.org master: Switch grafana cname to grafana02 https://review.opendev.org/c/opendev/zone-opendev.org/+/940668 | 17:53 |
| clarkb | the dns record is pointing me at the new server now re ^ | 18:07 |
| fungi | same | 18:08 |
| fungi | nodepool dashboard lgtm still, aside from the angular warning | 18:08 |
| *** iurygregory_ is now known as iurygregory | 18:55 | |
| melwitt | just fyi I have noticed odd (automated I assume) comments by this user recently https://review.opendev.org/dashboard/37702 for some reason when I try to search commentby: it doesn't bring up any results | 20:44 |
| Clark[m] | melwitt: we noticed too and I believe fungi disabled the account due to their spam. Disabling the account likely broke search for it | 20:53 |
| melwitt | ah ok cool | 20:55 |
| corvus | oh shoot, i wanted to highlight https://review.opendev.org/940547 the jeepyb zuul.yaml change in the meeting | 21:04 |
| corvus | i feel like that's ready to either move forward or abandon | 21:04 |
| corvus | do we think that's sufficiently socialized, or should i put it on the agenda for next week? | 21:05 |
| tonyb | ooops I missed the meeting because TZs are hard :( | 21:07 |
| Clark[m] | corvus I think it's fine. The main thing is it will rebuild our Gerrit image so we may want to restart to avoid surprises..however the Gerrit image isn't necessary for your change maybe we want to stop jeepyb from doing that and instead manually trigger rebuilds when we know we make a change that does | 21:20 |
| Clark[m] | Though now that I've said that I wonder if we run jeepyb out of the image for manage projects. I don't think we do | 21:25 |
| fungi | melwitt: https://review.opendev.org/q/commentby:37702 still brings up anything they commented on, but because they're disabled (as of ~1700 utc yesterday) any of their other ids are now invalid | 21:28 |
| fungi | we also observed launchpad bugs they fiddled with, github pull requests they opened, et cetera | 21:29 |
| JayF | this looks almost like really bad pen testing | 21:29 |
| fungi | so they may still continue messing with stuff, but they were also fiddling with aws, npm, and similar orgs elsewhere | 21:29 |
| JayF | oh wait, I know this | 21:29 |
| JayF | hoold on a sec I have a spicy addition I think | 21:29 |
| fungi | at one point they asked eclipse foundation to add a project namespace for them too, but then ghosted | 21:30 |
| fungi | maybe ai training related | 21:30 |
| melwitt | fungi: oh interesting. I was trying to search using the email rather than the user id. good to know | 21:30 |
| JayF | nope, https://github.com/pypa/packaging-problems/issues/872 is what I was thinking of, but the names/account ids don't match | 21:30 |
| fungi | they seem to have some ai-oriented projects in their github org that don't look like direct forks of stuff on github, at least | 21:31 |
| fungi | JayF: ooh, that's an exciting case | 21:32 |
| JayF | ironic-secureboot-driver does exist, btw, github.com/yahoo/ironic-secureboot-driver and is already in pypi | 21:33 |
| JayF | which is whole bucket of unfortunate all it's own | 21:33 |
| * JayF dislikes anytime there's an openstack-* or ironic-* that isn't capital-O-OpenStack | 21:33 | |
| JayF | https://github.com/openstack-exporter/openstack-exporter being a recent painful example (it's not well maintained and patches don't merge there, but it says openstack on it :C) | 21:34 |
| fungi | looks like it's not in pypi under the exact name ironic-secureboot-driver at least | 21:35 |
| fungi | maybe it was deleted from pypi at some point, but yeah definitely looks like an attempt to exploit references to names for defunct projects | 21:36 |
| JayF | I got that link from someone almost certainly empowered to act on it accordingly, fwiw | 21:36 |
| JayF | but I saw the ironic-* and it raised an eyebrow | 21:37 |
| fungi | as one does | 21:38 |
| Clark[m] | Discord is apparently having a large outage if anyone else is using it for things | 21:44 |
| corvus | i would not have known if you didn't mention it | 21:50 |
| Clark[m] | I guess it was fixed by the time I posted about it | 21:52 |
| corvus | well, i just meant i'm not using it for things :) | 21:52 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Remove grafana01 from config management https://review.opendev.org/c/opendev/system-config/+/940752 | 22:45 |
| opendevreview | Clark Boylan proposed opendev/zone-opendev.org master: Cleanup grafana01 DNS records https://review.opendev.org/c/opendev/zone-opendev.org/+/940753 | 22:47 |
| clarkb | the promised grafana01 cleanup changes for when we're happy with grafana02 | 22:47 |
| clarkb | oh keycloak was the third mariadb update. I suspect we can do that one whenever as well since keycloak use is currently minimal | 22:48 |
| corvus | ++ | 23:22 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!