| tonyb | I tried the new etherpad and for me it's a regression from 2.5.1 that I tried yesterday. I'll do more debugging after the i18n PTG session | 00:06 |
|---|---|---|
| clarkb | tonyb: what sort of regression? | 00:22 |
| tonyb | I can't get any pads to load | 00:24 |
| tonyb | the home page works, but they added more text to the already full button on the home page | 00:25 |
| fungi | button label overrun | 00:26 |
| clarkb | tonyb: huh it worked much better for me in both ff and chrome. Did you do this in a clean brower with no existing cookies? | 00:26 |
| clarkb | no cookeies and no cached data | 00:26 |
| clarkb | sometimes the old version and new version conflict over that stuff | 00:26 |
| tonyb | I can look at that. | 00:28 |
| prometheanfire | JayF, clarkb: openrc generally uses netifrc and systemd generally uses networkd or networkmanager, depending on what the user wants | 00:46 |
| prometheanfire | it's hard to write support for everything :| | 00:46 |
| prometheanfire | one day maybe cross distro server stuff will be "standard" | 00:47 |
| fungi | clarkb: wrt the weblate api key, we have one recorded in our usual list so maybe it's that | 00:48 |
| clarkb | fungi: oh til | 00:48 |
| fungi | listed under "openstack.weblate.cloud" | 00:48 |
| JayF | prometheanfire: I've got a good idea, let's write an application suite to standardize all of this stuff across all Linux distributions. We can call it systemd 😂 | 00:50 |
| fungi | i thought you were going to say lsb | 00:52 |
| fungi | though i guess that was more about standardizing things that already existed, not replacing everything that exists and then declaring it a standard | 00:53 |
| JayF | I'm totally just making the joke there. I'm legitimately glad that lots of options exist even if I choose the boring one. | 00:54 |
| prometheanfire | what's debian's new one? I think that one is declaritive in yaml? | 00:54 |
| fungi | are you talking about netplan? i think that came out of canonical/ubuntu | 00:56 |
| fungi | someone was pushing for it to become the default in debian but that didn't get very far from what i saw | 00:56 |
| fungi | where i saw the discussion peter out is that networkmanager is great for roaming wireless clients, systemd-networkd is great for static server deployments, and trying to force "one true network management layer" was not sensible | 00:57 |
| fungi | though lately on my mobile clients i'm combining networkmanager (for wifi) and systemd-networkd (for wireguard vpn to my house) with no problem | 00:58 |
| fungi | apparently recent networkmanager versions now also support a yaml-based configuration model lifted from netplan too | 01:00 |
| fungi | i also still just use ifupdown on a lot of my debian systems, particularly in a cloud vm context | 01:02 |
| JayF | I've moved mostly to a dhcp-everything configuration | 01:03 |
| clarkb | networkmanager is my persistent nemesis. Latest problem is it cannot wireguard at all. | 01:03 |
| JayF | if my opnsense box is down I'm likely offline anyway | 01:03 |
| JayF | only exception are a couple of specific servers I want online in that specific case3 | 01:03 |
| clarkb | It at least detects when I configure wireguard separately | 01:03 |
| tonyb | clarkb: Okay after removing the cookies and restarting both chrome and ff the new etherpad is "fine". I still can't switch into light/dark mode as per 2.5.0 | 01:08 |
| Clark[m] | Ya I think light/dark modes aren't supported with this skin | 01:09 |
| prometheanfire | mobile I'm using networkmanager (with wireguard) and static I use networkd, ya | 01:16 |
| JayF | honestly my favorite thing about ussing networkd is how easy it is to rename interfaces | 01:17 |
| JayF | having a router interface names actually be wan/lan/san/etc is very, very useful | 01:17 |
| prometheanfire | clarkb: I haven't tried NM 1.54 with wireguard yet | 01:17 |
| JayF | I used to do it back in the day with RHEL5, it had syntax to do it in ... whatever /etc/sysconfig/network was :D | 01:18 |
| prometheanfire | renaming interfaces is very nice, yes, a few coworkers were surprised when I told them to configure the public interface with xyz IP and asked which one was public | 01:19 |
| tonyb | "update constraint for pbr to new release 7.0.2 | https://review.opendev.org/c/openstack/requirements/+/965358" is in the gate pipeline now so we should check in in about 90mins for any new failures that might be related | 01:39 |
| tonyb | infra-root: Should we land 964728: Force gitea http(s) connectivity through the load balancer early next week or after the gitea update. I suspect we don't want both changes to land at once | 01:42 |
| tonyb | mnasiadka, noonedeadpunk: I had some follow-up thoughts about constraints, stable branches and deployments projects. Other than ${this} channel is there a good channel you're both in to discuss it? | 06:50 |
| opendevreview | Mathieu Parent proposed opendev/glean master: Use systemd-networkd automatically when enabled https://review.opendev.org/c/opendev/glean/+/963010 | 07:07 |
| mnasiadka | Clark[m]:I don’t know if you’re still considering mjollnir - but it seems draupnir is the replacement(?) - https://github.com/the-draupnir-project/Draupnir | 09:13 |
| tonyb | mnasiadka: It certainly claims to be a drop-in replacement rewrite | 09:17 |
| tonyb | mnasiadka: but worth looking at Thanks | 09:18 |
| opendevreview | Merged openstack/diskimage-builder master: Fix openstack-ci-mirrors element for CentOS Stream https://review.opendev.org/c/openstack/diskimage-builder/+/965344 | 09:41 |
| opendevreview | ribaudr proposed opendev/irc-meetings master: Move Nova weekly meeting from Tue to Mon https://review.opendev.org/c/opendev/irc-meetings/+/965465 | 10:26 |
| *** jroll08 is now known as jroll0 | 10:46 | |
| stephenfin | clarkb: fungi: We're going to need a pbr 7.0.3. https://review.opendev.org/c/openstack/pbr/+/965474 is essential (see linked bug: I'd missed this change before now) and https://review.opendev.org/c/openstack/pbr/+/965470 is a pretty noticeable issue that's affecting pbr itself. https://review.opendev.org/c/openstack/pbr/+/965313 is not essential but might good to get in also | 11:31 |
| fungi | looking | 13:08 |
| fungi | stephenfin: looks like the py27 job is throwing ImportError: pbr._compat.commands.LocalEggInfo when doing an editable install of pbr. do you happen to know why yet? | 13:12 |
| fungi | on 965474 | 13:12 |
| fungi | at first i thought it might be related to clarkb's https://review.opendev.org/965397 but it looks like that failed out of the gate due to the depends-on not having merged yet | 13:28 |
| Clark[m] | fungi: stephenfin: that py27 error may be related to the change since 965470 passes? | 13:54 |
| Clark[m] | And yes the failed job ran on bionic so not due to switching to jammy | 13:55 |
| stephenfin | Hmm, that happened locally when I had a bug in the code. I'll take a look | 13:56 |
| stephenfin | ugh, python 2's awful relative import mechanism strikes again | 13:59 |
| stephenfin | >>> from pbr._compat import commands | 13:59 |
| stephenfin | ... | 13:59 |
| stephenfin | from setuptools.command import egg_info | 13:59 |
| stephenfin | ImportError: No module named command | 13:59 |
| * stephenfin rename module | 13:59 | |
| stephenfin | Clark[m]: fungi: Should be resolved now https://review.opendev.org/c/openstack/pbr/+/965474 | 14:10 |
| Clark[m] | Thanks I'll review all three changes properly in a bit. Still booting the morning. Do we need to warn the release team? | 14:12 |
| noonedeadpunk | tonyb: we can use #openstack-requirements as well I guess? seems we all are there | 14:38 |
| clarkb | stephenfin: that latest patchset hit ImportError: No module named setuptools | 14:48 |
| clarkb | I've approved the other two changes as they pass testing and seem fine | 14:49 |
| clarkb | I think we just need to s/setuptools/versions/ on that import line. If I decide that is the acse I can push an update | 14:51 |
| clarkb | ya I think that is it I'll go ahead and modify | 14:52 |
| fungi | ah yep thanks, i missed that reviewing | 14:54 |
| clarkb | tonyb: re etherpad, ya thats a known thing with updates being weird with overriding /etc/hosts. I typically use an incognito tab to work around it | 14:57 |
| clarkb | tonyb: for gitea I think we can focus on stability changes from our side before worrying too much about 1.25. Typicalyl they will release a .1 or .2 with many bugfixes which are worth waiting for. On the stability side of things we've got the limit access through the load balancer change and I also proposed a change to increase the memory available to memcached to hopefully mitigate | 14:59 |
| clarkb | some of the effects of crawling as a cache poison | 14:59 |
| clarkb | but ya maybe early next week we get an etherpad upgrade done to finally catch back up there, then figure out what if anything we want to do with the gitea deployment | 14:59 |
| clarkb | testing looks good on that third pbr chagne for the develop command so I've approved it now too. | 15:14 |
| fungi | thanks! | 15:15 |
| opendevreview | Merged opendev/irc-meetings master: Move Nova weekly meeting from Tue to Mon https://review.opendev.org/c/opendev/irc-meetings/+/965465 | 15:15 |
| fungi | i can propose pbr 7.0.3 as soon as it lands | 15:15 |
| clarkb | do you think it is worth direct enqueing to the gate or should we just ride it out? | 15:19 |
| clarkb | today is the day we all learn guthub releases are not immutable by default (they just added support for this) | 15:27 |
| fungi | i think we can just wait for it to merge normally, i'm buried under other things in the meantime anyway | 15:29 |
| clarkb | my main concern is that if setuptools makes a release in the next 5 minutes we might break the release pipeline to fix things in the first place. But I think we can figure out a workaround for that up to an including manual workarounds up to pypi upload | 15:29 |
| fungi | yeah | 15:31 |
| corvus | do we know what timezone the setuptools deadline is in? :) | 15:42 |
| clarkb | corvus: no, and we don't even know if they've remembered their own deadline | 15:42 |
| clarkb | that said historically they are fans of the weekend release. I wonder if today is the public deadline then we'll see a new release saturday or sunday | 15:42 |
| clarkb | ubuntu has announce x86-64-v3 support but will be doing so via additional packages (and possibly only for packages where the greatest impact will be made?) | 16:37 |
| fungi | heh, blizzard's dockenstack jenkins is e-mailing us about job failures again ;) | 17:26 |
| clarkb | I'll take the emails over the failed ssh connections ever few seconds :) | 17:27 |
| fungi | indeed | 17:28 |
| mnasiadka | There’s a question from my downstream how https://opendev.org/openstack/cinder/commit/49f386f3bc5d5a1e16c1b8cf2ef46c6bf453cd42 got in, if https://review.opendev.org/c/openstack/cinder/+/820375 is in merge conflict - I assume there’s a simple answer, but indeed that looks a bit weird from a user perspective | 17:58 |
| clarkb | mnasiadka: commits are replicated to gitea from gerrit. Whether or not that commit is merged to a branch is a separate question | 18:00 |
| clarkb | that ref doesn't show up in the master branch | 18:02 |
| fungi | mnasiadka: all refs in gerrit, in fact, are replicated to gitea | 18:02 |
| clarkb | (which is expected since it wasn't merged) | 18:02 |
| fungi | the named refs for changes/patchsets get replicated to gitea too (though they're harder to access than raw commit ids due to ui limitations), as well as git notes that contain a bunch of the gerrit review metadata | 18:03 |
| mnasiadka | Ah right, I just assumed something wrong happened, and this commit is not tied to any branch | 18:05 |
| fungi | basically everything in gerrit for a project, whether or not it's already merged, or can ever even merge at all (old patchsets for a change that got revised) appear in the repositories in gitea | 18:05 |
| fungi | we also make direct use of that in the gerrit ui, linking to commits in gitea from patchsets | 18:06 |
| mnasiadka | Right, thanks for the explanation - just the fact it’s not merged in any branch wasn’t that super easy visible :) | 18:08 |
| fungi | for example, https://review.opendev.org/c/openstack/pbr/+/965474/1 is an outdated patchset, and the "gitea" link immediately below the vote block takes you to it in gitea even though that will never be able to appear on any branch | 18:08 |
| mnasiadka | Actually I never noticed the gitea link in the Gerrit UI before ;) | 18:09 |
| fungi | mnasiadka: click the "..." button under the commit message and you'll see a "this commit is container in" list with branches and tags. for things that aren't on a branch it's empty | 18:09 |
| fungi | s/container/contained/ | 18:10 |
| fungi | so for example https://opendev.org/openstack/pbr/commit/562f4ebb5b3f1280beab1f7d1e3e56906e2169f7 says master branch and 7.0.2 tag | 18:10 |
| fungi | while https://opendev.org/openstack/pbr/commit/472f98878123b342558fb35351bb9f8920317cfe is empty there | 18:10 |
| mnasiadka | Yup, noticed that after your initial response :) | 18:11 |
| fungi | i agree it's rather hidden | 18:11 |
| mnasiadka | Great, I learned a new thing today ;) | 18:11 |
| mnasiadka | 7pm Friday - it’s been a long week , have a good weekend :) | 18:11 |
| tonyb | 5am Saturday.... it's been a long sleepless week | 18:16 |
| fungi | thanks! it's beer-o'clock here | 18:21 |
| fungi | always | 18:21 |
| mnasiadka | fungi: it’s always everywhere beer o’clock | 18:25 |
| fungi | see, i knew we agreed on something | 18:26 |
| mnasiadka | tonyb: for the requirements discussion - we can probably use #openstack-ansible or create a new one for this. You can also use the etherpad we used for cross deploy session | 18:27 |
| clarkb | I think gitea is slow again today and I'm not sure I have the energy to debug and identify who needs their ipaddresses blocked today | 18:31 |
| fungi | i'm headed out to beer^H^H^H^Hlunch now that the pbr 7.0.3 release is proposed, will be back in a while | 18:40 |
| clarkb | enjoy! | 18:42 |
| tonyb | clarkb: I'll look at gitea after breakfast. | 19:08 |
| clarkb | thanks. I'm going to pop out for a bike ride now myself | 19:24 |
| tonyb | clarkb: gitea looks okay to me. All servers have a load average between 1 and 2.5, though some are coming down from a higher value. | 19:52 |
| tonyb | grafana's view of the LB indicates that there was a spike about an hour ago | 19:52 |
| fungi | okay, back | 20:04 |
| Clark[m] | tonyb: ack thank you for checking | 21:13 |
| clarkb | as a sanity check all of the zuul components are currently running (since last week's restart failed due to a launcher being down) | 21:43 |
| fungi | fingers crossed they all restart again in a few hours | 21:44 |
| clarkb | I've run down the issue with the reestart too and will have a patch up shortly | 21:52 |
| clarkb | (I don't think we need to rush it in before these restarts, but good to have eventually) | 21:52 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Fix zuul service graceful shutdown routines https://review.opendev.org/c/opendev/system-config/+/965866 | 21:57 |
| clarkb | I think ^ that should address the problem. Its a docker-compose vs docker compose output problem when shutting things down | 21:57 |
| tonyb | what time does the weekly restart happen? I can be around to monitor/check afterwards | 23:19 |
| tonyb | nm I see it in crontab on bridge | 23:26 |
| clarkb | tonyb: it should start in like an hour? | 23:30 |
| clarkb | we've found that the late friday into saturday block tends to be quiet so its a good time to wind down the executors (we do so gracefully one by one which requires all jobs on each executor be stopped one after another) | 23:30 |
| clarkb | since the components list shows a full set of components I'm not too worried about it | 23:31 |
| tonyb | `1 0 * * 6` which I read as 00:01 on Saturday ... so 30mins from now | 23:31 |
| tonyb | okay. | 23:32 |
| clarkb | tonyb: and the easiest way to follow along is to just refresh the zuul /components page periodically | 23:32 |
| tonyb | noted | 23:33 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!