Friday, 2023-09-08

elodillesfrickler: there are some seconds already between branch deletions, but a timestamp can be added05:36
opendevreviewDmitriy Rabotyagov proposed openstack/project-config master: Report Vitrage changes to stable branches in the IRC
opendevreviewArtem Goncharov proposed openstack/project-config master: Rollback to LaunchPad for issuetracking
*** ykarel|away is now known as ykarel09:18
opendevreviewGuillaume Boutry proposed openstack/project-config master: Add OpenStack K8S Designate charms
fricklerTIL , maybe we should clone that to debug issues12:43
fricklerfungi: clarkb: regarding switching openstacksdk back to LP: doesn't exist yet, but there is . do you think you could try to contact one of the people in to see if they could hand it over?12:55
frickleror does that require the administrator (jnoller, never heard that name I think)?12:56
fricklerwould it make sense to talk to launchpad admins instead or if that fails?12:56
fricklerjamespage: or maybe going via you and canonical might be another option?12:58
fungifrickler: any of the members of that group should be able to change the maintainer/driver for the project to another group. i think i've seen bcurtin around still from time to time? if not, doug has always been amazingly responsive but i would avoid pestering him about it except as a last resort13:01
fungi(he's lurking in this channel, i'm just trying not to ping him unnecessarily)13:01
fungifrickler: do you have a group in lp you want to be the new owner? make sure the group itself is owned by openstack-admins too, so that we have a fallback escalation path should the same problem arise there in the future13:02
fricklerI don't have a group yet, is that what would create? not sure which details to consider there. maybe it would be best if gtema as ptl would create one called "openstacksdk"?13:12
gtemafrickler -
gtemaI invited admins13:15
gtemaI explicitly used plural form for future13:17
fungigtema: please set the owner of that team to if it's going to be a driver/maintainer/bug-supervisor for an openstack project on lp13:34
fungithat way we don't have the same problem you ran into with the abandoned team there13:34
gtemaI think first you should accept invite13:35
gtemacause I can't set anything while invite is not accepted13:35
fungican do13:39
fungilooks like i may need to be an administrator of openstack-admins in order to accept invites on behalf of that team. corvus ^ is that something you can help with?13:43
fricklercorvus: I also would not mind getting added to while you're at it ;)13:50
corvusfungi: done14:25
fungithanks corvus!14:30
fungilooks like now i have access to see all the teams it's been invited to, i'll go ahead and approve the relevant ones14:32
fricklergtema: I neglected to check this earlier, but the situation is looking better for , the associated team is owned by openstack-admins14:57
fricklerdo we need two different projects or can we use one for both sdk and osc?14:58
gtemaoh, ok14:58
fricklerfungi: do you happen to know if project renaming is supported on LP?14:58
gtemaI would say we should merge - this is anyway what we did on our side merging 2 teams/projects14:58
fungii have no idea, can't say i've ever tried to rename a project there14:58
fricklergtema: you should be able to request to "Join the team" on and I hope then fungi as openstack-admins member should be able to approve that and me?15:01
gtemaand what is this team?15:03
fricklerthat's the owner/driver for
gtemaah, ok15:03
fricklerand then we can check whether renaming is possible or otherwise I guess just create openstacksdk project with the same team15:03
fungifrickler: i've invited you to openstack-admins just now, so you should be able to try it yourself i think?15:05
fungier, added you i mean15:06
opendevreviewClark Boylan proposed opendev/system-config master: Update to gitea 1.20.4
clarkbcorvus: I found two small things on the Gerrit bookworm change do you want to take a look or should I go ahead and push a new patchset?15:17
clarkbI don't think either are critical but worth sorting out upfront if we can15:18
corvusclarkb: i think if you grok that it'd be great if you pushed the new ps15:19
clarkbwill do15:20
opendevreviewClark Boylan proposed opendev/system-config master: Update gerrit image to bookworm
opendevreviewClark Boylan proposed opendev/system-config master: DNM Forced fail on Gerrit to check bookworm/java 17 update
fricklerfungi: the addition has worked and I could approve both gtema and myself15:30
clarkbI went ahead and rotated the autohold for that gerrit change too15:30
fricklerediting the display name for the project seems possible, too, but it has a warning "Avoid changing the Name, because that will change the project group’s URL, which will break all links from outside Launchpad. "15:30
fricklerso maybe better create openstacksdk separately? or openstacksdks?15:31
gtemayou mean project or group?15:31
fricklerproject, sorry15:31
fricklerit is also being actively used by ubuntu as
gtemawell, then we should just create as many project as repos we have15:32
gtemaopenstacksdk, python-openstackclient (whether it is possible), osc-cli, ...15:32
fricklerI'd have argued to place osc-cli into python-openstackclient. and things like cliff maybe into openstacksdk15:33
fricklersimilar like neutron has all their subprojects under the neutron name on LP15:33
gtemacliff into sdk is a bad idea. It is not used there at all15:34
fricklerah, right, that's cli15:34
fricklerI was just looking for an example what could belong to sdk15:34
clarkbthe verification code for infra root is me trying to figure out that account16:31
clarkbwow it now wants me to provide a phone number to get a code.... I have a feeling this might forever associate my phone number with that account which isn't ideal16:32
clarkbbut here we go16:32
fungithis is for the g00g?16:33
clarkbany objections to me providing my numebr for that purpose?16:33
clarkblooks like they will actively send an sms code to it so a random number or landline won't work16:34
fungino objection16:34
fungiespecially since we have no idea what it was even used for16:34
fricklerwell you'll lock yourself in as opendev coordinator for life, but I'm fine with that, too ;)16:35
clarkbthe name on the account is "OpenDev Zuul" and I did login. That resets the December 1 account gets deleted timer at least16:36
clarkbstill have no idea what this was used for16:36
clarkbNote there is no positive confirmation this resets the account deletion timer but the warning email said the simplest way to avoid deletion is to log in to the account which I have done16:37
clarkbI am the only current session and the account appears to have last been managed in September 201916:38
clarkbmaybe if I look at irc logs from around then I'll find some clues16:39
clarkboh! I think this account might be used to do third party CI with gerrit?16:39
clarkb*Upstream gerrit16:40
clarkbthere is a connection to which is upstream gerrit and comparing to zuul config I think this is the case16:41
clarkbso ya we'll just need to manually login like I just did every couple of years to keep this connection with upstream gerrit working.16:41
clarkbI've logged out as I think that is mystery solved16:43
clarkbit didn't appear to add my phone number to the list of phone numbers associated with the account. I think they must be using that to help determine if you are a real human (possibly by checking the phone number against data they have on all of us humans)16:43
clarkbcorvus: ^ fyi since I think you set that up16:44
clarkbcorvus: you may also need to do similar for Gerrit's Zuul? I'm not sure16:55
yoctozeptomorning; do you have around some metrics related to the performance / resource consumption of the Hound instance at OpenDev? (both indexer and the query web ui) has it been problematic on this front? and what is your hw config for it?17:42
fungiyoctozepto: looks like we missed adding data collection for it to but i can pull some quick stats17:48
fungi8gb ram virtual machine currently using about 1gb for active pages plus most of the rest for buffers/cache17:49
fungiwe're doing all the indexing on the 40gb rootfs though it's starting to get full-ish17:51
fungithe hound container us using /var/lib/hound/data for that and i should have a du for it shortly17:51
fungi28gb in that directory17:52
fungiyoctozepto: our recipe for building the container image is at
clarkbits not something that gets continuous use. Instead its pretty spiky I think. When it starts up and indexes everything that takes a while and some queries with a lot of results can take a while.17:53
fungiyoctozepto: and is the ansible role we deploy it to the server with (compose file is under templates)17:53
clarkbdepending on your tolerance for that you many want more CPU/memory/IO but I'm not sure where it bottlenecks17:54
fungiyoctozepto: if you want to propose a change to add it to our cacti we could probably get some usage trends that way17:54
fungitook me a minute to find that, i forgot cacti is one of the last remaining services we're configuring with puppet still17:58
clarkbya we have a spec to replace it...17:58
fungiindeed we do17:59
yoctozeptothank you, fungi and clarkb for the thorough answers! I will be looking into your config and may add this cacti integration for you along the way ;-)18:01
fungimy pleasure18:02
yoctozeptooh, and how much code does it index? do you have a handy stat showing the count of bytes of all indexed repositories (code, not .git)?18:04
clarkbinfra-root is the held gerrit after my updates to the bookworm + java 17 change18:04
clarkbit seems to be working. If you agree and looks good we should land that today in prep for switching to it tomorrow18:05
clarkbyoctozepto: I don't think we have that data because it is only indexing a single branch for each repo18:05
yoctozeptoyeah, that I'm aware of18:06
clarkbbut it won't be more than the grand total of all the branch as an upper bound which might be easier to determine18:06
yoctozeptothough I can also download all "HEADs" myself and du that18:06
fungiwell, as i said, the index itself is occupying 28gb of disk18:07
funginot sure if that's what you need18:07
clarkb passed CI too to update gitea to 1.20.4. I'm going to run some errands around lunch but am otherwise around for either or both18:07
clarkbfungi: I'm guessing the idea is to figure out how large of an input produces the 28gb output index18:07
yoctozeptofungi: I mean the source data that led to that index now :D18:07
clarkbthen you can rough math on how large other instances might be based on their inputs18:08
fungioh, i can actually separate that because the directory contains the index and the worktrees it was built from18:10
fungiit does include .git directories but seems to be using shallow clones based on a quick survey of what's in there18:12
fungiso should be approximately representative of the worktree sizes18:13
yoctozeptowould not it be roughly "divide by 2" then?18:13
yoctozeptoI don't remember how shallow copies work18:14
fungimmm. yeah maybe it will be cleaner to just subtract the .git sizes18:15
fungiyoctozepto: du of all the vcs directories minus du of all their .git subdirs gives me 2151mib or 2.1gib18:18
yoctozeptotyvm, fungi!18:19
fungiso the data directory seems to contain approximately 13x the raw source code18:19
fungiactually more like 12x18:21
fungithe /var/lib/hound/data directory contains 25.3gib18:21
fungihuh, i'm rather disappointed in du's summarizations18:27
fungi`du -sm /var/lib/hound/data` should be mebibytes and reports 28305, yet `du -sb /var/lib/hound/data` reports 27169695799 bytes which is 25911 mebibytes18:28
fungiand it's not a mebibytes vs megabytes confusion i don't think because `du -sBMB /var/lib/hound/data` is supposed to be how you tell it to report in megabytes and that returns 2968118:30
fungioh, maybe i need --apparent-size18:30
fungiokay, that's it. if you ask in bytes it seems to return the apparent size automatically, while any other units do actual size rather than apparent size. how confusing18:32
fungiaha, i should read the manpage more carefully... "-b, --bytes: equivalent to '--apparent-size --block-size=1'"18:33
fungi"-m: like --block-size=1M"18:34
fungiso inconsistent, but at least documented18:34
clarkbfungi: did you still want to do gerrit tomorrow? we should review and possibly land if so. Also did we state a time for the things tomorrow?19:57
fungiclarkb: oh, i thought i mentioned but maybe i got the dates wrong. i won't actually be around this weekend (out of town for a concert)20:02
fungii might be able to squeeze in a restart if it's before 16:00 utc, but hitting the road about that time20:03
clarkboh I missed that20:13
clarkbin that case I'm also happy to stick with the zuul stuff. I think corvus might be out today so being on the safer side is fine20:13
clarkbfungi: maybe you can review the chagne but hold off approving and we can decide later?20:14
fungisure thing20:17
clarkbmy family has asserted they are done doing things for a bit. We have had a busy busy time over the last couple of months. That just means more time for biking and video games20:18
fungii probably said i would help and forgot i wasn't going to be around, sounds like something i'd do anyway20:18
fungichange lgtm, i'll check out the held node too20:20
fungiyeah, seems to be working as desired20:24
fungiclarkb: there's a couple of exceptions in the error_log, are those benign?20:24 java.lang.IllegalArgumentException: cannot convert null into a String20:24
clarkbgood question20:26
fungilooks like there's an occurrence in our production error_log.2023-08-23.gz20:26
fungidid we restart gerrit on that date?20:26
fungiindeed we did20:26
fungiso it seems to maybe only occur at startup and we saw it on the old images too, i think20:26
clarkb that error?20:27
clarkbI think it is related to us not explicitly enabling the plugin but having it installed and ya it occurs previously20:27
clarkbit is trying to load the plugin config and failing because it isn't present20:27
fungimakes sense then, yep20:27
dpanech_Hello, one of our gerrit reviews ( is failing a zuul gate ( I think we are missing a couple of OS packages on the zuul host. Would it be possible to install them?20:39
dpanech_It's a debian/bullseye system, I believe installing these packages should solve the problem: libsasl2-dev libldap2-dev libssl-dev20:40
clarkbdpanech_: if you need extra packages your job should install them20:41
clarkblooks like the job is running the bindep role so you can add dependencies to the repo's bindep.txt file20:43
clarkb is nova's file for example20:44
dpanech_clarkb: thanks!20:45
clarkbyou're welcome20:45
*** enzo is now known as Guest226120:46
corvusclarkb: oh im around20:49
clarkbcorvus: oh hi. Not sure if you saw the note about google accounts and zuul things but that might be useful to check for gerrit's zuul too.20:49
clarkbcorvus: and then I guess do we still want to do all the things tomorrow (if yes we should land the image update), but fungi won't be able to join20:50
corvusam debugging deduplication of circular dependencies in independent pipelines, so pretty long mental cycles today.  i'm up for air now.  :)20:50
corvusclarkb: yeah, i saw the gerrit thing; thanks.  tbh, i'm not sure if that applies... there is a zuul user in gerrit, but it's authenticated using the google cloud service account... so... <shrug?>  :)20:52
corvuslike... is it a real account even?  and if it is, does using the google cloud service to authenticate reset the login timer?20:53
clarkbcorvus: oh I see you're using the cloud account to do double duty there. My guess is as long as that account "pays" then its fine20:54
clarkbthe account timeout thing seems to be more of an issue for free idle accoutns20:54
corvusclarkb: plus, if they delete it, we get to have a fun conversation with people at google :)20:54
corvusclarkb: re tomorrow, that still works for me (did we specify a time?  or just "morning" which i'd interpret as sometime between 14:00 and 16:00) ?20:55
clarkb1500 should work for me. I'm also happy to wait on the Gerrit restart since those are much quicker these days we can do them another time20:56
corvusclarkb: okay, if we're not doing a coordinated gerrit restart, then i could probably just do the zuul thing when i wake up which might be a little before 1500; then you could come by at 1500 and maybe say "yup looks fine" or if everything is on fire help put it out.  how's that sound?20:58
corvus(if we are coordinating, then syncing at 15:00 sounds better)20:58
clarkbthat sounds great to me20:59
fungii might be around still at that point, though will probably be scarfing down a quick lunch before heading out the door21:09

Generated by 2.17.3 by Marius Gedminas - find it at!