Thursday, 2023-11-09

clarkbya I finally got out on the bike and I think I decided to do some refactoring even if it diverges from the documented commands for etherpad api management00:25
opendevreviewClark Boylan proposed opendev/system-config master: Improve etherpad functional testing
opendevreviewClark Boylan proposed opendev/system-config master: Improve etherpad functional testing
opendevreviewArtem Goncharov proposed openstack/project-config master: Add hashicorp-vault-plugin-secrets project
opendevreviewArtem Goncharov proposed openstack/project-config master: Add hashicorp-vault-plugin-secrets project
*** dhill is now known as Guest642215:21
JayFCan someone from infra help me with something?15:52
tonybAlmost certainly ;P15:52
JayFHow did bugfix/8.1 branch get recreated in openstack/ironic-python-agent gerrit?15:52
JayFIt was retired years ago, and now it appears to have zombied15:52
JayFalong with many other ironic bugfix branches15:52
JayFwhich also creates CI noise, zuul config errors, etc that I thought I had cleaned up :(15:52
clarkbwe don't create branches for openstack projects. Is that something in a openstack releases config file causing it to be recreated?15:53
JayFI'm trying to find the answer to that question. A dangling open change and gerrit did it? Releases automation?15:53
JayFI went to infra thinking you would have access to audit logs of some kind15:53
clarkbgerrit does not allow you to delete branches with open changes15:53
clarkbso maybe error checking for deletion requests missed that it failed due to open changes, or open changes were not a problem15:54
clarkbI believe we have 30 days of logs without going to backups15:54
clarkbbut I'd have to ssh in to be sure of that range15:55
tonybJayF: when did you notice they existed?15:55
JayFyesteday while trying to create periodic job dashboards15:55
JayFsaw a bunch of outta date tests running in bright red15:56
JayFdug it this morning, found zombie bugfix branches15:56
clarkbJayF: personally I would start here:
fungiJayF: bugfix/8.1 is still included in deliverables/xena/ironic-python-agent.yaml so that may be why it was recreated15:56
fungii recommend checking with the release managers to find out if branch references should be removed from deliverables before deleting15:57
fungii still see remotes/origin/bugfix/8.0 in the git branch list too15:59
tonybgerrit doens't have an 8.0:,branches16:02
JayFyeah, I'm using gerrit as source of truth16:03
fungii wonder if the branch deletion for bugfix/8.0 didn't replicate from gerrit to gitea for some reason16:04
fungigitea also seems to have bugfix branches for 6.2, 6.3, 6.5, 6.616:05
clarkbfungi: gitea's database tracking of refs is dijoint from the actual git repos16:05
clarkbits possible the gitea db thinks those refs exist but they aren't actually there in the git repo16:06
clarkbbecause gitea.... there is an issue around this and gitea sort of waved tehir hands around it and said not a problem16:06
fungii'm looking at the output of `git branch -a` after doing a remote update16:06
tonybI'm pretty sure that if the release tools were to recreate a branch it would be "on" at the tag/SHA in the "branches" config or at the very least the last release.  the bugfix/8.1 seems to be at $some_point16:06
fungii'm able to `git checkout origin/bugfix/8.0` too16:07
fungitonyb: good point16:07
fungithe tip of gerrit/bugfox/8.1 is which merged 2023-01-3116:08
clarkb was an open change until very recently16:09
tonybI at least have my local repo and gerrit agreeing:
clarkbmaybe my suggestion that branch deletion failed beacuse there were open changes is something16:09
fungiyeah, it's possible the branch was never actually deleted at all16:09
tonybclarkb: Yup I'm thinking that it plausible.16:10
fungiwhatever's going on with lingering branches in gitea is yet another issue, i guess. likely missed replication events if there was a burst of deletions all at once?16:10
tonybJayF: The sounds way more accusitory than I'd like or mean but, do you have any proof the branches were actually deleted?16:11
fungilike if we trigger re-replication of the repo in gerrit, they'll just disappear16:11
clarkbfungi: ya we might want to do semi regular full replications just to catch up any missed items16:11
JayFtonyb: I have the runbooks from when I did it, and I validated it at the time16:11
clarkbfungi: could also have been overlap in gitea load due to dos and events happenign together causing some of them to be missed16:11
JayFtonyb: and you can search that URL in the ML archives16:12
clarkbfwiw the example change I linked was cherry picked using gerrit if I'm reading the log correctly. When you cherry pick with gerrit it auto completes valid branches for you implying bugfix/8.1 was valid before cherry picking occured16:12
fungi was created targeting bugfix/8.1 on 2022-10-13, and looks like it would still have been open at the time branch deletion was attempted16:15
fungiso i agree, gerrit would not have allowed the bugfix/8.1 branch to be deleted with that change still open16:17
fungipcheli: not sure if you're around, but pinging you again since your ci now has 88 open connections to gerrit. at this rate, it will probably cease working some time next week. please look into whatever's breaking your ssh connections16:18
JayFfungi: note the date on my etherpad indicates 1/11/23 as retirement date16:18
tonybJayF: Do you have logs from the actual deletion?16:19
fungiJayF: right. change 861062 created 2022-10-13, branch deletion attempted 2023-01-11, change 861062 closed (abandoned) 2023-11-0916:20
clarkblooking at acls I think that ironic itself doesn't have perms ot create the branches. The release team or gerrit admin would haev to do that16:20
clarkbjust to rule out additional groups that may have taken action here16:20
JayFtonyb: no, I did it almost a year agop16:20
fungigerrit should have refused to delete the branch on 2023-01-11 because change 861062 was open for it at that time16:20
fungi861062 was opened over a year ago16:21
fungiand just abandoned today16:21
JayFit was in a weird state16:21
JayFW+1 but not landing16:22
JayFwhen I abandoned it this morning16:22
JayF(I thought that change had recreated it; but then I got digging, found more, and raised the questions here)16:22
fungiit's possible gerrit incorrectly reported deletion of that branch, i suppose, but it's not supposed to allow you to delete any branch with open changes16:23
clarkbJayF: I think there were no zuul jobs defined for it when it was pushed and +W'd as zuul took no action16:23
JayFfungi: It's also possible, because I was doing a large batch, and the middle batch failed for ACL issues, I missed that failure in the ACL failures16:23
clarkbthere are no check or gate jobs implying that no jobs were configured to run against that change. Possibly due to file matchers16:23
JayFfungi: one of the harms in having unautomated things (an issue I brought up with -ironic, and I think folks are taking up the mantle to remove this manual process entirely; and I am grateful)16:24
* JayF afk for a bit16:26
opendevreviewClark Boylan proposed opendev/system-config master: Improve etherpad functional testing
clarkbif that doesn't make things more clear I may have to hold a node16:35
clarkbaha docker compose by default tries to allocate a tty which is the inverse behavior of docker. That is why it complains about no tty16:57
tonybLOL, at least you know why.  switch to a call to docker instead?16:58
opendevreviewClark Boylan proposed opendev/system-config master: Improve etherpad functional testing
clarkbtonyb: I'm going to pass -T to disable tty allocation with docker-compose that way the test looks similar to our documented commands17:00
tonybSOunds good.17:01
tonybAlso +1 for your choice of 4-byte char and message :)17:01
clarkbtonyb: I was really happy when I found that one 17:01
* tonyb wonders how IRC/IRCCloud handles it .....17:02
clarkb🖖 here17:02
tonyb'🖖 Live long and prosper �17:02
clarkbthe second one in your message doesn't render for me for some reason17:03
clarkbmaybe the paste was truncated to fewer than 4 bytes?17:03
tonybI wonder if the copy/paste missed a byte or something17:03
tonybYeah that's what happened17:04
tonybhexdump says "20 ef bf bd 0a " so somehow it lost a byte and the "'"17:04
tonybYeah to get the full byte stream I need to include part of the next line17:07
tonybanyway I can see how you got nerdsniped17:07
clarkbtonyb: the real sniping happened when I ended up on a website with searchable glyphs by different category17:07
clarkbthere was a lot of scrolling and copy and paste into my termianl to see if my terminal font supported the glyphs :)17:07
clarkbrelated: years ago I was trying to buy a house and our mortage person at the bank always signed off their emails with 'J'. There were no J's in their name. I eventually figured out that wingdings J is equivalent to :) but in a single glyph. They used outlook which sent wingding chracter J which got rendered as a J and not the wingding because I'm not on windows17:11
clarkb has the mapping if you ever find weird characters in email and need ot translate17:21
fungii usually just open up the python repl and use or similar17:22
clarkbfungi: the problem here is that wingdings overloads ascii points17:23
fungiohhhh, got it17:23
clarkbI suspect the problem will slowly go away as utf8/unicode become more common17:23
fungii thought we were still talking about unicode17:23
fungii missed the transition to fonts17:23
opendevreviewClark Boylan proposed opendev/system-config master: Improve etherpad functional testing
clarkbturns out you must create the pad before you can set text on a pad17:28
clarkbbut this finally feels like progress17:28
tonybalso you seem to be having fun17:31
opendevreviewClark Boylan proposed opendev/system-config master: Improve etherpad functional testing
*** tosky_ is now known as tosky18:43
clarkbinfra-root passes testing now for functioanl testing of etherpad including utf8 4 byte character coverage18:55
clarkbtonyb: ^ infra-root is a highlight you may want ot add if you haven't already18:55
tonybyup I did it yesterday 18:56
*** dhill is now known as Guest645620:12
opendevreviewAmy Marrich proposed opendev/irc-meetings master: Updated D&I meeting to push back an hour due to time change
opendevreviewMerged opendev/irc-meetings master: [iwg] Update office hour date and time
opendevreviewMerged opendev/irc-meetings master: Updated D&I meeting to push back an hour due to time change
opendevreviewMerged opendev/system-config master: Improve etherpad functional testing
tonybclarkb: is still needed?23:28
tonybclarkb: Also

Generated by 2.17.3 by Marius Gedminas - find it at!