Friday, 2021-01-22

*** sshnaidm|ruck is now known as sshnaidm|afk00:08
*** tosky has quit IRC00:09
*** brinzhang_ has joined #opendev00:34
*** brinzhang has quit IRC00:37
*** lbragstad_ has joined #opendev00:38
*** lbragstad has quit IRC00:40
openstackgerritGhanshyam proposed opendev/subunit2sql master: Fix compatibility with latest oslo.config
*** _mlavalle_1 has quit IRC01:09
*** lbragstad_ is now known as lbragstad01:37
openstackgerritGhanshyam proposed opendev/subunit2sql master: Fix compatibility with latest oslo.config
*** lbragstad has quit IRC01:46
openstackgerritMerged openstack/project-config master: Set up access for #openinfra channel
*** hemanth_n has joined #opendev02:05
openstackgerritMerged opendev/system-config master: Set stop_grace_period on nodepool-builder containers
*** hamalq has quit IRC02:24
*** ysandeep|away is now known as ysandeep02:43
*** brinzhang_ has quit IRC02:44
*** brinzhang_ has joined #opendev02:45
*** ianw is now known as ianw_pto03:12
*** lbragstad has joined #opendev03:25
*** Mister_Q has joined #opendev03:53
*** Mister_Q has quit IRC04:28
*** ykarel has joined #opendev04:59
*** lbragstad has quit IRC05:11
*** marios has joined #opendev06:16
openstackgerritRico Lin proposed openstack/project-config master: Mark min-ready for ubuntu-focal-arm64
*** sboyron has joined #opendev06:32
*** rpittau|afk is now known as rpittau07:34
*** hashar has joined #opendev07:41
*** ralonsoh has joined #opendev07:48
*** eolivare has joined #opendev07:50
*** slaweq has joined #opendev07:54
*** ysandeep is now known as ysandeep|lunch08:00
openstackgerritMartin Kopec proposed opendev/system-config master: WIP Deploy refstack with ansible docker
*** andrewbonney has joined #opendev08:13
fricklermnaser: I finally got feedback from AS3320 regarding IPv6 from vexxhost. seems their main complaint is that there are no IRR records for your prefixes
openstackgerritMerged zuul/zuul-jobs master: Rename config repos file config for CentOS Stream
fricklermnaser: you should get that fixed, it is likely to affect more connectivity in the future, ping me if you need help with that08:16
*** fressi has joined #opendev08:16
*** tosky has joined #opendev08:44
*** jpena|off is now known as jpena08:57
*** klonn has joined #opendev09:08
*** zigo has quit IRC09:14
*** zigo has joined #opendev09:15
*** mordred has quit IRC09:21
*** Eighth_Doctor has quit IRC09:21
*** ysandeep|lunch is now known as ysandeep09:33
openstackgerritPranali Deore proposed openstack/project-config master: Add official-openstack-repo-jobs for openstack/glance-tempest-plugin
*** DSpider has joined #opendev09:40
openstackgerritMartin Kopec proposed opendev/system-config master: WIP Deploy refstack with ansible docker
*** dtantsur|afk is now known as dtantsur09:59
*** Eighth_Doctor has joined #opendev10:08
*** hemanth_n has quit IRC10:13
*** hemanth_n has joined #opendev10:13
*** mordred has joined #opendev10:27
openstackgerritMartin Kopec proposed opendev/system-config master: WIP Deploy refstack with ansible docker
*** hashar is now known as hasharAway10:44
*** hemanth_n has quit IRC10:57
*** hemanth_n has joined #opendev10:58
*** ykarel_ has joined #opendev11:08
*** ykarel has quit IRC11:08
*** ykarel__ has joined #opendev11:16
*** ykarel_ has quit IRC11:19
*** calcmandan has quit IRC11:22
*** calcmandan has joined #opendev11:22
*** ysandeep is now known as ysandeep|afk12:15
*** klonn has quit IRC12:21
*** iurygregory_ has joined #opendev12:28
*** iurygregory has quit IRC12:28
*** iurygregory_ is now known as iurygregory12:29
*** jpena is now known as jpena|lunch12:35
*** hasharAway is now known as hashar12:52
*** AJaeger has joined #opendev12:54
*** ysandeep|afk is now known as ysandeep12:56
*** hemanth_n has quit IRC13:11
*** fressi has quit IRC13:12
*** tkajinam_ has quit IRC13:16
*** sboyron has quit IRC13:16
*** sboyron has joined #opendev13:27
*** jpena|lunch is now known as jpena13:34
openstackgerritMaksim Malchuk proposed openstack/diskimage-builder master: Don't use hardcode while override base image file
*** lbragstad has joined #opendev13:42
*** ykarel__ is now known as ykarel13:44
*** AJaeger has quit IRC13:53
openstackgerritJavier Peña proposed openstack/diskimage-builder master: Fix CentOS Stream 8 base repo in centos element
openstackgerritJavier Peña proposed zuul/zuul-jobs master: Fix BaseOS repo file name for CentOS Stream
*** mlavalle has joined #opendev13:58
*** klonn has joined #opendev14:00
*** ysandeep is now known as ysandeep|away14:06
*** klonn has quit IRC14:18
*** klonn has joined #opendev14:18
openstackgerritGuillaume Chauvel proposed opendev/system-config master: Increase autogenerated comment width to avoid line wrap
openstackgerritGuillaume Chauvel proposed opendev/system-config master: [DNM] test comment width: review without autogenerated tag
openstackgerritGuillaume Chauvel proposed opendev/system-config master: Increase autogenerated comment width to avoid line wrap
openstackgerritGuillaume Chauvel proposed opendev/system-config master: [DNM] test comment width: review without autogenerated tag
openstackgerritSorin Sbârnea proposed opendev/git-review master: Allow choosing field for author in named branch
*** rpittau is now known as rpittau|afk14:58
openstackgerritMerged opendev/git-review master: Allow choosing field for author in named branch
*** mugsie has joined #opendev15:09
*** mugsie has quit IRC15:18
*** zbr3 has joined #opendev15:22
*** zbr has quit IRC15:24
*** zbr3 is now known as zbr15:24
*** mugsie has joined #opendev15:51
*** lpetrut has joined #opendev16:10
*** zul has quit IRC16:11
*** lpetrut has quit IRC16:14
openstackgerritJavier Peña proposed zuul/zuul-jobs master: Fix repo files for CentOS Stream
*** ykarel has quit IRC16:47
openstackgerritJavier Peña proposed zuul/zuul-jobs master: Fix repo files for CentOS Stream
*** marios has quit IRC17:15
openstackgerritMartin Kopec proposed opendev/system-config master: WIP Deploy refstack with ansible docker
*** eolivare has quit IRC17:18
fungipaladox: tristanC: do either of you have a basic understanding of how notedb functions? we've been trying to work out how to manually update an external-id in modern gerrit but questions to the ml have been met with silence17:20
clarkbin particular All-Users:refs/meta/external-ids doesn't appear to be a notes thing if you `git fetch origin refs/meta/external-ids:refs/meta/external-ids && git notes list refs/meta/external-ids`. So it isn't clear how one is meant to update the external ids notes in that ref17:21
clarkbif you git log refs/meta/external-ids you get what appears to be a normal git tree history17:22
fungishort story is a user's openid seems to have changed, we need to associate it with their prior account. in earlier gerrit we'd just update the row in the external_ids table, but with notedb it seems like we need to commit files in refs/meta/external-ids and then push (potentially racing other updates to external ids when doing so)17:22
fungithere's no rest api method we can find documented for adding or changing external ids, only fetching and deleting17:23
clarkb has some interesting related info (though not the answer to our questions as far as I can tell)17:24
clarkbsomeone found a way to do it then didn't write it down :(17:24
*** hashar has quit IRC17:25
corvusclarkb: i ran into a problem recently using a (non-opendev) current gerrit with openid (not sso, any openid) where i set a username for my account, then removed the original email address, and ended up with a new user on next login.  the only way i could correct the situation was to shut gerrit down and rewind all-users.17:26
corvusthat sounds similar (but maybe not exactly) like what your msg describes.17:26
fungiyeah, this could be similar17:27
corvusto be fair, i did think that there might be a way to correct it with the aid of a second admin account and three different transitions, but that sounded way harder than rewinding, and i didn't want to go make another account17:27
fungialso that wmf phab entry frightens me. i think once we do try to push this we're going to hit the same validation errors because i know for a fact that we have accounts with duplicate e-mail addresses17:27
clarkbfungi: ya17:28
fungiseems like it indicates we'll have to go through and clean up (likely hundreds) of old accounts which previous gerrit allowed to have duplicate addresses17:28
clarkbwhat is really confusing to me is that refs/meta/external-ids doesn't seem to be notes but a proper git tree17:28
clarkbhowever the contents of that tree look note like so I'm all kinds of confused17:29
corvusclarkb, fungi: the main thing i think was the issue in my case was the inequivalency of different external ids.  there seems to be a 'primary' id when you look at all-users, and doing certain operations changes that.  to the point where the actual openid didn't matter for the account anymore, it was the email address gerrit cared about.17:29
corvusclarkb: there's 2 things, let me dig them up.  1 sec17:29
clarkbcorvus: yes in this case we just want to change the account id in the note for the openid url I think17:29
clarkbwe're poking at it on review-test17:30
corvusclarkb: refs/meta/external-ids is a tree with each file the sha of the external id17:31
clarkbcorvus: yup17:31
corvusso you look up the external id "username:corvus" by sha-ing that and looking up the file with that name17:32
clarkbyup where we've gotten is roughly git fetch origin refs/meta/external-ids && git checkout FETCH_HEAD17:32
clarkbthen I can vim the files and edit the account id for the openid file I want17:32
openstackgerritGuillaume Chauvel proposed opendev/system-config master: Increase autogenerated comment width to avoid line wrap
openstackgerritGuillaume Chauvel proposed opendev/system-config master: [DNM] test comment width: review without autogenerated tag
clarkbbut thats a very normal git tree appraoch but gerrit implies these are git notes instead. I think the distinction matters because if I do a git commit after vim'ing some files and push that to refs/meta/external-ids I'm likely to conflict with any updates that happened between when I fetched and when I pushed?17:33
clarkbor at leasat the risk of racing is there17:33
corvusah right, then there's refs/users which has the account info (including preferred email)17:33
corvusi think they mostly wrote-off the idea of actually using git notes17:34
corvusclarkb: so i think you understand that file correctly.17:34
clarkbcorvus: hrm that would match what I observe except for the magical tree sharding which is supposedly something ti does automatically for objects?17:34
corvusi assumed that was just convention -- like, it's a ref pointing to a tree with a bunch of files and they didn't want a directory with millions of entries17:35
corvusbut i don't know for sure17:35
clarkbcorvus: well I pushed a docs update for it beacuse it was super confusing when I first ran into this and they told me no git does that automatically for notes17:35
fungibut apparently it's variable-depth, based on some algorithm git also uses for note storage17:35
clarkband the level varies based on total numbers of objects17:35
corvusclarkb: if you do decide to update the repo, remember the cache will need to be cleared if you do it while server is up.17:36
clarkbcorvus: ok, do you know if accounts need to be reindexed too?17:36
corvusclarkb: weird.  maybe they're using the underlying implementation for git notes just not actual notes.17:36
fungiwhich is not too dissimilar from needing to clear the cache after we updated tables in sql previously17:36
clarkb(that was another question on my thread that has been ignored)17:36
paladoxfungi you can git clone the repo and make changes and git push i think (then reindex).17:37
corvusclarkb: i don't know.  i would think that would be a good idea though.17:37
corvus(ie, i would guess 'yes' it needs to be reindexed)17:37
clarkbpaladox: do we make those changes with git notes add/edit or vim and then git commit?17:37
clarkbif we do it via git commit then the process is inherently racy I believe17:38
fungipaladox: do you know how (or if) it avoids undoing changes which occur in the seconds between pull/edit/commit and push?17:38
clarkbbut if you want the gerrit checks to run you have to push to gerrit and not directly when gerrit is off17:38
fungithough i suppose pushing to the filesystem with gerrit stopped is how we'd avoid needing to immediately clean up the mess of duplicate e-mail addresses alluded to at the beginning of
clarkbfungi: unless it somehow decides to not start up at all17:39
clarkbI feel like that is more risky17:39
corvusclarkb: why would there be duplicates?17:40
clarkbpaladox: maybe you can give us a high level overview from git fetch to checkout to edit to commit to push?17:40
clarkbcorvus: see
clarkbcorvus: fungi expects we're in the same situation17:40
corvusclarkb: i mean in our case17:40
clarkbcorvus: I think its a hunch based on the number of accounts we've got and general user ability to break things that are breakable :)17:40
corvuswhy would fixing this cause a startup error?17:41
clarkbcorvus: if we bypass gerrit's checking and push to git directly as a way to workaround conflicts then we potentially add other problems to the "db" that gerrit will refuse to start on17:42
paladoxclarkb i just git clone the repo, edit .git/config (replacing refs/heads/ with refs/*) then i checkout external-id and then i edit the file. Then i push back to the branch.17:42
clarkbgerrit checks more than just email addr conflicts. I'm not sure if any of those additioanl issues it checks for would be fatal if we introduced one via a side load17:42
paladoxyou can edit via the disk too17:42
corvusi guess i'm missing why changing one account is going to cause hundreds of new errors to appear on startup17:43
clarkbcorvus: no I'm suggesting that if we get the one account change wrong then perhaps gerrit will not start17:43
corvusor are you saying you're concerned that the "push to gerrit" path will fail, not the push to disk17:43
clarkbright if we pushed to gerrit gerrit is going to validate a number of potential issues with the edit17:43
clarkbif we push directly that does not happen and we could potentially break the whole thing (I don't have evidence that is a real concern, just that it seems possible)17:44
corvusclarkb: if the supposition is that we have 200 existing errors, i don't see how having 201 would cause it to fail to start.17:44
clarkbcorvus: because there are ~6 errors checekd for and the ~200 is only one of those 617:44
clarkbpaladox: ok, isn't that racy? if you fetch, then a user updates their rest api token, then you edit and push you'll overwrite their changes17:45
corvusthere's probably too many unknowns here.  seems like testing on review-test is warranted either way.17:45
clarkbpaladox: this was one of my questions on the mailing list from a couple of days ago17:45
clarkbcorvus: yup that was the paln all along. fungi I guess we do the edit, commit and push on review-test like paladox suggests and see what we learn?17:45
paladoxas long as you don't force git push, it *should* ask you to git pull if someone updated it.17:45
clarkbpaladox: ya I mentioend that you could git rebase and push in a loop until it goes through in my email too17:46
fungicorvus: sorry, troubleshooting something unrelated at the same time, but catching up the reason we have duplicate e-mail addresses is that the openid plugin used to create them, in newer gerrit it will properly refuse to, but under older gerrit it did not have that17:46
clarkbpaladox: mostly I was trying to call out that this process seems incredibly complicated compared to the old system and it isn't well documented and help would be appreciated :)17:46
paladoxYeh, agreed.17:46
clarkbpaladox: in particular I think that races like that should be called out in the docs17:47
clarkbbecause for many git users I bet git push -f is their first reaction :/17:47
paladoxi think that some tool should be created that allows you to do this without worry.17:47
clarkbanyway, fungi I think our next step is backup All-Users on review-test, commit then push and see what happens?17:47
funginot just a hunch that we have duplicate e-mail addresses, i know for a fact that there are a lot because i have workarounds in the openstack election tooling to work around it and combine/deduplicate account records when two or more accounts have the same e-mail addresses17:47
clarkbfungi: got it17:48
fungibasically normal account creation in gerrit would refuse to add an account with the same address as an existing account, or would refuse to let you add an address to an account if it was already set on another account, but the openid plugin didn't go through those checks when autocreating new accounts and would happily create a new account with the same address as an existing one (for example, if the openid17:49
fungichanged but the openid provider continued to assert the same e-mail address for it)17:49
fungialso new gerrit has gotten stricter it seems, i've seen evidence that it will now refuse to add an address to an account even if that address is only already used by a disabled (inactive) account17:51
*** ralonsoh has quit IRC17:51
fungiso our old method of just setting duplicate accounts inactive without deleting the addresses from them will likely provide even more duplicates17:51
clarkbfungi: for backing up All-Users I figure we stop Gerrit, then rsync the All-Users repo to another location, then restart Gerrit?17:51
clarkbjust to avoid any potential updates there too?17:51
clarkbpaladox: ya another thing I pointed out on the mailing list is you can get and delete external ids but not edit them via the rest api17:52
clarkbpaladox: having an edit api would be great then tools could use that17:52
fungipaladox: probably the "some tool" would be just extending the rest api to add methods for creating/updating external ids similar to the existing methods for getting and deleting them17:52
fungithe complexity there is likely in makign sure those go through the right validation codepaths17:53
clarkband avoiding races17:53
clarkbgerrit could sequence them internally if it was handling them all17:53
*** klonn has quit IRC17:54
*** jpena is now known as jpena|off17:55
clarkbfungi: I've gone ahead and committed my edits in my cloned All-Users17:58
clarkbwill stop gerrit on review-test and work to make a copy of prod All-Users next17:59
clarkbthen we can psuh and see what happens17:59
fungiyeah, as far as races go, presumably it already know how to avoid them when handling its own updates to the refs/meta/external-ids so that bit is presumably solved18:01
fungias far as the rest api would be concerned18:01
clarkbfungi: how does that rsync command in screen look?18:01
fungilooks legit. you've correctly omitted the trailing / on the source path so it should create /home/gerrit2/backups/All-Users.git18:04
fungii assume /home/gerrit2/backups/All-Users.git doesn't currently exist?18:04
fungiif it does you'll probably want to add --delete18:04
fungiso you don't end up with extra files in there18:04
clarkbits empty18:05
clarkbI'll run that comamdn now for real18:05
clarkbok starting gerrit back up again18:06
*** slaweq has quit IRC18:12
*** lbragstad has quit IRC18:16
*** dtantsur is now known as dtantsur|afk18:17
TheJuliais gerrit running a bit slow today?18:19
*** klonn has joined #opendev18:19
fungiload average was really high a little while ago18:19
fungi>40 on a 16-cpu vm18:20
TheJuliahmm, okay18:20
fungiit's not exactly low now either, but it's not as high as it was around 16:30z18:20
TheJuliayeah, I think it was a little before then I uploaded a batch of patches and was like "wow, this is taking a while"18:21
*** lbragstad has joined #opendev18:22
corvusTheJulia: you wrote so much code you really made the gerrit server work :)18:25
* TheJulia blushes18:26
*** klonn has quit IRC18:33
*** klonn has joined #opendev18:33
clarkbfwiw it is a fairly regular load spike around then18:36
clarkbat one time I thought it might be related to backups but we run backups ~12 hours later without the same issues18:36
clarkbpossibly some ci system doing periodic jobs or thats when we've got the most user overlap and we just hit the server harder then18:36
clarkbwill likely need diggign in logs to characterize ebtter18:36
fungiuser overlap has been my assumption. we get pacific coast of the americas getting to their desks around then, and europe hasn't quite finished for the day yet18:37
clarkbcorvus: paladox: ok fungi and I have discovered some new things. Gerrit will reject the push if it isn't fast forwardable so my hunch that we rebase and retry in a loop when pushing to a live server seems to eb a good one. We have 642 email address errors18:37
fungiglad to know that my estimate was on the low side... :/18:38
clarkbthese 642 email conflict errors seem to come in two versions, first is two different oepnid external ids sharing an email. The other is mailto and openid external ids sharing an email18:38
fungistill for different account ids in the latter case right?18:38
clarkbI'm going to pause here and write a response to myself on repo-discuss explaining what the process we'ev used is and what we've learned before it escapes my head18:39
fungii assume a mailto and an openid having the same addy is fine if they're for the same account id18:39
clarkbfungi: oh tahts a good question I can check that18:39
fungiclarkb: awesome, then you'll no longer be able to say nobody has responded to your questions! ;)18:39
clarkbbut then I think our process might become: push edits to live server to ensure that gerrit doesn't complain about the changes we are making (does it report all validations or just the first set it checks when failing). Once gerrit doesn't complain about our changes we can stop gerrit, rebase to ensure we are fast forwardable, then push into git directly under gerrit. Then start gerrit, reindex and clear18:40
clarkbI'll suggest that as a workaroudn on the list too to see if anyone complains18:40
clarkbin the meantime we might want to give DannyMassa the go ahead to use the new account or something? I don't know what the right answer is for that, but I don't think we'll have this sorted today18:41
*** bodgix has quit IRC18:41
*** bodgix_ has joined #opendev18:41
fungilonger term we'll probably want to work through the conflicts and try to clean them up, but in many cases that's likely to involve reaching out to users who are unaware they're using two accounts because their openid split from their ssh login at some point years ago and they've been pushing changes with one account and reviewing with a different one18:41
fungiin the past we really only ever found out if someone had trouble adding them to a review group or as a requested reviewer on a change18:42
clarkbfungi: first example I've checked is two different accounts18:43
mordredfungi: that sounds like a "fun" group of conflicts to rectify18:43
fungiyeah, that's what i'm expecting (or at least hoping) they'll all be18:43
clarkbfungi: that example is now in the screen if you want to check it18:43
fungiyeah, that example looks exactly like what i expected18:44
clarkbI'm going to put that list of conflicts in my homedir on review-test too18:44
clarkbthen I'm making tea18:44
fungiif we cross-reference against inactive accounts we can probably clean up a bunch, but i'm not sure if it will be a majority18:44
clarkb~clarkb/conflicting_emails if you want to check them18:45
clarkbpaladox: do you know how wikimedia worked around these conflicts?18:46
clarkbpaladox: the bug mentions it was worked around, but doesn't say how18:46
fungiwe might also just send a form letter to the preferred addresses for the accounts in that state letting them know we need to deactivate one of their accounts and if we don't hear back from them then deactivate both or pick one at random or something18:52
fungiwhere "deactivate" would imply marking them inactive and removing the conflicting external-ids18:52
*** andrewbonney has quit IRC18:56
fungiokay, my dance card for the day is topped up, i've got to get through stuff i've already committed to, so probably going relatively silent to try to get most of it donw18:57
paladoxclarkb i think they just edited on disk18:59
fungiso basically what we're looking at doing19:00
fungiat least that's confirmation that it's probably the easiest interim solution19:01
clarkbfungi: thank you for all the help on this19:01
paladoxyou push then reindex19:01
fungipaladox: figured. possibly also flush caches since accounts may be persisted in the disk cache between restarts?19:02
funginot all that different from what we used to have to do when editing the db behind gerrit's back19:03
clarkbpaladox: do you know if that was done with or without gerrit online?19:04
paladoxit was done with gerrit online19:04
paladoxsince the only difference is you are editing directly skipping gerrit19:05
paladoxso you just git clone <path to gerrit repo on disk>19:05
fungii suppose as long as gerrit checks the state of the repo on disk every time it commits to refs/meta/external-ids it won't get flustered. if it assumes nothing besides it is touching that ref, then it might throw errors until reindexed?19:06
fungior maybe it does the same thing we would do: try to push and if it fails rebase onto what's on disk and try again19:07
clarkb aha19:09
clarkbthat says offline though19:09
fungiTheJulia: here you can see it wasn't just gerrit that got busy around 16:30, we apparently topped out the zuul executors to the point they stopped accepting build requests, node requests shot up from almost nothing to nearly 1k, and so on...
fungiso my guess is a large dump of changes entered gerrit around that time19:24
clarkbalright email sent to repo-discuss. I also sent email to DannyMassa to give an update as well19:37
*** slaweq has joined #opendev19:46
fungisaw it, thanks!19:49
*** slaweq has quit IRC20:01
openstackgerritMerged opendev/subunit2sql master: Fix compatibility with latest oslo.config
*** auristor has quit IRC20:20
*** auristor has joined #opendev20:23
openstackgerritMerged opendev/bindep master: Tolerate missing bindep.txt when using --profiles
prometheanfireshould gean work when distro is defined as networkd on debian?21:26
prometheanfireI imagine the answer is yes21:26
clarkbI don't know that anyone has ever tested it21:26
clarkbwe use it with the e/n/i support21:26
prometheanfireprobably I'm having is when two interfaces come up both with default gateways...  debian network scripts go boom21:27
prometheanfireeni being networkmanager?21:27
fungibasically the difference between "should work" (we consider it a bug if not) and "does work" (we actually test and use it that way)21:39
openstackgerritArtom Lifshitz proposed zuul/zuul-jobs master: WIP: Generate Zuul temp key in PEM format
prometheanfirefungi: yep21:56
prometheanfireon debian glean outputs dhcp for static IP networkd, so gonna look at that22:18
*** CWalker has quit IRC22:20
*** klonn has quit IRC22:59
*** paladox has quit IRC23:14
*** paladox has joined #opendev23:17
clarkb#status log Removed email from DannyMassa's stale gerrit account so that a new account can be created with that email. This is admittedly a workaround but we need to figure out external-id edits more properly to properly solve this.23:39
openstackstatusclarkb: finished logging23:39
*** paladox has quit IRC23:40
clarkbthis was suggested by the user fwiw23:51
openstackgerritJames E. Blair proposed opendev/system-config master: Remove old service cleanups from zuul
clarkbwe might also consider disabling the preexisting accounts23:53
clarkbbut I think that can wait for when we're "happy" with the running state23:53
*** paladox has joined #opendev23:55
artomIs Depends-on: <zuul-jobs patch> allowed? Will it use the patch when deploying?23:57
artomI've Depends-on, but it doesn't look like it took hold...23:57
clarkbartom: it depends on whether or not that contents of the zuul-jobs cahnge are executed by a trusted or untrusted portion of a job23:58
artomOr I've grievously misunderstood something23:58
clarkbif they are part of a trusted job portion then they have to be landed first23:58
artomclarkb, setting up the ssh build keys?23:58
clarkbya thats part of opendev/base-jobs/playbooks/pre.yaml which is trusted23:59

Generated by 2.17.2 by Marius Gedminas - find it at!