| *** whoami-rajat has quit IRC | 00:15 | |
| *** tosky has quit IRC | 00:17 | |
| fungi | okay, hearing no objections i've pre-written the git-review 2.0.0 release announcement so will push the tag now and then approve 778056 so the docs get republished with the new tag present, and then send the announcement i've queued up | 00:23 |
|---|---|---|
| clarkb | sounds good | 00:24 |
| clarkb | I'm about to take advantage of some sun and give the brain a rest. Hope to get through the rest of those accounts tomororw | 00:24 |
| clarkb | ianw: its a whole 12C and that is some of the best weather we have had in a while :) | 00:25 |
| ianw | heh, yeah i can't brag ATM it's like 15C here ATM | 00:26 |
| fungi | looking at the test nodes and node requests graphs today, i think the extra nodes from inap have made a great improvement | 00:26 |
| fungi | we seem to have lpateaued a few times, but it was off and on, not all day | 00:27 |
| fungi | er, plateaued | 00:27 |
| fungi | new git-review is up now: https://pypi.org/project/git-review | 00:31 |
| openstackgerrit | Merged opendev/git-review master: Remove comments for unstaged/uncommitted tests https://review.opendev.org/c/opendev/git-review/+/778056 | 00:46 |
| fungi | docs promote job for ^ has completed, next vos release for that volume should occur in two minutes | 00:48 |
| *** SotK has quit IRC | 00:58 | |
| *** SotK has joined #opendev | 00:58 | |
| fungi | https://docs.opendev.org/opendev/git-review/latest/releasenotes.html looks correct now, so sending release announcement | 01:00 |
| fungi | and sent | 01:01 |
| *** mlavalle has quit IRC | 01:02 | |
| *** dries has quit IRC | 02:04 | |
| fungi | #status log released git-review 2.0.0 | 02:21 |
| openstackstatus | fungi: finished logging | 02:21 |
| *** hamalq has quit IRC | 02:25 | |
| *** dtantsur has quit IRC | 02:25 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] handle zuul-summary-results as .jar / per-project config https://review.opendev.org/c/opendev/system-config/+/778116 | 02:29 |
| *** dtantsur has joined #opendev | 02:29 | |
| ianw | clarkb: ^ that appears to be hanging doing gerrit init. i can't replicate it :/ i have a feeling it might have to do with pulling from the intermediate repository. anyway, still debugging but if you see anything similar ... | 02:33 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Fix up openafs-client job matching https://review.opendev.org/c/opendev/system-config/+/778353 | 02:43 |
| *** hemanth_n has joined #opendev | 02:49 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: install-ansible: ensure stevedore https://review.opendev.org/c/opendev/system-config/+/778354 | 03:10 |
| ianw | clarkb / frickler: ^ | 03:11 |
| *** dtantsur has quit IRC | 03:14 | |
| *** dtantsur has joined #opendev | 03:15 | |
| ianw | ok, i've in-place ugpraded afsdb01 to focal now. went ok, i manually ran base and just had to clear the ansible cache because it picked it up wrong. i'll do 02 in a bit | 04:14 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] handle zuul-summary-results as .jar / per-project config https://review.opendev.org/c/opendev/system-config/+/778116 | 04:20 |
| *** ykarel has joined #opendev | 04:23 | |
| *** dviroel has quit IRC | 04:55 | |
| *** whoami-rajat has joined #opendev | 04:58 | |
| *** brinzhang has joined #opendev | 05:41 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [wip] handle zuul-summary-results as .jar / per-project config https://review.opendev.org/c/opendev/system-config/+/778116 | 06:01 |
| *** marios has joined #opendev | 06:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/project-config master: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/778358 | 06:07 |
| openstackgerrit | Merged openstack/project-config master: Normalize projects.yaml https://review.opendev.org/c/openstack/project-config/+/778358 | 06:54 |
| *** mrunge has quit IRC | 06:56 | |
| *** mrunge has joined #opendev | 06:57 | |
| *** eolivare has joined #opendev | 07:22 | |
| ianw | #status log afsdb01 and afsdb02 in-place upgraded to focal | 07:32 |
| openstackstatus | ianw: finished logging | 07:32 |
| ianw | i've manually run the base & afs playbooks against them, so i'm confident they will just keep ticking along now | 07:32 |
| *** ralonsoh has joined #opendev | 07:37 | |
| *** gnuoy has joined #opendev | 07:37 | |
| *** fressi has quit IRC | 07:45 | |
| *** fressi has joined #opendev | 07:46 | |
| *** rpittau|afk is now known as rpittau | 07:52 | |
| *** sboyron has joined #opendev | 07:57 | |
| *** andrewbonney has joined #opendev | 08:14 | |
| *** iurygregory_ has joined #opendev | 08:17 | |
| *** iurygregory has quit IRC | 08:18 | |
| *** tosky has joined #opendev | 08:34 | |
| *** jpena|off is now known as jpena | 08:57 | |
| *** hashar has joined #opendev | 09:40 | |
| *** fressi has quit IRC | 09:52 | |
| openstackgerrit | Mateusz Kowalski proposed openstack/diskimage-builder master: Change paths for bootloader files in iso element https://review.opendev.org/c/openstack/diskimage-builder/+/777606 | 09:57 |
| openstackgerrit | Mateusz Kowalski proposed openstack/diskimage-builder master: Change paths for bootloader files in iso element https://review.opendev.org/c/openstack/diskimage-builder/+/777606 | 10:02 |
| *** sshnaidm|afk is now known as sshnaidm | 10:06 | |
| *** ykarel_ has joined #opendev | 10:20 | |
| *** lpetrut has joined #opendev | 10:22 | |
| *** ykarel has quit IRC | 10:23 | |
| *** fressi has joined #opendev | 10:27 | |
| *** ykarel_ is now known as ykarel | 10:31 | |
| openstackgerrit | David Ostrovsky proposed opendev/system-config master: Remove obsolete Bazel spawn strategies https://review.opendev.org/c/opendev/system-config/+/778404 | 10:32 |
| *** fressi has quit IRC | 10:39 | |
| *** fressi has joined #opendev | 10:57 | |
| *** fressi has quit IRC | 11:02 | |
| *** dviroel has joined #opendev | 11:05 | |
| *** zoharm has joined #opendev | 11:11 | |
| *** iurygregory_ is now known as iurygregory | 11:19 | |
| *** fressi has joined #opendev | 11:20 | |
| *** bhagyashris is now known as bhagyashris|rove | 11:30 | |
| *** bhagyashris|rove is now known as bhagyashri|rover | 11:30 | |
| *** hashar is now known as hasharLucnh | 11:37 | |
| *** hasharLucnh is now known as hasharLunch | 11:37 | |
| *** hasharLunch has quit IRC | 11:59 | |
| *** fressi has quit IRC | 12:08 | |
| *** eolivare_ has joined #opendev | 12:19 | |
| *** eolivare has quit IRC | 12:23 | |
| *** hashar has joined #opendev | 12:29 | |
| *** jpena is now known as jpena|lunch | 12:30 | |
| *** fressi has joined #opendev | 12:31 | |
| *** hemanth_n has quit IRC | 12:32 | |
| *** hemanth_n has joined #opendev | 12:36 | |
| *** hemanth_n has quit IRC | 12:40 | |
| *** eolivare_ has quit IRC | 12:43 | |
| *** hashar has quit IRC | 12:45 | |
| *** ykarel_ has joined #opendev | 12:46 | |
| *** ykarel has quit IRC | 12:49 | |
| fungi | ianw: thanks, so general rule of thumb is we need to clear the ansible cache immediately following an in-place upgrade? | 12:50 |
| *** ykarel__ has joined #opendev | 12:55 | |
| *** ykarel__ is now known as ykarel | 12:55 | |
| *** ykarel_ has quit IRC | 12:57 | |
| *** eolivare_ has joined #opendev | 13:26 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| *** ykarel_ has joined #opendev | 13:39 | |
| *** hemanth_n has joined #opendev | 13:41 | |
| *** ykarel has quit IRC | 13:42 | |
| *** ykarel_ is now known as ykarel | 13:42 | |
| *** hemanth_n has quit IRC | 13:45 | |
| *** hashar has joined #opendev | 13:49 | |
| *** toomer has joined #opendev | 13:53 | |
| openstackgerrit | Merged opendev/irc-meetings master: Remove usused Vitrage meeting slot https://review.opendev.org/c/opendev/irc-meetings/+/777193 | 14:12 |
| *** iurygregory has quit IRC | 14:27 | |
| *** iurygregory has joined #opendev | 14:29 | |
| *** zoharm has quit IRC | 14:47 | |
| *** zoharm has joined #opendev | 14:57 | |
| fungi | popping out to run some errands but should be back by 16:00z | 15:01 |
| *** lpetrut has quit IRC | 15:09 | |
| *** fressi has quit IRC | 15:14 | |
| openstackgerrit | Sorin Sbârnea proposed openstack/project-config master: Add tripleo-ci-health-queries to zuul https://review.opendev.org/c/openstack/project-config/+/778489 | 15:39 |
| clarkb | fungi: we need to clear out the ansible cache anytime the details of a particular host change dramaticly. Another example would be replacing a server with like for like under the same inventory name aiui | 15:53 |
| *** zoharm has quit IRC | 15:55 | |
| *** ykarel has quit IRC | 16:05 | |
| fungi | yep, got it | 16:05 |
| fungi | makes total sense | 16:05 |
| clarkb | ianw: thank you for pushing on the afs stuff. I'm trying to finish up the gerrit account stuff but then will take a look at the gerrit init thing | 16:19 |
| auristor | ianw: the vlserver and ptservers on afsdb01 and afsdb02 look good from here | 16:21 |
| mordred | have I mentioned that I think it's super neat that auristor can look at things like that? | 16:22 |
| auristor | I'm only taking advantage of the lack of privacy features of openafs :-) | 16:23 |
| fungi | public transparency is one of the things i appreciate about it. then again, i miss the days when i could telnet to just about any site as guest and request an account | 16:24 |
| auristor | the remote administration from anywhere features is one of the strengths of the afs-family architecture | 16:25 |
| *** smcginnis has quit IRC | 16:25 | |
| mordred | ++ | 16:30 |
| mordred | fungi: I remember thinking the web was stupid (just pages of links to other pages of links) compared to all the lovely ftp sites | 16:31 |
| mordred | I might not be the fastest adopter of new tech | 16:31 |
| fungi | i remember thinking it was a mild improvement over gopher, but that was about it | 16:32 |
| fungi | then again, i also thought embedded images and media files were also a passing fad, same for html e-mail | 16:33 |
| *** smcginnis has joined #opendev | 16:34 | |
| *** Dmitrii-Sh has quit IRC | 16:47 | |
| *** Dmitrii-Sh has joined #opendev | 16:47 | |
| *** mlavalle has joined #opendev | 16:51 | |
| *** hashar has quit IRC | 16:56 | |
| *** hashar has joined #opendev | 16:59 | |
| *** ralonsoh has quit IRC | 17:01 | |
| *** stand has quit IRC | 17:01 | |
| *** mkowalski has quit IRC | 17:01 | |
| *** redrobot has quit IRC | 17:01 | |
| *** JohnnyRa1 has quit IRC | 17:01 | |
| *** slittle1 has quit IRC | 17:01 | |
| *** mgoddard has quit IRC | 17:01 | |
| *** openstackgerrit has quit IRC | 17:02 | |
| *** ralonsoh has joined #opendev | 17:05 | |
| *** stand has joined #opendev | 17:05 | |
| *** marios is now known as marios|out | 17:06 | |
| *** slittle1 has joined #opendev | 17:07 | |
| *** mkowalski has joined #opendev | 17:07 | |
| *** redrobot has joined #opendev | 17:07 | |
| *** JohnnyRa1 has joined #opendev | 17:07 | |
| *** mgoddard has joined #opendev | 17:07 | |
| clarkb | fungi and I have run the external id cleanups for the ~35 identified inactive accounts | 17:13 |
| clarkb | I am running consistency checks now to do a diff against | 17:13 |
| clarkb | these were all the accounts similar to smcginnis' where one account is active and the other is inactive. We should've only modified the inactive side (and the script has a check for active accoutns and will skip if active) | 17:14 |
| clarkb | also the logs for that made it into review alonside the logs for previous cleanups | 17:27 |
| *** eolivare_ has quit IRC | 17:57 | |
| *** jpena is now known as jpena|off | 18:00 | |
| yoctozepto | morning | 18:07 |
| yoctozepto | does the zuul "eager run" of newly added jobs does not apply to project templates? | 18:08 |
| yoctozepto | for ref see https://review.opendev.org/c/openstack/python-masakariclient/+/778513 | 18:08 |
| yoctozepto | I added the project template and thought I would get its job run | 18:08 |
| yoctozepto | (asking to make sure I understand it right) | 18:09 |
| clarkb | it should apply to anything in untrusted config | 18:09 |
| clarkb | https://opendev.org/openstack/openstackclient/src/branch/master/.zuul.yaml#L35-L40 defines the template in an untrusted context so I would've expected that to run | 18:10 |
| clarkb | yoctozepto: does that job filter files? | 18:10 |
| yoctozepto | clarkb: checked, it filters branches | 18:10 |
| yoctozepto | does not run on stable branches | 18:11 |
| yoctozepto | but this is master | 18:11 |
| clarkb | it shouldn't filter branches that way | 18:11 |
| clarkb | I don't think that is the problem but those branch filters never work the way people expect | 18:11 |
| *** rpittau is now known as rpittau|afk | 18:11 | |
| yoctozepto | yeah, it does not trigger on the followup patch I tried | 18:12 |
| yoctozepto | so the project template is busted | 18:12 |
| yoctozepto | oh well | 18:12 |
| clarkb | yoctozepto: what happens if you try to add the job directly without the template? if that doesn't work then I would look at the job, if that does work then look at the template | 18:12 |
| yoctozepto | clarkb: yeah, I think I will try that too | 18:13 |
| yoctozepto | though I am supposed to use the template | 18:13 |
| clarkb | oh osc does not have branches so the branch exclusion for stable is probably ok in this context (it causes problems when you have branches and do exclusions that conflict with the current branch) | 18:13 |
| clarkb | yoctozepto: I think I see why this is happening | 18:14 |
| clarkb | yoctozepto: the openstackclient .zuul.yaml is broken | 18:14 |
| yoctozepto | oh gosh | 18:14 |
| yoctozepto | end of the world | 18:14 |
| clarkb | yoctozepto: https://zuul.opendev.org/t/openstack/config-errors search for openstackclient | 18:15 |
| yoctozepto | oh, nice! | 18:15 |
| yoctozepto | I am really glad to be the one to find global issues lol | 18:15 |
| yoctozepto | :D | 18:15 |
| clarkb | Unknown projects: openstack/python-karborclient | 18:15 |
| clarkb | seems to be the root cause | 18:16 |
| yoctozepto | yeah, need to add more instructions for retirement | 18:16 |
| yoctozepto | clarkb: are you proposing the fix now? | 18:16 |
| clarkb | I am not (sorry still digging into gerrit account stuff) | 18:18 |
| yoctozepto | fwiw, the project is here but obviously retired https://opendev.org/openstack/python-karborclient | 18:18 |
| yoctozepto | ah yes | 18:19 |
| yoctozepto | it no longer has any zuul config | 18:19 |
| yoctozepto | makes total sense | 18:19 |
| clarkb | fungi: I've uploaded the newer audit results to review now. As expected the active + inactive set is now empty. We have about 140 accounts that have pushed or reviewed code recently and the rest have not pushed or reviewed code recently | 18:19 |
| clarkb | I want to followup with weshay|ruck on the tripleo ruck rover account before moving too much furhter aheada s that is in the no pushes and reviews group and I think will give us good insight on further identifying recent usage patterns | 18:20 |
| fungi | thanks! | 18:20 |
| *** toomer has quit IRC | 18:21 | |
| clarkb | I'm now going to try changing the recency period to 2 years and then 6 months and see if the data drastically changes | 18:22 |
| *** andrewbonney has quit IRC | 18:23 | |
| *** ralonsoh has quit IRC | 18:23 | |
| yoctozepto | clarkb: https://review.opendev.org/c/openstack/openstackclient/+/778536 | 18:25 |
| yoctozepto | now it runs | 18:25 |
| yoctozepto | clarkb: btw, gerrit shows you still have "Turkey time" | 18:28 |
| yoctozepto | that's one big turkey there | 18:28 |
| clarkb | ya I always forget to update it :) | 18:30 |
| clarkb | but also turkey time is a good time | 18:30 |
| zbr | clarkb: fungi: https://review.opendev.org/c/openstack/project-config/+/778489 please and thanks. | 18:30 |
| yoctozepto | I agree | 18:30 |
| zbr | I was wondering about the same thing about clarkb timezone. | 18:31 |
| yoctozepto | it's not timezone though | 18:31 |
| yoctozepto | it's status, like vacationing :-) | 18:32 |
| zbr | if the timezone was correct, he would have being a night-turkey | 18:32 |
| weshay|ruck | clarkb, aye.. so we've confirmed tripleo-ci.ruck.rover@gmail is only for listening to gerrit events | 18:32 |
| weshay|ruck | it would never push a review | 18:32 |
| weshay|ruck | https://review.opendev.org/q/owner:tripleo.ci.ruck.rover%2540gmail.com | 18:32 |
| weshay|ruck | and has none | 18:32 |
| weshay|ruck | clarkb, does that answer the question well enough? | 18:33 |
| clarkb | weshay|ruck: yes, I think that basically means that we should be looking to see recent ssh logins as well to determine recent usage | 18:33 |
| clarkb | weshay|ruck: I may dig up mroe account details in a bit (want to finish up comparing different time deltas), and will bring up any additional questions if they arise | 18:33 |
| fungi | right, i think for anything in that bucket, just grepping the ssh api log for the username is sufficient. we have like a month of retention, should be plenty | 18:34 |
| weshay|ruck | cool, not a problem.. thanks as always for the help | 18:34 |
| clarkb | I suspect we want something like: if username external id is set then check ssh logs for use of that in sshd logs | 18:34 |
| fungi | i agree. we could check whether there's an ssh key configured too if desired | 18:34 |
| clarkb | gerrit has a timezone? | 18:36 |
| *** lpetrut has joined #opendev | 18:38 | |
| *** hashar is now known as hasharDinner | 18:40 | |
| *** marios|out has quit IRC | 18:41 | |
| *** lpetrut has quit IRC | 18:49 | |
| mordred | it does? | 18:58 |
| clarkb | I couldn't find one | 18:58 |
| mordred | I would have thought it would just be the UTC from the server, right? | 18:58 |
| clarkb | there are about 50 more "recently used" accounts if I switch the recency period from 1 year to 2 yaers | 18:58 |
| clarkb | it does show you timestamps in your browser specified timezeon | 18:58 |
| clarkb | but I can't see a way to tell the server that such that other people will see it | 18:59 |
| mordred | yeah | 18:59 |
| clarkb | now to see what a 6 month period looks like | 18:59 |
| *** sshnaidm is now known as sshnaidm|afk | 19:00 | |
| *** gothicserpent has quit IRC | 19:08 | |
| clarkb | if we look at a 6 month as the recency then we lose 36 accounts | 19:32 |
| corvus | i did not see an expected gerritbot msg; sorry i have to run right now and don't have time to check on it | 19:32 |
| clarkb | 17:02:23 <-- | openstackgerrit (trim) has quit (Quit: Changing servers) | 19:33 |
| clarkb | I'll restart it | 19:34 |
| clarkb | that is done | 19:34 |
| fungi | thanks! | 19:36 |
| clarkb | having 3 data points for "recently used accounts" probably doesn't actually make a trend btu it does seem there may be an attrition rate there | 19:38 |
| clarkb | that actually makes me more comfortable with using a year because it seems that is a good balance between reducing problem set and "these accounts are unlikely to ever notice" | 19:40 |
| clarkb | 6 months further reduces the problem set but those accounts are probably more likely to try and push code tomorrow | 19:41 |
| clarkb | er maybe I've got that backwards. I should eat lunch then do thinking | 19:42 |
| *** whoami-rajat has quit IRC | 19:50 | |
| *** slaweq has quit IRC | 20:04 | |
| *** openstackgerrit has joined #opendev | 20:15 | |
| openstackgerrit | Merged openstack/project-config master: Add tripleo-ci-health-queries to zuul https://review.opendev.org/c/openstack/project-config/+/778489 | 20:15 |
| johnsom | Hello opendev neighbors. I just wanted to mention an oddity I noticed on the nodejs jobs where zuul has to retry a few times. | 20:26 |
| johnsom | https://zuul.openstack.org/builds?job_name=horizon-nodejs10-run-test | 20:26 |
| johnsom | It seems to be not happy trying to get a chromium package | 20:27 |
| johnsom | https://zuul.openstack.org/build/991677a0cbc849a0be2939a4b904efd7 | 20:27 |
| johnsom | My guess is the remote side for the snap store is having network issues or such. "Download snap "core18" (1988) from channel "stable" (unexpected EOF)" | 20:28 |
| johnsom | From what I can see we are moving away from using chrome in the nodejs tests, so this might go away, but thought I would mention it. | 20:29 |
| clarkb | johnsom: I had no idea that anyone was using snaps for anything. Is chrome not properly packaged anymore (we mirror the packages) | 20:30 |
| johnsom | clarkb Me either, but I think that is the "magic" of focal | 20:30 |
| fungi | ouch | 20:31 |
| johnsom | When I track the task back: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/nodejs-test-dependencies/tasks/main.yaml | 20:31 |
| johnsom | It's an apt install call via ansible, but snap seems to be getting involved | 20:31 |
| fungi | someone in another channel started asking me about snaps this morning, and i went looking for my torch and pitchfork | 20:32 |
| fungi | yikes! https://packages.ubuntu.com/focal-updates/chromium-chromedriver "Transitional package - chromium-chromedriver -> chromium snap" | 20:33 |
| johnsom | I will not start the "debate" about "Is chrome not properly packaged anymore". grin. | 20:33 |
| fungi | seems like that debate is already over anyway | 20:34 |
| clarkb | weshay|ruck: ok, I've found that the two accounts that conflict over the single tripleo rover ruck email address both have usernames and ssh keys configured. However, grepping sshd logs the account with username os-tripleo-ci seems to be actively used but the one with tripleo.ci as the username may not be used | 20:35 |
| clarkb | weshay|ruck: do you know if that is the case? If so we'd want to retire the tripleo.ci account and remove its conflicting external ids. If you can help confirm that my investigation makes sense that would be great as we can apply rules like this to other accounts | 20:35 |
| clarkb | fungi: ^ I'm thinking the next good update to the audit will be to check if there are accounts with no username and or no sshkeys as those can probably be safely retired if they have also not been recently used | 20:36 |
| clarkb | johnsom: well in this case I mean in a package and not a container | 20:36 |
| clarkb | johnsom: since we mirror the packages but not the snap containers | 20:36 |
| johnsom | Yeah, I know. It's a "hot" topic these days. | 20:37 |
| johnsom | Not one I have cycles to get in the middle of. grin | 20:38 |
| clarkb | https://packages.ubuntu.com/focal/chromium-browser it looks like the package is a transitional package that pull the snap | 20:38 |
| weshay|ruck | clarkb, ah.. ok this makes sense now | 20:38 |
| weshay|ruck | sec | 20:38 |
| fungi | i find it interesting that ubuntu would decide to punt their chromium packages to a snap instead of just using debian's (which is also newer than any snap ubuntu seems to have) | 20:38 |
| clarkb | fungi: they may rely on chrom* updating itself within the snap/container | 20:39 |
| fungi | https://packages.debian.org/sid/chromium 88.0.4324.182-1, https://packages.ubuntu.com/hirsute/chromium-browser 1:85.0.4183.83-0ubuntu2 | 20:39 |
| fungi | yeah maybe | 20:39 |
| fungi | but seems silly to install chromium 85 only to immediately upgrade it | 20:39 |
| clarkb | if it gets you out of the business of doing frequent updates I can see the argument for it | 20:40 |
| johnsom | https://www.theregister.com/2020/06/02/linux_mint_team_snap/ | 20:40 |
| johnsom | Just for an outside reference | 20:41 |
| weshay|ruck | clarkb, ok.. so os-tripleo-ci is used w/ our tripleo zuul reproducer | 20:42 |
| weshay|ruck | you can nuke tripleo.ci. | 20:42 |
| weshay|ruck | but please leave os-tripleo-ci | 20:42 |
| clarkb | weshay|ruck: cool, that means my investigation produced what appears to be reliable results :) | 20:43 |
| clarkb | weshay|ruck: also fwiw I think what we may do is set a bunch of these accounts inactive, then wait a few days for screaming before doing the more invasive external id removals | 20:43 |
| weshay|ruck | ++ | 20:43 |
| clarkb | just to be sure we haven't missed anything | 20:43 |
| fungi | just good to know they're not actively using both accounts | 20:43 |
| weshay|ruck | k.. if we can avoid setting os-tripleo-ci innactive, that would be appreciated.. even if I scream as loud as I can, not everyone will get the message | 20:45 |
| clarkb | weshay|ruck: yup we only have to pick one of the two to deactivate so we will deactivate the one you aren't using | 20:45 |
| weshay|ruck | ++ | 20:45 |
| weshay|ruck | thanks! | 20:45 |
| clarkb | and now I've got more info on improving our audit scripts to better find accounts like those two and classifythem | 20:46 |
| clarkb | just 607 more to figure out :) | 20:46 |
| clarkb | fungi: I'm thinking the next pass might be "no reviews, no pushes, no username, no sshkeys" and see what that produces | 20:47 |
| clarkb | then the next set will be no reviews, no pushes, and no recent sshd log entries | 20:48 |
| fungi | where "recent" is roughly a month's log retention, i think | 20:48 |
| clarkb | yes | 20:48 |
| clarkb | the reviews and pushes should catch those activities outside of the sshd log | 20:49 |
| clarkb | so we'd really be isolating accounts like teh tripleo one used only for pulling events and not responding back again | 20:49 |
| fungi | i think we discussed this previously, and talked about possibly preserving logs or pulling some from backups if we wanted to check a longer timeframe | 20:49 |
| clarkb | I feel like cross checking against pushes and reviews is probably good enough, but may depend on the size of those datasets | 20:49 |
| fungi | yeah, if they're not leaving comments or pushing changes, then the only cnoceivable things they could be doing are listening to the event stream or running queries over ssh (the relevant queries over rest api wouldn't need auth anyway) | 20:51 |
| fungi | and i expect those would be frequent periodic or continuous access anyway | 20:51 |
| clarkb | fungi: do you have time to review https://review.opendev.org/c/opendev/system-config/+/778227 as the next step in zuul server replacements? | 20:52 |
| fungi | we might also want to compare against active connections too though? possible our server is so stable they haven't had to reconnect to the event stream for longer than our log retention | 20:52 |
| clarkb | fungi: ya exactly re frequent or continuous | 20:52 |
| clarkb | fungi: we restarted gerrit recently, like 10 days ago? | 20:52 |
| fungi | yeah, so should be good enough if we do it in the next 20 days | 20:52 |
| clarkb | but also it logs queries made not just connections | 20:52 |
| fungi | true, but if the only thing they're doing is connecting to the event stream... meh that probably fairly unlikely you're right | 20:53 |
| *** slaweq has joined #opendev | 20:57 | |
| *** slaweq has quit IRC | 21:07 | |
| openstackgerrit | Gomathi Selvi Srinivasan proposed zuul/zuul-jobs master: Create a template for ssh-key and size https://review.opendev.org/c/zuul/zuul-jobs/+/773474 | 21:13 |
| openstackgerrit | Gomathi Selvi Srinivasan proposed zuul/zuul-jobs master: Create a template for ssh-key and size https://review.opendev.org/c/zuul/zuul-jobs/+/773474 | 21:29 |
| *** sboyron has quit IRC | 21:30 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: install-ansible: ensure stevedore https://review.opendev.org/c/opendev/system-config/+/778354 | 21:30 |
| *** ianw has quit IRC | 21:33 | |
| *** kopecmartin has quit IRC | 21:33 | |
| *** tristanC has quit IRC | 21:33 | |
| *** amotoki has quit IRC | 21:33 | |
| *** ozzzo has quit IRC | 21:33 | |
| *** odyssey4me has quit IRC | 21:33 | |
| *** janders has quit IRC | 21:33 | |
| *** ianw has joined #opendev | 21:34 | |
| *** kopecmartin has joined #opendev | 21:34 | |
| *** tristanC has joined #opendev | 21:34 | |
| *** amotoki has joined #opendev | 21:34 | |
| *** ozzzo has joined #opendev | 21:34 | |
| *** odyssey4me has joined #opendev | 21:34 | |
| *** janders has joined #opendev | 21:34 | |
| ianw | did we have some known issues with the limestone mirror? e.g. https://zuul.opendev.org/t/openstack/build/6db7cccd761940068993b40a11c4f787/log/job-output.txt#728 | 21:35 |
| clarkb | it appears to be up now, and no wasn't aware of any | 21:36 |
| ianw | weird | 21:38 |
| ianw | also seems we lost the gerrit announcer | 21:38 |
| *** gothicserpent has joined #opendev | 21:40 | |
| clarkb | hrm I had to restart the gerritbot earlier today because it said it was switching servers then never came back | 21:41 |
| clarkb | oh no it saw your stevedore push | 21:41 |
| clarkb | but you weren't in here | 21:41 |
| clarkb | was that the one you expected to see? Maybe you were split away too | 21:42 |
| clarkb | ya looks like there was a netsplit, possible the bot got caught in it again though | 21:43 |
| *** roman_g has joined #opendev | 21:51 | |
| openstackgerrit | Merged opendev/system-config master: Remove ze01.openstack.org https://review.opendev.org/c/opendev/system-config/+/778227 | 21:54 |
| jrosser | ianw: i had a few jobs with the same ipv6 fail on the limestone mirror today | 21:54 |
| gothicserpent | clarkb, i had an issue with that.. might be a k-line possibly | 22:07 |
| gothicserpent | was the bot on a vpn? | 22:08 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: cabal-test: add install_args and build_args role var https://review.opendev.org/c/zuul/zuul-jobs/+/777653 | 22:08 |
| *** hasharDinner has quit IRC | 22:15 | |
| fungi | gothicserpent: nope, no vpn. it's just running from a virtual machine in rackspace connecting to chat.freenode.net directly with the python irc module | 22:22 |
| fungi | rackspace's dfw pop to be precise | 22:22 |
| gothicserpent | ah ok | 22:24 |
| *** gothicserpent has quit IRC | 22:26 | |
| *** gothicserpent has joined #opendev | 22:26 | |
| openstackgerrit | Lee Yarwood proposed openstack/project-config master: Add custom cirros image with ahci module enabled to cache https://review.opendev.org/c/openstack/project-config/+/778590 | 22:28 |
| ianw | mirror-update is quiescent, so i'm think the best idea for upgrading the afs fileserver is to just shut it down for a bit and cycle through one by one | 22:32 |
| *** gothicserpent has quit IRC | 22:34 | |
| ianw | although docs may write out i guess | 22:34 |
| ianw | i think we can start with ord anyway | 22:35 |
| *** gothicserpent has joined #opendev | 22:35 | |
| fungi | sounds great | 22:50 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Remove obsolete Bazel spawn strategies https://review.opendev.org/c/opendev/system-config/+/778404 | 23:39 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: system-config-roles: only match jobs on roles tested https://review.opendev.org/c/opendev/system-config/+/778593 | 23:39 |
| *** roman_g has quit IRC | 23:46 | |
| clarkb | I've got an audit run going that checks for accounts without usernames and ssh keys now | 23:54 |
| clarkb | if that loosk good I'll push up my chagnes to the audit script and test new git-review in the process :) | 23:54 |
| clarkb | I'm hopeful this pass will give us another good set of accounts to retire and cleanup | 23:55 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!