| opendevreview | Merged opendev/system-config master: Cleanup force merging docs https://review.opendev.org/c/opendev/system-config/+/864798 | 00:02 |
|---|---|---|
| ianw | clarkb: thanks! | 01:04 |
| ianw | #status log Status logs now appearing at https://fosstodon.org/@opendevinfra | 01:07 |
| opendevstatus | ianw: finished logging | 01:07 |
| ianw | the admins were kind enough to unmute the account after my efforts yesterday :) | 01:08 |
| *** yadnesh|away is now known as yadnesh | 04:37 | |
| *** ysandeep|out is now known as ysandeep | 07:25 | |
| *** akahat|ruck is now known as akahat|ruck|lunch | 08:14 | |
| *** jpena|off is now known as jpena | 08:29 | |
| opendevreview | Lajos Katona proposed openstack/project-config master: Add toggleWipState to networking projects https://review.opendev.org/c/openstack/project-config/+/864563 | 08:32 |
| *** yadnesh is now known as yadnesh|afk | 08:34 | |
| *** yadnesh|afk is now known as yadnesh | 08:53 | |
| *** akahat|ruck|lunch is now known as akahat|ruck | 09:22 | |
| *** soniya29|rover is now known as soniya29|rover|lunch | 09:29 | |
| dpawlik | clarkb, fungi: hey, i'm going on vacation on monday and I'll be away for about 2 weeks. The logscraper/logsender seems to work normally, the Opensearch have enough space, there was no alert also in AWS console so it should be ok. If something will happening with the logscraper/logsender, ping nhicher, fbo[m] | 10:03 |
| *** ysandeep is now known as ysandeep|afk | 10:08 | |
| *** soniya29|rover|lunch is now known as soniya29|rover | 10:16 | |
| *** soniya29|rover is now known as soniya29|rover|afk | 10:29 | |
| *** soniya29|rover|afk is now known as soniya29|rover | 10:35 | |
| *** dviroel_ is now known as dviroel | 11:09 | |
| *** rlandy|out is now known as rlandy | 11:18 | |
| *** ysandeep|afk is now known as ysandeep | 11:38 | |
| fungi | thanks for the heads up, dpawlik! enjoy your vacation | 12:22 |
| *** pojadhav is now known as pojadhav|afk | 12:27 | |
| dpawlik | thanks | 12:37 |
| *** soniya29|rover is now known as soniya29|rover|afk | 12:55 | |
| *** pojadhav|afk is now known as pojadhav | 13:07 | |
| opendevreview | Merged openstack/project-config master: Add toggleWipState to networking projects https://review.opendev.org/c/openstack/project-config/+/864563 | 13:11 |
| *** soniya29|rover|afk is now known as soniya29|rover | 13:21 | |
| *** yadnesh is now known as yadnesh|away | 13:30 | |
| *** ysandeep is now known as ysandeep|afk | 13:41 | |
| *** ysandeep|afk is now known as ysandeep | 14:06 | |
| *** dasm|off is now known as dasm | 14:22 | |
| priteau | Hello. Is something going on with opendevmeet? It took 90 seconds to start a meeting just now. | 15:02 |
| fungi | priteau: which channel? | 15:03 |
| priteau | In #openstack-blazar | 15:04 |
| fungi | the meeting which just started in #openstack-meeting-alt seems to have begun straight away | 15:04 |
| fungi | trying to track down the service log for it to see if there's anything going on | 15:06 |
| fungi | priteau: https://meetings.opendev.org/irclogs/%23openstack-blazar/%23openstack-blazar.2022-11-17.log shows that it responded straight away. maybe there was lag for your startmeeting command making it into the channel? | 15:08 |
| priteau | OK, maybe something on my bouncer side | 15:09 |
| priteau | I issued the command at 15:00 exactly | 15:09 |
| fungi | i've had things like packet loss between my client and an irc server result in lengthy delays for a message making it into channels before | 15:11 |
| fungi | so it's not impossible | 15:11 |
| fungi | my irc client shows a lag indicator letting me know how long it's been since a message was sent with no ack from the server, but some clients may not make that obvious (and going through a proxy/bouncer service likely makes that harder to notice) | 15:12 |
| fungi | also, connectivity issues between irc servers can cause messages to lag. so it's possible the server your client connection is into got your message immediately but then didn't forward it on to the irc server the meetbot is connecting to due to some problem | 15:15 |
| fungi | that would manifest in some people in the channel seeing your message right away and others not seeing it until after some delay | 15:15 |
| fungi | infra-root: i got around to testing manual removal of a row from the repo_redirect table in gitea and it seems to have worked as hoped without any obvious negative consequences. if there are no objections, i'm ready to move forward deleting from repo_redirect where lower_name="ansible-role-zookeeper" | 15:35 |
| fungi | for the record, all 8 gitea servers have this same identical row i plan to delete: id=691 owmer_id=8 lower_name="ansible-role-zookeeper" redirect_repo_id=308 | 15:36 |
| Clark[m] | fungi: maybe do a db dump first? gitea01 is the server that does the db backups and crontab should have an example. Also I thought the column name was something else when I checked? | 15:37 |
| fungi | yeah i planned to do mysqldumps too, but worst case, if we run into some issue which didn't arise on the held node i tested with, we can just put that row back into the repo_redirect table | 15:38 |
| Clark[m] | Oh it's the source side that is lower_name. I was thinking of redirect_repo_id but that may not be stable across instances | 15:38 |
| fungi | right, in this case it happens to have a consistent id in all 8 databases (i manually checked them all), but i didn't want to rely on that | 15:38 |
| fungi | every one looks just like https://paste.openstack.org/show/b5Zo0g6dtQjEQkVQUGcQ/ | 15:38 |
| fungi | and yeah, the owner_id in that row corresponds to the users.id row for "openstack" while the redirect_repo_id corresponds to the repository.id for windmill/ansible-role-zookeeper (just to be clear about the schema) | 15:42 |
| Clark[m] | fungi: my only other thought is to check owner Id against the users table to double check it is windmill | 15:42 |
| fungi | other way around, but yes | 15:43 |
| Clark[m] | Er I had the source and dest backwards. That all lgtm | 15:43 |
| fungi | also there's a ~fungi/2022-11-17_before_edit.sql.gz on all 8 servers now of the full db dump from before any editing is done | 15:45 |
| fungi | okay, i ran this query on all 8 servers: | 15:48 |
| fungi | delete from repo_redirect where lower_name="ansible-role-zookeeper"; | 15:48 |
| fungi | each reported one row affected | 15:48 |
| *** pojadhav is now known as pojadhav|out | 15:49 | |
| fungi | however, https://opendev.org/openstack/ansible-role-zookeeper is returning a 404 rather than the repository | 15:51 |
| fungi | that's something i didn't test on the held node... creating a new project shadowed by the redirect | 15:51 |
| Clark[m] | Do we know if the project exists? | 15:52 |
| fungi | i think it doesn't | 15:52 |
| fungi | select * from repository where name="ansible-role-zookeeper"; | 15:53 |
| Clark[m] | It's possible it wasn't created properly due to the redirect and we just need a rerun of the project config playbook? | 15:53 |
| fungi | that returns only one result, and it's the one in windmill | 15:53 |
| fungi | so yes, that's my suspicion as well, gitea probably refused to create the project because there was a conflicting redirect | 15:53 |
| fungi | i guess i can rerun the project creation playbook against just gitea01 and see what happens | 15:54 |
| Clark[m] | The playbook does query for projects before creating them iirc. It should find the project isn't there and then create it I think | 15:57 |
| fungi | happen to recall which playbook it is i want for this? | 15:58 |
| fungi | service-gitea or something else? | 15:58 |
| fungi | sync-gitea-projects? | 15:58 |
| fungi | oh, manage-projects | 16:00 |
| Clark[m] | Let me see | 16:00 |
| Clark[m] | https://opendev.org/opendev/system-config/src/branch/master/zuul.d/infra-prod.yaml#L101 | 16:00 |
| *** dviroel is now known as dviroel|brb|lunch | 16:00 | |
| fungi | yeah, manage-projects includes the gitea-git-repos role but doesn't turn on the gitea_always_update var | 16:01 |
| Clark[m] | https://opendev.org/opendev/system-config/src/branch/master/playbooks/manage-projects.yaml#L22 | 16:01 |
| Clark[m] | I don't think you need the gitea_always_update var because the project doesn't exist | 16:01 |
| fungi | right | 16:02 |
| fungi | that's what i'm saying, the sync playbook turns that on and will end up taking forever updating all the projects' metadata | 16:02 |
| fungi | but the manage-projects playbook should be quick and just add the missing project | 16:02 |
| Clark[m] | Ya that is my expectation | 16:02 |
| fungi | i have a root screen session on bridge01 about to run this: ansible-playbook --limit gitea01.opendev.org /home/zuul/src/opendev.org/opendev/system-config/playbooks/manage-projects.yaml | 16:03 |
| fungi | then i'll check the db and see if the project has appeared | 16:03 |
| fungi | sound good? | 16:04 |
| Clark[m] | Yes, I think that is roughly equivalent to what zuul would run when a change lands | 16:05 |
| fungi | in progress now | 16:05 |
| Clark[m] | (note I'm not in the screen but can attach if that becomes necessary) | 16:05 |
| Clark[m] | You can also address the host directly with https://gitea01.opendev.org:3081/openstack/ansible-role-zookeeper to check | 16:06 |
| fungi | yep, sql query on that name returns two repos now (one in openstack, one in windmill) and the direct url returns a readme file | 16:09 |
| fungi | may need to re-trigger replication from gerrit | 16:10 |
| fungi | but i'll get it created on the other 7 servers first | 16:10 |
| fungi | how much parallelism does ansible-playbook do by default? i forgot to add the -f option | 16:12 |
| Clark[m] | I think 5 | 16:13 |
| fungi | okay, good enough | 16:13 |
| fungi | i do see it doing more than one at a time anyway | 16:13 |
| fungi | i was worried for a moment that it serialized them all by default | 16:13 |
| Clark[m] | Might be worth also double checking nothing unexpected occurs in the Gerrit side of that playbook when it is done too | 16:14 |
| fungi | yeah, and i'll kick off replication for that repo too | 16:14 |
| Clark[m] | It should noop right because it did create in Gerrit? | 16:14 |
| fungi | in theory, yes | 16:14 |
| *** ysandeep is now known as ysandeep|out | 16:15 | |
| fungi | Clark[m]: oh, also i did --limit gitea so it's not doing the gerrit plays anyway | 16:16 |
| Clark[m] | Aha | 16:17 |
| fungi | okay, it's finished. i'll remind myself how to retrigger replication next | 16:19 |
| fungi | looks like it's `replication start openstack/ansible-role-zookeeper` | 16:23 |
| Clark[m] | Ya I think the replication command doesn't have the Gerrit prefix | 16:23 |
| fungi | that looks better | 16:23 |
| fungi | https://opendev.org/openstack/ansible-role-zookeeper | 16:23 |
| fungi | now has our initial .gitreview file instead of gitea's default readme | 16:24 |
| fungi | noonedeadpunk: ^ | 16:24 |
| fungi | i'll work on the opendev/project-config rename history update next | 16:24 |
| *** marios is now known as marios|out | 16:31 | |
| opendevreview | Jeremy Stanley proposed opendev/project-config master: Comment openstack/ansible-role-zookeeper rename https://review.opendev.org/c/opendev/project-config/+/864919 | 16:32 |
| noonedeadpunk | fungi: nice, thanks! | 16:32 |
| fungi | you're welcome | 16:35 |
| opendevreview | Rodolfo Alonso proposed opendev/irc-meetings master: Update Neutron meetings chairs https://review.opendev.org/c/opendev/irc-meetings/+/864925 | 16:58 |
| *** rlandy is now known as rlandy|ruck | 17:04 | |
| opendevreview | Merged opendev/irc-meetings master: Update Neutron meetings chairs https://review.opendev.org/c/opendev/irc-meetings/+/864925 | 17:07 |
| *** dviroel|brb|lunch is now known as dviroel | 17:12 | |
| *** jpena is now known as jpena|off | 17:36 | |
| *** dviroel_ is now known as dviroel | 18:16 | |
| ianw | fungi: weirdly your comments didn't seem to line up as responses in the webui for https://review.opendev.org/c/opendev/system-config/+/861284 | 21:15 |
| *** dviroel is now known as dviroel|out | 21:15 | |
| ianw | they're like separate comments | 21:15 |
| fungi | oh, yeah gertty doesn't know how to reply to a comment (that functionality was added to gerrit more recently) | 21:20 |
| fungi | also doesn't show replies to inline comments any differently from new inline comments, so i never notice the difference | 21:21 |
| opendevreview | James E. Blair proposed zuul/zuul-jobs master: Add AUTOGENERATED comment to helm-jobs https://review.opendev.org/c/zuul/zuul-jobs/+/864947 | 21:43 |
| corvus | wip for that: https://review.opendev.org/860588 (not quite 100% yet) | 21:54 |
| fungi | oh, cool! i missed that, i think | 21:56 |
| corvus | it kinda barely works. lots of buttons don't do anything and things disappear when you move around (but they're really there still!) i can't recommend it. but i am running it. :) | 21:59 |
| opendevreview | Merged zuul/zuul-jobs master: helm: Add job for linting helm charts https://review.opendev.org/c/zuul/zuul-jobs/+/861799 | 22:21 |
| opendevreview | Merged zuul/zuul-jobs master: Add AUTOGENERATED comment to helm-jobs https://review.opendev.org/c/zuul/zuul-jobs/+/864947 | 22:25 |
| *** dasm is now known as dasm|off | 22:47 | |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 23:30 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!