| opendevreview | Merged opendev/system-config master: Connect Zuul to review01.opendev.org https://review.opendev.org/c/opendev/system-config/+/798243 | 00:11 |
|---|---|---|
| ianw | ok, that has deployed. i'm going to save queues and restart zuul & gerrit with new image | 00:51 |
| fungi | thanks! | 00:52 |
| ianw | interesting, the docker-compose.yaml on gerrit appears to have a syntax error of some sort | 00:56 |
| ianw | it appears to have got itself indented slightly | 00:57 |
| ianw | oh i see it. i'll fix by hand and get things going | 00:58 |
| ianw | pulling new gerrit image now | 00:59 |
| ianw | https://hub.docker.com/layers/opendevorg/gerrit/3.3/images/sha256-e9b4909ca19f4609e8486666062c87d846300d5c1f42c38b1606e05c137791b1?context=explore is the one we want | 01:01 |
| ianw | no sorry, 3.2 | 01:02 |
| ianw | https://hub.docker.com/layers/opendevorg/gerrit/3.2/images/sha256-e03d7652bb63f516611cb11d3642e9e3659548fbc99fdf9e265cbac225471459?context=explore | 01:02 |
| ianw | opendevorg/gerrit 3.2 sha256:e03d7652bb63f516611cb11d3642e9e3659548fbc99fdf9e265cbac225471459 847424025943 5 days ago 811MB | 01:02 |
| ianw | that matches, ergo we have the correct image on the production server. so i'll start it now | 01:02 |
| ianw | Powered by Gerrit Code Review (3.2.11-2-gf500af8500-dirty) | 01:05 |
| ianw | ok, zuul is restarting | 01:06 |
| ianw | ok, looks like some events coming in via gerrit, so the connection should be ok | 01:08 |
| opendevreview | Ian Wienand proposed opendev/system-config master: gerrit: fix docker-compose template https://review.opendev.org/c/opendev/system-config/+/800358 | 01:16 |
| ianw | everything seems to be working so i think we're done | 01:36 |
| ianw | #status log pulled gerrit 3.2.11 image and restarted, restarted zuul to use full review01.opendev.org name when connecting | 01:36 |
| opendevstatus | ianw: finished logging | 01:37 |
| opendevreview | Ian Wienand proposed opendev/zone-opendev.org master: Add paste01 https://review.opendev.org/c/opendev/zone-opendev.org/+/799735 | 01:45 |
| opendevreview | Merged opendev/zone-opendev.org master: Add paste01 https://review.opendev.org/c/opendev/zone-opendev.org/+/799735 | 02:09 |
| ianw | hrm, there must be a new ffi release that we haven't wheeled yet, and the gerrit testing is failing | 03:27 |
| opendevreview | chzhang8 proposed openstack/project-config master: add acls for tricirlce under x namespaces https://review.opendev.org/c/openstack/project-config/+/800404 | 03:43 |
| ianw | hrm | 04:10 |
| ianw | last time it ran it got | 04:10 |
| ianw | Collecting cffi>=1.1 | 04:10 |
| ianw | Downloading http://mirror-int.dfw.rax.opendev.org/pypifiles/packages/6c/a6/1662485aee65761f09707220d9e847baac4a912d419ee0139a9de22f34c4/cffi-1.14.5-cp35-cp35m-manylinux1_x86_64.whl (401 kB) | 04:10 |
| ianw | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_07c/798400/24/check/system-config-run-review-3.2/07c59a4/bridge.openstack.org/ara-report/results/284.html | 04:11 |
| ianw | ahh | 04:12 |
| ianw | Collecting cffi>=1.1 | 04:12 |
| ianw | Downloading http://mirror-int.ord.rax.opendev.org/pypifiles/packages/2e/92/87bb61538d7e60da8a7ec247dc048f7671afe17016cd0008b3b710012804/cffi-1.14.6.tar.gz (475 kB) | 04:12 |
| ianw | it looks like they ahve a new release management process | 04:27 |
| ianw | https://groups.google.com/g/python-cffi/c/Qz7Ill0Ndlc | 04:29 |
| ianw | and python35 wheels have been dropped | 04:29 |
| opendevreview | Ian Wienand proposed opendev/system-config master: gerrit: fix docker-compose template https://review.opendev.org/c/opendev/system-config/+/800358 | 04:49 |
| opendevreview | Ian Wienand proposed opendev/system-config master: install-docker: install build-essential for cffi on Xenial https://review.opendev.org/c/opendev/system-config/+/800409 | 04:49 |
| opendevreview | Merged opendev/system-config master: Sync project-config before deploying accessbot https://review.opendev.org/c/opendev/system-config/+/800314 | 05:57 |
| *** ysandeep- is now known as ysandeep | 05:59 | |
| *** bhagyashris__ is now known as bhagyashris|ruck | 06:51 | |
| *** amoralej|off is now known as amoralej | 07:01 | |
| *** rpittau|afk is now known as rpittau | 07:35 | |
| opendevreview | chzhang8 proposed openstack/project-config master: register and bring back tricircle under x namespaces https://review.opendev.org/c/openstack/project-config/+/800427 | 07:39 |
| opendevreview | Takashi Kajinami proposed openstack/project-config master: tripleo-common-tempest-plugin - Step 2: End project Gating https://review.opendev.org/c/openstack/project-config/+/800154 | 07:51 |
| opendevreview | Takashi Kajinami proposed openstack/project-config master: tripleo-common-tempest-plugin - Step 4: Remove Project https://review.opendev.org/c/openstack/project-config/+/800157 | 07:53 |
| opendevreview | Takashi Kajinami proposed openstack/project-config master: tripleo-common-tempest-plugin - Step 4: Remove Project https://review.opendev.org/c/openstack/project-config/+/800157 | 08:11 |
| *** ysandeep is now known as ysandeep|lunch | 08:16 | |
| *** ykarel is now known as ykarel|lunch | 08:17 | |
| opendevreview | Merged openstack/diskimage-builder master: Migrate from testr to stestr https://review.opendev.org/c/openstack/diskimage-builder/+/789246 | 08:43 |
| *** rm_work| is now known as _rm_work | 08:54 | |
| *** _rm_work is now known as ]rm_work | 08:55 | |
| *** ]rm_work is now known as [rm_work | 08:55 | |
| *** [rm_work is now known as zrm_work | 08:55 | |
| *** cgoncalves_ is now known as cgoncalves | 09:13 | |
| opendevreview | chzhang8 proposed openstack/project-config master: register and bring back tricircle under x namespaces https://review.opendev.org/c/openstack/project-config/+/800442 | 09:15 |
| *** ykarel|lunch is now known as ykarel | 09:47 | |
| opendevreview | Merged openstack/diskimage-builder master: cache-url : turn down verbose curl https://review.opendev.org/c/openstack/diskimage-builder/+/797791 | 09:55 |
| *** ysandeep|lunch is now known as ysandeep | 09:58 | |
| opendevreview | Merged openstack/diskimage-builder master: Add a growvols utility for growing LVM volumes https://review.opendev.org/c/openstack/diskimage-builder/+/791083 | 10:12 |
| *** ysandeep is now known as ysandeep|afk | 10:57 | |
| *** dviroel|out is now known as dviroel | 11:24 | |
| *** bhagyashris_ is now known as bhagyashris|ruck | 11:32 | |
| *** ysandeep|afk is now known as ysandeep | 11:52 | |
| opendevreview | Vishal Manchanda proposed openstack/project-config master: Retire django-openstack-auth https://review.opendev.org/c/openstack/project-config/+/800459 | 12:08 |
| *** ysandeep is now known as ysandeep|afk | 12:25 | |
| *** amoralej is now known as amoralej|lunch | 12:33 | |
| tristanC | corvus: mordred: the curl notes have been moved to unclutter the readme to https://softwarefactory-project.io/cgit/software-factory/gerritbot-matrix/tree/API_NOTES.md | 12:38 |
| tristanC | corvus: yes i can have a look | 12:40 |
| *** needssleep is now known as TheJulia | 12:44 | |
| *** ysandeep|afk is now known as ysandeep | 13:05 | |
| fungi | ianw: yeah, i similarly proposed https://review.opendev.org/800325 to get gear's tests worknig again with new cffi | 13:14 |
| fungi | infra-root: i'm sort of around today, in and out doing travel prep and errands. i'll be out of communication most of tomorrow and then much of the time through the 19th, but hope to be around again as usual on the 20th or 21st worst case | 13:16 |
| *** amoralej|lunch is now known as amoralej | 13:31 | |
| walshh_ | Hi all, a quick question on Ocata and Newton if I may. Both are EOL but I am wondering is it possible to still update the online docs e.g. https://docs.openstack.org/ocata/config-reference/block-storage/drivers/emc-vmax-driver.html | 14:32 |
| fungi | walshh_: nope, not as far as i understand, the branches those were built from have been deleted so there's no way to merge new commits to them | 14:33 |
| walshh_ | Thank you fungi for the quick reply | 14:37 |
| fungi | walshh_: someone could probably hand modify the html content for those in afs if there's something which badly needs to be fixed, but i recommend talking with the openstack cinder team and openstack technical committee to determine if it's serious enough to warrant that | 14:38 |
| walshh_ | Thanks, as it is only applicable to the VMAX driver and is more | 14:41 |
| walshh_ | It is about hardware OS support so more informational than anything else | 14:43 |
| clarkb | fungi: can you review ianw's gerrit fixes stack at https://review.opendev.org/c/opendev/system-config/+/800358 if you have time? | 14:46 |
| rosmaita | fungi: you know you there's that tag line that shows up on the pages, for example, the train release : "This is maintained, but not the current release. The current supported release is Wallaby." | 14:49 |
| rosmaita | i don't see something like that on newton | 14:49 |
| rosmaita | would be helpful to say "This release is End of Life and the features and configuration described herein may no longer be accurate" | 14:50 |
| clarkb | rosmaita: the opendev team doesn't manage the docs or their contents. As fungi mentioned we could potentially do hand edits but we wouldn't be constructing those they would need to be provided to us | 14:51 |
| clarkb | might also provide feedback to the openstack release team to have them double check stuff like that is in place before cleaning up branches | 14:52 |
| rosmaita | clarkb: thanks ... didn't realize the release team supplied the wrapper for the docs, but that makes sense | 14:53 |
| clarkb | well I don't know that they do. But they have a number of checks they make when doing releases and retiring release series | 14:54 |
| clarkb | and this could be one of them | 14:54 |
| rosmaita | ok, thanks | 14:55 |
| fungi | yeah, i really have no idea how all that works. would need to consult the technical writers sig, the oslo team (who maintain the docs theme), maybe the release team and tc | 14:55 |
| fungi | it's a bigger discussion | 14:56 |
| clarkb | its possible when that was added it was never backported | 14:57 |
| fungi | or it was backported but nothing merged to regenerate/publish those particular docs | 15:00 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update gitea to 1.14.4 https://review.opendev.org/c/opendev/system-config/+/800274 | 15:02 |
| *** ykarel is now known as ykarel|away | 15:03 | |
| *** dviroel is now known as dviroel|lunch | 15:05 | |
| clarkb | tristanC: corvus: I was thinking it would be a good idea to test the bots you are working on in oftc bridged channels too. That way we don't have to run multiple copies of the bots (in fact I would consider not running multiple copies of bots for different protocols are requirement for this) | 15:21 |
| corvus | clarkb: the log bot will need a little extra work to strip the "oftc" stuff from the nicks, but otherwise, should be fine. | 15:29 |
| corvus | clarkb: note that limnoria currently provides meetbot+logging, and i have only written a logging replacement, not a meetbot replacement. it's probably possible to turn off the channel logging in limnoria without disabling meetings, but i don't know. just something to consider. | 15:33 |
| *** ysandeep is now known as ysandeep|away | 15:35 | |
| clarkb | yes, I think you can disable the limnoria logging. It is provided by a plugin and you remove/disable the plugin | 15:37 |
| opendevreview | Tristan Cacqueray proposed opendev/system-config master: Run matrix-gerritbot on eavesdrop https://review.opendev.org/c/opendev/system-config/+/800506 | 15:46 |
| tristanC | corvus: ^ should contains the tasks required to setup the gerritbot-matrix, it includes a task to convert a yaml based configuration, but i think it would be better to keep the dhall format | 15:50 |
| tristanC | (otherwise we need to repeat the default settings such as branch name and gerrit events) | 15:52 |
| *** jgwentworth is now known as melwitt | 16:07 | |
| *** dviroel|lunch is now known as dviroel | 16:10 | |
| *** ysandeep|away is now known as ysandeep | 16:20 | |
| *** rpittau is now known as rpittau|afk | 16:21 | |
| *** amoralej is now known as amoralej|off | 16:26 | |
| *** ysandeep is now known as ysandeep|away | 16:26 | |
| opendevreview | Clark Boylan proposed opendev/system-config master: DNM force gitea failure for interaction https://review.opendev.org/c/opendev/system-config/+/800516 | 16:44 |
| clarkb | I've put a hold in place for ^ so that we can look at it | 16:46 |
| opendevreview | Tristan Cacqueray proposed opendev/system-config master: Run matrix-gerritbot on eavesdrop https://review.opendev.org/c/opendev/system-config/+/800506 | 17:02 |
| opendevreview | Vishal Manchanda proposed openstack/project-config master: Retire django-openstack-auth https://review.opendev.org/c/openstack/project-config/+/800459 | 17:37 |
| opendevreview | Tristan Cacqueray proposed opendev/system-config master: Run matrix-gerritbot on eavesdrop https://review.opendev.org/c/opendev/system-config/+/800506 | 17:50 |
| clarkb | infra-root I think https://review.opendev.org/c/opendev/system-config/+/800274/2 is ready for review. I left a comment on there with a held node link which you can use to poke at the updated system | 17:52 |
| clarkb | this is the gitea upgrade change | 17:52 |
| clarkb | oh wait the logos aren't right | 17:52 |
| clarkb | I think our logo override isn't working :/ | 17:52 |
| clarkb | I'll dig into that. I think you can review the other bits of the change safely without that interferring too much though | 17:54 |
| mordred | clarkb: fwiw - the matrix well known files *are* serving as expected | 17:57 |
| mordred | so that's good at least | 17:58 |
| opendevreview | Merged opendev/system-config master: install-docker: install build-essential for cffi on Xenial https://review.opendev.org/c/opendev/system-config/+/800409 | 18:00 |
| opendevreview | Vishal Manchanda proposed openstack/project-config master: Retire django-openstack-auth https://review.opendev.org/c/openstack/project-config/+/800532 | 18:01 |
| clarkb | mordred: yup, and the matrix side verified one half of them. The other half is failing due to cors but it would only give us element urls under opendev.org so not super important | 18:02 |
| clarkb | I think I have osrted out the logo issue in the test build. I'll get udpatse up shortly and recycle the hold | 18:02 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Update gitea to 1.14.4 https://review.opendev.org/c/opendev/system-config/+/800274 | 18:05 |
| opendevreview | Clark Boylan proposed opendev/system-config master: DNM force gitea failure for interaction https://review.opendev.org/c/opendev/system-config/+/800516 | 18:05 |
| opendevreview | Merged opendev/system-config master: gerrit: fix docker-compose template https://review.opendev.org/c/opendev/system-config/+/800358 | 18:05 |
| opendevreview | Vishal Manchanda proposed openstack/project-config master: Retire django-openstack-auth https://review.opendev.org/c/openstack/project-config/+/800532 | 18:10 |
| opendevreview | Ghanshyam proposed openstack/project-config master: End project gate for retired neutron-lbaas https://review.opendev.org/c/openstack/project-config/+/800537 | 18:17 |
| clarkb | tristanC: a lot of our users do override the branches and gerrit events that they want to see | 18:30 |
| clarkb | tristanC: as far as making this easier for humans to consume I think the existing yaml config is pretty straightforward. People will probably figure out dhall too if we preseed with enough examples (and I assume we would port our config over either way) | 18:31 |
| opendevreview | Tristan Cacqueray proposed opendev/system-config master: Run matrix-gerritbot on eavesdrop https://review.opendev.org/c/opendev/system-config/+/800506 | 18:31 |
| clarkb | maybe the defaults should be in the application and not config if you feel strongly about them? | 18:31 |
| tristanC | clarkb: i'd rather not manage the configuration default in the application, and if you don't mind the repetition then we can keep the yaml format | 18:38 |
| clarkb | I don't mind. Its easy enough to capture them with an anchor and repeat them where applicable and just redefine them where it isn't (and i think that may happen a lot more than expected) | 18:39 |
| efoley | fungi, clarkb: I've been locked out of my account for gerrit, can you help me please? I removed an old e-mail address from gerrit/ubuntu one and now can't log in with my current e-mail address. | 18:50 |
| clarkb | efoley: let me see. Chances are gerrit is preventing your openid from being used because it conflicts with another account | 18:51 |
| clarkb | yup that is exactly it | 18:52 |
| clarkb | efoley: let me gather more data on the gerrit accounts at play here so that we can understand it better | 18:52 |
| efoley | clarkb, I first removed the account on gerrit, and then tried to log in again with ubuntu one, which sent my old e-mail and created a new gerrit account | 18:52 |
| clarkb | efoley: can you maybe be a bit more specific ( you don't need to use identifiable identifers). I am having a hard time parsing that | 18:54 |
| efoley | clarkb: sequence of events? | 18:54 |
| clarkb | efoley: yes the sequence of events | 18:54 |
| efoley | clarkb: 1- I removed the old e-mail from gerrit | 18:55 |
| efoley | clarkb: 2- I logged out of gerrit and tried to log in via ubuntu one using my old account details which had not yet changed - Which should not have worked, and (I thought) should confirm that the old address was removed | 18:56 |
| efoley | clarkb: 3- Logging into gerrit with the old address actually worked, and a new gerrit account was created, with my gerrit username and old e-mail address | 18:57 |
| efoley | clarkb: 4- I set my ubuntu one account to use my current e-mail address, to see if I could log into gerrit with my current e-mail and username -- this failed. | 18:59 |
| clarkb | thanks that helps. | 18:59 |
| clarkb | So there is a bit of difficulty in fixing these things. The first issue is taht gerrit isn't currently allowing us to edit external account ids individually because we have a few hundred similar conflicts (though in the case of existing problems it is because old gerrit allowed the conflcit that new gerrit prevents. basiclly old gerrit would've given you a third account at step 4, | 19:02 |
| clarkb | new gerrit errors) | 19:02 |
| clarkb | I've been slowly working through those and doing surgery on them so that we can get to a point where we have no oustanding issues in the account config allowing us to push changes for individual edits/fixups. There are about 174 in lfight right now in fact. But that still leaves us with something like 80 more to go | 19:03 |
| clarkb | I think there are two easy options available to us right now (but I'll need todig into the account configs to confirm). 1) remove the external ids from the old accounts and disable them then roll forward on the new account or 2) reset your ubuntu one account to the old email so that you get the old openid back. This should log you into the old account | 19:04 |
| clarkb | in your sequence of events above I think step 3 logged you into your old account not a new one (but I'm not 100% of that). The reason I think this is that gerrit forces you to set the username manually and doesn't populate that from the openid info. If you logged in and saw the same username then it was likely the same account | 19:05 |
| efoley | clarkb: step 3 logged me into a new account; the "account created" value was "x minutes ago" and none of my reviews were there | 19:06 |
| clarkb | efoley: ok, I suspect the username wasn't set then | 19:06 |
| clarkb | in that case option 2) above may be functionaly similar to option 1) basically require a roll forward. I'll need to dig into the accounts information and try to sort out what gerrit did | 19:08 |
| efoley | clarkb: can confirm: old e-mail/new account = no username set | 19:08 |
| opendevreview | Vishal Manchanda proposed openstack/project-config master: Retire django-openstack-auth https://review.opendev.org/c/openstack/project-config/+/800532 | 19:08 |
| clarkb | ok cool that helps. efoley can you PM me the account id number(s) you see in the gerrit ui if you are able to get into the gerrit ui? | 19:08 |
| clarkb | I need to eat lunch but when I get back I can collect all the relevant info for the accounts in question then we can make more concrete statements on what our options are | 19:09 |
| efoley | clarkb: okie dokie | 19:09 |
| clarkb | Alright I think I see what hapepend. The old account had old email removed from it. When you logged in again the openid that was provided was provided with the old email address. Gerrit decided that it needed a new account because no account matches that openid + email pair. It then created the new account and updated the existing openid externalid record to point to the new account | 19:37 |
| clarkb | and removed it from the old | 19:37 |
| opendevreview | Tristan Cacqueray proposed opendev/system-config master: Run matrix-gerritbot on eavesdrop https://review.opendev.org/c/opendev/system-config/+/800506 | 19:38 |
| clarkb | Then when you updated the ubuntu one account to have the new email address logins fail because the old account's preferred email address is the new email address and the conflict | 19:38 |
| efoley | clarkb: So now my old account doesn't have an openid associated with it. | 19:38 |
| clarkb | correct | 19:38 |
| clarkb | with old gerrit we would do a quick and easy sql statement to move the external id back t oyour old account and then disable teh new account | 19:39 |
| clarkb | with new gerrit all of that information is stored in the git notedb and we are not currently allowed to push updates to gerrit for those records without fixing all of the other existing conflicts first :/ | 19:40 |
| clarkb | (there is a not great workaround for that which I'll get to in a bit) | 19:40 |
| clarkb | I think that means our easy option is 1) roll forward on the new account. 2) won't work because I assumed the openid had changed (another openid related issue we run into), but in reality the openid has stayed the same gerrit just happily moved it for us :( | 19:42 |
| clarkb | The not great workaround we can try (we have yet to attempt it) is to stop gerrit, push into the git repo directly without gerrit verification, offline reindex accounts (and groups), then start gerrit and hope we did proper surgery | 19:42 |
| clarkb | The downside to this workaround is that it requires a gerrit downtime. And not a super short one due to the reindexing though reindexing of accounts and groups does go reasonably quikcly | 19:43 |
| efoley | clarkb: Option 2 sounds disruptive to everyone | 19:44 |
| clarkb | The "good" news is that ianw has been working on a gerrit downtime to move servers and planned to reindex things anyway. It is possible that ianw would be open to incorporating an external id cleanup in that move | 19:44 |
| efoley | clarkb: Option 1 sounds like I accept the loss and move to the new account, and am still stuck with the old e-mail address which I need to change. | 19:45 |
| clarkb | efoley: ^ if ianw is open to that and you are willing to wait a week (downtime is scheduled to begin at 23:00 UTC July 18) that may be a reasonable option | 19:45 |
| clarkb | efoley: if you do 1) we would remove the new address from the old account which should allow you to use it on the new account | 19:45 |
| efoley | clarkb: What does that mean for patchsets and review associated with the old account? | 19:49 |
| efoley | clarkb: they will still be owned by the old account and mean that I can't easily search for them? | 19:49 |
| clarkb | they will continue to be associated with the old account. We allow anyone to push new patchsets to existing changes regardless of ownership so that shouldn't be a big problem. Probably the biggest issues you'll have are searching as well as needing to use a different username to push | 19:50 |
| efoley | clarkb: I'm going on vacation at the end of the week, and don't expect to push changes this week, however, there are some reviews I wanted to do (in theory, I can use the command line to apply votes to reviews, but maybe can't comment). I'm open to option 2 and can wait a week, if ianw is open to option 2. | 19:51 |
| clarkb | efoley: you should be able to vote and commetn via the command line iirc. | 19:52 |
| clarkb | it may be clunky to do inline comments though | 19:53 |
| efoley | clarkb: I can deal with a bit of annoyance since it seems like I can piggyback on the cleanup that's on the TODO list anyway. | 19:55 |
| clarkb | (thinking out loud here) I wonder what would happen if we essentially retire and remove identities from the new account then add your old email address to the old account. Then you login again. Will it simply shift the account associating back to the old account? | 19:55 |
| efoley | clarkb: Wouldn't that require verifying the old e-mail address? | 19:56 |
| clarkb | efoley: if you're going to be around for the next half hour or so I don't mind trying ^ I don't think we can make this any worse (it might create a third account) | 19:56 |
| clarkb | efoley: it shouldn't because I think I can set it directly as admin without verification | 19:56 |
| fungi | clarkb: i'm only semi-around but what you describe sounds plausible, worth a shot anyway | 19:57 |
| efoley | clarkb: I'm curious, I was thinking that something like that might work IF I could skip the e-mail verification. | 19:57 |
| clarkb | efoley: give me a few minutes to read up on api calls and put together a list of curl commands to run but I'm game to try it | 19:58 |
| fungi | i'm not too sure to what extent they've tied e-mail verification to the external-ids association | 19:58 |
| fungi | i wouldn't be surprised if they just punted and it works (in which case, shhh, that could make it possible to squat other people's addresses if they don't have accounts with them yet) | 19:59 |
| clarkb | I suspect it will error because the external id has that email and belongs to another account | 20:00 |
| fungi | oh, if it's still associated with the other account, yes | 20:01 |
| clarkb | yup because I can't delete the external id, if I do that it will for sure create a new account | 20:01 |
| clarkb | but I also can't edit it. So I'm hoping we can simply convince it to sldie it back over like it did in the first place if I can convince it to add the email addr to the other account. But I expect this to error | 20:02 |
| clarkb | worth trying though | 20:02 |
| clarkb | ok confirmed "Email 'foo' is in use by another account" | 20:05 |
| clarkb | it is so frustrating that the rest api has methods for updating just about everything but external ids. This forces you to push external ids to gerrit via git which doesn't work because of completely unrelated conflicts | 20:09 |
| clarkb | efoley: only key thing if we do the update during the downtime is that you'll want to stick to an email address on the ubuntu one side then we will update the external id's email address to match when we update the account id | 20:11 |
| clarkb | because if you log in with the old email id after we do that we'll be right back where we started I Think | 20:11 |
| clarkb | I guess I could add an email address externalid to the new account now for the old email address to prevent it causing this problem again. That seems like a reasonable idea | 20:12 |
| efoley | You mean make sure that my new e-mail address is on the ubuntuone account? | 20:12 |
| clarkb | efoley: yes, beacuse if we update the gerrit account to use the new address then you log back in with the old address I think it may create a new account again | 20:14 |
| clarkb | not 100% sure of that but I think it is a risk | 20:14 |
| clarkb | also I cannot add the old email to the new account anymore than it is already there. Api says "email exists" | 20:14 |
| clarkb | efoley: ianw's day should be starting in an hour or two. We probably don't need to keep you around any longer. I'll see what ianw has to say about this and we can sync up tomorrow? | 20:15 |
| clarkb | ianw: efoley: also maybe we can test it on the new server before we do the cut over? | 20:17 |
| clarkb | https://bugs.chromium.org/p/gerrit/issues/detail?id=9318 is the related upstream bug for these issues fwiw | 20:17 |
| efoley | clarkb: Thanks for your help so far! | 20:20 |
| clarkb | but ya as is we unfortunately have no great answers. Just mediocre options :( | 20:20 |
| clarkb | efoley: you're welcome | 20:20 |
| efoley | clarkb: I would have thought this was a semi-common occurance, with people switching companies and still working on openstack. | 20:21 |
| clarkb | efoley: it sort of is. You're not the first to run into this. That said I think a lot of people use personal addresses to identify themselves then the work is an attribute against that rather than the identifier | 20:22 |
| clarkb | if you think about it from gerrit's perspective the issue is we're giving gerrit a new idnetifier and expecting it to not make a new account | 20:22 |
| clarkb | the inability to easily edit things to fix that up after the fact is definitely a gerrit issue | 20:23 |
| clarkb | and one that I wish was taken a bit more seriously upstream :/ | 20:23 |
| opendevreview | Ghanshyam proposed openstack/project-config master: Use publish-to-pypi-stable-only template for deprecated repo https://review.opendev.org/c/openstack/project-config/+/800558 | 20:23 |
| clarkb | I should probably fiel a new bug about the shift of an openid though | 20:26 |
| clarkb | since that is a subtle behavior that can probably stop as well | 20:26 |
| efoley | clarkb: It's unfortunate that is isn't taken seriously. We can't think from the perspective of the software that is used in the infra, so one should expect a few mistakes now and then. If it's difficult to fix mistakes, it can be offputting. Especially if someone's takeaway is that they should have known in advance not to try and change it, or to use a persistent e-mail address. | 20:29 |
| efoley | clarkb: anyway, I should be taking off, thanks for the help so far. | 20:30 |
| fungi | efoley: gerrit upstream is somewhat blind to these sorts of problems since they don't rely on openid to authenticate (i think they use ldap backended by google's auth service, because of being a google project) | 20:31 |
| opendevreview | Ghanshyam proposed openstack/project-config master: Update deprecated repo description and release job template https://review.opendev.org/c/openstack/project-config/+/800559 | 20:40 |
| clarkb | ya they force everyone to use a google account and those tend to map 1:1 with humans | 20:40 |
| clarkb | regardless of who your employer is | 20:40 |
| clarkb | fwiw I think it would work to update the ubuntu one side first, then clean up the gerrit side. Doing the other order does not | 20:40 |
| clarkb | infra-root ok https://review.opendev.org/c/opendev/system-config/+/800274 shoudl actually be ready for review now after I fixed the favicon thing | 20:42 |
| clarkb | actually no you need to update the gerrit account to include both sets of email addresses, update the ubuntu one side, then update gerrit to remove the old addresses to do this safely | 20:48 |
| clarkb | neat I'm not able to file issues in the gerrit issue tracker right now | 20:55 |
| * clarkb gets on their slack | 20:55 | |
| mordred | clarkb: you know, corvus has a matrix bridge for that | 20:59 |
| clarkb | mordred: ya, I have a tab for it in my browser that I have to restore after restarts | 21:00 |
| mordred | :) | 21:00 |
| clarkb | mordred: corvus: how does the matrix bridge handle slack threading? I've noticed the gerrit slack isntance users like to make use of threading whcih means that basically 90% of all messages are completely lost to time for me | 21:04 |
| clarkb | does matrix/element handle that in a sane way? | 21:04 |
| mordred | it does - it does the message reply interface | 21:05 |
| clarkb | hrm not sure I like that better :P but I'll at least see the messages in that case | 21:05 |
| mordred | lemme find an example | 21:05 |
| fungi | weeslack annotates the message in the main channel which started the thread, and then you can use /thread with the id to open a new buffer for it | 21:06 |
| fungi | but yeah, it's a pain and easy to miss | 21:06 |
| clarkb | fungi: yup the official slack clients aren't much better than weeslack in that regard | 21:06 |
| * mordred uploaded an image: (604KiB) < https://matrix.org/_matrix/media/r0/download/inaugust.com/85924b79438afdf4f3bfb894e9c3b931a71d2e2f/Screen%20Shot%202021-07-12%20at%2016.05.53.png > | 21:06 | |
| mordred | there's the current gerrit slack channel in matrix | 21:06 |
| clarkb | they basically do a little annotation forcing you to take action and the messages remain hidden otherwise. You can also go to "your threads" or something called something like that and it shows you all the threads you've participated in | 21:06 |
| mordred | yeah. threads in slack drive me CRAZY - it's the worst UI ever | 21:07 |
| mordred | with the matrix bit - I can at least follow and notice when something happened | 21:07 |
| clarkb | ya seems like you always get the messages front and center with what matrix does. You lose some context which may be a problem if people expect you to see hundreds of messages, but that is probably fine for gerrit slack since they don't do hundreds of messages in threads | 21:08 |
| *** dviroel is now known as dviroel|out | 21:41 | |
| ianw | o/ | 22:07 |
| clarkb | ianw: morning! | 22:07 |
| clarkb | I've been digging into the gerrit code and what I think happens is if you delete an email address in the web ui and that is the email address associated with the openid it deletes the openid externalid | 22:08 |
| clarkb | then when you try to log back in again it creates the openid externalid file again but with a new account id. | 22:08 |
| clarkb | Anyway gerrit slack has given me a workaround for the issue filing problem (i have to use a different template) so I'll try that | 22:09 |
| clarkb | ianw: re scrollback tl;dr is that efoley managed to orphan their old account because gerrit moved their old externalid record to a new account | 22:09 |
| clarkb | ianw: I've suggested that one way to fix this would be to do a All-Users:refs/meta/external-ids edit during the upgrade process since we will be reindexing then anyway | 22:09 |
| clarkb | ianw: I'm happy to help with that and be around (and probably test it on review02 in the next few days) if you think that is reasonable. Otherwise I've suggested that efoley can roll forward with the new account or we can maybe delay another week and do it during the project rename outage | 22:10 |
| clarkb | ianw: but wanted to gauge your comfort with that and will totally defer to you if you think that is too risky | 22:10 |
| clarkb | also I'm almost ready to send out the meeting agenda, if you have anything to add now is a good time :) | 22:12 |
| ianw | i think we can do some edits as part of the update. worse case roll-back scenario if edits cause problems it seems we re-rsync the data from review01 | 22:12 |
| clarkb | ianw: resync and probably reindex yup | 22:12 |
| clarkb | and I agree that is why I felt you might be ok with it | 22:12 |
| ianw | so is the solution gerrit saying "you can't delete that email"? | 22:18 |
| clarkb | ianw: we can delete the email from the old account and retire/disable it. Then the new account would work as the email conflict would go away | 22:19 |
| ianw | looks like cffi could produce 35 wheels without too much fuss and put them back https://foss.heptapod.net/pypy/cffi/-/issues/503 ... not that we want to use them for much longer, but gives us some more runway | 22:19 |
| clarkb | But if we want to restore the old account we have to edit the externalids for efoley's openid in All-Users:refs/meta/external-ids to set the accountId back to the old account and update the email address to match the account | 22:20 |
| clarkb | This second thing is what we need to do during a gerrit outage followed by reindexing of accounts (and possibly groups) | 22:20 |
| clarkb | https://bugs.chromium.org/p/gerrit/issues/detail?id=14776 is the related issue I've just filed upstream | 22:21 |
| clarkb | ianw: let me get the meeting agenda out then I'll put some notes on review02 with actual account numbers and emails and stuff | 22:21 |
| ianw | ah, thanks, that bug answers my next question of "how do you update your email" :) | 22:22 |
| ianw | oh, and that also explains why my issue went in under "polygerrit" too | 22:23 |
| clarkb | they default to polygerrit, but if you chagne it to "bug report" it just doesn't work | 22:23 |
| clarkb | ianw: another thing that came to mind today is we should probably send reminders of the server move. Maybe one todayish and one your Fridayish | 22:25 |
| ianw | sure, can send follow-up mail | 22:25 |
| ianw | another thing i thought is that i should pre-setup a maintenance splash page for review01's apache | 22:27 |
| fungi | ianw: i think the bits from when we put one up for the breach cleanup are still there | 22:29 |
| fungi | may just need to flip the switch to redirect to it | 22:29 |
| fungi | pretty sure i put it all in our config management | 22:29 |
| fungi | tweak the message to suit, obviously | 22:30 |
| clarkb | yup I think that is in config management still | 22:30 |
| ianw | ++ will check that out and add to checklist | 22:30 |
| clarkb | ianw: review02:~clarkb/efoley-notes.txt for my notes with concrete info | 22:32 |
| clarkb | ianw: if you've been testing with a running gerrit on review02 then we should be able to test that on 02 too. Maybe we can run through taht tomorrow? | 22:33 |
| ianw | sure, it hasn't been synced in a while. i can poke at that today, also sync to the latest gerrit images for good measure | 22:34 |
| clarkb | sounds good | 22:34 |
| clarkb | far less urgent and maybe worth punting on this week given everything else happening is https://review.opendev.org/c/opendev/system-config/+/800274 to upgrade gitea | 22:34 |
| clarkb | If you find yourself with extra time taking a look at that would be good but I suspect we won't land it until next week anyway | 22:35 |
| clarkb | ianw: the other thing we can test is my suspicion on 14776 | 22:37 |
| ianw | cool i noticed you had a held node, will poke at it | 22:37 |
| clarkb | I don't want to test that with my account on prod because I like being able to log in :) | 22:37 |
| clarkb | I'm going to go and read more gerrit code now to see if I can narrow that down | 22:38 |
| clarkb | ok I think that my suspicion is exactly the problem | 22:42 |
| clarkb | there is a accountBelongsToRealm() method that gets implemented on realm implementations to filter out externalids that belong to external account authentication | 22:42 |
| clarkb | oauth and ldap do this but openid does not appear to | 22:43 |
| * clarkb updates the bug | 22:44 | |
| clarkb | that explains why upstream doesn't hit this problem at least | 22:50 |
| ianw | that's some good java readin' | 23:02 |
| clarkb | I think to fix this we can subclass DefaultRealm in a OpenIDRealm that only overrides accountBelongsToRealm() then update java/com/google/gerrit/server/config/AuthModule.java to set the OpenID realm to that OpenIDRealm instaed of DefaultRealm | 23:06 |
| clarkb | where it gets weird is there is no openid external id scheme so I guess you have to assume anything that is an http(s):// prefixed external id name is an openid (this is probably fine as other external ids get username:foo) prefixes | 23:07 |
| clarkb | I don't have a gerrit dev env set up, but I guess if I end up with time after server upgrades and upstream hasn't volunteered to fix it I can give it a go | 23:13 |
| clarkb | public abstract @Nullable String scheme(); public boolean isScheme(String scheme) { return scheme.equals(scheme()); } | 23:18 |
| clarkb | I suspect I'm about to learn something about java name scoping | 23:19 |
| clarkb | this.scheme and scheme would be distinct in that function call | 23:20 |
| clarkb | I guess maybe it is smart based on the fact that one is accessed like an object and the other like a function | 23:21 |
| ianw | sure looks like it is calling a string as a function | 23:26 |
| clarkb | ianw: that apparently becomes an autogenerated getter function due to @AutoValue annotation on the class | 23:30 |
| clarkb | the public abstract @Nullable String scheme() definition tells the code generation that you want a getter for the attribute called scheme | 23:30 |
| clarkb | and ya I think since you can't treat the passed argument scheme as a method the parser figures it out | 23:31 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
