| *** sboyron has quit IRC | 00:07 | |
| clarkb | fungi: looks like your change has entered the gate now | 00:30 |
|---|---|---|
| fungi | thanks, and yeah i've been stalking it | 00:31 |
| clarkb | we should double check there isn't any unexpected changes to the other files that are being modified too before restarting though I reviewed the change multiple times and I expect htat is fine | 00:32 |
| fungi | i mean, i'll keep an eye on it during the restart too. in the unlikely event that it balks, i can manually unroll stuff | 00:36 |
| *** weshay|ruck is now known as weshay|pto | 00:41 | |
| *** mlavalle has quit IRC | 00:56 | |
| ianw | i manged to a "Zuul Summary" tab @ https://104.130.172.52/c/openstack/diskimage-builder/+/561003 just as a proof of concept. filling stuff into the tab, etc. is obviously to-do, but i think it's more the way plugins are supposed to work | 00:58 |
| ianw | it builds with bazel, which, i don't think is a blocker, but again is just more "gerrity" way to do things | 01:00 |
| clarkb | ianw: we're building gerrit with bazel. Also maybe this is somethign we can host upstream and then the upstream zuul can bazel build it for us or something | 01:00 |
| ianw | yeah, sorry i meant that *not* using bazel probably isn't a blocker; but it does make it look more just like everything else in the gerrit tree | 01:01 |
| ianw | it's all cribbed from the code editor and checks plugins. i'll clean it up and add a link to that mail, see if anyone wants to collaborate | 01:04 |
| fungi | mmm, the jgit config change hot a timeout on system-config-run-review | 01:41 |
| fungi | i wonder if it's causing gerrit to not start | 01:41 |
| fungi | of so, yay testing! | 01:41 |
| chengsheng1 | I am a new contributor. My code has been submitted for a long time, but there is no review. I ask for help if there is a problem with which process. https://review.opendev.org/c/openstack/nova/+/762330/ | 01:44 |
| clarkb | chengsheng1: your best bet will be to ask in #openstack-nova | 01:54 |
| clarkb | chengsheng1: the people there are responsible for reviewing changes to that repo | 01:54 |
| clarkb | fungi: looks like it timed out more "normally". It appears the deployment happened but then there wasn't enough time to do the testinfra testing | 01:55 |
| chengsheng1 | ok, thank you. | 01:56 |
| clarkb | fungi: task install ansible seems to not have been fast. Also there is a large gap after 01:24 that is odd | 01:56 |
| clarkb | chengsheng1: one thing to keep in mind is that last week was a big holiday week for those of us in the USA | 01:57 |
| clarkb | not sure if that would impact nova reviews but it seems possible | 01:57 |
| chengsheng1 | maybe this is the reason | 02:01 |
| fungi | clarkb: ahh, yeah, digging into the logs it looks like it was just unusually slow/stalling at that point. i'll recheck it | 02:02 |
| *** ysandeep|away is now known as ysandeep | 02:18 | |
| *** iurygregory|pto has quit IRC | 03:01 | |
| ianw | sigh, it hit a limestone mirror error? | 03:08 |
| *** whoami-rajat__ has quit IRC | 03:29 | |
| *** whoami-rajat__ has joined #opendev | 04:06 | |
| fungi | yeah, looks like it | 04:21 |
| fungi | if it actually merges this time, i should still be around to do a restart after it's deployed | 04:59 |
| openstackgerrit | Merged opendev/system-config master: Move jgit autogc disablement into correct config https://review.opendev.org/c/opendev/system-config/+/765004 | 05:06 |
| *** ykarel has joined #opendev | 05:23 | |
| fungi | yay | 05:24 |
| fungi | the deploy claims to have been successful too, checking configs now | 05:25 |
| fungi | looks right | 05:28 |
| fungi | doing a quick gerrit restart nw | 05:28 |
| fungi | now | 05:28 |
| fungi | on its way back up now | 05:29 |
| fungi | #status log restarted the gerrit service on review.o.o for the config change from https://review.opendev.org/765004 | 05:29 |
| openstackstatus | fungi: finished logging | 05:29 |
| fungi | looks like it's up and working fine | 05:30 |
| fungi | ianw: ^ | 05:32 |
| fungi | i'll stick around for a bit longer in case anyone reports problems | 05:32 |
| ianw | ++ | 05:32 |
| fungi | okay, suspending my higher order cognitive functions and entering low-power standby for a while. i'll wake on interrupt in 8 hours or so | 05:55 |
| *** marios has joined #opendev | 06:12 | |
| *** zbr has quit IRC | 06:13 | |
| *** zbr has joined #opendev | 06:14 | |
| *** hamalq has quit IRC | 06:17 | |
| *** hamalq has joined #opendev | 06:18 | |
| *** sboyron has joined #opendev | 06:42 | |
| *** brinzhang0 has joined #opendev | 06:46 | |
| *** brinzhang_ has quit IRC | 06:49 | |
| *** sboyron has quit IRC | 06:55 | |
| *** eolivare has joined #opendev | 07:27 | |
| *** brinzhang_ has joined #opendev | 07:32 | |
| *** brinzhang0 has quit IRC | 07:35 | |
| *** lpetrut has joined #opendev | 07:38 | |
| *** fressi has joined #opendev | 07:49 | |
| *** slaweq has joined #opendev | 07:56 | |
| *** hashar has joined #opendev | 08:00 | |
| *** hashar has quit IRC | 08:04 | |
| *** hashar has joined #opendev | 08:04 | |
| *** sboyron has joined #opendev | 08:05 | |
| *** rpittau|afk is now known as rpittau | 08:28 | |
| *** andrewbonney has joined #opendev | 08:34 | |
| *** DSpider has joined #opendev | 08:36 | |
| *** zbr has quit IRC | 08:44 | |
| *** zbr has joined #opendev | 08:45 | |
| *** zbr has quit IRC | 08:47 | |
| *** zbr has joined #opendev | 08:47 | |
| *** zbr has quit IRC | 08:49 | |
| *** tkajinam has quit IRC | 08:49 | |
| *** zbr has joined #opendev | 08:49 | |
| *** zbr has quit IRC | 08:52 | |
| *** zbr has joined #opendev | 08:52 | |
| *** tosky has joined #opendev | 09:05 | |
| *** marios has quit IRC | 09:06 | |
| *** sshnaidm is now known as sshnaidm|ruck | 09:34 | |
| *** sshnaidm|ruck has quit IRC | 10:01 | |
| *** sshnaidm has joined #opendev | 10:01 | |
| *** mklyus has joined #opendev | 10:08 | |
| mklyus | Hi, I need some one from core reviewers could take a look on this change https://review.opendev.org/c/openstack/project-config/+/764054 and help to get +2 and merge it | 10:09 |
| openstackgerrit | Merged openstack/project-config master: Remove openstack-python and openstack-docs job for x/vmtp project. https://review.opendev.org/c/openstack/project-config/+/764054 | 10:22 |
| *** ykarel_ has joined #opendev | 10:27 | |
| *** ykarel_ is now known as ykarel|lunch | 10:28 | |
| *** ykarel has quit IRC | 10:29 | |
| *** hashar is now known as hasharLunch | 10:32 | |
| *** icey_ has joined #opendev | 10:43 | |
| *** icey has quit IRC | 10:44 | |
| mklyus | Thanks a lot Ian | 10:52 |
| openstackgerrit | Stephen Finucane proposed opendev/elastic-recheck master: Add query for bug 1906428 https://review.opendev.org/c/opendev/elastic-recheck/+/765108 | 10:53 |
| openstackgerrit | Stephen Finucane proposed opendev/elastic-recheck master: gitignore: Ignore .eggs directory https://review.opendev.org/c/opendev/elastic-recheck/+/765109 | 10:53 |
| openstack | bug 1906428 in OpenStack Compute (nova) "test_cold_migrate_unshelved_instance failing with cat: can't open '/mnt/timestamp': No such file or directory" [Critical,Confirmed] https://launchpad.net/bugs/1906428 | 10:53 |
| *** sshnaidm is now known as sshnaidm|afk | 10:55 | |
| *** ysandeep is now known as ysandeep|afk | 11:04 | |
| *** marios has joined #opendev | 11:07 | |
| *** mklyus has quit IRC | 11:11 | |
| *** dtantsur|afk is now known as dtantsur | 11:19 | |
| *** ykarel|lunch is now known as ykarel | 11:30 | |
| *** ykarel_ has joined #opendev | 11:35 | |
| *** ykarel has quit IRC | 11:38 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 11:39 | |
| *** icey_ has quit IRC | 11:44 | |
| openstackgerrit | Sorin Sbârnea proposed zuul/zuul-jobs master: Enable ensure ansible to install local collection https://review.opendev.org/c/zuul/zuul-jobs/+/765125 | 11:50 |
| *** brinzhang_ has quit IRC | 11:52 | |
| *** brinzhang_ has joined #opendev | 11:52 | |
| *** icey has joined #opendev | 11:58 | |
| *** iurygregory has joined #opendev | 12:03 | |
| openstackgerrit | Sorin Sbârnea proposed zuul/zuul-jobs master: Enable ensure ansible to install local collection https://review.opendev.org/c/zuul/zuul-jobs/+/765125 | 12:06 |
| openstackgerrit | Sorin Sbârnea proposed zuul/zuul-jobs master: Enable ensure ansible to install local collection https://review.opendev.org/c/zuul/zuul-jobs/+/765125 | 12:09 |
| *** rishabhhpe has joined #opendev | 12:14 | |
| openstackgerrit | Sorin Sbârnea proposed opendev/gerritbot master: Skip notifications about WIP changes https://review.opendev.org/c/opendev/gerritbot/+/765130 | 12:24 |
| *** dtantsur is now known as dtantsur|brb | 12:31 | |
| *** toma4 has quit IRC | 12:55 | |
| *** rishabhhpe has quit IRC | 12:55 | |
| *** hamalq has quit IRC | 13:01 | |
| *** hamalq has joined #opendev | 13:05 | |
| *** ykarel_ is now known as ykarel | 13:30 | |
| *** ysandeep|afk is now known as ysandeep | 13:30 | |
| *** hasharLunch is now known as hashar | 13:43 | |
| *** dtantsur|brb is now known as dtantsur | 13:46 | |
| *** eharney has joined #opendev | 13:56 | |
| *** auristor has quit IRC | 14:10 | |
| openstackgerrit | Merged opendev/gerritlib master: Avoid crash due to POLLIN event check https://review.opendev.org/c/opendev/gerritlib/+/729966 | 14:29 |
| *** hashar is now known as hasharAway | 15:05 | |
| openstackgerrit | Sorin Sbârnea proposed opendev/gerritbot master: Skip notifications about WIP changes https://review.opendev.org/c/opendev/gerritbot/+/765130 | 15:07 |
| *** auristor has joined #opendev | 15:09 | |
| *** ysandeep is now known as ysandeep|away | 15:23 | |
| *** mlavalle has joined #opendev | 15:28 | |
| openstackgerrit | Sorin Sbârnea proposed opendev/gerritbot master: Skip notifications about WIP changes https://review.opendev.org/c/opendev/gerritbot/+/765130 | 15:37 |
| *** lpetrut has quit IRC | 15:41 | |
| openstackgerrit | Sorin Sbârnea proposed opendev/gerritbot master: Use yaml.safe_load instead of load. https://review.opendev.org/c/opendev/gerritbot/+/765164 | 15:46 |
| *** slaweq has quit IRC | 15:54 | |
| *** slaweq has joined #opendev | 15:56 | |
| *** redrobot has quit IRC | 16:11 | |
| *** brinzhang0 has joined #opendev | 16:14 | |
| *** brinzhang_ has quit IRC | 16:16 | |
| fungi | we're around 2/3 full on the changeid_project cache after the gerrit restart from just before i went to sleep. 39% cache hit ratio at the moment | 16:38 |
| *** marios is now known as marios|out | 16:38 | |
| *** marios|out has quit IRC | 16:44 | |
| openstackgerrit | Merged zuul/zuul-jobs master: upload-logs-s3: fix test-jobs files: attribute https://review.opendev.org/c/zuul/zuul-jobs/+/764476 | 16:45 |
| openstackgerrit | Sorin Sbârnea proposed opendev/gerritbot master: Use yaml.safe_load instead of load. https://review.opendev.org/c/opendev/gerritbot/+/765164 | 16:51 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: GZIPCompressedStream default read length to -1 https://review.opendev.org/c/zuul/zuul-jobs/+/764478 | 16:52 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: GZIPCompressedStream default read size to -1 https://review.opendev.org/c/zuul/zuul-jobs/+/764478 | 16:53 |
| openstackgerrit | Merged opendev/elastic-recheck master: Add query for bug 1906428 https://review.opendev.org/c/opendev/elastic-recheck/+/765108 | 17:02 |
| openstack | bug 1906428 in OpenStack Compute (nova) "test_cold_migrate_unshelved_instance failing with cat: can't open '/mnt/timestamp': No such file or directory" [Critical,In progress] https://launchpad.net/bugs/1906428 | 17:02 |
| clarkb | melwitt: ^ reminded me to check the health of the processing pipeline and we still have all 80 workers reporting to gearman says gearman status. I think your change made a big improvement. THank you! | 17:03 |
| *** ykarel is now known as ykarel|away | 17:04 | |
| *** rpittau is now known as rpittau|afk | 17:04 | |
| melwitt | clarkb: that's great to hear :) | 17:06 |
| openstackgerrit | Merged opendev/elastic-recheck master: gitignore: Ignore .eggs directory https://review.opendev.org/c/opendev/elastic-recheck/+/765109 | 17:07 |
| *** ykarel|away has quit IRC | 17:09 | |
| *** ykarel|away has joined #opendev | 17:17 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 17:18 |
| *** ykarel|away has quit IRC | 17:22 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 17:38 |
| *** eolivare has quit IRC | 17:44 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 17:47 | |
| *** toma4 has joined #opendev | 17:48 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 17:48 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 18:04 |
| *** lpetrut has joined #opendev | 18:09 | |
| *** hasharAway is now known as hashar | 18:15 | |
| *** andrewbonney has quit IRC | 18:19 | |
| openstackgerrit | Merged opendev/system-config master: gerrit: set ownership on ~gerrit2/.ssh directory https://review.opendev.org/c/opendev/system-config/+/764395 | 18:25 |
| openstackgerrit | Merged opendev/system-config master: gerrit: fix db/ mount for gate testing https://review.opendev.org/c/opendev/system-config/+/764396 | 18:29 |
| openstackgerrit | Jeremy Stanley proposed opendev/system-config master: Update gerrit docker image to java 11 https://review.opendev.org/c/opendev/system-config/+/763656 | 18:34 |
| openstackgerrit | Jeremy Stanley proposed opendev/system-config master: Build Gerrit 3.3 images https://review.opendev.org/c/opendev/system-config/+/765021 | 18:40 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: ensure-podman: Use official podman repos for ubuntu https://review.opendev.org/c/zuul/zuul-jobs/+/765177 | 18:43 |
| *** dtantsur is now known as dtantsur|afk | 18:47 | |
| openstackgerrit | Merged zuul/zuul-jobs master: GZIPCompressedStream default read size to -1 https://review.opendev.org/c/zuul/zuul-jobs/+/764478 | 18:49 |
| *** toma4 has quit IRC | 19:00 | |
| fungi | i'm heading out to run some errands, should be back in less than an hour. hopefully by then the openjdk 11 update should have made it into our gerrit images | 19:01 |
| melwitt | with the new gerrit, I read that there's no integration with updating launchpad for blueprints, but what about bugs? I noticed my bug didn't get a comment and state update on it | 19:28 |
| *** iurygregory has quit IRC | 19:29 | |
| fungi | melwitt: yes, see https://etherpad.opendev.org/p/gerrit-3.2-post-upgrade-notes currently around line #60 | 19:48 |
| fungi | we need to rewrite some chunks of jeepyb which used to perform database queries. there's no more rdbms containing that data so some bits need to switch to using gerrit rest api calls | 19:48 |
| fungi | it's a priority, but the gerrit performance problems have been taking more priority for now | 19:49 |
| melwitt | thanks fungi | 19:51 |
| *** hashar has quit IRC | 20:23 | |
| *** lpetrut has quit IRC | 20:29 | |
| *** fressi has quit IRC | 20:47 | |
| fungi | looks like we've got a sizeable node request backlog | 20:48 |
| fungi | one of the jobs for the openjdk 11 change took nearly an hour to get a node assignment in the gate, as the only change in that queue | 20:49 |
| fungi | looks like tripleo may be suffering some gate thrash | 20:49 |
| clarkb | oh ya change near the top had a failure | 20:50 |
| *** brinzhang_ has joined #opendev | 20:58 | |
| *** brinzhang0 has quit IRC | 21:01 | |
| openstackgerrit | Merged opendev/system-config master: Update gerrit docker image to java 11 https://review.opendev.org/c/opendev/system-config/+/763656 | 21:10 |
| fungi | yay! deploy running now | 21:11 |
| fungi | oh, and promote | 21:11 |
| fungi | i wonder if those race one another | 21:11 |
| fungi | clarkb: ^ do you think? | 21:11 |
| ianw | i think they're all ordered? | 21:12 |
| fungi | in this particular case i guess it didn't | 21:12 |
| fungi | ianw: well, we tag the image in dockerhub in promote and also pull in deploy | 21:12 |
| fungi | it's conceivable that we may pull before the image is tagged | 21:13 |
| fungi | ahh, though we also run the promote job in the deploy pipeline | 21:13 |
| fungi | so maybe that's how we get around it | 21:13 |
| ianw | although some jobs in there have dependencies on their image promote, and others don't | 21:15 |
| fungi | so the image promotion in the promote pipeline isn't guaranteed to run before the deploy job pulls it, but the copy of the (presumably idempotent?) promote job which also runs in the deploy pipeline is strictly ordered before the deploy job which would pull it | 21:15 |
| ianw | like all these have dependencies explicitly listed : https://opendev.org/opendev/system-config/src/branch/master/zuul.d/project.yaml#L236 | 21:17 |
| ianw | but not review https://opendev.org/opendev/system-config/src/branch/master/zuul.d/project.yaml#L267 | 21:18 |
| ianw | i'm responsible for a bunch of them in there, i've just been copy-pasting | 21:18 |
| fungi | copy-pasting is the best kind of pasting | 21:19 |
| mordred | mmm pasta | 21:20 |
| clarkb | for some reason I thought we didn't run them in the promote pipeline at all | 21:20 |
| clarkb | but I also think that job is indempotent? it scans tags and does the right thing | 21:21 |
| fungi | maybe not, i saw it the change show up in the promote pipeline for a little while but didn't check what jobs were run for it | 21:21 |
| ianw | gerrit we have templates for https://opendev.org/opendev/system-config/src/branch/master/zuul.d/project.yaml#L267 | 21:21 |
| fungi | it was done by the time i realized it was also already in deploy and started to question my assumptions | 21:22 |
| ianw | it might be nice to split all the services up into templates in separate files like that | 21:22 |
| fungi | anyway, infra-prod-service-review is finally running in deploy for it, so i'll check the docker image ls shortly | 21:22 |
| fungi | and assuming that looks good we can start thinking about when will be a good time for yet still another service restart | 21:23 |
| clarkb | fungi: we should double check ianw's chagnes landed happily too | 21:23 |
| fungi | oh, yup | 21:23 |
| fungi | i also need to start cooking dinner soon | 21:24 |
| fungi | but i'm confident i can have this restarted on openjdk 11 before i head to bed | 21:24 |
| ianw | do we know why infra-prod-manage-projects failed? | 21:25 |
| clarkb | I do not | 21:25 |
| clarkb | fwiw I did manually test project creation on review-test with jeepyb but maybe something isn't working around that? | 21:26 |
| clarkb | ianw: do you have a log link? | 21:26 |
| ianw | i have to run out for about 30 mins now, but will then be around to monitor for explosions. i guess worse case we roll back a tag | 21:26 |
| clarkb | wfm | 21:26 |
| clarkb | I too am juggling some other stuff | 21:26 |
| clarkb | but am somewhat around | 21:26 |
| ianw | 2020-12-02 18:40:16,860: manage_projects - INFO - Writing cache file /opt/lib/jeepyb/project.cache | 21:27 |
| ianw | 2020-12-02 18:40:16,911: manage_projects - ERROR - 1 problems has been caught during run: | 21:27 |
| ianw | ['Problems creating openstack/tripleo-ci, moving on.'] | 21:27 |
| ianw | via https://zuul.opendev.org/t/openstack/build/46e0e0ee3f9f42f49e5c9f5739181826 | 21:27 |
| *** iurygregory has joined #opendev | 21:28 | |
| ianw | Exception: Gerrit error executing gerrit create-group --visible-to-all "tripleo-ci-core" | 21:28 |
| clarkb | hrm | 21:28 |
| fungi | that's decidedly odd | 21:28 |
| clarkb | so I think maybe that file stopped getting bind mounted in maybe when we cleaned up the local replication | 21:29 |
| fungi | oh! yes manage-projects may have wanted to init repos for the local replica? | 21:30 |
| ianw | i have to get kids to school, bib | 21:30 |
| clarkb | hrm we bind mount in that path in the manage-projects script | 21:30 |
| fungi | and this is about the create-group api call too | 21:31 |
| clarkb | we only dropped the local replica not /opt/lib/jeepyb | 21:31 |
| clarkb | fungi: we should set the jeepyb setting to not do a local replica | 21:31 |
| fungi | tripleo-ci-core also definitely already exists in the existing gerrit groups | 21:33 |
| fungi | i too need to step away momentarily to start cooking dinner | 21:34 |
| clarkb | ya I think the bit about the project.cache confused me I thought it was related to the error but the error is in creating that group for some reason | 21:34 |
| *** sboyron has quit IRC | 21:43 | |
| clarkb | the tripleo-ci.config update landed recently. jeepyb sees that as an udpated config (properly so) so thats all good. I think the issue is in doing group listings to determine if we need to create the group or not | 21:56 |
| clarkb | when I tested I tested the new project creation and not udpate old project situation | 21:57 |
| clarkb | I think I figured it out. We use -q and that returns -q is not a valid option | 21:58 |
| fungi | ooh, we used to rely on gerrit db checks for the group existing to avoid race conditions, maybe that's still in there? | 21:58 |
| clarkb | no I "fixed" that by querying the api | 21:59 |
| fungi | ahh, that's far more straightforward | 21:59 |
| clarkb | but the documented -q option doesn't seem to be working | 21:59 |
| fungi | yeah | 21:59 |
| clarkb | (nad i just checked current gerrit docs show -q as valid, maybe it is an order problem I'll check that next) | 21:59 |
| clarkb | nerp seems that -q is just not valid not to rtfs and see if it was renamed | 21:59 |
| clarkb | rtfs'ing seems to show only -v is a valid option. So I think we need to update gerritlib to do a full listing then filter from there | 22:01 |
| clarkb | which is less than ideal | 22:01 |
| clarkb | though I'm still poking around in the source | 22:02 |
| fungi | i'm poking chicken and mushrooms around in a skillet, sorry | 22:02 |
| fungi | can help take a look after we eat though | 22:02 |
| clarkb | I don't think it is urgent | 22:04 |
| clarkb | this seems to be failing in a safe way | 22:04 |
| clarkb | more just trying to sort out what gerrit expects you to do now | 22:04 |
| ianw | clarkb: do you have a quick link to what you're looking at, for those of us who maybe get a little lost in these layers? :) | 22:05 |
| clarkb | ianw: ya one sec | 22:08 |
| clarkb | ianw: https://opendev.org/opendev/jeepyb/src/branch/master/jeepyb/cmd/manage_projects.py#L559 this condition is true beacuse the acl was updated ( change merged last week). We then call process_acls which calls create_groups_file which calls get_group_uuid which calls _get_group_uuid. | 22:10 |
| clarkb | we fail to get the group uuid because the -q used in https://opendev.org/opendev/gerritlib/src/branch/master/gerritlib/gerrit.py#L352 is no longer a valid argument to that command in gerrit | 22:11 |
| clarkb | we then bubble back up to _get_group_uuid which tries to create teh group when the group does not exist | 22:11 |
| clarkb | I think new project creations may also not work right because we won't get their group uuids properly | 22:11 |
| clarkb | the flag is now -g | 22:13 |
| clarkb | I don't know when that changed :/ | 22:13 |
| clarkb | but I guess I'll try to figure that out next so we can check the gerrit version and then switch on that? | 22:14 |
| ianw | https://gerrit-review.googlesource.com/Documentation/cmd-ls-groups.html does show -q right? | 22:18 |
| clarkb | yup the docs are wrong. They depreacted -q in 2.14 and 2.14 supported -q and -g | 22:20 |
| clarkb | I think the docs just never updated | 22:20 |
| clarkb | I can make a docs fix for gerrit 3.3 in a bit too | 22:20 |
| clarkb | ya confirmed not depreactedin 2.13 | 22:21 |
| clarkb | is depreacted in 2.14 and the docs are just wrong | 22:21 |
| ianw | yay! already got a -3 on http://sweng.the-davies.net/Home/rustys-api-design-manifesto and it's not even 9:30am! :) | 22:22 |
| clarkb | just working on improving our testing around this then I'll push | 22:27 |
| clarkb | as a side note we appaer to still be managing projects on review-dev despite it being in emergency.yaml | 22:32 |
| clarkb | I don't think we want that behavior because one reason we put things in emergency.yaml is so that eg manage projects doesn't run? maybe someone else can look at that | 22:32 |
| ianw | the match looks right ... - hosts: "review:review-dev:!review-test.opendev.org:!disabled" | 22:34 |
| ianw | review-dev01.opendev.org is being managed, emergency has openstack.org | 22:36 |
| ianw | - review-dev01.openstack.org # mordred 2019/09/19 working on dockerification | 22:38 |
| ianw | i have the file open, so if you want me to update i can | 22:38 |
| openstackgerrit | Clark Boylan proposed opendev/gerritlib master: Handle newer Gerrit ls-groups command syntax https://review.opendev.org/c/opendev/gerritlib/+/765212 | 22:39 |
| clarkb | ianw: no we want it disabled | 22:39 |
| clarkb | beacuse its on the odl code and is effectively a dead server | 22:39 |
| clarkb | I didn't expect that manage projects would run on it | 22:40 |
| clarkb | is review-dev group not matching review-dev01.opendev.org ? | 22:40 |
| clarkb | ianw: fungi: I believe https://review.opendev.org/c/opendev/gerritlib/+/765212 should hopefully fix this problem and I added a bit more robust testing to the tests we run there | 22:40 |
| clarkb | But I still think we should only land that once we're happy there aren't other gremlins in manage-projects and we should avoid adding more projects or acl updates until we land this fix | 22:41 |
| ianw | clarkb: i mean in emergency it's listed as review-dev01.openstack.org; so that's why review-dev01.opendev.org isn't being skipped | 22:41 |
| clarkb | oh! | 22:41 |
| clarkb | ianw: yes please fix it :) | 22:42 |
| ianw | ok, done, noted that it was the old server in the comment | 22:43 |
| fungi | okay, dinner is done, i'm catching up | 22:45 |
| clarkb | I'm going to work on a upstream docs fix now then I promised I would help with some foundation stuff then I need to help kids with lights on a tree | 22:45 |
| ianw | clarkb: do you think it's worth a manual run with that fix to confirm? | 22:46 |
| clarkb | ianw: ya that would probably be a good idea, we can limit it to just that project too | 22:47 |
| clarkb | ianw: I think to do that we need to put review in emergency, then manually pull the new image once it lands, then run manage-projects? | 22:47 |
| clarkb | otherwise our daily run will hit it ? (I think we have ad aily run) | 22:47 |
| fungi | and now i've discovered rusty's fir the first time ever, my day is complete | 22:47 |
| corvus | fungi: i need way more context for that last | 22:49 |
| corvus | my mental pattern matching is coming up with lots of possibilities but most are very weird with very little discrimination between the levels of weirdness | 22:50 |
| ianw | clarkb: ok, i can probably figure out babysitting a manual run if we're happy with the change. it lgtm, though i claim no expertise | 22:51 |
| clarkb | https://gerrit-review.googlesource.com/c/gerrit/+/290603 is the upstream fix | 22:51 |
| clarkb | ianw: ya I dunno. manage-projects always scares me. That makes me both want to be around and not be around when we run things :) | 22:52 |
| clarkb | ianw: the testing is getting better though so maybe we want to focus on that a bit more if we think we're missing any other cases we shoudl check | 22:53 |
| clarkb | I think the reason that testing didn't have problems with this before is we only used Registered Users in the test which is a built in group and handled differently iirc | 22:55 |
| clarkb | that is why I added the two new groups | 22:55 |
| clarkb | those should get created "normally" and be more representative | 22:55 |
| fungi | corvus: sorry, s/fir/for/ and responding to ianw's link to http://sweng.the-davies.net/Home/rustys-api-design-manifesto | 22:56 |
| fungi | it's apparently like a bingo card for api design | 22:57 |
| ianw | oh, i think in looking, i've answered my question on creating the initial admin gerrit | 22:59 |
| ianw | you just do a GET on http://localhost:8080/a/accounts/admin/sshkeys and that creates the initial user? | 22:59 |
| ianw | that is way easier than i thought | 23:00 |
| *** tkajinam has joined #opendev | 23:00 | |
| corvus | ianw: we do more than that in the zuul quickstart | 23:00 |
| fungi | jeepyb fix lgtm, approved | 23:00 |
| clarkb | fungi: should we put review in emergency so that we can manually run it before the daily run? | 23:01 |
| clarkb | or are we happy with the testing and goign to go for it? | 23:01 |
| * clarkb is running out of day so will defer to others on that | 23:01 | |
| fungi | the test adjustments you added satisfied me | 23:01 |
| *** slaweq has quit IRC | 23:02 | |
| ianw | clarkb: yeah, i feel like we assume the best and monitor it | 23:04 |
| clarkb | though suddenly I wonder if we need a new gerritlib release too? | 23:04 |
| clarkb | and/or need to update jeepyb or gerrit to get the image updated | 23:04 |
| clarkb | its possible just landing that chagne won't get the code where we need it | 23:05 |
| ianw | hrm, i made gerritbot use gerritlib master, but not sure about other paths | 23:07 |
| fungi | also i +2'd a change toady which would make gerritlib master uninstallable on ubuntu xenial, just a heads up that if we want to continuously deploy gerritlib we probably should stick to doing it in containers (so for jeepyb with our manage-projects run it's probably fine) | 23:10 |
| clarkb | fungi: whcih change is that? | 23:10 |
| clarkb | if it is the python3.6 chagne i meant to -2 it | 23:11 |
| clarkb | I don't recall if I did or not | 23:11 |
| openstackgerrit | Merged opendev/gerritlib master: Handle newer Gerrit ls-groups command syntax https://review.opendev.org/c/opendev/gerritlib/+/765212 | 23:11 |
| fungi | https://review.opendev.org/754784 | 23:11 |
| fungi | please -2 if you have concerns | 23:11 |
| fungi | i'm trying not to stand in the way of progress like arthur dent and the bulldozer | 23:12 |
| clarkb | in this particular care I think the order of operations should be "update all the things running gerritlib on xenial" first then | 23:13 |
| * fungi checks his bathrobe pocket for an anaklgesic | 23:13 | |
| fungi | analgesic | 23:13 |
| clarkb | s/care/case/ | 23:14 |
| clarkb | from where I'm sitting we don't gain much in dropping python3.5 support on that library | 23:14 |
| clarkb | we can add new python support easily enough | 23:14 |
| fungi | seems like the oprimary concern raised by that change is that we weren't testing it on 3.5 | 23:14 |
| clarkb | we are testing on 35 | 23:15 |
| clarkb | or we were when that was first pushed up | 23:15 |
| clarkb | maybe the bug is taht it got removed elsewhere and side effected gerritlib | 23:15 |
| ianw | corvus: thanks for pointing that out. i'm not sure we need to deploy https://opendev.org/zuul/zuul/src/branch/master/doc/source/examples/playbooks/project.config for the gate testing scenario but the bones there are very helpful | 23:17 |
| clarkb | looking at jeepyb I think we do need a release of gerritlib and then a jeepyb update (or gerrit update) to update the code in thei mage | 23:18 |
| corvus | ianw: yeah i would expect this is what we'd need to do: https://opendev.org/zuul/zuul/src/branch/master/doc/source/examples/playbooks/setup.yaml#L28-L45 | 23:23 |
| clarkb | I think there is a small bug in my testing for gerritlib too https://zuul.opendev.org/t/openstack/build/2686beace42b4d90807319c16bc10f43/log/job-output.txt#767 shows that test-repo1 isn't getting updated acls. but if we look below in the log testrepo2 is updated. Does lineinfile only modify the last occurence? I expected it to do all of them | 23:25 |
| clarkb | the testing seems to show things are still working as expected, but we're not getting the behavior out of the test itself that I expected | 23:25 |
| clarkb | yup ansible docs confirm it "only the last line found will be replaced | 23:26 |
| fungi | that's... unfortunate | 23:27 |
| clarkb | I'm getting a brute force hack to amke it work better :) | 23:28 |
| openstackgerrit | Clark Boylan proposed opendev/gerritlib master: Test manage-projects repos harder https://review.opendev.org/c/opendev/gerritlib/+/765217 | 23:29 |
| clarkb | fungi: ianw corvus ^ that should replace both lines and then do the more aggressive testing that I expected | 23:29 |
| clarkb | however, I think the checks against just test-repo-2 show that the change should work as is | 23:30 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!