| *** holser has joined #zuul | 00:03 | |
| *** shanemcd has quit IRC | 00:03 | |
| *** shanemcd has joined #zuul | 00:08 | |
| *** jamesmcarthur has quit IRC | 00:09 | |
| *** jamesmcarthur has joined #zuul | 00:13 | |
| *** jamesmcarthur has quit IRC | 00:18 | |
| *** jamesmcarthur has joined #zuul | 00:36 | |
| *** holser has quit IRC | 00:45 | |
| *** jamesmcarthur has quit IRC | 01:23 | |
| *** jamesmcarthur has joined #zuul | 01:30 | |
| *** jamesmcarthur has quit IRC | 01:35 | |
| *** jamesmcarthur has joined #zuul | 01:56 | |
| *** jamesmcarthur has quit IRC | 02:13 | |
| *** jamesmcarthur has joined #zuul | 02:18 | |
| *** ikhan has joined #zuul | 02:28 | |
| *** jamesmcarthur has quit IRC | 02:33 | |
| *** jamesmcarthur has joined #zuul | 02:35 | |
| *** jamesmcarthur has quit IRC | 02:35 | |
| *** jamesmcarthur has joined #zuul | 02:35 | |
| *** ikhan has quit IRC | 03:46 | |
| *** ikhan has joined #zuul | 03:47 | |
| *** ikhan has quit IRC | 03:53 | |
| *** vishalmanchanda has joined #zuul | 04:14 | |
| *** jamesmcarthur has quit IRC | 04:16 | |
| *** jamesmcarthur has joined #zuul | 04:16 | |
| *** jamesmcarthur has quit IRC | 04:21 | |
| *** ykarel has joined #zuul | 04:25 | |
| *** jamesmcarthur has joined #zuul | 04:32 | |
| *** jamesmcarthur has quit IRC | 04:42 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** ykarel_ has joined #zuul | 05:51 | |
| *** ykarel has quit IRC | 05:53 | |
| *** ikhan has joined #zuul | 06:15 | |
| *** jfoufas1 has joined #zuul | 06:25 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Align ZooKeeper builds and merger API https://review.opendev.org/c/zuul/zuul/+/773031 | 06:30 |
|---|---|---|
| *** ykarel_ is now known as ykarel | 06:40 | |
| *** jamesmcarthur has joined #zuul | 06:43 | |
| *** gmann has quit IRC | 06:45 | |
| *** gmann has joined #zuul | 06:46 | |
| *** jamesmcarthur has quit IRC | 06:48 | |
| *** ikhan has quit IRC | 07:26 | |
| *** jcapitao has joined #zuul | 07:41 | |
| *** holser has joined #zuul | 07:44 | |
| *** ikhan has joined #zuul | 07:58 | |
| *** jcapitao has quit IRC | 08:00 | |
| *** jcapitao has joined #zuul | 08:02 | |
| *** rpittau|afk is now known as rpittau | 08:11 | |
| *** ykarel is now known as ykarel|lunch | 08:34 | |
| *** holser has quit IRC | 08:36 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Implement ZooKeeper builds API https://review.opendev.org/c/zuul/zuul/+/770901 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Switch to ZooKeeper backed job execution and result events https://review.opendev.org/c/zuul/zuul/+/770902 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Merge ZooKeeper connection and client classes https://review.opendev.org/c/zuul/zuul/+/771442 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Refactor pipeline processing in run handler https://review.opendev.org/c/zuul/zuul/+/771452 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Dequeue superceded items via management event https://review.opendev.org/c/zuul/zuul/+/771453 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Text stream API for sharded Zookeeper data https://review.opendev.org/c/zuul/zuul/+/771454 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Cache unparsed config files in Zookeeper https://review.opendev.org/c/zuul/zuul/+/771455 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Store tenants in unparsed abide as dict https://review.opendev.org/c/zuul/zuul/+/771456 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Refactor config/tenant (re-)loading https://review.opendev.org/c/zuul/zuul/+/771457 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Tenant read/write lock in Zookeeper https://review.opendev.org/c/zuul/zuul/+/771458 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Lock pipelines during processing https://review.opendev.org/c/zuul/zuul/+/771459 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Lock global event queues during processing https://review.opendev.org/c/zuul/zuul/+/771460 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Store tenant layout state in Zookeeper https://review.opendev.org/c/zuul/zuul/+/771461 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Configure unique command socket path per scheduler https://review.opendev.org/c/zuul/zuul/+/771462 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Support cross scheduler config loading https://review.opendev.org/c/zuul/zuul/+/771463 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Add UUID for queue items https://review.opendev.org/c/zuul/zuul/+/772512 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Store semaphore state in Zookeeper https://review.opendev.org/c/zuul/zuul/+/772513 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Fix test_gerrit.TestPolling.test_config_update https://review.opendev.org/c/zuul/zuul/+/773023 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Move serialization helper methods to ZooKeeperBase class https://review.opendev.org/c/zuul/zuul/+/773024 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Implement ZooKeeper backed merge jobs https://review.opendev.org/c/zuul/zuul/+/773025 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Switch to ZooKeeper backed merge jobs https://review.opendev.org/c/zuul/zuul/+/773026 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Collect statsd information from ZooKeeper rather than gearman https://review.opendev.org/c/zuul/zuul/+/773027 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Remove remaining gearman parts from merger https://review.opendev.org/c/zuul/zuul/+/773028 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Provide zk_client to merger client rather than the whole scheduler https://review.opendev.org/c/zuul/zuul/+/773029 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Remove history from RecordingMergeClient https://review.opendev.org/c/zuul/zuul/+/773030 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Align ZooKeeper builds and merger API https://review.opendev.org/c/zuul/zuul/+/773031 | 08:38 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: TODO Improve connection events https://review.opendev.org/c/zuul/zuul/+/773304 | 08:38 |
| *** holser has joined #zuul | 08:40 | |
| *** tosky has joined #zuul | 08:43 | |
| *** hashar has joined #zuul | 08:46 | |
| *** jpena|off is now known as jpena | 08:58 | |
| *** ikhan has quit IRC | 09:01 | |
| *** nils has joined #zuul | 09:04 | |
| *** reiterative has quit IRC | 09:10 | |
| *** reiterative has joined #zuul | 09:13 | |
| *** piotrowskim has joined #zuul | 09:29 | |
| *** holser has quit IRC | 09:38 | |
| *** ykarel|lunch is now known as ykarel | 09:44 | |
| openstackgerrit | Clément Mondion proposed zuul/zuul master: [UI][Nodes] use patternfly 4 https://review.opendev.org/c/zuul/zuul/+/768192 | 09:48 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: gitlab: Add access token name, Update docs, Fix webhook https://review.opendev.org/c/zuul/zuul/+/771184 | 10:20 |
| *** sshnaidm|off is now known as sshnaidm|ruck | 10:35 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: gitlab: Add access token name, Update docs, Fix webhook https://review.opendev.org/c/zuul/zuul/+/771184 | 10:48 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: REST API: Implement nodes filtering https://review.opendev.org/c/zuul/zuul/+/736042 | 10:50 |
| *** holser has joined #zuul | 11:17 | |
| mhu | Hello zuul-maint, if this edit to zuul-client's installation doc is fine with you, let's merge it and tag a new version of zuul-client with the fixes to the encrypt command | 11:25 |
| mhu | https://review.opendev.org/c/zuul/zuul-client/+/767592 | 11:25 |
| mhu | so I can do a release on the rpm package for Fedora too | 11:25 |
| *** jamesmcarthur has joined #zuul | 11:41 | |
| *** hashar is now known as hasharLunch | 11:41 | |
| *** jamesmcarthur has quit IRC | 11:46 | |
| *** jcapitao is now known as jcapitao_lunch | 12:03 | |
| *** rlandy has joined #zuul | 12:16 | |
| *** ikhan has joined #zuul | 12:19 | |
| *** jpena is now known as jpena|lunch | 12:30 | |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Update tests/base.py to use proper git data https://review.opendev.org/c/zuul/zuul/+/742746 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Fix gerrit merge commit change with zuul configuration https://review.opendev.org/c/zuul/zuul/+/762886 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Fix zuul-client enqueue-ref when oldrev/newrev aren't provided https://review.opendev.org/c/zuul/zuul/+/765767 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Improve merger getFilesChanges, Fix edge cases https://review.opendev.org/c/zuul/zuul/+/762887 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Tenant reconfiguration: Add tests https://review.opendev.org/c/zuul/zuul/+/742747 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Tenant reconfiguration: Reorg scheduler event process loop https://review.opendev.org/c/zuul/zuul/+/742748 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Tenant reconfiguration: Scheduler ref-updated create/delete https://review.opendev.org/c/zuul/zuul/+/739198 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Tenant reconfiguration: Allow ref-updated newrev+oldrev reconfiguration https://review.opendev.org/c/zuul/zuul/+/742749 | 12:37 |
| openstackgerrit | Guillaume Chauvel proposed zuul/zuul master: Tenant reconfiguration: Get ref-updated modified files https://review.opendev.org/c/zuul/zuul/+/739078 | 12:37 |
| *** jcapitao_lunch is now known as jcapitao | 12:58 | |
| *** hasharLunch is now known as hashar | 13:03 | |
| *** EmilienM has joined #zuul | 13:04 | |
| *** ikhan has quit IRC | 13:06 | |
| *** jpena|lunch is now known as jpena | 13:24 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Move the testing framework under zuul https://review.opendev.org/c/zuul/zuul/+/760361 | 13:34 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Move functional tests to zuul-client https://review.opendev.org/c/zuul/zuul-client/+/760363 | 13:34 |
| *** ikhan has joined #zuul | 13:40 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: bindep: remove set_fact usage when converting string to list https://review.opendev.org/c/zuul/zuul-jobs/+/771585 | 14:10 |
| openstackgerrit | Merged zuul/zuul-jobs master: Use urlencoded filenames in test fixtures https://review.opendev.org/c/zuul/zuul-jobs/+/771566 | 14:11 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Move functional tests to zuul-client https://review.opendev.org/c/zuul/zuul-client/+/760363 | 14:23 |
| mhu | would there be some interest in a demo of the web ui authentication changes? is there any way I can help with moving forward on this feature? | 14:26 |
| tobiash | mhu: that's still on my todo list for review but I unfortunately didn't come to this yet due to ops tasks | 14:27 |
| mhu | tobiash, I understand everybody's busy and it's not a trivial feature to review, so that's why I am asking if there's anything I can do to make it easier to get into it | 14:30 |
| tobiash | mhu: if it's not a high effort for you a demo would be awesome | 14:31 |
| tobiash | mhu: 768115 is the top of the stack if I want to try it in my own environment (against our central identity management)? | 14:32 |
| mhu | tobiash, yes | 14:34 |
| mhu | tobiash, does your IAM support openid connect? | 14:34 |
| tobiash | great, I'll try to test this this week in my environment to see if it fits | 14:35 |
| tobiash | mhu: we're currently setting up a new IAM based on keycloak as identity broker | 14:35 |
| mhu | tobiash, great, I was using keycloak for testing so I have some experience with it - I might be able to help if needed | 14:36 |
| tobiash | awesome, thanks! | 14:36 |
| tobiash | mhu: is this supposed to show a login button after it's configured an auth provider? | 15:04 |
| tristanC | tobiash: speaking of list for review, i forgot what is missing for zuul-v4, is there a topic or pad with the necessary changes? | 15:08 |
| *** hashar is now known as hasharAway | 15:08 | |
| mhu | tobiash, you should get a login button as you are scoped to a tenant | 15:09 |
| mhu | it's on the upper right side | 15:09 |
| *** jamesmcarthur has joined #zuul | 15:09 | |
| tobiash | mhu: I get a '-' sign where I guess there should be a login button: https://paste.pics/a99f88097ce30df9c88fa58cc43508a8 | 15:10 |
| mhu | tobiash, zuul's config should look like this https://review.opendev.org/c/zuul/zuul/+/769943/4/doc/source/examples/etc_zuul/zuul.conf | 15:13 |
| *** jamesmcarthur has quit IRC | 15:14 | |
| tobiash | it looks like that, but I have a second local provider, just removing that to see if that fixes it | 15:15 |
| openstackgerrit | Sorin Sbârnea proposed zuul/zuul-jobs master: Upgrade ansible-lint to 5.0 https://review.opendev.org/c/zuul/zuul-jobs/+/773245 | 15:15 |
| mhu | tobiash, maybe the local provider was set to default too? | 15:16 |
| tobiash | removed now the local provider and it still looks the same | 15:16 |
| mhu | have you followed https://review.opendev.org/c/zuul/zuul/+/734082/19/doc/source/howtos/openid-with-keycloak.rst ? | 15:16 |
| tobiash | oh I guess I missed restarting the scheduler | 15:19 |
| tobiash | oh there is now a sign in button :D | 15:20 |
| mhu | \o/ | 15:20 |
| *** ykarel has quit IRC | 15:22 | |
| tobiash | now the hard part, how to setup group based claims | 15:23 |
| tobiash | login itself worked out of the box after restarting the scheduler :) | 15:23 |
| mhu | tobiash, looking at the config in docker might help you. Keycloak is set up to define groups for users, and there's an admin rule on the groups claim here: https://review.opendev.org/c/zuul/zuul/+/769943/4/doc/source/examples/etc_zuul/main.yaml | 15:27 |
| mhu | well, except the groups are predefined in the realm import file, so it might not be very helpful if you're setting this up via keycloak's gui | 15:28 |
| tobiash | mhu: can the admin rules also be defined inline in the tenant object instead of referencing them by name? | 15:32 |
| tobiash | if not I'll have to patch my tenant config generation script first | 15:32 |
| mhu | tobiash, no - the idea was to make them reusable, but that could be an improvement to make | 15:33 |
| *** nils has quit IRC | 15:47 | |
| *** jamesmcarthur has joined #zuul | 15:58 | |
| *** ikhan has quit IRC | 15:59 | |
| *** jfoufas1 has quit IRC | 16:09 | |
| tobiash | mhu: yay, login button works, admin rule works, dequeue button works :) | 16:24 |
| openstackgerrit | Matthieu Huin proposed zuul/nodepool master: Move zuul integration testing to nodepool project https://review.opendev.org/c/zuul/nodepool/+/760564 | 16:26 |
| mhu | tobiash, good to hear! I hope it wasn't too hard to get it to work | 16:27 |
| tobiash | was easier than expected | 16:31 |
| *** vishalmanchanda has quit IRC | 16:32 | |
| *** nils has joined #zuul | 16:43 | |
| *** vishalmanchanda has joined #zuul | 17:05 | |
| zbr | do we have a policy for zuul-jobs that get rotten and not fixed for long time? | 17:14 |
| corvus | zbr: no; is there one that's troubling you? | 17:15 |
| zbr | https://zuul.opendev.org/t/zuul/builds?job_name=zuul-jobs-test-registry-docker-multiarch&project=zuul/zuul-jobs | 17:15 |
| zbr | that one, looking at all i think there are less than ten brokended | 17:16 |
| zbr | sadly it did not watch the weekly-pipeline closely as I wanted | 17:16 |
| corvus | that on is important for both zuul and opendev; someone is going to need to fix it | 17:17 |
| zbr | clearly some should be fixed, but not all | 17:17 |
| zbr | yep but “someone” is as good as nobody and preventing other changes from happening because one jobs is broken can cause a problematic loop. | 17:18 |
| zbr | sooner or later we endup under a big pile of broken jobs | 17:18 |
| corvus | sure. i'm just saying *that job* is absolutely critical | 17:19 |
| zbr | maybe we should look for some kind of alternative? | 17:19 |
| zbr | sure | 17:19 |
| corvus | we can either fix that job, or remove the arm builds of nodepool and the arm architecture from the opendev test platform entirely | 17:20 |
| zbr | one job to fix should not be too much, i was afraid that i may not be able to keep them all green | 17:20 |
| corvus | yep, understood | 17:20 |
| zbr | probably alternatives are temporary NV | 17:21 |
| *** rpittau is now known as rpittau|afk | 17:21 | |
| zbr | imho something that is broken for 4 weeks is good candidate to be moved to NV | 17:21 |
| corvus | i've only learned that it's broken now | 17:22 |
| zbr | yep, me only and hour before you | 17:22 |
| corvus | i think this should be raised with the opendev team and the openstack tact sig | 17:23 |
| zbr | i need to add a mailing notification on broken weekly pipeline | 17:23 |
| corvus | because this is a risk to those teams most | 17:23 |
| zbr | we can talk at tomorrow meeting | 17:23 |
| corvus | fungi: ^ fyi https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5 has bitrotted which puts arm builds at risk | 17:24 |
| *** jcapitao has quit IRC | 17:24 | |
| zbr | while is zuul-jobs the problem is generic enough for infra IMHO. | 17:24 |
| corvus | zbr: i don't understand that sentence, can you rephrase? | 17:24 |
| clarkb | looks like it is talking to the local buildset registry and failing from there? | 17:27 |
| clarkb | hrm I wonder though if it is actually hitting docker hub when "load metadata for docker.io/upstream/image:latest" happens? | 17:27 |
| clarkb | and ya not surprising that doesn't exist | 17:27 |
| fungi | fwiw, i did pull out the gentoo role test job when it was blocking folks, while gentoo representatives get that platform back on track in opendev so it can be used again | 17:29 |
| corvus | fungi: yep, and that makes sense under the circumstances. | 17:29 |
| clarkb | I think it is supposed to be finding that image entirely locally, is it possible the error is happening earlier (eg we aren't uploading that image because the build fails ro something) | 17:30 |
| fungi | i hadn't noticed the arm64 failures before today, but can take a look after my current meeting concludes | 17:30 |
| corvus | i believe that if we let this bitrot continue, we will lose arm builds and testing altogether and it will take a lot of effort to get it back. this job is at the bottom of a house of cards. that's why i think it's very important to get it fixed. | 17:30 |
| corvus | fungi: no i'm not saying there are arm64 build failures | 17:31 |
| corvus | i'm saying that this test job is failing, which means there's something wrong with our multi-arch build system, which, if that problem spreads, could take out the nodepool multi-arch build job, which would mean we wouldn't have multi-arch nodepool-builder images which could take out our ability to support arm images in opendev. | 17:31 |
| tobiash | mhu: what I noticed is that on page reload after login the login button returns | 17:32 |
| corvus | i'm saying that unlike most other jobs, this one failing is exceptionally high risk and fixing it should be prioritized | 17:32 |
| corvus | (if arm testing is valued) | 17:32 |
| fungi | ohhh, this is the thing where we have arm64 emulated on amd64 via qemu or something? | 17:33 |
| corvus | yes | 17:35 |
| corvus | (though i doubt that's the part that broke) | 17:35 |
| clarkb | it appears to be somethign to do with registries | 17:35 |
| corvus | yes, there could be some behavior change wrt registries in docker buildx | 17:36 |
| *** jamesmcarthur has quit IRC | 17:37 | |
| clarkb | https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5/log/builder/docker/buildset_registry.txt#154-162 | 17:38 |
| clarkb | I think that lines up with the log at: https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5/log/job-output.txt#1640-1670 | 17:38 |
| clarkb | which I think says it isn't talking to upstream docker hub (good) | 17:39 |
| clarkb | hrm though the error happens almost a half a second after the buildset registries last log | 17:40 |
| clarkb | so maybe buildx is being given a 404 by the local stuff then looking at the intermediate or up to docker hub? | 17:41 |
| clarkb | (I don't fully understand what the desired lookup pattern is there) | 17:41 |
| clarkb | ok ya looking at the earlier portion of that log the buildx image is also looked up in the buildset registry and is not there. So it must be falling back to docker hub | 17:44 |
| clarkb | That makes me wonder if we aren't loading upstream:latest into the buildset registry properly | 17:44 |
| clarkb | corvus: "HEAD /v2/upstream/image/manifests/latest?ns=docker.io HTTP/1.1" <- do you know what the ?ns=docker.io parameter means? | 17:47 |
| *** jamesmcarthur has joined #zuul | 17:47 | |
| fungi | namespace? | 17:48 |
| corvus | clarkb: that does not immediately ring a bell | 17:48 |
| clarkb | earlier in the buildset registry there is a "HEAD /v2/upstream/image/manifests/latest HTTP/1.1" and then a "GET /v2/upstream/image/manifests/latest HTTP/1.1" both successful and it seems to be able to fetch the image at that point | 17:49 |
| clarkb | is it possible that this is some new parameter sent by buildx and the buildset registry doesn't know how to handle it resulting in a 404? | 17:49 |
| corvus | clarkb: i can't find any handling of an "ns" parameter in zuul-registry; perhaps that's the new behavior | 17:49 |
| corvus | yeah | 17:49 |
| clarkb | ya that is my hunch at this point. we see the iamge is fetchable from the buildset registry about a minute prior to the error. The difference in requests appears to be that parameter | 17:50 |
| clarkb | I would've expected cherrypy to ignore the parameter, but maybe if your python signatures don't match the params list it is a problem? or perhaps even more oddly the /ns=... could be considered part of the image name? | 17:51 |
| corvus | clarkb: https://zuul.opendev.org/api/info?ns=1 that returns 404 (just as an example of cherrypy behavior when given an unexpected arg) | 17:51 |
| clarkb | https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5/log/builder/docker/buildset_registry.txt#95-112 <- is the happy log portion fwiw | 17:52 |
| clarkb | corvus: oh cool so that lines up pretty well | 17:52 |
| mhu | tobiash, the login info probably needs to be serialized to localstorage to be kept over reloads or opening new tabs | 17:59 |
| mhu | I didn't get into that yet | 17:59 |
| tobiash | mhu: apart from that it looks great | 18:00 |
| tobiash | mhu: I'll try to do a review tomorrow then | 18:01 |
| clarkb | corvus: it looks like cherrypy infers GET parameters from function argument lists. I'm not quite sure reading the docs if/how that interacts with our routes dispatching | 18:03 |
| *** jpena is now known as jpena|off | 18:04 | |
| clarkb | we can apparently use **kwargs then ignore them | 18:05 |
| clarkb | (I haven' tested that though, just foudn a random blog referring to it) | 18:05 |
| corvus | clarkb: yeah, or add a ns param | 18:05 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-registry master: Handle ns parameter on manifest GET/HEAD https://review.opendev.org/c/zuul/zuul-registry/+/773461 | 18:09 |
| clarkb | totally untested and not sure if we need ti on the blob gets too | 18:09 |
| clarkb | but it may be that simple? (I looked for tests too but didn't see any for the http layer) | 18:09 |
| *** Eighth_Doctor has quit IRC | 18:10 | |
| openstackgerrit | Matthieu Huin proposed zuul/nodepool master: Move zuul integration testing to nodepool project https://review.opendev.org/c/zuul/nodepool/+/760564 | 18:11 |
| clarkb | corvus: it also looks like we figure out namespace from the repository name. Is that something that we might be able to short circuit if ?ns is set? | 18:12 |
| clarkb | hrm no, not in this case because we look in _local by default but that would override to looking at docker.io | 18:12 |
| corvus | clarkb: namespace in zuul-registry is a vestigal concept; | 18:13 |
| corvus | everything is stored in _local as you noted | 18:13 |
| clarkb | ok, in that case I think ignoring the ns is correct | 18:13 |
| corvus | clarkb: what we ended up doing for the multi-namespace jobs was to just configure registries.conf or whatever to add the namespace to the start of the repo name. zuul-registry doesn't need to understand anything about it in that case. | 18:14 |
| corvus | so we effectively run zuul-registry in "single namespace mode" regardless of whether we are using it to hold one or more than one namespace. and in the case that we're using it for more than one, all the namespaces are *externally* mapped to repo prefixes | 18:15 |
| corvus | (generally speaking, by podman) | 18:15 |
| clarkb | corvus: also would you expect a change that runs the multiarch builds to depends on https://review.opendev.org/c/zuul/zuul-registry/+/773461 and exercise the new registry? | 18:16 |
| clarkb | looks like we do pull the zuul registry out of the intermediate registry? so maybe that would work | 18:17 |
| corvus | clarkb: yes, that's intended to work | 18:18 |
| corvus | (inception feature is implemented :) | 18:19 |
| clarkb | ya I'm working on a change to do that now | 18:19 |
| clarkb | just need to figure out which file to edit to trigger the appropraite job | 18:19 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: DNM: Testing zuul-registry changes https://review.opendev.org/c/zuul/zuul-jobs/+/773463 | 18:21 |
| clarkb | that should give us an idea | 18:21 |
| clarkb | zbr: ^ fyi | 18:21 |
| clarkb | if that fails on blob gets with the same error then we'll know to add an ns parameter to those methods too | 18:22 |
| clarkb | I don't expect the blobs to need namespacing because they are sha256 addressed and those should be unique enough to not bother | 18:23 |
| clarkb | you only need it at the human readable side of things which is the manifest lookup | 18:23 |
| *** Eighth_Doctor has joined #zuul | 18:25 | |
| clarkb | corvus: https://zuul.opendev.org/t/zuul/build/742ca02d17184b68bf27529c06303a4a/log/docker/functionaltest_registry_1.txt my registry update seems to have failed due to the traceback there | 18:40 |
| *** jfoufas1 has joined #zuul | 18:41 | |
| clarkb | hrm did jwt also change their api | 18:41 |
| corvus | clarkb: yeah, i think so | 18:41 |
| clarkb | ya they just made a 2.0 release | 18:42 |
| clarkb | I'm trying to find a changelog | 18:42 |
| clarkb | but I think we may just be able to drop the decode() | 18:42 |
| corvus | clarkb: see https://review.opendev.org/768312 | 18:42 |
| clarkb | https://pyjwt.readthedocs.io/en/stable/changelog.html#jwt-encode-return-type | 18:42 |
| corvus | clarkb: and yes that looks like the main diff | 18:43 |
| *** hamalq has joined #zuul | 18:46 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-registry master: Handle ns parameter on manifest GET/HEAD https://review.opendev.org/c/zuul/zuul-registry/+/773461 | 18:46 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-registry master: Update pyjwt version to >=2.0.0,<3.0.0 https://review.opendev.org/c/zuul/zuul-registry/+/773466 | 18:46 |
| *** nils has quit IRC | 18:49 | |
| *** jamesmcarthur has quit IRC | 18:50 | |
| *** jamesmcarthur has joined #zuul | 18:52 | |
| *** jamesmcarthur has quit IRC | 18:56 | |
| *** jamesmcarthur has joined #zuul | 18:58 | |
| *** hasharAway is now known as hashar | 19:17 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-registry master: Handle ns parameter on manifest GET/HEAD https://review.opendev.org/c/zuul/zuul-registry/+/773461 | 19:17 |
| *** jfoufas1 has quit IRC | 19:21 | |
| *** GomathiselviS has joined #zuul | 19:26 | |
| openstackgerrit | Gomathi Selvi Srinivasan proposed zuul/zuul-jobs master: Create a template for ssh-key and size https://review.opendev.org/c/zuul/zuul-jobs/+/773474 | 19:32 |
| openstackgerrit | Gomathi Selvi Srinivasan proposed zuul/zuul-jobs master: Create a template for ssh-key and size. https://review.opendev.org/c/zuul/zuul-jobs/+/773474 | 19:37 |
| openstackgerrit | Gomathi Selvi Srinivasan proposed zuul/zuul-jobs master: updated vars https://review.opendev.org/c/zuul/zuul-jobs/+/773475 | 19:40 |
| clarkb | corvus: hrm it doesn't look like the depends on worked there. We pull latest from docker hub if I'm reading the log here correctly: https://3525f1c73d59ef5d5b98-485374e596f765d9f96c9ac94e680c34.ssl.cf2.rackcdn.com/773463/1/check/zuul-jobs-test-registry-docker-multiarch/b506539/job-output.txt | 19:42 |
| clarkb | corvus: the two zuul-registry changes do pass testing now, I wonder if we should land them and see if that helps? or do we need to disect this further? | 19:42 |
| openstackgerrit | Matthieu Huin proposed zuul/nodepool master: Move zuul integration testing to nodepool project https://review.opendev.org/c/zuul/nodepool/+/760564 | 19:45 |
| clarkb | I still have a hard time understanding image id and image digest and why docker hub shows you one and docker image list shows you the other and that makes both of them largely useless outputs :/ | 19:45 |
| *** zbr5 has joined #zuul | 19:46 | |
| clarkb | luckily the image pull step seems to emit the digest and that is what docker hub shows as well and we seem to have the current release running in that job not the latest from the parent change | 19:46 |
| *** zbr has quit IRC | 19:47 | |
| *** zbr5 is now known as zbr | 19:47 | |
| clarkb | I'm reasonably confident that the registry change is an improvement and not a regression. Just can't be sure that it is sufficient to make buildx's image client completely happy | 19:48 |
| *** hashar has quit IRC | 19:52 | |
| fungi | i have a maybe slightly off-topic question about the quickstart setup: i can see in some tasks we're relying on a gerrit user named "admin" with a password of "secret" but i can't find where that account is initially created. is that a built-in account when gerrit is initialized in "dev" mode (with the become any account config option)? | 19:58 |
| corvus | fungi: yes | 19:59 |
| fungi | got it, so no need to create the account "admin" it just exists automatically? | 20:01 |
| fungi | trying to fix up the git-review functional tests to be able to work with newer gerrit, and bootstrapping an account added to the administrators group is the present challenge (post gerrit 2.11 where ssh keys could still be injected directly into its rdbms) | 20:03 |
| corvus | fungi: can you bring it up in a container like we do with q-s? | 20:05 |
| fungi | probably, but that will be a much larger testing overhaul. at the moment i just want to be able to confirm git-review runs with python 3.9 before i tag a new release | 20:06 |
| fungi | i mean, i know it runs with 3.9 because that's what i use it with, but i want to see the job list for it reflect that | 20:06 |
| fungi | and to do that (for a few reasons) we need to use a newer version of gerrit to do that | 20:06 |
| fungi | mostly because openssh on ubuntu focal won't work with the mina-sshd in gerrit 2.11 | 20:07 |
| fungi | and upgrading gerrit past 2.11 means changing how the test account is bootstrapped | 20:07 |
| *** gundalow_ has joined #zuul | 20:18 | |
| *** guilhermesp__ has joined #zuul | 20:18 | |
| *** gundalow has quit IRC | 20:26 | |
| *** guilhermesp has quit IRC | 20:26 | |
| *** guilhermesp__ is now known as guilhermesp | 20:26 | |
| *** gundalow_ is now known as gundalow | 20:26 | |
| fungi | aha, thanks! after some hunting i found they actually documented the default username/password: https://gerrit-review.googlesource.com/Documentation/dev-readme.html#localdev | 20:29 |
| fungi | but yeah, ultimately we do want to switch up how the functional testing works, it's just at the moment it's designed with the expectation a new gerrit will be started for each and every "unit" test defined and it does gymnastics to start them all on random tcp ports so they can run in parallel, not sure if we should try to keep the same model for a containerized version of the job | 20:34 |
| fungi | it tortures my workstation every time i try to run it locally | 20:35 |
| *** jamesmcarthur has quit IRC | 20:42 | |
| *** hamalq has quit IRC | 20:43 | |
| *** hamalq has joined #zuul | 20:43 | |
| clarkb | I've rechecked https://review.opendev.org/c/zuul/zuul-jobs/+/773463 thinking that maybe it will pick up the artifact for https://review.opendev.org/c/zuul/zuul-registry/+/773461 on a rerun now that all the jobs are done running from the first pass. | 20:50 |
| clarkb | if that doesn't work what do you think about just landing https://review.opendev.org/c/zuul/zuul-registry/+/773466 and https://review.opendev.org/c/zuul/zuul-registry/+/773461 and seeing how we do? That first one should be totally safe to approve anyway since it is just the jwt update | 20:51 |
| corvus | clarkb: i'd be okay with that | 20:51 |
| corvus | i imagine you'd recheck the test change again after it lands and we'd revert if it borked | 20:52 |
| clarkb | yup | 20:52 |
| *** jamesmcarthur has joined #zuul | 20:54 | |
| *** jamesmcarthur has quit IRC | 20:54 | |
| openstackgerrit | Matthieu Huin proposed zuul/nodepool master: Move zuul integration testing to nodepool project https://review.opendev.org/c/zuul/nodepool/+/760564 | 20:59 |
| clarkb | corvus: yup we're still pulling the published on docker hub version: 2021-02-01 20:53:22.609408 | builder | Digest: sha256:1c3f3aedb53ad0276ecbbdd239703fe49a74f0b09bd1c69d7aaff48be7e35dd0 | 21:00 |
| clarkb | if you have time for a quick review on those two zuul-registry changes that would be appreciated. I can recheck https://review.opendev.org/c/zuul/zuul-jobs/+/773463 once published and revert if necessary | 21:01 |
| *** GomathiselviS has quit IRC | 21:36 | |
| *** vishalmanchanda has quit IRC | 21:45 | |
| openstackgerrit | Matthieu Huin proposed zuul/nodepool master: Move zuul integration testing to nodepool project https://review.opendev.org/c/zuul/nodepool/+/760564 | 21:55 |
| mhu | corvus, I've addressed your comments on the doc in https://review.opendev.org/c/zuul/zuul-client/+/767592 - if that's fine with you let's +3 it and get the next tag of zuul-client out? | 21:58 |
| corvus | mhu: yep, i saw your msg from earlier, but fires happened; it's on the list | 21:58 |
| mhu | no worries, take your time! | 21:59 |
| openstackgerrit | Matthieu Huin proposed zuul/nodepool master: Move zuul integration testing to nodepool project https://review.opendev.org/c/zuul/nodepool/+/760564 | 22:09 |
| *** jamesmcarthur has joined #zuul | 22:18 | |
| *** jamesmcarthur has quit IRC | 22:36 | |
| openstackgerrit | Merged zuul/zuul-registry master: Update pyjwt version to >=2.0.0,<3.0.0 https://review.opendev.org/c/zuul/zuul-registry/+/773466 | 22:51 |
| ianw | clarkb: is the "namespace" parameter like "/v2" in URL? sorry, trying to bootstrap myself on what's going on | 23:15 |
| clarkb | ianw: no its ?ns=docker.io let me get you a link illustrating it | 23:17 |
| clarkb | ianw: https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5/log/builder/docker/buildset_registry.txt#154-162 | 23:17 |
| clarkb | ianw: https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5/log/builder/docker/buildset_registry.txt#95-112 shows an earlier set of downloads using docker proper | 23:18 |
| clarkb | none of those have ?ns=foo and they work. | 23:18 |
| clarkb | ianw: also 17:51:48 corvus | clarkb: https://zuul.opendev.org/api/info?ns=1 that returns 404 (just as an example of cherrypy behavior when given an unexpected arg) | 23:18 |
| corvus | it's unclear to me what docker intends to use that for, but i think it would be interesting to learn | 23:20 |
| ianw | "ns=docker.io" has like two google hits ... you'd think if it was in a lot of urls it would be more common | 23:21 |
| ianw | so yeah, what exactly it is, is unclear at present | 23:22 |
| fungi | seems like it appeared *very* recently | 23:22 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Required SQL reporters https://review.opendev.org/c/zuul/zuul/+/630472 | 23:24 |
| *** piotrowskim has quit IRC | 23:29 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: Format multi-line log entries https://review.opendev.org/c/zuul/nodepool/+/773540 | 23:30 |
| ianw | i believe this is our ns= parameter ... https://github.com/containerd/containerd/commit/adeba792f16bff6d8f68d560566ed50202634bac | 23:39 |
| clarkb | ianw: that seems to imply it should be handled on the get blobs handler too | 23:39 |
| clarkb | whcih is really weird to me since blobs are sha256sum addressed. But maybe they just didn't think that through | 23:40 |
| corvus | well, depending on how deep down the content-addressable-storage rabbithole they are | 23:40 |
| ianw | it only adds it if refhost != "docker.io" || h.Host != "registry-1.docker.io" { | 23:40 |
| clarkb | "this query parameter can safely be ignored if multiple upstreams are not supported" | 23:40 |
| corvus | of course it does :) | 23:41 |
| clarkb | we've constructed a corner case catcher | 23:41 |
| corvus | possible podman doesn't obey that and always sends it? | 23:41 |
| clarkb | I think we would've caught it earlier if podman was sending it? | 23:41 |
| clarkb | this is buildx doing it if I read things correctly (not podman) | 23:42 |
| corvus | good point | 23:42 |
| corvus | wonder why it's sent then | 23:42 |
| clarkb | beacuse our buildset registry's refhost is localhost:9000 or whatever the port is I bet | 23:42 |
| corvus | oh that's what refhost is. got it. | 23:42 |
| ianw | the UA in https://zuul.opendev.org/t/zuul/build/7be904b221ab402fab74ded9de20dae5/log/builder/docker/buildset_registry.txt#158 is for containerd | 23:42 |
| clarkb | thats my hunch anyway | 23:42 |
| ianw | docker is more ... docker/20.10.2 go/go1.13.15 git-commit/8891c58 kernel/4.15.0-135-generic | 23:43 |
| clarkb | should I add the ns=None parameter to head_blob and get_blob in zuul-registry too then? | 23:43 |
| corvus | i'd be inclined to add it only if we're sure it's being sent.... | 23:45 |
| clarkb | corvus: the diff ianw linked indicated it does it for GET on manifests and GET on blobs. We've seen it do it for HEAD against manifests as well | 23:46 |
| clarkb | but ya we could do a follow on change too if we prefer | 23:46 |
| ianw | yeah, req.addNamespace is called for "manifests" and "blobs" | 23:46 |
| openstackgerrit | Merged zuul/zuul-registry master: Handle ns parameter on manifest GET/HEAD https://review.opendev.org/c/zuul/zuul-registry/+/773461 | 23:49 |
| corvus | clarkb: ^ maybe recheck and see since that landed? | 23:50 |
| openstackgerrit | Merged zuul/zuul-client master: Reorganize the installation documentation, specify versions https://review.opendev.org/c/zuul/zuul-client/+/767592 | 23:51 |
| clarkb | corvus: ++ | 23:51 |
| *** tosky has quit IRC | 23:52 | |
| clarkb | recheck has been posted | 23:53 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!