| pabelanger | I'll dig more into it tomorrow for ipv6 | 00:00 |
|---|---|---|
| *** holser has quit IRC | 00:05 | |
| *** rfolco has joined #zuul | 00:21 | |
| *** jhesketh has joined #zuul | 00:49 | |
| ianw | corvus: do you think /var/log/zuul is still worth backing up in it's entirety on zuul.opendev.org? | 00:58 |
| corvus | ianw: (probably an #opendev q) no; the only reason would be for extra paranoia in case of compromise (extra auditable logs) | 01:00 |
| corvus | /var/lib/zuul is important | 01:00 |
| corvus | keys etc | 01:00 |
| ianw | yeah it is of course a trade-off. it's working set is about 20gb in that directory | 01:01 |
| pabelanger | corvus: do you remember what python-passlib is for in the run-buildset-registry role? | 01:02 |
| pabelanger | I'm trying to add centos support for the role | 01:03 |
| *** rfolco has quit IRC | 01:03 | |
| corvus | pabelanger: i think setting the password (it's a bcrypt hash iirc) | 01:04 |
| corvus | pabelanger: i think lookup('password', '/dev/null') uses it | 01:05 |
| corvus | wait, hrm, no that'd be on the executor | 01:05 |
| pabelanger | I wonder if 3f01dc11f70a9475bd09c1a00477506c64e66c9a might have been why | 01:06 |
| corvus | pabelanger: it may be obsolete | 01:07 |
| pabelanger | ack, lets see what testing says | 01:08 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: WIP: Drop python-passlib from run-buildset-registry https://review.opendev.org/762444 | 01:08 |
| corvus | pabelanger: yeah, if you look at 2292ce9ae which added passlib, there's an htpasswd module use with bcrypt. but that's gone now. so i'd guess passlib isn't necessary. | 01:09 |
| * corvus dinner | 01:09 | |
| pabelanger | ++ | 01:09 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: WIP: Drop python-passlib from run-buildset-registry https://review.opendev.org/762444 | 01:12 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Update run-buildset-registry for readability https://review.opendev.org/762447 | 01:12 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Add zuul-jobs-test-registry-podman-centos-8 job https://review.opendev.org/762448 | 01:17 |
| *** ikhan has joined #zuul | 01:23 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Install podman in container for container-based builds https://review.opendev.org/756643 | 01:44 |
| *** Goneri has quit IRC | 02:15 | |
| pabelanger | okay, moving back to build-docker-image, and using quay.io in context, and podman on test-job side works too | 02:21 |
| pabelanger | next up is to stand up intermediate registry to finish off the new jobs | 02:22 |
| *** ikhan has quit IRC | 02:59 | |
| *** bhavikdbavishi has joined #zuul | 03:02 | |
| *** bhavikdbavishi1 has joined #zuul | 03:04 | |
| *** bhavikdbavishi has quit IRC | 03:06 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:06 | |
| *** hamalq has quit IRC | 03:07 | |
| *** rlandy has quit IRC | 03:10 | |
| *** bhavikdbavishi has quit IRC | 04:09 | |
| *** bhavikdbavishi1 has joined #zuul | 04:09 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 04:11 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** reiterative has quit IRC | 05:42 | |
| *** reiterative has joined #zuul | 05:42 | |
| *** vishalmanchanda has joined #zuul | 05:48 | |
| *** cloudnull has quit IRC | 06:01 | |
| *** cloudnull has joined #zuul | 06:02 | |
| *** bhavikdbavishi1 has joined #zuul | 06:26 | |
| *** jfoufas1 has joined #zuul | 06:27 | |
| *** bhavikdbavishi has quit IRC | 06:27 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 06:27 | |
| *** bhavikdbavishi has quit IRC | 07:48 | |
| *** bhavikdbavishi1 has joined #zuul | 07:48 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 07:50 | |
| *** bhavikdbavishi has quit IRC | 08:07 | |
| *** hashar has joined #zuul | 08:08 | |
| *** jcapitao has joined #zuul | 08:09 | |
| *** jpena|off is now known as jpena | 08:09 | |
| openstackgerrit | Daniel Pawlik proposed zuul/nodepool master: Changed informations provided in openshiftpods driver https://review.opendev.org/756671 | 08:16 |
| *** bhavikdbavishi has joined #zuul | 08:18 | |
| *** rpittau|afk is now known as rpittau | 08:19 | |
| *** bhavikdbavishi1 has joined #zuul | 08:21 | |
| *** bhavikdbavishi has quit IRC | 08:23 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 08:23 | |
| avass | corvus: sure, I don't have a strong opinion as long as you don't have to scroll down the entire page to reach them | 08:31 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Fix urllib3 dependency https://review.opendev.org/762473 | 08:36 |
| openstackgerrit | Merged zuul/zuul-jobs master: Remove unecessary files attributes from child jobs https://review.opendev.org/761802 | 09:06 |
| *** yolanda has quit IRC | 09:29 | |
| *** yolanda has joined #zuul | 09:30 | |
| *** bhavikdbavishi1 has joined #zuul | 09:32 | |
| *** bhavikdbavishi has quit IRC | 09:33 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 09:33 | |
| *** sean-k-mooney1 has joined #zuul | 09:42 | |
| *** bhavikdbavishi1 has joined #zuul | 09:42 | |
| *** sean-k-mooney has quit IRC | 09:43 | |
| *** bhavikdbavishi has quit IRC | 09:44 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 09:44 | |
| *** bhavikdbavishi has quit IRC | 10:10 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues https://review.opendev.org/761170 | 10:17 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher https://review.opendev.org/761171 | 10:17 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues https://review.opendev.org/761172 | 10:17 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed management event queues https://review.opendev.org/761738 | 10:17 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Generalize tenant event queue registry https://review.opendev.org/762491 | 10:17 |
| avass | I'm gonna hit you with a niche use-case again. I think we might have repositories on the same gerrit instances but with different labels that should be set for the gating process to start | 10:19 |
| avass | I can't see a way zuul handles this and the best fix would be to make sure they use the same config. but the only way to support that if it's needed is to add the same instance as two different connections to handle different repositories | 10:21 |
| tobiash | avass: zuul should automatically know which labels are required | 10:23 |
| tobiash | just add all labels to the triggers | 10:24 |
| tobiash | avass: or do you mean the gate needs to report different labels? | 10:25 |
| tobiash | avass: if you can keep at least the label that is reported by the gate pipeline constant it should automagically work | 10:26 |
| tobiash | if not you probably need different gate pipelines | 10:26 |
| *** zenkuro has joined #zuul | 10:50 | |
| avass | tobiash: right we could use different pipelines as well | 11:08 |
| *** wuchunyang has joined #zuul | 11:08 | |
| avass | it's different 'code-review' | 11:09 |
| tobiash | if the gate pipeline reports the same for all projects it should just work with a single pipeline | 11:09 |
| tobiash | regardless of different required labels | 11:10 |
| avass | yeah but it's the trigger/requirements config that is different | 11:10 |
| avass | so far it looks like all that was need to be done was change the name of a label | 11:11 |
| tobiash | avass: you don't need different requirement configs | 11:11 |
| tobiash | zuul auto-detects them from the project config | 11:11 |
| tobiash | also for the trigger, just add all of them | 11:12 |
| tobiash | the magic happens here: https://opendev.org/zuul/zuul/src/branch/master/zuul/driver/gerrit/gerritconnection.py#L962 | 11:12 |
| avass | so if there's a label say called 'Maintainer-Review' and that is blocking zuul will autodetect that? | 11:12 |
| avass | if so that's great and can be handled completely in gerrit | 11:13 |
| tobiash | the change knows which labels are missing, so the gate doesn't need to require them explicitly | 11:13 |
| avass | great | 11:13 |
| *** bhavikdbavishi has joined #zuul | 11:13 | |
| avass | tobiash: thanks! :) | 11:15 |
| *** bhavikdbavishi1 has joined #zuul | 11:16 | |
| avass | actually remember debugging that part when we encountered the 'verified' vs 'Verified' error a long time ago | 11:16 |
| *** bhavikdbavishi has quit IRC | 11:17 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 11:17 | |
| *** zenkuro has quit IRC | 11:27 | |
| *** zenkuro has joined #zuul | 11:28 | |
| *** sean-k-mooney1 is now known as sean-k-mooney | 11:36 | |
| openstackgerrit | Merged zuul/nodepool master: Fix fake image build under Mac OS https://review.opendev.org/760543 | 11:36 |
| *** rfolco has joined #zuul | 11:39 | |
| *** jcapitao is now known as jcapitao_lunch | 11:40 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler https://review.opendev.org/717269 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Mandatory Zookeeper connection for ZuulWeb in tests https://review.opendev.org/721254 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Merge Zookeeper connection methods https://review.opendev.org/754360 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Connect merger to Zookeeper https://review.opendev.org/716221 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Connect fingergw to Zookeeper https://review.opendev.org/716875 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Connect executor to Zookeeper https://review.opendev.org/716262 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Improve typings in context of 744416 https://review.opendev.org/753578 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Make Zookeeper mandatory for Scheduler https://review.opendev.org/756716 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Make ConnectionRegistry mandatory for Scheduler https://review.opendev.org/757095 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Improve typings in context of 756716 and 757095 https://review.opendev.org/757148 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Instantiate executor client, merger, nodepool and app within Scheduler https://review.opendev.org/757149 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Improve typings in context of 756304 https://review.opendev.org/757097 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Driver event ingestion https://review.opendev.org/717299 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Component Registry in ZooKeeper https://review.opendev.org/759187 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Move management and result events to model https://review.opendev.org/761163 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Allow (de-)serialization of management events https://review.opendev.org/761164 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Allow (de-)serialization of result events https://review.opendev.org/761165 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Add and fix fields in driver trigger event models https://review.opendev.org/761166 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Allow (de-)serialization of trigger events https://review.opendev.org/761167 | 11:44 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Interface to get a driver's trigger event class https://review.opendev.org/761168 | 11:44 |
| *** zenkuro has quit IRC | 11:53 | |
| *** zenkuro has joined #zuul | 11:53 | |
| *** tosky has joined #zuul | 12:18 | |
| *** rlandy has joined #zuul | 12:32 | |
| *** jpena is now known as jpena|lunch | 12:32 | |
| *** holser has joined #zuul | 12:34 | |
| *** wuchunyang has quit IRC | 12:37 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Prepare Zookeeper for scale-out scheduler https://review.opendev.org/717269 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Mandatory Zookeeper connection for ZuulWeb in tests https://review.opendev.org/721254 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Merge Zookeeper connection methods https://review.opendev.org/754360 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Connect merger to Zookeeper https://review.opendev.org/716221 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Connect fingergw to Zookeeper https://review.opendev.org/716875 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Connect executor to Zookeeper https://review.opendev.org/716262 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Improve typings in context of 744416 https://review.opendev.org/753578 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Make Zookeeper mandatory for Scheduler https://review.opendev.org/756716 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Make ConnectionRegistry mandatory for Scheduler https://review.opendev.org/757095 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Improve typings in context of 756716 and 757095 https://review.opendev.org/757148 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Instantiate executor client, merger, nodepool and app within Scheduler https://review.opendev.org/757149 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Improve typings in context of 756304 https://review.opendev.org/757097 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Driver event ingestion https://review.opendev.org/717299 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Component Registry in ZooKeeper https://review.opendev.org/759187 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Move management and result events to model https://review.opendev.org/761163 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Allow (de-)serialization of management events https://review.opendev.org/761164 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Allow (de-)serialization of result events https://review.opendev.org/761165 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Add and fix fields in driver trigger event models https://review.opendev.org/761166 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Allow (de-)serialization of trigger events https://review.opendev.org/761167 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Interface to get a driver's trigger event class https://review.opendev.org/761168 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Clear list of Zookeeper connections after tests https://review.opendev.org/761169 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implementation of Zookeeper backed event queues https://review.opendev.org/761170 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implementation of Zookeeper event watcher https://review.opendev.org/761171 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed trigger event queues https://review.opendev.org/761172 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to Zookeeper backed management event queues https://review.opendev.org/761738 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Generalize tenant event queue registry https://review.opendev.org/762491 | 12:47 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to using zookeeper instead of gearman for jobs https://review.opendev.org/762506 | 12:47 |
| openstackgerrit | Albin Vass proposed zuul/nodepool master: Fix indentation for azure driver documentation https://review.opendev.org/762514 | 12:49 |
| avass | corvus: there was a missing whitespace in the azure driver ^ | 12:50 |
| avass | oh that diff is not fun | 12:50 |
| *** jcapitao_lunch is now known as jcapitao | 12:54 | |
| *** jpena|lunch is now known as jpena | 13:29 | |
| *** Goneri has joined #zuul | 13:53 | |
| *** Phoenikzz has joined #zuul | 13:54 | |
| corvus | https://groups.google.com/g/repo-discuss/c/72ginQvhPEM/m/yvT_rSa5CQAJ | 13:55 |
| corvus | looks like an issue with the google log upload? | 13:55 |
| *** bhavikdbavishi has quit IRC | 13:57 | |
| *** nils has joined #zuul | 14:02 | |
| avass | corvus: I don't think there's been any recent changes to the roles that could cause that | 14:09 |
| swest | corvus: we are currently looking at the event processing for the scale out scheduler. the target architecture in the spec talks about having separate event queues per pipeline in order to parallelize the processing. | 14:10 |
| swest | we think that it might be better as a first step to parallelize the processing on a per-tenant level as that would require less changes to how zuul currently works and be a lot easier e.g regarding coordination (we would only need a single tenant lock). | 14:11 |
| swest | event queues (management, trigger, result) would then be partitioned by tenant and we'd have less event duplication with that solution. | 14:11 |
| swest | with pipeline event queues there might also be some unforseen issues related to inter-pipelines actions (supercedes). | 14:11 |
| swest | we could still try per-pipeline events in a later phase if we see the need. any thoughts on this? would you be ok with diverging from the spec here? | 14:12 |
| fungi | those are good points. i didn't consider event duplication where you might trigger multiple pipelines on the same event, and i think that spec predated addition of the supercedes relationships between pipelines | 14:15 |
| fungi | supercedes could probably be approached asynchronously, i don't think there's any expectation that an item is necessarily dequeued from the superceded pipeline at the same moment it's enqueued into the superceding pipeline... i considered it more of a best-effort efficiency measure to not continue to waste resources on builds which are no longer considered relevant | 14:20 |
| swest | fung: yep the case with supercedent pipelines could be solved by decoupling them w/ e.g. events. | 14:22 |
| corvus | swest: i'm concerned that if we lock at the tenant level we won't have done as much to make zuul more scalable. it addresses the HA case without addressing the scalability case. | 14:23 |
| fungi | for sure, if we only parallelize by tenant, then it doesn't help single-tenant deployments at all | 14:24 |
| corvus | swest: the supercede action could be enqueued as a high-priority management event for the other pipeline | 14:26 |
| corvus | swest: oh you and fungi just said that :) | 14:26 |
| fungi | yeah with other words | 14:26 |
| fungi | as long as we're okay with supercedes dequeuing not being atomic/synchronous then that's not hard to solve | 14:27 |
| swest | I think my main concern is the event duplication | 14:28 |
| fungi | yeah, the trivial implementation would result in n-way multiplication of the events to have a copy for every pipeline | 14:29 |
| fungi | maybe there's cheap ways to filter those? | 14:29 |
| fungi | i guess the question is whether that's really a problem needing optimization, or if, say, a 100-pipeline tenant needing 100 copies of all events in a high-churn environment will just fall over | 14:30 |
| swest | filtering might be possible by doing the eventMatches() check already in the driver. and the event also needs to be added to pipelines containing an item for that change (abandon, old version, ...) | 14:35 |
| corvus | fungi, swest: yeah, if this code just changes from "add event to pipeline" to "add event to pipeline queue" then i think we'll have duplication to a minimum: https://opendev.org/zuul/zuul/src/branch/master/zuul/scheduler.py#L1369-L1370 | 14:36 |
| corvus | swest: and yes similar with old vers (eg: if there is an old version, or it's abandoned, or it matches, add this event to the pipeline queue and let the pipeline deal with it) | 14:37 |
| fungi | one option might be to extend the queue data structure to no longer be a queue, but some sort of heap with pipeline-specific flags and garbage collection to remove events which have been handled by all pipelines | 14:37 |
| fungi | that could require less memory but increases complexity | 14:37 |
| fungi | basically virtual per-pipeline queues setting/removing flags on a common set | 14:38 |
| fungi | also may be considerably less efficient for the pipelines to process | 14:38 |
| fungi | if we're within the same process, python could do that for us i guess by using copies of the common queue with python objects for the queue entries, then once the refcount for an object in the common queue is down to 1, explicitly del it? | 14:41 |
| fungi | does that work across threads though? | 14:41 |
| fungi | i still find multi-threaded sharing of objects hard to reason about | 14:41 |
| swest | fungi: we need to store/sync the pipelines in Zookeeper | 14:41 |
| fungi | ahh, nevermind then | 14:41 |
| corvus | fungi: we need some level of persistence in the queue though in case the scheduler process fails | 14:41 |
| fungi | yep, i had forgotten about that tiny detail ;) | 14:42 |
| corvus | easiest way to do that is what swest says and keep everything in there; however, we could do some in-memory processing if we work to make it durable | 14:42 |
| corvus | but that would likely be difficult | 14:42 |
| fungi | well, also the whole point is that this can be distributed across multiple servers, so in-memory anything with regard to queue management is probably off the table | 14:43 |
| corvus | (i mean, i don't have a plan for that; it's just a theoretical possibility) | 14:43 |
| fungi | unless we do a lot of locking | 14:43 |
| swest | corvus: so I guess that's a soft no regarding per-tenant queues? :) | 14:47 |
| *** vishalmanchanda has quit IRC | 14:47 | |
| corvus | swest: yeah, i feel like we should still target per-pipeline queues. now, if the change to get there is too big, i think it's fine for per-tenant queue to be a step along the way. but at least in my mental model, i think i'd probably stick with going straight to per-pipeline. | 14:49 |
| swest | corvus: ok. I'll do some prototyping to see how this could work. I see your point with the per-tenant queues not being of any help for single tenant deployments, but e.g. in our case w/ ~35 tenants this would already be a significant speedup. my main reasoning was that the implementation would be pretty straight forward with the option to do the per-pipelines queues in later step. | 14:53 |
| corvus | swest: maybe i'm missing something you're seeing, but at least in my head, i think the current 'dispatch event to pipeline' mechanism matches pretty well to 'dispatch event to pipeline queue' so i hope it's not too big of a change. | 14:55 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Fix error in google log storage https://review.opendev.org/762530 | 14:56 |
| corvus | avass, fungi: can you speedy review that ^ ? i haven't tested it against gcs; i'd like to merge it and see if that fixes gerrit's ci; if it doesn't then i'll set up my test environment for attempt #2 | 14:58 |
| avass | corvus: already did :) | 14:58 |
| swest | corvus: k, I think some protoyping will help clear things up for me. thanks! | 15:00 |
| avass | corvus: linting passed and I don't think we have any tests for upload-logs except for s3 but I have a hard time seeing how that could break anything anyway so should I go ahead and +3 that? | 15:07 |
| avass | fungi: oh you already did :) | 15:08 |
| avass | tobiash: and you | 15:08 |
| fungi | i only +2'd it because it was already approved | 15:09 |
| fungi | by the time i finished reviewing | 15:10 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Drop python-passlib from run-buildset-registry https://review.opendev.org/762444 | 15:10 |
| *** yolanda has quit IRC | 15:11 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix error in google log storage https://review.opendev.org/762530 | 15:17 |
| tristanC | looking at promote failure of https://review.opendev.org/762530 , it seems like the executor image is missing unzip and/or tar | 15:22 |
| tristanC | or it is missing another unarchive command | 15:25 |
| *** hashar has quit IRC | 15:28 | |
| *** bhavikdbavishi has joined #zuul | 15:30 | |
| *** rlandy is now known as rlandy|rover | 15:40 | |
| *** bhavikdbavishi1 has joined #zuul | 15:43 | |
| *** bhavikdbavishi has quit IRC | 15:44 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 15:44 | |
| *** hashar has joined #zuul | 15:52 | |
| *** jfoufas1 has quit IRC | 16:08 | |
| corvus | tobiash: when you re-org'd the log upload code, did you try running any of the scripts from the cli? | 16:15 |
| tobiash | corvus: hrm, actually I think I forgot that | 16:16 |
| corvus | (they're designed to be run from the cli for testing as well as imported) | 16:16 |
| corvus | tobiash: ok. it looks like that may not work anymore because of ansible_utils imports | 16:16 |
| corvus | i'll dig into that in a bit (in a meeting now) | 16:17 |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: Add tox-py39 job https://review.opendev.org/762192 | 16:36 |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: Add tox-py39 job https://review.opendev.org/762192 | 16:36 |
| *** hashar has quit IRC | 16:51 | |
| *** rlandy|rover is now known as rlandy|rover|brb | 17:03 | |
| *** jcapitao has quit IRC | 17:05 | |
| tobiash | corvus: actually this is intended to handle that case: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/upload-logs-base/library/zuul_google_storage_upload.py#L54 | 17:23 |
| tobiash | maybe there is something missing there | 17:24 |
| corvus | from ..module_utils.zuul_jobs.upload_utils import ( | 17:30 |
| corvus | ValueError: attempted relative import beyond top-level package | 17:30 |
| tobiash | yeah just seeing that | 17:31 |
| corvus | tobiash: ^ unit tests handle path manipulation for us | 17:31 |
| corvus | tobiash: i'm working around it for the moment with a 3rd import statement that does sys.path.append('..') | 17:31 |
| corvus | i don't love that but it works for now :) | 17:31 |
| *** bhavikdbavishi has quit IRC | 17:44 | |
| *** bhavikdbavishi has joined #zuul | 17:45 | |
| *** rlandy|rover|brb is now known as rlandy|rover | 17:45 | |
| avass | corvus: you don't need to do that | 17:48 |
| avass | corvus: you should be able to run it from the roles directory like: python -m upload-logs-base.library.zuul_google_storage_upload | 17:49 |
| *** jpena is now known as jpena|off | 17:49 | |
| *** rpittau is now known as rpittau|afk | 17:51 | |
| avass | upload-logs-base has an __init__.py file so it's treated as it's own module | 17:52 |
| avass | actually it works even if you remove the __init__.py | 17:56 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Support the project argument in GCS https://review.opendev.org/762560 | 17:58 |
| avass | you just need to make sure your 'top level package' is a level above what you're trying to import | 17:58 |
| corvus | avass: yep you're right | 17:59 |
| corvus | avass: i almost tried that but thought 'upload-logs-base' would barf. but it doesn't | 17:59 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Support the project argument in GCS https://review.opendev.org/762560 | 18:00 |
| corvus | avass, fungi, tobiash: ^ | 18:00 |
| corvus | i've manually tested that for real (turns out there were 2 problems) | 18:01 |
| avass | corvus: should the fix you made earlier be removed? or did that fix something? | 18:03 |
| corvus | avass: it fixed problem #1 of 2. | 18:03 |
| corvus | (problem #1 was masking problem #2) | 18:04 |
| corvus | so they should both stay | 18:04 |
| avass | alright | 18:04 |
| avass | corvus: the azure driver doc is fixed btw, the indation was slightly wrong and that made the entire document render badly | 18:22 |
| avass | https://review.opendev.org/#/c/762514/ | 18:22 |
| openstackgerrit | Merged zuul/zuul-jobs master: Support the project argument in GCS https://review.opendev.org/762560 | 18:22 |
| corvus | avass: looks great, thanks! | 18:24 |
| avass | zbr: do you have an idea how to fetch the latest version from a github release? (looking at the kind change) | 18:25 |
| zbr | there api call to return releases but i do not remember a simple url for latest | 18:26 |
| avass | zbr: I think I found a solution: https://gist.github.com/lukechilds/a83e1d7127b78fef38c2914c4ececc3c :) | 18:28 |
| zbr | long live the gists | 18:30 |
| fungi | if you want to order by version, gnu sort has a -V (version sort) option which works well | 18:30 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add option to install kubernetes with kind https://review.opendev.org/740935 | 18:37 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add option to install kubernetes with kind https://review.opendev.org/740935 | 18:39 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add option to install kubernetes with kind https://review.opendev.org/740935 | 18:39 |
| *** wuchunyang has joined #zuul | 18:40 | |
| avass | oh.. | 18:40 |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Add option to install kubernetes with kind https://review.opendev.org/740935 | 18:40 |
| avass | zbr: how about that ^? | 18:40 |
| *** wuchunyang has quit IRC | 18:44 | |
| pabelanger | corvus: I'm starting to poke into upload-container-image role, I see https://zuul-ci.org/docs/zuul-jobs/container-roles.html#rolevar-build-container-image.container_registry_credentials exists today. Were you expecting we'd loop over the registry credentials for the upload? | 18:46 |
| *** nils has quit IRC | 18:48 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Update ensure-helm to always use latest version unless a specific version is specified. https://review.opendev.org/762564 | 18:48 |
| avass | do we need an announcement when we bump which version a role installs? | 18:48 |
| avass | I'd expect the user to pin the package to a version if they don't want that to change | 18:49 |
| *** bhavikdbavishi has quit IRC | 18:54 | |
| corvus | pabelanger: i don't think i would use the word loop; i would think we would use the creds for the appropriate registry (it's a dictionary keyed by registry) | 18:54 |
| pabelanger | k, need to think about now to login / push / logout for each registry key | 18:55 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Always log out of docker registry after upload https://review.opendev.org/762566 | 19:03 |
| pabelanger | just looking at docker-image, I think ^ should help prevent other playbooks from interacting with dockerhub by mistake | 19:06 |
| avass | pabelanger: ++ | 19:06 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add with_scopes method to GCS credentials class https://review.opendev.org/762568 | 19:10 |
| corvus | avass, fungi: so sorry ^ one more fix (one of the modules i had locally was a bit out of date, so there's a 3rd problem i didn't see in my local testing; i think i have that corrected now and tested this) | 19:11 |
| avass | corvus: that makes more sense when looking at the traceback :) | 19:11 |
| corvus | avass: yeah, it's almost the same traceback as problem #2 | 19:12 |
| openstackgerrit | Merged zuul/nodepool master: Fix indentation for azure driver documentation https://review.opendev.org/762514 | 19:22 |
| pabelanger | I had a question about nodepool-upload-image and nodepool-promote-image. Why does nodepool-upload-image publish tags to dockerhub directly, and intermediate? then promote the tag on dockerhub side? Could the job not work like publish to intermediate, then on promote-docker-image, copy from intermediate to dockerhub as proper tag? The only thing I can think of, is it could be possible for intermediate | 19:25 |
| pabelanger | to delete the tag before copy / promote happened | 19:25 |
| corvus | that plus there's less to go wrong in the promote job | 19:26 |
| corvus | oh, and promote can happen on the executor since it's light-weight | 19:27 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: Add with_scopes method to GCS credentials class https://review.opendev.org/762568 | 19:31 |
| corvus | linter ^ | 19:31 |
| corvus | i'm going to self-approve that since gerrit's dead in the water; fungi, avass ^ can you retro-review | 19:32 |
| avass | ++ | 19:33 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add with_scopes method to GCS credentials class https://review.opendev.org/762568 | 19:46 |
| fungi | yep, sorry, heads-down on gerrit upgrade testing but can look in a bit | 19:52 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: WIP: Create upload-container-image role https://review.opendev.org/762581 | 21:18 |
| pabelanger | okay, first stab at upload-container-image role, now to see how to test | 21:18 |
| pabelanger | the main things to figure out, is how to loop over the different registries, if container_registry_credentials has multiple endpoints | 21:19 |
| corvus | pabelanger: i think you would loop over the images? | 21:22 |
| pabelanger | yah, trying to see if I can group them some how, to avoid login / logout muliple times | 21:22 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: WIP: Create upload-container-image role https://review.opendev.org/762581 | 21:33 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: WIP: Create upload-container-image role https://review.opendev.org/762581 | 21:35 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: WIP: Create upload-container-image role https://review.opendev.org/762581 | 21:39 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 21:41 |
| corvus | pabelanger: the zuul-jobs-test-build-docker-image-release job tests the upload-docker-image role (when it's used as a plain upload rather than as prep for a promote); you can probably make a similar test for upload-container-image | 21:48 |
| corvus | pabelanger: (it runs another local publication registry to stand in for dockerhub) | 21:49 |
| pabelanger | hmm, I don't see where upload-docker-image is tested | 21:55 |
| pabelanger | will poke around in test-playbooks | 21:55 |
| pabelanger | I see it now | 21:56 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:01 |
| openstackgerrit | Ian Wienand proposed zuul/zuul master: web: make build table details more clickable https://review.opendev.org/762588 | 22:04 |
| *** Goneri has quit IRC | 22:07 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:13 |
| pabelanger | I'm going to have to split that out into multiple PRs | 22:14 |
| pabelanger | changes* | 22:14 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:21 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:25 |
| openstackgerrit | Merged zuul/zuul-jobs master: Always log out of docker registry after upload https://review.opendev.org/762566 | 22:27 |
| *** persia has quit IRC | 22:31 | |
| *** persia has joined #zuul | 22:32 | |
| ianw | i wonder why https://zuul.opendev.org/t/zuul/build/e3aa703bb19146499e89147f02631387 that just ran picked up requests 2.24.0 instead of 2.25.0 | 22:36 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:37 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:39 |
| pabelanger | okay, need to break for day. will pick up ^ again tomorrow | 22:39 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:41 |
| *** wuchunyang has joined #zuul | 22:42 | |
| ianw | https://mirror.iad.rax.opendev.org/pypi/simple/requests/ shows 2.25.0 ; i dunno | 22:43 |
| *** tosky has quit IRC | 22:44 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Add nimble roles and job https://review.opendev.org/747865 | 22:44 |
| *** wuchunyang has quit IRC | 22:46 | |
| fungi | for some reason https://zuul.opendev.org/t/zuul/build/e3aa703bb19146499e89147f02631387/log/tox/zuul_client-1.log wants to kill my browser | 22:47 |
| fungi | oh no wonder... 88k lines?!? | 22:50 |
| fungi | aha, yarn is verbosely echoing every single file move it makes for every package it npm installs | 22:50 |
| fungi | that seems... excessive | 22:51 |
| fungi | ianw: according to that log, something is specifying a dependency on requests==2.24.0 | 22:53 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 22:53 |
| clarkb | nodepool added acap didnt it? | 22:53 |
| fungi | this isn't a cap, it's pinned to a single version somewhere | 22:54 |
| fungi | maybe in one of our dependencies | 22:54 |
| fungi | unfortunately pip isn't saying where it got the requests==2.24.0 versionspec | 22:54 |
| fungi | i don't think it's directly in anything we're maintaining, based on a quick look in codesearch | 22:55 |
| fungi | oh, nevermind, i see it even though codesearch doesn't | 23:00 |
| fungi | ianw: https://opendev.org/zuul/zuul-client/src/branch/master/requirements.txt#L2 | 23:00 |
| fungi | bingo | 23:00 |
| ianw | hrrm, why does it only affect that job | 23:01 |
| ianw | i guess that's all that installs client | 23:02 |
| fungi | because theh tox env for that job is explicitly installing zuul-client? | 23:02 |
| fungi | yeah | 23:02 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-client master: requirements: uncap requests https://review.opendev.org/762591 | 23:04 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 23:13 |
| *** Goneri has joined #zuul | 23:18 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 23:20 |
| ianw | interesting, still failed. i wonder if the test isn't installing zuul-client from source | 23:35 |
| ianw | this uses some sort of pip wrapper in the gate, which doesn't actually seem to install zuul-client from source :/ | 23:45 |
| ianw | ok, sigh, wow. so the tox role first does "Run tox without tests" before installing the siblings. So it tries to pull in zuul-client from pypi which has the bad dependency | 23:56 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Create upload-container-image role https://review.opendev.org/762581 | 23:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!