| *** tosky has quit IRC | 00:02 | |
| *** zenkuro has quit IRC | 00:24 | |
| *** zenkuro has joined #zuul | 00:25 | |
| *** jamesmcarthur has joined #zuul | 00:46 | |
| *** jamesmcarthur has quit IRC | 00:52 | |
| *** jamesmcarthur has joined #zuul | 01:19 | |
| *** jamesmcarthur has quit IRC | 01:19 | |
| *** jamesmcarthur has joined #zuul | 01:20 | |
| *** jamesmcarthur has quit IRC | 01:20 | |
| *** jamesmcarthur has joined #zuul | 01:26 | |
| *** jamesmcarthur has quit IRC | 01:57 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: mirror-workspace-git-repos: run clean as root https://review.opendev.org/c/zuul/zuul-jobs/+/771297 | 02:17 |
|---|---|---|
| *** jamesmcarthur has joined #zuul | 02:18 | |
| *** zenkuro has quit IRC | 02:20 | |
| *** jamesmcarthur has quit IRC | 02:23 | |
| *** jamesmcarthur has joined #zuul | 02:23 | |
| *** jamesmcarthur has quit IRC | 02:29 | |
| *** jamesmcarthur has joined #zuul | 02:33 | |
| *** jamesmcarthur has quit IRC | 02:34 | |
| *** jamesmcarthur has joined #zuul | 02:56 | |
| *** sanjayu_ has joined #zuul | 02:59 | |
| *** jhesketh_ has joined #zuul | 03:40 | |
| *** jhesketh has quit IRC | 03:41 | |
| *** jhesketh_ is now known as jhesketh | 03:43 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: mirror-workspace-git-repos: run clean as root https://review.opendev.org/c/zuul/zuul-jobs/+/771297 | 04:01 |
| *** ykarel has joined #zuul | 04:18 | |
| *** bhavikdbavishi has joined #zuul | 04:26 | |
| *** bhavikdbavishi has quit IRC | 04:35 | |
| *** bhavikdbavishi has joined #zuul | 04:59 | |
| *** bhavikdbavishi1 has joined #zuul | 05:02 | |
| *** bhavikdbavishi has quit IRC | 05:04 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 05:04 | |
| *** bhavikdbavishi has quit IRC | 05:15 | |
| *** bhavikdbavishi has joined #zuul | 05:15 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: mirror-workspace-git-repos: run clean as root https://review.opendev.org/c/zuul/zuul-jobs/+/771297 | 05:21 |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #zuul | 05:33 | |
| *** jfoufas1 has joined #zuul | 05:39 | |
| *** guillaumec has quit IRC | 05:40 | |
| *** guillaumec has joined #zuul | 05:44 | |
| *** bhavikdbavishi has quit IRC | 05:58 | |
| *** bhavikdbavishi has joined #zuul | 05:58 | |
| *** bhavikdbavishi has quit IRC | 06:05 | |
| *** bhavikdbavishi has joined #zuul | 06:10 | |
| *** ykarel_ has joined #zuul | 06:16 | |
| *** ykarel has quit IRC | 06:19 | |
| *** vishalmanchanda has joined #zuul | 06:20 | |
| *** bhavikdbavishi has quit IRC | 06:25 | |
| *** ykarel_ is now known as ykarel | 06:29 | |
| *** bhavikdbavishi has joined #zuul | 07:04 | |
| *** jamesmcarthur has quit IRC | 07:06 | |
| *** bhavikdbavishi has quit IRC | 07:43 | |
| *** jcapitao has joined #zuul | 07:57 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Implement ZooKeeper builds API https://review.opendev.org/c/zuul/zuul/+/770901 | 08:05 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Switch to ZooKeeper backed job execution and result events https://review.opendev.org/c/zuul/zuul/+/770902 | 08:05 |
| *** rpittau|afk is now known as rpittau | 08:07 | |
| *** bhavikdbavishi has joined #zuul | 08:13 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Use logical timestamp to detect outdated changes https://review.opendev.org/c/zuul/zuul/+/763755 | 08:21 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Make buildset mandatory on build https://review.opendev.org/c/zuul/zuul/+/770900 | 08:21 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Implement ZooKeeper builds API https://review.opendev.org/c/zuul/zuul/+/770901 | 08:21 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Switch to ZooKeeper backed job execution and result events https://review.opendev.org/c/zuul/zuul/+/770902 | 08:21 |
| *** hashar has joined #zuul | 08:22 | |
| *** tosky has joined #zuul | 08:39 | |
| *** jpena|off is now known as jpena | 08:58 | |
| *** jamesmcarthur has joined #zuul | 09:06 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: zuul-client: test create-auth-token subcommand https://review.opendev.org/c/zuul/zuul/+/771226 | 09:07 |
| *** jamesmcarthur has quit IRC | 09:11 | |
| *** imtiazc has quit IRC | 09:16 | |
| *** harrymichal has joined #zuul | 09:30 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: zuul-client: test create-auth-token subcommand https://review.opendev.org/c/zuul/zuul/+/771226 | 09:35 |
| *** nils has joined #zuul | 09:36 | |
| *** hashar is now known as hasharOut | 09:59 | |
| *** harrymichal has quit IRC | 10:03 | |
| *** harrymichal has joined #zuul | 10:50 | |
| *** Phoenikzz has joined #zuul | 10:55 | |
| *** bhavikdbavishi has quit IRC | 11:02 | |
| *** jcapitao is now known as jcapitao_lunch | 11:26 | |
| *** hanson has joined #zuul | 11:38 | |
| *** hanson has quit IRC | 11:38 | |
| *** zenkuro has joined #zuul | 11:44 | |
| *** dpawlik has quit IRC | 11:57 | |
| *** logan- has quit IRC | 11:57 | |
| *** smyers has quit IRC | 11:57 | |
| *** avass has quit IRC | 11:57 | |
| *** paladox has quit IRC | 11:57 | |
| *** tflink has quit IRC | 11:57 | |
| *** paladox has joined #zuul | 11:57 | |
| *** avass has joined #zuul | 11:58 | |
| *** bhavikdbavishi has joined #zuul | 11:58 | |
| *** smyers_ has joined #zuul | 11:59 | |
| *** smyers_ is now known as smyers | 11:59 | |
| *** tflink has joined #zuul | 11:59 | |
| openstackgerrit | Gorka Eguileor proposed zuul/zuul-jobs master: Fix CentOS wheel mirror URL https://review.opendev.org/c/zuul/zuul-jobs/+/771428 | 12:02 |
| *** logan- has joined #zuul | 12:02 | |
| *** bhavikdbavishi1 has joined #zuul | 12:02 | |
| *** bhavikdbavishi has quit IRC | 12:03 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 12:03 | |
| *** dpawlik has joined #zuul | 12:15 | |
| *** ajitha_ has joined #zuul | 12:23 | |
| *** jpena is now known as jpena|lunch | 12:34 | |
| *** ajitha_ is now known as ajitha | 12:34 | |
| *** jcapitao_lunch is now known as jcapitao | 12:44 | |
| *** hasharOut is now known as hashar | 12:45 | |
| *** ttx has quit IRC | 12:51 | |
| *** rlandy has joined #zuul | 12:51 | |
| *** bhavikdbavishi has quit IRC | 12:58 | |
| *** ykarel has quit IRC | 13:01 | |
| *** ttx has joined #zuul | 13:04 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Make buildset mandatory on build https://review.opendev.org/c/zuul/zuul/+/770900 | 13:10 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Implement ZooKeeper builds API https://review.opendev.org/c/zuul/zuul/+/770901 | 13:10 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Switch to ZooKeeper backed job execution and result events https://review.opendev.org/c/zuul/zuul/+/770902 | 13:10 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Merge ZooKeeper connection and client classes https://review.opendev.org/c/zuul/zuul/+/771442 | 13:10 |
| openstackgerrit | Gorka Eguileor proposed zuul/zuul-jobs master: Fix CentOS wheel mirror URL https://review.opendev.org/c/zuul/zuul-jobs/+/771428 | 13:13 |
| *** ykarel has joined #zuul | 13:24 | |
| *** jamesmcarthur has joined #zuul | 13:26 | |
| *** jpena|lunch is now known as jpena | 13:34 | |
| *** _erlon_ has joined #zuul | 13:49 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Refactor pipeline processing in run handler https://review.opendev.org/c/zuul/zuul/+/771452 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Dequeue superceded items via management event https://review.opendev.org/c/zuul/zuul/+/771453 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Text stream API for sharded Zookeeper data https://review.opendev.org/c/zuul/zuul/+/771454 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Cache unparsed config files in Zookeeper https://review.opendev.org/c/zuul/zuul/+/771455 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Store tenants in unparsed abide as dict https://review.opendev.org/c/zuul/zuul/+/771456 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Refactor config/tenant (re-)loading https://review.opendev.org/c/zuul/zuul/+/771457 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Tenant read/write lock in Zookeeper https://review.opendev.org/c/zuul/zuul/+/771458 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Lock pipelines during processing https://review.opendev.org/c/zuul/zuul/+/771459 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Lock global event queues during processing https://review.opendev.org/c/zuul/zuul/+/771460 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Store tenant layout state in Zookeeper https://review.opendev.org/c/zuul/zuul/+/771461 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Configure unique command socket path per scheduler https://review.opendev.org/c/zuul/zuul/+/771462 | 13:52 |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Support cross scheduler config loading https://review.opendev.org/c/zuul/zuul/+/771463 | 13:52 |
| avass | zuul-maint: It would be good if we could get more reviews on: https://review.opendev.org/c/zuul/zuul/+/766748 to get support for gerrit 3.3.0 | 14:01 |
| *** sanjayu_ has quit IRC | 14:05 | |
| avass | tristanC: thanks! | 14:10 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: encrypt: add --public-key argument https://review.opendev.org/c/zuul/zuul-client/+/765313 | 14:12 |
| tristanC | avass: you are welcome, thank you for the reminder | 14:12 |
| zbr | tristanC: maybe you can also have a look at https://review.opendev.org/c/opendev/gerritbot/+/765130 ? | 14:14 |
| zbr | thanks | 14:19 |
| openstackgerrit | James Bailey proposed zuul/nodepool master: add ansible from 2018 https://review.opendev.org/c/zuul/nodepool/+/771470 | 14:56 |
| guillaumec | avass, tristanC: thanks, gerrit currently has an intermediate fix which makes patchset level comments as "old-style" comment by default https://bugs.chromium.org/p/gerrit/issues/detail?id=13800#c12 the complete gerrit fix from Luca https://gerrit-review.googlesource.com/c/gerrit/+/291225 is not yet merged | 14:59 |
| openstackgerrit | Merged zuul/zuul master: Gerrit: add support for patchset level comments https://review.opendev.org/c/zuul/zuul/+/766748 | 15:10 |
| avass | guillaumec: I think my brain is kaputt for today. Does that mean ^ fix won't be enough for it to work? | 15:12 |
| guillaumec | avass, it means that merging https://review.opendev.org/c/zuul/zuul/+/766748 was not mandatory for gerrit 3.3, as stated by Luca : "@DavidO's solution is still very good for backward compatibility with legacy systems and for managing the migration. I would expect to be in place for Gerrit v3.3 and being dropped in the next v3.4 version." | 15:13 |
| *** hashar is now known as hasharKids | 15:28 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 15:37 | |
| avass | guillaumec: ok got it | 15:41 |
| *** ikhan has quit IRC | 15:48 | |
| *** ykarel is now known as ykarel|away | 16:00 | |
| *** ikhan has joined #zuul | 16:03 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 16:03 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add builds subcommand https://review.opendev.org/c/zuul/zuul-client/+/750999 | 16:04 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add build-info subcommand https://review.opendev.org/c/zuul/zuul-client/+/751070 | 16:04 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add buildsets, buildset-info to subcommands https://review.opendev.org/c/zuul/zuul-client/+/752909 | 16:04 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add console-stream subcommand https://review.opendev.org/c/zuul/zuul-client/+/751238 | 16:04 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add object models and support for formatters, document API https://review.opendev.org/c/zuul/zuul-client/+/755292 | 16:04 |
| *** bhavikdbavishi has joined #zuul | 16:09 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: encrypt: add --public-key argument https://review.opendev.org/c/zuul/zuul-client/+/765313 | 16:11 |
| clarkb | tobiash: tristanC do we have any thoughts on working around what https://review.opendev.org/c/zuul/zuul-jobs/+/771297 is trying to address? Opendev's CD stuff is currently dead in the water beacuse of that issue | 16:15 |
| clarkb | it does seem like maybe the CI side of the house and CD side of the house are at odds with each other here | 16:15 |
| fungi | short of reverting tobiash's https://review.opendev.org/771220 fix, maybe we can just ignore the exit code from the git clean? | 16:16 |
| *** ykarel|away has quit IRC | 16:17 | |
| fungi | if zuul can't remove some untracked files from the worktree, that's probably fine. if new tracked files get added to the repo with the same names, then the checkout will still exit nonzero and cause the task to fail | 16:17 |
| tristanC | clarkb: could your CD job clean-up the workspace after it runs? | 16:18 |
| clarkb | tristanC: yes, but that wouldn't be reliable in the case of timeouts I think | 16:18 |
| avass | couldn't 'git reset' technically get stuck with the same issue? | 16:18 |
| clarkb | avass: only if the git managed files have the wrong perms | 16:18 |
| fungi | avass: if git reset causes a conflict with untracked files then it's probably fine that it fails | 16:19 |
| fungi | short of somebody committing bytecode files into our repo i don't expect us to hit that condition | 16:19 |
| clarkb | tristanC: we could use a cleanup playbook probably, but even that would be at risk of zuul restarts | 16:19 |
| avass | I guess that's true | 16:20 |
| tristanC | clarkb: why is the CD job running ansible as root? | 16:20 |
| clarkb | tristanC: it isn't it is running other utilities | 16:21 |
| avass | clarkb: couldn't it still use a cleanup-run ? | 16:21 |
| clarkb | though maybe ansible too? but ansible isn't what is polluting things | 16:21 |
| clarkb | avass: yes, but then if we restart zuul those won't fire and we'll have to manualyl cleanup | 16:21 |
| avass | uh oh, got it. missed that :9 | 16:22 |
| clarkb | tristanC: fwiw I am only just now catching up on this issue and may not understand all the intricate details. I just know that we cannot run CD jobs currently because zuul fails almost immediately on a git clena due to permissions issues | 16:22 |
| clarkb | I do think the new behavior is correct for CI | 16:22 |
| clarkb | but it creates this interesting conflict with cd running out of long lived repos | 16:23 |
| tristanC | clarkb: well i guess we could revert tobiash change, but i don't think we should requires sudo privilege for prepare-workspace, that would break job running without sudo | 16:24 |
| clarkb | tristanC: yes I agree on that, which is why I'm asking if anyone has better ideas :) | 16:24 |
| clarkb | fungi's suggestion seems like maybe ar easonable halfway step | 16:24 |
| fungi | yes, i've suggested instead of using root to do the git clean, just ignoring the exit code for git clean | 16:24 |
| clarkb | maybe we can go with fungi's idea in the short term and try to work up something better long term? | 16:24 |
| tristanC | how about we test if sudo is working, and use become if it can? | 16:24 |
| clarkb | tristanC: if it can't you still have the same problem (maybe not for us though, I'm not sure) | 16:25 |
| clarkb | the context of a job setting up the workspace with its ability to sudo doesn't necesarily accurate reflect some other processes ability to do so and pollute that workspace | 16:25 |
| fungi | like i said, the typical cases you want git clean to save you from will cause failures elsewhere in the role if they're really problematic and the git clean didn't work | 16:26 |
| avass | maybe the workspace should be separate from where zuul pushes the repos to the node | 16:28 |
| fungi | yes, i can think of half a dozen ways we can completely rework our continuous deployment to work around the problem, but for now we're unable to update systems because of a regression in zuul-jobs | 16:30 |
| *** jfoufas1 has quit IRC | 16:30 | |
| fungi | and trying to come to an amenable compromise rather than reverting | 16:30 |
| tristanC | fungi: what if the undeletable files are not in the index, is it going to cause failure elsewhere? | 16:31 |
| fungi | tristanC: if it does, that's what we want, right? | 16:32 |
| tristanC | which was one of the previous issue, a PR added test file, then follow-up PR would use the left-over test file without the clean | 16:32 |
| clarkb | ya I think the concern is that we may improperly succeed | 16:32 |
| fungi | so say there's a file created by root in the workspace which conflicts with a tracked file, and the git clean doesn't cause the role to fail because the exit code is ignored, the git reset or checkout will still cause a failure, which ought to be good enough | 16:32 |
| clarkb | also I believe running the clean will clean up everything it can | 16:33 |
| clarkb | it will then fail if it didn't clean up everything it wanted to | 16:33 |
| clarkb | (which narrows the scope of concern a bit more) | 16:33 |
| fungi | the previous problem which 771220 solves is that we were ignoring all the git actions in the role. i feel like it's safe to only ignore the result of the git clean but continue to error on the rest | 16:35 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: zuul-client: Test buildsets, buildset-info subcommands https://review.opendev.org/c/zuul/zuul/+/758985 | 16:36 |
| avass | probably good to ignore_error the git clean so it's visible at least | 16:36 |
| avass | so it's a separate task and not ignore the exit code of git clean in a multiline | 16:37 |
| clarkb | ya that will help on the reporting side | 16:38 |
| fungi | right, we can likely reuse the structure if ianw's 771297 but instead of using become use ignore_error? | 16:38 |
| avass | sounds llike a good compromise | 16:38 |
| tristanC | that's sounds good to me, at least that's similar to the previous behavior | 16:39 |
| fungi | i'll push an update to 771297 proposing that instead | 16:39 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul-client master: Add change-status subcommand https://review.opendev.org/c/zuul/zuul-client/+/759838 | 16:40 |
| tristanC | though it seems like if the root owned file is not conflicting with a tracked file, then ignoring the git clean error might result in undesirable behavior (e.g. a test loading data not part of the change being tested) | 16:43 |
| clarkb | yes, I think what fungi and I are saying is that we can acknowledge it isn't ideal but return the preexisting behavior and then sort out a proper way to address this with less immediate (unexpected) impact | 16:45 |
| fungi | it might, but there are also plenty of other ways running things on a reused job node could cause lingering impact on subsequent builds | 16:45 |
| clarkb | and maybe one way to address that is "accept that multi use nodes may have state leaks" | 16:45 |
| *** zenkuro has quit IRC | 16:47 | |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul-jobs master: mirror-workspace-git-repos: ignore errors from clean https://review.opendev.org/c/zuul/zuul-jobs/+/771297 | 16:47 |
| avass | tristanC: I thin something like that is why I added the git clean to begin with :) | 16:47 |
| *** zenkuro has joined #zuul | 16:48 | |
| *** rlandy_ has joined #zuul | 16:48 | |
| *** rlandy has quit IRC | 16:49 | |
| avass | fungi, clarkb: there could also be a variable to toggle whether to clean or not. In case there's a reason why files should be kept | 16:49 |
| *** rlandy_ is now known as rlandy | 16:50 | |
| clarkb | avass: ya something like that is what I'm startign to think may make sense longer term | 16:50 |
| fungi | my opinion on the matter is, if you run things as not the zuul user which creates untracked files in the work tree and this breaks subsequent builds, that's your choice | 16:50 |
| fungi | there's only so much zuul can do to return a reused node to its original state | 16:51 |
| avass | well yes but I think that zuul should at least try to reset the state of the repositories, and preferably error if it fails. | 16:53 |
| fungi | we can avoid it by setting sys.bytecodebase in some scripts, for sure, but if we're running ansible or something we can still wind up with problems like https://zuul.opendev.org/t/openstack/build/ea9b8fbb985b456fa0721f71de6f4f44 | 16:54 |
| tristanC | avass: in that situation, in particular if zuul can't use sudo, it's unlikely the clean will not remove the left over test data, but it would be ideal indeed to know if it can't | 16:55 |
| fungi | in that situation we're telling ansible to load filter plugins from the worktree, and running ansible as a different user than zuul | 16:55 |
| clarkb | ah this is the nested ansible | 16:55 |
| fungi | we could work around that by cloning from the zuul deployed copy of the repo into a random tempdir and telling ansible to use that instead, but would probably run into other leaks like slow accumulation of those tempdirs over time as some runs timed out | 16:56 |
| clarkb | fungi: it also complicates manual sysadmin intervention | 16:57 |
| fungi | also we'll need to ferret out all the various places where we have similar cd patterns | 16:57 |
| mhu | hello zuul-maint, here are two fixes that need the final +3: https://review.opendev.org/c/zuul/zuul/+/768229 and https://review.opendev.org/c/zuul/zuul/+/768312 | 16:58 |
| avass | it doesn't technically have to be a random tempdir, it could just be a separate workspace. but a toggle whether to clean or not (or a toggle whether to error on a clean or not) could be better than a silent error | 17:00 |
| avass | I'm fine with: https://review.opendev.org/c/zuul/zuul-jobs/+/771297 however to get your CD running | 17:02 |
| fungi | once we've got python 3.8 in our cd environment, we might be able to set PYTHONPYCACHEPREFIX to some place which isn't the worktree | 17:02 |
| fungi | but that's not an immediate solution unfortunately | 17:02 |
| *** jamesmcarthur has quit IRC | 17:03 | |
| *** jamesmcarthur has joined #zuul | 17:03 | |
| *** jcapitao has quit IRC | 17:05 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 17:11 | |
| *** _erlon_ has quit IRC | 17:13 | |
| corvus | clarkb, fungi, avass: hold on 771297 while i look into something | 17:15 |
| fungi | k | 17:15 |
| *** zbr3 has joined #zuul | 17:18 | |
| *** zbr has quit IRC | 17:20 | |
| *** zbr3 is now known as zbr | 17:20 | |
| tobiash | do we need to pull the clean into its own task? | 17:22 |
| fungi | tobiash: 771297 does that, yes | 17:23 |
| tobiash | I'm asking because especially with many projects the per task in a loop penalty is quite high which was one reason to initially keep those actions in a single task | 17:23 |
| corvus | remote: https://review.opendev.org/c/opendev/base-jobs/+/771496 Force ownership of zuul repo dirs in opendev CD jobs [NEW] | 17:24 |
| corvus | clarkb, fungi, avass, tobiash: ^ | 17:24 |
| fungi | tobiash: oh, got it. we could go back to using a monolithic script, but as avass points out ignoring errors from a shell command in that script is less discoverable | 17:24 |
| tobiash | fungi: I guess both are tradeoffs | 17:25 |
| corvus | i think that's the logical extension of fungi's "it's opendev's problem, opendev should fix it" line of thought | 17:25 |
| tobiash | is the clean problem more widespread or just opendev? | 17:25 |
| corvus | consider that a rough draft; i'm not 100% sure that's correct | 17:25 |
| fungi | corvus: yep, there may also be other places where we directly invoke python scripts | 17:26 |
| clarkb | corvus: I think taht still has the issue of using become | 17:26 |
| tobiash | sorry for asking but the backlog is quite large ;) | 17:26 |
| clarkb | oh I see it is in opendev/base-jobs though | 17:26 |
| clarkb | so we've isolated it to where we can assume root | 17:26 |
| avass | corvus: you might even be able to skip the loop and chown ~/src | 17:26 |
| corvus | avass: ++ | 17:26 |
| corvus | clarkb: yep. it's a narrowly targeted solution to a narrow problem | 17:26 |
| fungi | tobiash: i'm only aware of opendev being impacted, but it's only been changed to error in zuul-jobs for one day so there could conceivably be other zuul cd sites doing similar patterns | 17:26 |
| clarkb | I wouldn't be surprised if others using static nodes have a similar problem with this | 17:27 |
| clarkb | but I don't know who that group is off the top of my head | 17:27 |
| tobiash | I guess anyone using static nodes would need something like 771496 in any case to be reliably reproducable | 17:28 |
| avass | clarkb: us if we hadn't moved to k8s :) | 17:28 |
| avass | tobiash: I think it might be better for git clean to be configurable | 17:28 |
| corvus | "reuse of git repos on static nodes after having created files that can't be deleted by the zuul user" is the failing case; so it's not a given that any use on static nodes or even any CD use is affected currently | 17:29 |
| corvus | i don't think our opendev CD jobs are (and i think i can say this as one of the authors) extremely well designed models of how it should be done. :) so i'm inclined to consider this an eccentric opendev edge case until we hear otherwise. | 17:30 |
| tobiash | :) | 17:30 |
| corvus | (i think our CD jobs are getting better though!) | 17:31 |
| *** jamesmcarthur has quit IRC | 17:31 | |
| *** jamesmcarthur has joined #zuul | 17:32 | |
| corvus | i updated https://review.opendev.org/c/opendev/base-jobs/+/771496 with avass's suggestion | 17:33 |
| corvus | clarkb, fungi, avass, tobiash: what do you think, should we try that solution? or do we want to address it in zuul-jobs? | 17:34 |
| avass | I think that's a better solution :) | 17:34 |
| fungi | i'm willing to give it a shot in opendev and see what else is still broken at least | 17:35 |
| tobiash | corvus: commented | 17:35 |
| fungi | but we may want to keep a change to zuul-jobs in our back pocket in case other affected users come calling | 17:35 |
| clarkb | ya I don't mind trying that, does it need a base-test iteration though? | 17:36 |
| corvus | tobiash: hah i had the other change open when i wrote that :) | 17:36 |
| corvus | clarkb: normally, yes, but since evrything is broken, not this time | 17:36 |
| corvus | clarkb: it's a base job only used for infra-prod | 17:36 |
| avass | probably only needs the syntax verified because of the ignore_errors | 17:36 |
| *** jamesmcarthur has quit IRC | 17:37 | |
| tobiash | corvus: I've commented that also on the other change :) | 17:37 |
| corvus | tobiash: yeah, i saw :) | 17:37 |
| *** rlandy is now known as rlandy|brb | 17:37 | |
| tobiash | lgtm | 17:38 |
| *** jamesmcarthur has joined #zuul | 17:38 | |
| clarkb | corvus: oh I see | 17:38 |
| clarkb | everythign in this case is constrained :) | 17:38 |
| corvus | yep | 17:38 |
| corvus | cool, if we've established we want a localized fix for now, we can probably take further discussion to #opendev then | 17:39 |
| fungi | thanks corvus! | 17:39 |
| *** hamalq has joined #zuul | 17:40 | |
| *** jamesmcarthur has quit IRC | 17:43 | |
| *** jamesmcarthur has joined #zuul | 18:02 | |
| *** jpena is now known as jpena|off | 18:18 | |
| *** rlandy|brb is now known as rlandy | 18:20 | |
| *** hasharKids has quit IRC | 18:34 | |
| *** sanjayu_ has joined #zuul | 18:37 | |
| *** rpittau is now known as rpittau|afk | 18:41 | |
| *** bhavikdbavishi has quit IRC | 18:58 | |
| *** jamesmcarthur has quit IRC | 18:59 | |
| *** jamesmcarthur has joined #zuul | 18:59 | |
| *** zenkuro has quit IRC | 19:20 | |
| *** zenkuro has joined #zuul | 19:21 | |
| openstackgerrit | Merged zuul/zuul master: Re-elect James Blair as project lead https://review.opendev.org/c/zuul/zuul/+/769420 | 19:29 |
| tristanC | congratulation corvus! | 19:38 |
| *** ajitha has quit IRC | 20:01 | |
| *** openstackgerrit has quit IRC | 20:12 | |
| *** zbr5 has joined #zuul | 20:14 | |
| *** zbr has quit IRC | 20:16 | |
| *** zbr5 is now known as zbr | 20:16 | |
| *** openstackgerrit has joined #zuul | 20:36 | |
| openstackgerrit | James Bailey proposed zuul/nodepool master: fix pep8 https://review.opendev.org/c/zuul/nodepool/+/771514 | 20:36 |
| *** vishalmanchanda has quit IRC | 20:39 | |
| *** holser has quit IRC | 21:09 | |
| *** holser has joined #zuul | 21:09 | |
| *** zenkuro has quit IRC | 21:17 | |
| *** zenkuro has joined #zuul | 21:17 | |
| *** jamesmcarthur has quit IRC | 21:17 | |
| *** jamesmcarthur has joined #zuul | 21:19 | |
| *** jamesmcarthur has quit IRC | 21:42 | |
| *** jamesmcarthur has joined #zuul | 21:46 | |
| *** ikhan has quit IRC | 22:04 | |
| *** jamesmcarthur has quit IRC | 22:16 | |
| *** sanjayu_ has quit IRC | 22:17 | |
| *** harrymichal_ has joined #zuul | 22:20 | |
| *** jamesmcarthur has joined #zuul | 22:23 | |
| *** harrymichal has quit IRC | 22:24 | |
| *** harrymichal_ is now known as harrymichal | 22:24 | |
| *** nils has quit IRC | 22:24 | |
| *** iurygregory has quit IRC | 22:28 | |
| *** jamesmcarthur has quit IRC | 22:30 | |
| *** jamesmcarthur has joined #zuul | 22:33 | |
| *** iurygregory has joined #zuul | 22:37 | |
| *** snapiri has quit IRC | 22:58 | |
| *** openstackgerrit has quit IRC | 22:59 | |
| *** snapiri has joined #zuul | 23:03 | |
| *** snapiri has quit IRC | 23:08 | |
| *** CrayZee has joined #zuul | 23:08 | |
| *** jamesmcarthur has quit IRC | 23:25 | |
| *** jamesmcarthur has joined #zuul | 23:44 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!