| *** sgw has quit IRC | 00:06 | |
| *** tosky has quit IRC | 00:51 | |
| *** rfolco has quit IRC | 01:32 | |
| *** dangtrinhnt has joined #zuul | 02:23 | |
| dangtrinhnt | Hi, I'm trying to convince my company to use Zuul. Hopefully I can also help with the development. | 02:24 |
|---|---|---|
| *** dangtrinhnt has quit IRC | 02:38 | |
| *** dangtrinhnt_ has joined #zuul | 02:38 | |
| *** swest has quit IRC | 02:42 | |
| *** jamesmcarthur has joined #zuul | 02:50 | |
| *** dangtrinhnt_ has quit IRC | 02:51 | |
| *** bhavikdbavishi has joined #zuul | 02:54 | |
| *** bhavikdbavishi1 has joined #zuul | 02:56 | |
| *** swest has joined #zuul | 02:57 | |
| *** bhavikdbavishi has quit IRC | 02:58 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 02:58 | |
| *** michael-beaver has quit IRC | 03:03 | |
| *** jamesmcarthur has quit IRC | 03:08 | |
| *** jamesmcarthur has joined #zuul | 03:15 | |
| *** dangtrinhnt has joined #zuul | 03:27 | |
| *** dangtrinhnt has joined #zuul | 03:27 | |
| *** bhavikdbavishi has quit IRC | 03:45 | |
| *** bhavikdbavishi has joined #zuul | 03:45 | |
| *** rlandy|bbl is now known as rlandy | 03:57 | |
| *** jbryce has joined #zuul | 04:01 | |
| *** jamesmcarthur has quit IRC | 04:21 | |
| *** jamesmcarthur has joined #zuul | 05:23 | |
| *** jamesmcarthur has quit IRC | 05:28 | |
| *** swest has quit IRC | 05:29 | |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #zuul | 05:34 | |
| *** raukadah is now known as chandankumar | 05:36 | |
| *** swest has joined #zuul | 06:14 | |
| *** wxy-xiyuan has joined #zuul | 06:16 | |
| *** dangtrinhnt has quit IRC | 06:48 | |
| *** jhesketh has quit IRC | 06:48 | |
| *** jhesketh has joined #zuul | 06:50 | |
| *** saneax has joined #zuul | 06:53 | |
| *** bhavikdbavishi has quit IRC | 06:57 | |
| *** felixedel has joined #zuul | 06:58 | |
| *** dangtrinhnt has joined #zuul | 06:58 | |
| *** bhavikdbavishi has joined #zuul | 06:58 | |
| *** bolg has joined #zuul | 07:08 | |
| *** jhesketh has quit IRC | 07:15 | |
| *** jhesketh has joined #zuul | 07:25 | |
| *** Defolos has joined #zuul | 07:40 | |
| *** dpawlik_off is now known as dpawlik | 07:58 | |
| *** dangtrinhnt has quit IRC | 08:06 | |
| *** zxiiro has quit IRC | 08:23 | |
| *** tosky has joined #zuul | 08:36 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Don't set OS_LOG_DEFAULTS if unset https://review.opendev.org/707783 | 08:37 |
| *** avass has joined #zuul | 08:44 | |
| *** jpena|off is now known as jpena | 08:46 | |
| *** jcapitao has joined #zuul | 08:55 | |
| *** maxamillion[m] has quit IRC | 09:00 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Clarify RequirementError message https://review.opendev.org/707620 | 09:25 |
| *** masterpe has joined #zuul | 09:29 | |
| *** hashar has joined #zuul | 09:31 | |
| *** bhavikdbavishi has quit IRC | 09:42 | |
| *** bhavikdbavishi has joined #zuul | 09:43 | |
| frickler | does zuul ignore files that are deleted completely when calculating which jobs to run based on files/irrelevant-files? I'm having a hard time finding out why no devstack-based jobs were running on https://review.opendev.org/#/c/707635/ , causing it to get merged and breaking devstack. in particular tempest-full and grenade-py3 were missing | 10:02 |
| frickler | ah, nevermind, I found it | 10:12 |
| *** sugaar has quit IRC | 10:27 | |
| *** felixedel has quit IRC | 10:36 | |
| *** felixedel has joined #zuul | 10:36 | |
| *** sugaar has joined #zuul | 10:39 | |
| tobiash | zuul-maint: there is a test race fix that needs a second review: https://review.opendev.org/706566 | 10:55 |
| mnaser | before i start diving so badly into this but | 11:06 |
| mnaser | i probably cant install an ansible collection in the executor, could i? | 11:07 |
| mnaser | i have a role that's going to be nothing more than http requests and i was hoping to get it to be a 0 node job | 11:07 |
| tobiash | mnaser: currently you can only import roles via job definition, not a collection | 11:23 |
| tobiash | but if you have a role that only does http requests it might be possible to run it as a 0 node job | 11:23 |
| tobiash | uri module can be used locally in an untrusted context | 11:24 |
| *** felixedel has joined #zuul | 11:27 | |
| *** felixedel has quit IRC | 11:27 | |
| *** hashar has quit IRC | 11:28 | |
| openstackgerrit | Merged zuul/zuul master: Adjust io-level logging in gerrit/git drivers https://review.opendev.org/707728 | 11:41 |
| mnaser | tobiash: right, but i wonder then if i can just checkout that collection manually (just via a shell command) and using it | 11:42 |
| tobiash | mnaser: I guess that won't work, how does such a collection repo look like? | 11:43 |
| mnaser | tobiash: https://github.com/netbox-community/ansible_modules | 11:43 |
| mnaser | but then i realize i also need to pip install some packages, im not sure if i can do that on the executor | 11:44 |
| tobiash | that looks like it contains modules, no role, modules cannot be executed locally (in untrusted context) | 11:45 |
| mnaser | would have been neat for a 0 node job but i guess it wont hurt | 11:45 |
| *** jcapitao is now known as jcapitao_lunch | 11:52 | |
| *** hashar has joined #zuul | 11:57 | |
| *** felixedel has joined #zuul | 12:17 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Support pausing merge jobs https://review.opendev.org/707192 | 12:18 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Centralize merge handling https://review.opendev.org/707692 | 12:18 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Support pausing merge jobs https://review.opendev.org/707192 | 12:21 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Centralize merge handling https://review.opendev.org/707692 | 12:21 |
| *** jpena is now known as jpena|lunch | 12:40 | |
| *** rfolco has joined #zuul | 12:47 | |
| *** dSrinivas has joined #zuul | 12:50 | |
| *** rlandy has joined #zuul | 12:50 | |
| dSrinivas | Hi All, When I run our CI on particular Patch it is not triggering the Job http://paste.openstack.org/show/789572/ | 12:56 |
| dSrinivas | When run the zuul show running-jobs, it showing this error http://paste.openstack.org/show/789575/ | 12:56 |
| AJaeger | dSrinivas: please ask only on one channel, not on multiple! | 13:06 |
| * AJaeger answered on #openstack-infra | 13:06 | |
| *** jcapitao_lunch is now known as jcapitao | 13:15 | |
| *** plaurin has joined #zuul | 13:16 | |
| *** plaurin has quit IRC | 13:18 | |
| *** tjgresha has quit IRC | 13:20 | |
| *** tjgresha has joined #zuul | 13:20 | |
| *** avass has quit IRC | 13:20 | |
| *** jpena|lunch is now known as jpena | 13:36 | |
| *** sshnaidm|afk has quit IRC | 13:50 | |
| *** sshnaidm has joined #zuul | 13:50 | |
| *** sshnaidm is now known as sshnaidm|off | 14:05 | |
| *** zxiiro has joined #zuul | 14:08 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: wip: Set misfire grace time of apscheduler job https://review.opendev.org/707842 | 14:10 |
| *** dSrinivas has quit IRC | 14:19 | |
| *** hashar has quit IRC | 14:24 | |
| *** hashar has joined #zuul | 14:30 | |
| mordred | mnaser: we'd like to extend zuul's concept of zuul roles to include collections so that you can configure a job to use a collection and have zuul check it out. we've been waiting for the dust to settle with collections first | 14:40 |
| *** plaurin has joined #zuul | 14:52 | |
| plaurin | Hello! | 14:52 |
| plaurin | I encountered an issue in zuul when a long running task (ansible shell) returns MODULE FAILURE after 4 hours. | 14:53 |
| plaurin | Yes 4 hours is a long time this is hardware testing, it might get over 6 hours at some point ;-) | 14:53 |
| pabelanger | you may want to look at asynchronous task, for long running commands | 14:55 |
| pabelanger | but, you should be able to see the error in executor logs | 14:55 |
| pabelanger | or need to enable more version to see failure | 14:55 |
| plaurin | ya that was my idea, but I cannot use async because I need to see the std output of the task | 14:55 |
| tobiash | plaurin: did you also set a job timeout of > 4 hours? | 14:55 |
| plaurin | I cannot registed the shell output after an async and I cannot see live logging from the shell command as I use kubernetes/nodepool plugin | 14:56 |
| plaurin | yes it's set to 8 hours tobiash | 14:56 |
| tobiash | k | 14:56 |
| tobiash | just asking because it's easy to forget this ;) | 14:56 |
| plaurin | yep, happened to me many times | 14:57 |
| plaurin | executor logs are non visible because of kubernetes that doesn't stream the outputs of each tasks :( | 14:58 |
| tobiash | can it be that the output was too large? I think I remember someone had that problem in our deployment. | 15:01 |
| plaurin | no it's not that much of a long output | 15:03 |
| *** saneax has quit IRC | 15:03 | |
| tobiash | then I'd guess for a connection issue | 15:03 |
| plaurin | It seems "precisely" a 14400 second issue (4 hours) | 15:03 |
| plaurin | 2020-02-13 16:21:55.380369 | TASK [Run tests] | 15:03 |
| plaurin | 2020-02-13 20:21:55.810957 | embedded-software-pytest-k3s | MODULE FAILURE | 15:03 |
| tobiash | plaurin: the json log or executor log should give you more details on that task | 15:04 |
| *** armstrongs has joined #zuul | 15:04 | |
| plaurin | non executor doesn't show anything because it's running on kubernetes, so no stream output from the command | 15:04 |
| plaurin | I have deactivated json a long time ago, hum maybe I should reactivate | 15:05 |
| pabelanger | zuul-executor verbose will enable more logging | 15:05 |
| tobiash | I wasn't even aware that this can be deactivated | 15:05 |
| pabelanger | https://zuul-ci.org/docs/zuul/discussion/components.html#id11 | 15:05 |
| tobiash | yes, but be cautios on a busy system | 15:06 |
| pabelanger | yup, agree | 15:06 |
| pabelanger | will likely fill up HDD | 15:06 |
| tobiash | Once I've filled the hard disk of all executors over a lunch break ;) | 15:06 |
| plaurin | ya it's a bit busy | 15:06 |
| plaurin | so kind of in a bind :( | 15:06 |
| pabelanger | I've always wanted to add support to autohold, that also enables verbose, so you can do it for single job | 15:07 |
| plaurin | btw I don't mind the async run however I need to hack my way to having the log output of the shell command in the executor log | 15:08 |
| plaurin | I know ansible generates log files on the target build machine while running, I wonder if I could print one of those | 15:08 |
| plaurin | oh maybe I could redirect the stdout 2>&1 of my shell command and then print it after the async task | 15:15 |
| *** felixedel has quit IRC | 15:15 | |
| plaurin | stdout to a file I mean | 15:15 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Offload repo reset to processes https://review.opendev.org/707095 | 15:17 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Don't set untouched refs of the repo state twice. https://review.opendev.org/707857 | 15:17 |
| mordred | tobiash: I'm finding your scaling related patches fascinating | 15:27 |
| tobiash | :) | 15:28 |
| openstackgerrit | Merged zuul/zuul master: Fix test race with paused build https://review.opendev.org/706566 | 15:33 |
| *** bhavikdbavishi has quit IRC | 15:43 | |
| *** dpawlik has quit IRC | 15:53 | |
| plaurin | ah yes my solution works. from my previous comments | 15:56 |
| plaurin | run shell: command &> testlog.txt asynchronously | 15:56 |
| plaurin | wait for shell command to complete | 15:57 |
| plaurin | cat testlog.txt and register variable | 15:57 |
| *** Goneri has joined #zuul | 15:57 | |
| plaurin | run debug task to see content of testlog.stdout_lines | 15:57 |
| plaurin | good ol' bash to the rescue | 15:58 |
| mordred | plaurin: everybody always wants to get rid of shell because "it's too hard" ... but man it's good at its job | 16:02 |
| plaurin | hehe. For me it's a swiss knife. Downside is that I sometime use ansible shell module when I should not need to lol. But it's so reliable for me | 16:03 |
| *** hashar has quit IRC | 16:11 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: WIP: Store unparsed branch config in Zookeeper https://review.opendev.org/705716 | 16:11 |
| *** jamesmcarthur has joined #zuul | 16:16 | |
| *** sgw has joined #zuul | 16:30 | |
| corvus | mordred: what do you think of vitaly's comment in https://gerrit-review.googlesource.com/c/zuul/jobs/+/254994/9 ? | 16:37 |
| mordred | corvus: I definitely like using .gitmodules as a source of truth | 16:44 |
| *** Defolos has quit IRC | 16:44 | |
| mordred | corvus: and I think that would also address david's question about branches that don't have all of the modules | 16:45 |
| *** rlandy has quit IRC | 16:45 | |
| mordred | that said - I'm not 100% sure how to square that with making sure required-projects gets all the right repos | 16:45 |
| corvus | mordred: yes (though david's question is actually pointing to a test job that will only run on master) the system we made will copy unecessary projects into gerrit, vitaliy's would only copy the ones needed | 16:46 |
| mordred | corvus: I wonder (thinking out loud) if we could learn from the gitmodules approach he's describing and teach zuul how to infer required-projects relationships somehow | 16:46 |
| mordred | starting off just having it be in the job and settig required-projects ourselves while we learn of course | 16:47 |
| corvus | mordred: and yes, i think i would still like leave required-projects in the gerrit base job alone. i don't want the jobs fetching from upstream, i only want them using zuul prepped repos. | 16:47 |
| mordred | corvus: yes. I agree about not having them fetch anything | 16:48 |
| *** rlandy has joined #zuul | 16:48 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Don't set untouched refs of the repo state twice. https://review.opendev.org/707857 | 16:49 |
| corvus | so the thig i like about his approach is that it uses gitmodules to figure out which repos to copy in. the thing i don't like is that it will fetch from upstream. so i kinda think i want to halfway adopt it. | 16:49 |
| corvus | mordred: (and yes, i suspect there is a future where we could infer required-projects from the contents of .gitmodules with the help of a merger) | 16:49 |
| mordred | corvus: ++ I agree | 16:50 |
| corvus | k, lemme write up a response and run it by you | 16:51 |
| mordred | corvus: in fact, once we know enough to infer required projects, we could even have zuul prepare the super-project repo with the submodules in place directly - but I think that's still a few steps away | 16:51 |
| corvus | ya | 16:52 |
| tobiash | some of our projects do this via a post job | 16:53 |
| corvus | tobiash: a post job sounds too late for this? | 16:53 |
| tobiash | (pushing the update to the super repo) | 16:54 |
| corvus | oh | 16:54 |
| corvus | tobiash: that sounds like a replacement for gerrit submodule subscription | 16:54 |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Clarify RequirementError message https://review.opendev.org/707620 | 16:54 |
| tobiash | so the submodules are gated and the master repo is updated in post to refer to the gated sub repos | 16:54 |
| tobiash | I think some projects also use branch tracking submodules instead | 16:55 |
| tobiash | corvus: could we maybe early next week do a gear release so we can use the mac support? | 16:58 |
| corvus | tobiash: yes! | 17:00 |
| tobiash | awesome, thanks! | 17:00 |
| corvus | mordred: https://etherpad.openstack.org/p/cgcAHVgFud how's that look? | 17:09 |
| mordred | looks good - maybe add a "nice to meet you" or something since he started his with a "let me introduce myself" - but the content looks good | 17:14 |
| corvus | mordred: i kinda want to add a footnote ("if we add .gitmodule parsing to zuul itself, we can automatically populate required-projects") | 17:16 |
| mordred | ++ | 17:16 |
| mordred | corvus: I really like the idea of zuul eventually being able to understand submodule relationships in some reasonable manner | 17:20 |
| mordred | because they're hard - and one of the things zuul is extra valuable for is understanding git better than its users | 17:20 |
| *** rlandy is now known as rlandy|brb | 17:20 | |
| corvus | mordred: yes. i think it's authors first need to understand git submodules better than zuul will need to, and i personally haven't felt that way yet, but i think i may be getting close | 17:23 |
| corvus | mordred, tobiash: but then there are so many ways of using submodules -- gated with gerrit subscriptions may be very different than other systems (which may have a local subscription system like tobiash has, or may not). so it's still hard to wrap one's head around all the possibilities. | 17:24 |
| corvus | anyone have an ensure-java role handy? :) | 17:26 |
| corvus | mordred: looks like you just did FROM docker.io/library/openjdk:8 to build old gerrit | 17:27 |
| mordred | corvus: yeah - specifically because I didn't want to try to deal with actually installing java :) | 17:28 |
| corvus | looks like RUN apt-get install -y openjdk-8-jdk && update-alternatives --set java /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java | 17:29 |
| corvus | is what the current gerrit ci is doing | 17:29 |
| mordred | corvus: then I thnik that's an excellent approach | 17:32 |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #zuul | 17:34 | |
| *** gmann is now known as gmann_afk | 17:42 | |
| *** igordc has joined #zuul | 17:43 | |
| *** rlandy|brb is now known as rlandy | 17:46 | |
| *** jpena is now known as jpena|off | 17:50 | |
| fungi | corvus: so thinking you could use .gitmodules as a sort of sieve to determine which registered projects are required? i wonder if it would be possible (i.e. safe) to have zuul discover additional projects by reading .gitmodules files in repos | 17:52 |
| *** jamesmcarthur has quit IRC | 17:54 | |
| *** jcapitao is now known as jcapitao_off | 17:56 | |
| *** jamesmcarthur has joined #zuul | 17:58 | |
| corvus | fungi: yes, the first part of that is more or less the idea from vitaliy on that change, and the second is something i think we could have zuul do (with the help of its mergers). whatever zuul might eventually natively do with submodules, that's probably the first step. | 18:05 |
| *** jamesmcarthur has quit IRC | 18:07 | |
| *** jamesmcarthur has joined #zuul | 18:09 | |
| fungi | makes sense. thanks! | 18:12 |
| fungi | my grasp of how submodules are expected to work in normal use is at best thin | 18:13 |
| *** Defolos has joined #zuul | 18:16 | |
| corvus | fungi: they are supposed to solve all of your problems without creating any more, but usually the opposite is true | 18:18 |
| * mordred has to afk for a a few - we had our dining room table legs fixed (one got broken in the move) and they're ready to be picked up - back in a few | 18:22 | |
| fungi | pretty sure the goal of any technology is to solve all existing problems without creating new ones | 18:24 |
| fungi | yet the end result is most often the opposite | 18:24 |
| fungi | mordred: happy (non-wobbly) dining! | 18:24 |
| *** jamesmcarthur has quit IRC | 18:25 | |
| *** jamesmcarthur has joined #zuul | 18:26 | |
| *** felixedel has joined #zuul | 18:29 | |
| *** jamesmcarthur has quit IRC | 18:31 | |
| *** jamesmcarthur has joined #zuul | 18:36 | |
| *** jamesmcarthur has quit IRC | 18:48 | |
| *** jcapitao_off has quit IRC | 18:53 | |
| corvus | wow, i think we built the checks plugin! https://ci.gerritcodereview.com/t/gerrit/build/1851e1f671194dd4a8fb36fac0a13c24 | 19:18 |
| *** armstrongs has quit IRC | 19:22 | |
| tristanC | corvus: that url yields 500' | 19:28 |
| corvus | tristanC: sorry, i was restarting the scheduler | 19:34 |
| *** tosky has quit IRC | 19:36 | |
| tristanC | corvus: it works now, that's nice, great work! | 19:36 |
| mnaser | hey all | 20:11 |
| mnaser | i'm doing really wild things with zuul+ansible to gitops our netbox deployment | 20:12 |
| *** armstrongs has joined #zuul | 20:12 | |
| *** gmann_afk is now known as gmann | 20:12 | |
| mnaser | i'm trying to get a netbox deployment up using docker-compose to run against it | 20:12 |
| mnaser | i noticed that zuul quick start jobs install docker-compose too so i'm wondering if i can just refactor that out | 20:13 |
| mnaser | do we feel like maybe adding a flag to install-docker to include compose for things that need it? | 20:13 |
| pabelanger | mnaser: tristanC has a patch up https://review.opendev.org/687135/ | 20:14 |
| pabelanger | for podman | 20:14 |
| *** igordc has quit IRC | 20:15 | |
| tristanC | pabelanger: oh my, i think this patch needs a lot of work... i should probably abandon it | 20:15 |
| mnaser | o neat, im wondering about making the zuul/zuul-jobs install-docker role letting you install docker-compose (or at least having maybe another role that does it) | 20:15 |
| mnaser | just cause i feel like that might be a common scenario in ci | 20:16 |
| tristanC | and it seems like mnaser just wants to update the zuul-jobs install-docker role | 20:16 |
| mnaser | ++ | 20:16 |
| pabelanger | I thought we had something started for that too | 20:16 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: install-docker: enable setting docker userland proxy https://review.opendev.org/702753 | 20:16 |
| tristanC | mnaser: why not using a dedicated install-docker-compose ? | 20:17 |
| mnaser | tristanC: really open for either or tbh, the main reasoning initially was because install-docker plays/touches with repo config on the host | 20:17 |
| mnaser | so i figured that it would make sense to have it there since it might be confusing if someone installs from upstream _but_ doesnt set that other setting in the other role | 20:19 |
| *** armstrongs has quit IRC | 20:22 | |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: install-docker: option to install docker-compose https://review.opendev.org/707902 | 20:32 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: install-docker: option to install docker-compose https://review.opendev.org/707902 | 20:34 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul master: cleanup: install docker-compose via role https://review.opendev.org/707903 | 20:36 |
| mnaser | corvus: what do you think about setting <gerrit connection>.auth_type to required with no default inside zuul, the most recent default switch broke our conncetion to opendev b/c we didnt have it explicitly set | 20:53 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: buildset-registry-k8s-crio: mark job as non-voting https://review.opendev.org/707908 | 21:01 |
| corvus | mnaser: i don't think it's necessary -- i suspect that will be the last time this happens (zuul was catching up with a gerrit default that changed like 5 years ago, and opendev is on a catastrophically old gerrit) | 21:02 |
| mnaser | corvus: ok cool seems fair, i'm just not familiar with the landscape of the gerrits | 21:02 |
| mnaser | so i was confused for a while why the depends-on weren't working | 21:02 |
| corvus | mnaser: yeah, digest is not even supported now, so there's basically one right answer :) | 21:03 |
| mnaser | ah gotcha | 21:03 |
| corvus | mnaser: we probably should have thought to change opendev's gerrit's default | 21:03 |
| mnaser | corvus: we need to revisit the whole -f implies debug thing, i can imagine you are also probably dealing with a *ton* of logs in your zuul if you're using the helm charts | 21:03 |
| corvus | mnaser: yes. tobiash just pushed up some patches for that, btw. i also pushed up some log tuning patches. | 21:04 |
| mnaser | oh really? nice. it's been quite hard to look over the logs these days from an 'operator' pov | 21:04 |
| corvus | mnaser: oh, it looks like my patch merged: If0d5877f65296ee2a9f7f0298f8bc6664e6c2a4c | 21:04 |
| * mnaser check | 21:05 | |
| corvus | i look forward to reviewing tobiash's patch, but i'll need a moment to think about that. | 21:05 |
| corvus | basically my patch will get the *traffic* out of the debug log, and tobiash will get us to not having a debug log :) | 21:06 |
| corvus | (but mine was only aimed at the excess gerrit traffic) | 21:06 |
| tristanC | corvus: interesting failure in https://review.opendev.org/#/c/707908/ . it seems like we can't do that without also touching the zuul-jobs-test-registry-buildset-registry job | 21:07 |
| tristanC | corvus: also, would you know why the cri-o job stopped working? | 21:07 |
| mnaser | i'm thinking add -f which foregrounds but doesnt necessary enable debug, and -d stays the same, but yes, i remember us discussing some much more fine details | 21:07 |
| corvus | mnaser: yeah, i believe that's what tobiash has been working on | 21:08 |
| tobiash | mnaser: from an operators pov I bet you'll want debug logs (with central log aggregation) | 21:08 |
| corvus | tobiash: i hope not -- i'd like for an operator to run at info unless they are debugging or writing zuul code | 21:09 |
| tristanC | tobiash: that sounds odd, sounds like debug should be promoted to info then | 21:09 |
| mnaser | tobiash: could be, but yeah, i agree with corvus there. it's also hard to find data when we're connected to 2 really busy gerrits | 21:09 |
| mnaser | opendev and gerrithub just generate *so* much data | 21:10 |
| corvus | i'm sure we're not there, because i wrote most of the log lines, and i'm always looking at debug logs, because the only reason i look at logs is to find bugs in zuul :) so i know we need help sorting out the levels | 21:10 |
| tristanC | mnaser: you can make the zuul.GerritConnection.io quietter though, e.g. https://softwarefactory-project.io/cgit/software-factory/sf-config/tree/ansible/roles/sf-zuul/templates/logging.yaml.j2#n43 | 21:11 |
| tobiash | well let me rephrase, as an operator with zuul dev background you'll want that at least now | 21:11 |
| mnaser | tristanC: i think corvus merged the patch to quiet it down 'natively' | 21:11 |
| corvus | but the original idea is that any action zuul takes should be at info | 21:11 |
| mnaser | s/natively/within zuul/ | 21:11 |
| mnaser | this might sound silly but is it possible to have a repo that's within required-projects which the github app is not installed on? | 21:12 |
| tobiash | actually I have the feeling that I needed almost every part of the debug log at least once | 21:12 |
| mnaser | im trying to use this https://github.com/netbox-community/netbox-docker in my job and trying to let zuul check it out instead of me doing that | 21:13 |
| corvus | tobiash: me too, but i hope we're not typical :) | 21:13 |
| tobiash | so I find it hard to reason in many cases if it needs th be debug or info | 21:13 |
| corvus | mnaser: you'll need to add it to the zuul tenant config, but i don't think you need to install the app | 21:13 |
| corvus | mnaser: (and maybe add it with "include: []") | 21:14 |
| tobiash | at least we're dependent on the debug logs because our users find all sorts of unexpected side effects ;) | 21:14 |
| mnaser | thinking out loud, is there any particular reason why we couldn't just let users arbitrarily use required-projects to any connections thats available? | 21:14 |
| corvus | mnaser: see commit message at https://review.opendev.org/613143 | 21:15 |
| mnaser | seems like a nice and native/useful thing if you depend on a project in github, but might be short sighted quick impulse idea of mine that may have reasons why it wouldn't work | 21:15 |
| mnaser | aah | 21:15 |
| mnaser | gotcha. | 21:15 |
| tobiash | mnaser: we do (within the limits of configured projects in the tenant) | 21:15 |
| kmalloc | corvus: that's a nice feature | 21:15 |
| mnaser | required-projects: ['github.com/some-org/my-secrets'] | 21:15 |
| kmalloc | hehe | 21:16 |
| mnaser | some-org has the app installed and it's a bad time, makes sense | 21:16 |
| kmalloc | mnaser: quick store all your secrets in github in a public repo :P | 21:16 |
| mnaser | kmalloc: :D | 21:17 |
| mnaser | maybe if it's not listed inside config/untrusted projects, it doesnt use the github api credentials to fetch info | 21:17 |
| mnaser | i dunno if github lets you do that | 21:17 |
| kmalloc | hm. honestly, i don't know if github allows that. | 21:17 |
| corvus | tristanC: you found an interesting edge case in 707908. i'd call that a bug, and it'll need a little bit of code to fix (if we're ignoring file matchers for a job, we should ignore file matchers for all of its dependencies) | 21:18 |
| corvus | tristanC: and i don't know what might have changed to cause it to fail | 21:19 |
| corvus | mordred: it looks like the opendev build of gerrit checks is failing: https://zuul.opendev.org/t/zuul/build/d52491b15ffa44ea8545055909db3276 | 21:30 |
| mnaser | tristanC: i tihnk i know why cri-o is failing | 21:31 |
| mnaser | we're installing them from the ppas, right? | 21:31 |
| mnaser | yes, project atomic ppa's don't ship a registries.conf file. | 21:32 |
| mnaser | also why are we installing docker in the cri-o job? | 21:33 |
| tristanC | mnaser: you mean it stopped shipping a registries.conf file? | 21:34 |
| mnaser | im pretty sure it did, we use the ppas and we have to drop that file in :\ | 21:34 |
| mnaser | the odd thing is they stopped shipping it for a while now | 21:35 |
| mnaser | im unsure if cri-o jobs run all the time or if there is a specific set of files that they run on? | 21:35 |
| mnaser | tristanC: eg https://github.com/containers/podman.io/issues/127 | 21:36 |
| mordred | corvus: yay! I love it when that fails | 21:38 |
| mordred | corvus: what's with all the waiting for logger lines? | 21:38 |
| mordred | corvus: joy. it's a permissions issue with the new npm based build | 21:39 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: crio: add missing registries.conf https://review.opendev.org/707910 | 21:40 |
| mordred | corvus: I'll poke and find a solution. it's possible we need to also pre-install yarn (I think we're only installing npm as for right now) | 21:41 |
| tristanC | mnaser: thanks for the reference | 21:41 |
| mnaser | tristanC: np! just curious as to why we run install-docker in that job btw -- historical reasons or? | 21:43 |
| tristanC | mnaser: iiuc, install-docker affects buildset-registry which needs to be tested with cri-o too. though perhaps the docker <-> crio dependencies could be relaxed | 21:44 |
| mnaser | also fwiw we have this in our registries.search: `registries = ['registry.access.redhat.com', 'docker.io', 'registry.fedoraproject.org', 'quay.io', 'registry.centos.org']` | 21:44 |
| mnaser | we can probably drop the redhat/fedora ones and leave docker.io and quay.io only, as i think without those, it won't try and check for docker.io/foo/bar natively if you request foo/bar | 21:45 |
| corvus | yeah, for better or worse, i think the ship has sailed on registry search, and it should either be docker.io or nothing -- anything else is dangerous. the right way to level the playing field is to fully-qualify every image name (which is something we started doing in opendev and zuul). | 21:49 |
| mordred | yup. there's a great write up on how horrible of a security hole it is | 21:49 |
| mordred | I am very disappointed in my employer for MITMing people with that registry search path there | 21:49 |
| corvus | ok, on the gerrit front: i have a job that ran 'bazelisk test' on the delete-project plugin and succeeded! https://gerrit-review.googlesource.com/c/zuul/jobs/+/255214/ | 21:50 |
| mordred | \o/ | 21:50 |
| mordred | corvus: I'm working on reproducing our build failure locally | 21:51 |
| corvus | and i've added a checker to plugins/checks and i have a job running there: https://gerrit-review.googlesource.com/c/plugins/checks/+/255215 | 21:51 |
| mordred | corvus: although I'm starting to think that maybe changing our image build process to just do the build not in a container using bazelisk like your job does and then copy the war into a container might be more stable | 21:51 |
| corvus | mordred: yeah, i was wondering if that might be a win | 21:52 |
| mordred | since the thing I was trying to avoid in using that bazel image as a build image was to not have to download bazel each time - but it leaves us chasing a build setup which is not how gerrit builds things | 21:52 |
| mordred | so we don't get the efficiency - and we're more prone to breakages :) | 21:52 |
| mordred | and it's not like we locally build these images frequently anyway | 21:52 |
| mordred | corvus: medium-term I think it might be nice to figure out how to grab upstream built release war and just build the plugins we care about | 21:53 |
| mordred | but that was also complex last time I looked | 21:53 |
| corvus | mordred: yeah, do you need a whole source tree still? | 21:53 |
| corvus | because the build process i'm following is basically "copy plugin into gerrit repo and then build the plugin" | 21:54 |
| mordred | corvus: yeah. it's complicated | 21:56 |
| mordred | corvus: apparently sometimes you do and sometimes you don't and some plugins support it and some don't and I'm confused | 21:57 |
| mordred | corvus: which is why I went with "copy it all in and just build one war" | 21:57 |
| mordred | but I bet we could work with someone to understand a better approach | 21:57 |
| corvus | ++ | 21:57 |
| *** rlandy has quit IRC | 21:59 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: crio: add missing registries.conf https://review.opendev.org/707910 | 22:02 |
| *** rfolco has quit IRC | 22:02 | |
| mnaser | this might sound trivial as a question but there isn't really a way of passing some value to the base tox jobs via vars other than reusing the roles to build your own job? | 22:09 |
| mnaser | im trying to get a url passed from zuul vars to tox and i could think of many ways to do it by using the role directly but not by simply making a child job | 22:10 |
| mnaser | oh, `tox_environment` | 22:11 |
| mnaser | ill go back to hiding | 22:11 |
| corvus | mnaser: ok cool. ftr i was typing a reply and not ignoring you. :) | 22:12 |
| mnaser | corvus: hehe thanks. i'm working on doing something to gitops-ify our infra inventory using zuul. we use netbox and they have a very nice ansible collection for all their modules, so going to use it to have a bunch of jobs that add/update all resources into netbox (a dcim tool) | 22:13 |
| mnaser | and then in our gating, we actually use docker-compose to bring up an actual instance of netbox and apply everything against it too to make sure its all working before ending up in post | 22:13 |
| mnaser | i plan to write up something about this once its cooked up | 22:13 |
| corvus | mnaser: that sounds awesome :) | 22:14 |
| mnaser | (and i was also thinking if maybe we should come up with zuul-contrib-jobs or something for kindof non-standard jobs like this (or i mean, we could throw it in zuul-jobs but it might be too specific) | 22:15 |
| mnaser | but i'm just trying to get it to work first :) | 22:15 |
| corvus | i feel like the standard for inclusion in zuul jobs should be: 1) regarding subject matter, low: would anyone else be able to use this job? 2) regarding construction, probably a bit higher: is it well documented, follow relevant zuul-jobs policios, and does it come with a test. | 22:16 |
| corvus | basically, if you feel like it could go in zuul-jobs-contrib, it probably belongs in zuul-jobs :) | 22:17 |
| corvus | sounds like "test my config against netbox" is something that more than one project/org in the world might find useful :) | 22:19 |
| corvus | zuul-maint, infra-root: since the job is currently broken, i have disabled the opendev/zuul third-party testing of the gerrit checks plugin by disabling the checker in gerrit's gerrit. this will both avoid spamming the checks plugin repo with invalid results, and also avoid having our checker show up at all there [my real motivation here is for people not to be confused by the two different zuuls reporting | 22:22 |
| corvus | results, one passing, one failing]. | 22:22 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: crio: add missing registries.conf https://review.opendev.org/707910 | 22:22 |
| *** plaurin has quit IRC | 22:28 | |
| tristanC | corvus: mnaser: it seems like setting the test image fqn fixed the crio job, could you please have a look at https://review.opendev.org/707910 to unblock the install-docker test failure | 23:08 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: install-docker: enable setting docker userland proxy https://review.opendev.org/702753 | 23:08 |
| mnaser | tristanC: lgtm and its nice that we dont have to add those aliases on top of it | 23:09 |
| tristanC | corvus: mnaser: thanks for the review, then could we also land https://review.opendev.org/702753 please | 23:11 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: install-docker: option to install docker-compose https://review.opendev.org/707902 | 23:15 |
| mnaser | i rebased that change on tristanC fix and https://review.opendev.org/#/c/707903/ tests it (the failure seems unrelated) | 23:15 |
| *** Defolos has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!