| *** rfolco has quit IRC | 01:06 | |
| *** pcaruana has joined #zuul | 04:22 | |
| *** raukadah is now known as chandankumar | 04:52 | |
| *** saneax has quit IRC | 05:04 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/nodepool master: openstack: document the key-name parameter https://review.opendev.org/661677 | 06:07 |
|---|---|---|
| *** yolanda has quit IRC | 06:38 | |
| *** themroc has joined #zuul | 07:02 | |
| *** jhesketh has quit IRC | 07:11 | |
| *** saneax has joined #zuul | 07:17 | |
| *** gtema has joined #zuul | 07:20 | |
| *** jhesketh has joined #zuul | 07:30 | |
| *** logan- has quit IRC | 07:33 | |
| *** tosky has joined #zuul | 07:35 | |
| *** logan- has joined #zuul | 07:36 | |
| *** jpena|off is now known as jpena | 07:52 | |
| *** tflink has quit IRC | 08:33 | |
| *** tflink has joined #zuul | 08:36 | |
| *** jesusaur has quit IRC | 09:17 | |
| *** hashar has joined #zuul | 09:18 | |
| *** jesusaur has joined #zuul | 09:20 | |
| *** themroc has quit IRC | 09:50 | |
| *** themroc has joined #zuul | 09:51 | |
| *** bjackman has joined #zuul | 10:04 | |
| bjackman | Do I understand correctly that the raw git driver doesn't support authentication at all? | 10:07 |
| bjackman | (i.e. it can only work on public repos) | 10:08 |
| *** gtema has quit IRC | 10:48 | |
| *** gtema has joined #zuul | 10:50 | |
| *** AJaeger has quit IRC | 11:24 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 11:27 |
| *** gtema has quit IRC | 11:32 | |
| *** jpena is now known as jpena|lunch | 11:33 | |
| *** AJaeger has joined #zuul | 11:36 | |
| *** sean-k-mooney has joined #zuul | 11:56 | |
| *** rfolco has joined #zuul | 11:59 | |
| sean-k-mooney | o/ this is more to satify my own curiosity but i have a dumbish question about the zuul mergers. if i was to deploy 2+ zuul mergers and use an nfs share for the git repos would that break things? e.g. can concurrent git operations/mergers be perfromed on the same repo or should they all have there own stoarge | 12:01 |
| *** electrofelix has joined #zuul | 12:02 | |
| pabelanger | sean-k-mooney: you'd need different storage for each | 12:02 |
| sean-k-mooney | ok that is what i assumed | 12:03 |
| sean-k-mooney | i dont currently have my home zuul deployment currently as i reinstalled teh server it was on but i was debating if i redeployed it in the future would i have to use kubernetes deployment (one volume shared betwen all mergers) or statefule sets (each pod gets its own volume) if i redeploy it under kubernetes | 12:05 |
| sean-k-mooney | i used deployment before but i also only use 1 instance or each component | 12:05 |
| sean-k-mooney | mainly because i assumed git would break if i didnt. | 12:06 |
| *** rf0lc0 has joined #zuul | 12:09 | |
| *** rfolco has quit IRC | 12:10 | |
| *** rlandy has joined #zuul | 12:26 | |
| *** hashar has quit IRC | 12:33 | |
| *** rf0lc0 has quit IRC | 12:34 | |
| *** rfolco has joined #zuul | 12:34 | |
| *** rf0lc0 has joined #zuul | 12:35 | |
| *** rfolco has quit IRC | 12:38 | |
| mordred | sean-k-mooney: we're also working on a zuul operator for k8s - spec (and discussion) here: https://review.opendev.org/#/c/659180/ | 12:49 |
| fungi | bjackman: that sounds right, i expect it wouldn't be tough to add, just its initial implementation was first and foremost intended as a lightweight means for zuul deployments to be able to consume the zuul-jobs standard library in a zuul-native fashion | 12:51 |
| fungi | (as opposed to feeding it an out-of-band fork of the repo you maintain independently) | 12:51 |
| sean-k-mooney | mordred: that could be useful. i wrote my own yaml files partly as a learning excersize | 12:52 |
| mordred | sean-k-mooney: learning is good. or so they tell me :) | 12:57 |
| sean-k-mooney | well i and not done much with k8s and deploying zuul + node pool + gerit+ logs server (e.g. a full ci) was a good way to figure out how to laywer the different piceies | 12:58 |
| sean-k-mooney | *had | 12:58 |
| sean-k-mooney | *layer | 12:59 |
| fungi | it is indeed, which is also why our quickstart includes them | 13:00 |
| sean-k-mooney | well i have now deployed zuul 3-4 times. first using zuul form scratch guide, then againg a few monts later, then the quickstart docker compose and finally with my custom k8s scripts | 13:01 |
| sean-k-mooney | there are pros and cons to all of the above but its not a partcalarly difficult task | 13:02 |
| sean-k-mooney | but its also not totally trivial so its a good thing to try | 13:03 |
| *** bjackman has quit IRC | 13:06 | |
| fungi | sounds like an interesting exercise, for sure | 13:12 |
| *** jpena|lunch is now known as jpena | 13:16 | |
| *** gtema has joined #zuul | 13:18 | |
| *** tosky__ has joined #zuul | 13:21 | |
| *** tosky is now known as Guest72073 | 13:21 | |
| *** tosky__ is now known as tosky | 13:21 | |
| sean-k-mooney | while im here where would i find where future fetature for zuul are planned. e.g. do ye use specs blueprints storyboard? im partly intersted in the k8s stuff but also the zuul runner comandline features | 13:31 |
| *** rf0lc0 is now known as rfolco | 13:32 | |
| fungi | sean-k-mooney: https://zuul-ci.org/docs/zuul/developer/specs/index.html | 13:35 |
| fungi | so yes, specs | 13:35 |
| fungi | they just live in the docs tree of the zuul repo at the moment | 13:35 |
| sean-k-mooney | ah ok | 13:36 |
| fungi | (even though some could be specs involving other zuul repos) | 13:36 |
| *** themroc has quit IRC | 13:41 | |
| *** sanjayu_ has joined #zuul | 13:44 | |
| *** sanjayu_ has quit IRC | 13:46 | |
| *** saneax has quit IRC | 13:46 | |
| *** rf0lc0 has joined #zuul | 13:46 | |
| *** rfolco has quit IRC | 13:48 | |
| *** themroc has joined #zuul | 13:56 | |
| *** rf0lc0 is now known as rfolco | 13:57 | |
| *** bjackman has joined #zuul | 14:00 | |
| *** chandankumar is now known as raukadah | 14:25 | |
| corvus | i'm going to try to send out a project update email today, so if you have a moment to leave notes in https://etherpad.openstack.org/p/zuul-update-email that would help | 14:25 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 14:44 |
| *** bjackman has quit IRC | 14:48 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 14:49 |
| *** hashar has joined #zuul | 14:50 | |
| Shrews | corvus: wow, 2 months since the last? can't remember last weekend :) | 15:03 |
| Shrews | thank goodness for 'git log' | 15:03 |
| *** themroc has quit IRC | 15:06 | |
| fungi | doesn't seem that long, but i guess there have been conferences and whatnot in the interim | 15:08 |
| *** hashar has quit IRC | 15:08 | |
| Shrews | fungi: yeah, been a busy couple of months | 15:10 |
| *** hashar has joined #zuul | 15:11 | |
| corvus | oh yeah, we probably don't have to note everything that's happened in 2 months; more interested in what's going on now or has recently happened :) | 15:14 |
| corvus | mostly to give us a chance to resync | 15:14 |
| *** tosky has quit IRC | 15:16 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 15:17 |
| openstackgerrit | Slawek Kaplonski proposed zuul/zuul-jobs master: Add role to fetch journal log from test node https://review.opendev.org/643733 | 15:22 |
| openstackgerrit | Slawek Kaplonski proposed zuul/zuul-jobs master: Add role to fetch journal log from test node https://review.opendev.org/643733 | 15:26 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 15:36 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 16:00 |
| *** flepied has quit IRC | 16:01 | |
| *** gtema has quit IRC | 16:28 | |
| *** manjeets has joined #zuul | 16:53 | |
| *** hashar has quit IRC | 16:58 | |
| *** mattw4 has joined #zuul | 17:00 | |
| aspiers | hi folks, just had an idea for a feature which you'll probably tell me is already implemented | 17:00 |
| aspiers | would be nice if there was a magic string in commit messages which prevented zuul from merging | 17:02 |
| clarkb | aspiers: zuul really relies on the code review system to determine merge validity outside of testing | 17:03 |
| clarkb | so I don't think that is implemented | 17:03 |
| aspiers | IIUC, W-1 is not sticky across patchsets. Even if -2 is sticky, it would still be useful if non-cores were able to achieve the same persistent block for WIP patches, as an extra safety net | 17:03 |
| aspiers | OK, so maybe it's a feature for Gerrit | 17:03 |
| aspiers | probably easily doable via config, or maybe a plugin | 17:04 |
| clarkb | you can make workflow sticky via gerrit settings | 17:06 |
| clarkb | I think we chose not too because drafts weren't sticky and we basically wanted drafts without the privacy aspects | 17:06 |
| fungi | yeah, it's definitely more of a gerrit conversation... in short there is a desire for a sticky way to block merges and am ephemeral one which vanishes on the next patchset without having to manually clear it | 17:16 |
| *** irclogbot_0 has quit IRC | 17:17 | |
| *** irclogbot_0 has joined #zuul | 17:19 | |
| flaper87 | is there a way for me to manually generate the keys for projects and just mount them in the right place? We put our secrets in vault and then mount them where we need them so I'd rather do that than having a volume for the project keys | 17:20 |
| flaper87 | To do that, tho, I have to generate them myself for each project and make sure they exist in the container | 17:20 |
| flaper87 | tobiash: ^ any idea on what the best way to do this is? | 17:20 |
| flaper87 | also, my fetch-output task keeps failing with this error: https://paste.fedoraproject.org/paste/kUmtSmIjqVirZtZ-h3Tcmg | 17:23 |
| flaper87 | it's supposed to be a permission error | 17:24 |
| flaper87 | but if I run the command manually (using the same user the executor is using) it seems to work | 17:24 |
| *** jpena is now known as jpena|off | 17:26 | |
| clarkb | flaper87: zuul won't generate the keys if they exist so yes you can generate them and put them in place. I expect the issue is how to generate them though? | 17:31 |
| flaper87 | clarkb: correct | 17:32 |
| flaper87 | The first generation wouldn't be so much of a problem but rather generating keys for new projects. I'd like this process to be fully automated | 17:32 |
| clarkb | flaper87: I think tobiash uses a master key to seed with. Maybe you can get away with putting that in vault and then let zuul manage the somewhat more ephemeral project keys? | 17:33 |
| flaper87 | mmh, I guess that could work, yeah | 17:37 |
| flaper87 | how would that work if I need to generate secrets for a project? Would I use the master key for that? | 17:38 |
| *** themroc has joined #zuul | 17:39 | |
| clarkb | yes iirc zuul can take a master key which it uses as a seed for the other keys. | 17:39 |
| * clarkb looks at docs | 17:39 | |
| clarkb | hrm grep isn't finding it | 17:40 |
| clarkb | tobiash: ^ do you have those details avialable? | 17:40 |
| flaper87 | cool! I'll try to give this a try today (or just wait for tobiash :) | 17:42 |
| clarkb | digging in zuul/lib/encryption.py I don't see that the generate private key method takes a seed. Did I imagine this feature. I thought tobiash was using it for sure | 17:42 |
| flaper87 | now I need to understand why fetch-output isn't working | 17:43 |
| clarkb | in any case they are 4096 bit rsa keys | 17:43 |
| clarkb | so are easy to generate yourslef if you go that route | 17:43 |
| flaper87 | yeah, I think that will probably be the easiest path. Generate them in advance and then put them in the right place | 17:44 |
| tobiash | clarkb:, flaper87 I'm not at pc atm, I'll be back in an hour or so | 17:44 |
| flaper87 | tobiash: no rush | 17:44 |
| *** electrofelix has quit IRC | 17:49 | |
| tobiash | flaper87: re private keys | 18:26 |
| tobiash | so we have a separate script that runs before zuul startup and before we trigger full reconfigs to pre-populate the private keys based on a master key | 18:26 |
| SpamapS | that seems like a useful thing to have built-in | 18:28 |
| flaper87 | SpamapS: ++ | 18:28 |
| flaper87 | tobiash: and then you use the master key to create secrets for each project, correct? | 18:30 |
| *** jesusaur has quit IRC | 18:30 | |
| tobiash | yes | 18:30 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add script to pre-populate private keys https://review.opendev.org/661824 | 18:31 |
| tobiash | that's the script we use ^ | 18:31 |
| tobiash | SpamapS: yeah, at the dublin ptg we talked about this already but I haven't found time so far to integrate this into zuul itself | 18:32 |
| flaper87 | tobiash: <3 nice, thanks a bunch | 18:32 |
| *** rlandy is now known as rlandy|brb | 18:33 | |
| tobiash | SpamapS: the main hurdle for integration into zuul is (as clarkb already found out) that cryptography has no means to seed private key generation (but pycrypto has) | 18:34 |
| tobiash | and I'd like to avoid having a dependency in zuul to both pycrypto and cryptography | 18:34 |
| fungi | depending on pycrypto is also really not a great idea these days | 18:39 |
| fungi | just in general | 18:39 |
| tobiash | whoops, pycrypto's latest release looks to be from 2013... | 18:43 |
| fungi | yeah, we took a while to get it ripped out of all the openstack projects | 18:44 |
| tobiash | but that's the only lib I found a year ago that supported seeding private key generation with a master key | 18:44 |
| fungi | there were unfixed vulnerabilities in it which sat unaddressed for years because upstream had all but abandoned the lib | 18:44 |
| fungi | pycryptodome attempts to be a drop-in replacement for pycrypto i think, presumably more actively maintained https://pycryptodome.readthedocs.io/ | 18:44 |
| tobiash | yeah, latest release of that is this year | 18:45 |
| flaper87 | tobiash: how do you deal with the logs? Do you upload them from the worker nodes ? or do you sync them in a volume that is mounted in the executor container and then upload them? | 18:46 |
| tobiash | flaper87: I upload them from the executor to swift | 18:47 |
| tobiash | imho for bigger deployments I'd recommend using swift for storing logs | 18:47 |
| tobiash | since we switched to swift last august we never had any issue with logs storage anymore... | 18:49 |
| pabelanger | +1, same | 18:49 |
| tobiash | it was just switching to swift and never ever think about logs again | 18:49 |
| corvus | ++ openstack will someday soon i hope :) | 18:50 |
| tobiash | especially you can also attach a ttl to each file in swift so you also won't have to worry about cleanup | 18:51 |
| tobiash | flaper87: but to be more precise you your actual question, we download the logs to the executor and upload from there | 18:52 |
| fungi | the ttl idea is compelling since you can also indicate in the build metadata what the artifact expiration dates are | 18:53 |
| *** rlandy|brb is now known as rlandy | 18:53 | |
| tobiash | yes, we let the jobs choose via a job variable (with a max enforced by the base job) | 18:53 |
| flaper87 | tobiash: ++ | 18:54 |
| openstackgerrit | Merged zuul/zuul-jobs master: validate-zone-db : add job and make more generic https://review.opendev.org/661138 | 18:58 |
| tobiash | clarkb: do you want to re-review https://review.opendev.org/616306 (resource usage stats)? | 19:09 |
| clarkb | tobiash: I can though running the infra meeting now so will have to be a little bit | 19:10 |
| tobiash | clarkb: no rush | 19:11 |
| *** jlk has quit IRC | 19:15 | |
| *** jlk has joined #zuul | 19:24 | |
| *** pcaruana has quit IRC | 19:30 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Annotate logs around build completion and cancellation https://review.opendev.org/660806 | 19:32 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Annotate logs around build states https://review.opendev.org/661489 | 19:32 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Annotate logs around reporting https://review.opendev.org/661490 | 19:32 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Annotate logs around finished builds https://review.opendev.org/661491 | 19:32 |
| *** themroc has quit IRC | 19:48 | |
| openstackgerrit | Merged openstack-infra/zone-zuul-ci.org master: Add zone-check job https://review.opendev.org/660967 | 20:03 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 20:22 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 20:24 |
| evgenyl | Hi everyone, I have a 3rd party CI (Zuul) for a project hosted on opendev infra, this project already has .zuul.yaml file which contains the configuration for upstream opendev zuul installation. How do I make my 3rd party CI to ignore .zuul.yaml? Is there a way I can specify a custom path to the zuul configuration file (e.g. .zuul_3rd_party.yaml) in the repository? Is there a better, "standard" way to approach multi-zuul gating? | 21:36 |
| *** tosky has joined #zuul | 21:36 | |
| corvus | evgenyl: see include/exclude for the tenant config here: https://zuul-ci.org/docs/zuul/admin/tenants.html#attr-tenant.untrusted-projects.%3Cproject%3E.include | 21:38 |
| corvus | evgenyl: (depending on how they are written, you may be able to include "jobs" and exclude everything else in order to re-use upstream jobs) | 21:39 |
| mattw4 | Hi all, How do I configure a project's .zuul.yaml to use a role (devstack/roles/run-devstack) in a test? | 21:39 |
| corvus | mattw4: .zuul.yaml defines jobs which run playbooks according to the 'run' attribute here: https://zuul-ci.org/docs/zuul/user/config.html#attr-job.run -- those playbooks can use any roles in the current repo or adjacent to the playbook; if the role is in a different repository, you can add that to the job with https://zuul-ci.org/docs/zuul/user/config.html#attr-job.roles -- in the future we hope to directly | 21:46 |
| corvus | support ansible galaxy, but we don't yet. | 21:46 |
| evgenyl | corvus: Does it only support excluding/including the configuration items by type? I will need to define a couple of jobs that would get executed only on 3rd party CI, and it should not execute anything else, the later can probably be achieved with excludes. | 21:49 |
| corvus | evgenyl: yes, only by type. but remember that it's okay to define jobs you don't use, so you could have a mixture of upstream and third-party jobs in the repo (if the upstream project was okay with that), and in the third-party ci you can exclude loading 'project' elements and configure it to run those jobs in your third-party-ci config-project | 21:53 |
| clarkb | I've approved the usage statsd change. Thanks tobiash! There was only minor docs thing I found but otherwise looked great | 21:55 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 21:58 |
| evgenyl | corvus: Oh, so I can just redefine 'project' to run anything I want.. I was also thinking if `nodeset` is something I can use for that? I can define nodepool with a custom set of labels for my 3rd party CI and this would help (?) to make sure that nothing related to upstream jobs/gates/etc gets executed, and I could potentially define 3rd party jobs with these unique labels. | 22:01 |
| mattw4 | Thanks corvus! I've add "roles: zuul/zuul-jobs" to the job definition (run.yaml); can I use the same syntax to add devstack roles? I tried to add another item in the "roles" list like "- openstack: devstack", but get errors saying "extra keys not allowed" | 22:02 |
| clarkb | mattw4: can you share your config with a paste? | 22:02 |
| clarkb | as well as the full error? | 22:02 |
| mattw4 | sure clarkb just a sec.. | 22:02 |
| corvus | evgenyl: yes, in your third-party ci, you can mix-and-match between elements defined upstream and locally however you want. you almost certainly don't want to include the 'project' element from upstream, and instead take complete control of that locally. similarly, you will have local pipeline definitions. you may be able to share nodeset definitions with upstream, as long as you have matching local | 22:06 |
| corvus | nodepool image labels (eg "ubuntu-bionic", etc) | 22:06 |
| evgenyl | corvus: awesome, thank you! | 22:10 |
| mattw4 | corvus: let me know if you need more info: http://paste.openstack.org/show/752193/ | 22:10 |
| corvus | mattw4: i think it should look like this: http://paste.openstack.org/show/752194/ | 22:12 |
| clarkb | https://opendev.org/openstack/tempest/src/branch/master/.zuul.yaml#L13-L14 is an example from tempest | 22:12 |
| corvus | yeah, the canonical host part of that ('opendev.org/') is optional | 22:13 |
| mattw4 | Thanks corvus! That makes a lot of sense | 22:13 |
| mattw4 | and thanks clarkb for the example! | 22:14 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 22:20 |
| openstackgerrit | James E. Blair proposed zuul/zuul-jobs master: WIP: registry test job https://review.opendev.org/661327 | 22:30 |
| mattw4 | corvus, are there any additional configuration items needed to enable those jobs? I'm still seeing the "ERROR! the role 'run-devstack' was not found" in the job trace | 22:35 |
| *** tosky has quit IRC | 23:04 | |
| aspiers | any ideas why https://review.opendev.org/#/c/638680/ didn't check out the requested version of os-traits in the Depends-On? | 23:30 |
| clarkb | aspiers did the jobs install os-traits from pypi because it is a lib? | 23:39 |
| clarkb | jobs have to know to install from source | 23:39 |
| aspiers | clarkb: https://review.opendev.org/#/c/638680/19/.zuul.yaml | 23:39 |
| aspiers | AFAICS it installed from source, just the wrong commit | 23:40 |
| clarkb | aspiers I dont think the legacy jobs work that way | 23:40 |
| clarkb | the zuul native jobs got the magic required projects support but not the legacy ports | 23:40 |
| aspiers | I was just doing what the nova gurus suggested, no idea how this works really :) | 23:40 |
| aspiers | you mean legacy-dsvm-base, whatever that is? | 23:41 |
| aspiers | anyway, gotta sleep - will continue tomorrow. Thanks! | 23:41 |
| *** manjeets has quit IRC | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!