| opendevreview | Ian Wienand proposed openstack/project-config master: project-config-grafana: filter opendev-buildset-registry https://review.opendev.org/c/openstack/project-config/+/847870 | 03:44 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/project-config master: Add job to ansible-config_template to Galaxy https://review.opendev.org/c/openstack/project-config/+/881930 | 21:48 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/project-config master: Add job to ansible-config_template to Galaxy https://review.opendev.org/c/openstack/project-config/+/881930 | 21:49 |
| fungi | noonedeadpunk: for 881930 i think the job itself needs to be defined in project-config, not merely its addition to the pipeline. if the secret is going to be shared by uploads for multiple projects, it will need to be committed to the project-config repo too | 21:56 |
| noonedeadpunk | fungi: as long as project-config is trusted repo - this should not be needed? | 21:57 |
| noonedeadpunk | At least that kind of flow works just nicely downstream... | 21:57 |
| fungi | is the secret going to be used for multiple projects? | 21:57 |
| fungi | clarkb: ianw: ^ maybe i'm misreading that | 21:58 |
| noonedeadpunk | Well, yes, it's possible, but only when job using some foreign secret is added to another project in trusted one | 21:58 |
| noonedeadpunk | So downstream we have a secrets defined for each region with access to uploading images | 21:59 |
| fungi | is a region like a project then? | 21:59 |
| noonedeadpunk | yup | 21:59 |
| noonedeadpunk | And then we do have a repo with set of ansible stuff that does execute upload | 21:59 |
| fungi | okay, so not sharing the same secret, each project has a separate secret and is supplying it via pass-to-parent? | 22:00 |
| noonedeadpunk | So in trusted project it was enough to say like that https://paste.openstack.org/show/b03x50jsTxWyXkuP2ztR/ | 22:00 |
| noonedeadpunk | Parent is base for these jobs | 22:01 |
| noonedeadpunk | so technically they're not childs | 22:01 |
| clarkb | I don't think we should manage that in project-config thats a lot of reviews we have to do that shouldn't be necessary | 22:01 |
| noonedeadpunk | or well, they're childs of same parent but not to each other | 22:01 |
| fungi | i guess i'm unclear on what this is working around | 22:01 |
| clarkb | ya I don't think it is necessary and creates overhead for project-config reviewers | 22:02 |
| noonedeadpunk | Well, I kinda don't see any way then on how to use `openstack` namespace that is created in Ansible galaxy | 22:02 |
| noonedeadpunk | As right now this access is hold by zuul and 1 individual | 22:03 |
| fungi | the commit message indicates that the ansible-collections-openstack project contains a publication secret you're planning to use in jobs run for other projects | 22:03 |
| fungi | which you're saying isn't true? | 22:03 |
| noonedeadpunk | the way to work around through zuul is to define job that uses a secret to project that you want to publicize | 22:03 |
| noonedeadpunk | So I'm reffering to 2nd paragraph of https://zuul-ci.org/docs/zuul/latest/config/job.html#attr-job.allowed-projects basically | 22:05 |
| noonedeadpunk | I didn't say it's not true? did I? | 22:06 |
| clarkb | the secret needs to be in the same repo as the job which uses it | 22:08 |
| fungi | i asked "is a region like a project then?" and you said "yup" which i took to mean you were saying you had a different secret in each project | 22:08 |
| noonedeadpunk | Oh, yes, dowstream each project has own secret | 22:09 |
| fungi | i had forgotten about the possibility of adding a secrets-using job from an untrusted project to a pipeline for another project from a config repo, i don't think we've leveraged that exception before | 22:10 |
| noonedeadpunk | And then we add all jobs that have their own secrets to image-manager repo through trusted repo | 22:10 |
| clarkb | if each downstream repo has its own secret I don't think you need to define those jobs in project-config. You might define a single parent job for ease of use there but that wouldn't go in the pipeline config | 22:10 |
| noonedeadpunk | clarkb: I was explaining that such concept worked downstream | 22:10 |
| noonedeadpunk | that you technically can in zuul to achieve that | 22:11 |
| clarkb | whether or not it works I don't want to create extra work for project-config reviewers which I think this does? | 22:11 |
| clarkb | as we would need to approve any changes to these jobs in every repo that wants to use them | 22:11 |
| noonedeadpunk | but in current scenario only ansible-collections-openstack contain secret with access to the galaxy | 22:11 |
| fungi | it sounds like the "downstream" example was dissimilar to this case | 22:11 |
| fungi | in that this case does actually need to share a secret between builds for multiple projects | 22:12 |
| noonedeadpunk | So then either I need somehow to force SIG folks to create or re-encrypt secret for each other project we'd need | 22:12 |
| noonedeadpunk | or do that through trusted project in zuul | 22:12 |
| noonedeadpunk | or give up and create another `openstack-alt` namespace in ansible-galaxy | 22:13 |
| noonedeadpunk | each option is annoying to a degree | 22:13 |
| clarkb | right I think it comes down to management of the credentials. I think you have said it is already a different credential for every project os I don't think it is any extra work on the projects to manage this in their own trees? | 22:13 |
| noonedeadpunk | clarkb: um, no, it's not different. It should be the same | 22:13 |
| noonedeadpunk | ideally | 22:13 |
| fungi | sounds like the "downstream" example used different credentials per project, but in this case it's multiple untrusted projects wantig to reuse a single credential defined in another untrusted project | 22:14 |
| clarkb | in that case I think you can define the secret and the job(s) in project-config but not the pipelines. The projects would manage the pipelines directly | 22:14 |
| noonedeadpunk | so sig folks was kinda in agreement to re-encrypt secret for this specific repo, but failed to do that in a year (and likely already forgot) | 22:14 |
| clarkb | that reduces project-config overhead | 22:14 |
| noonedeadpunk | I don't have the value of the secret, just in case. Also adding jobs was rejected already | 22:16 |
| clarkb | ok ew may need to start over and describe what we are trying to accomplish here and ignore any downstream work or historical items | 22:16 |
| fungi | i think the "rejection" in 850664 was that the bulk of the job could be generalized somewhere, and then a child of that job in project-config would supply the secret | 22:17 |
| clarkb | because I'm really confused if you don't have the secret then how do we even start | 22:17 |
| noonedeadpunk | clarkb: so secret and job are here https://opendev.org/openstack/ansible-collections-openstack/src/branch/master/.zuul.yaml#L285-L306 | 22:17 |
| noonedeadpunk | there are 2 ways to add same/simmilar job elsewhere: 1. ask the person who has created the secret to re-encrypt it for any other project and create child jobs. 2. add job to the required project through trusted repo | 22:19 |
| fungi | yeah, an example of the case clarkb is describing is the github mirroring job for openstack projects. there's one credential for the openstack org on github, and it's defined as a secret in the project-config repo but the job in project-config which uses that secret runs ansible that's defined in an untrusted repo | 22:19 |
| noonedeadpunk | And in 850664 I think I got slightly confused about generalizing and overhead that was required to implement that... | 22:20 |
| noonedeadpunk | And the way on how to really test that | 22:20 |
| noonedeadpunk | Also "owner" of the secret refused to send it to project-config back then | 22:21 |
| clarkb | so there isn't anything we can do about who owns/knows a secret (and that is by design) | 22:21 |
| clarkb | I think adding the secret and job that uses it in project-config is fine. But we should avoid adding unnecessary config to project-config just because the job and secret are there | 22:22 |
| noonedeadpunk | But it still can be added to another project with job in trudted-repo :) | 22:22 |
| fungi | was there a reason for the refusal? if it's that the project-config maintainers could decrypt the secret, well most of them are also the zuul sysadmins and could technically do it anyway | 22:22 |
| noonedeadpunk | Well, I'm not sure to be frank... Maybe that or maybe they wanted to have more control over waht will get published to the galaxy as "openstack" - there wasn't anything clear stated | 22:23 |
| clarkb | fungi: you cannot decrypt fwiw | 22:24 |
| clarkb | I guess you can write a zuul change that does it for you in proxy though | 22:25 |
| fungi | well, by policy we state that we won't decrypt it | 22:25 |
| fungi | but the project keys are in zk | 22:26 |
| clarkb | right I was trying to say that being a core reviewer doesn't give you direct decrypt abilities | 22:26 |
| noonedeadpunk | Yeah, while technically it's possible I'm not sure it's worth it... | 22:26 |
| fungi | if you mean solely by virtue of being a project-config core reviewer (not a sysadmin), then yes i assume "decrypt" in this case means merge a change to a job that leaks or exfiltrates it | 22:26 |
| clarkb | you still need to push and land/review a chnage to have zuul do it for you | 22:27 |
| fungi | correct | 22:27 |
| clarkb | which has a papertrail | 22:27 |
| fungi | yes, it's auditable, but that's not the same thing as impossible | 22:27 |
| fungi | anyway, i guess we're veering off-topic | 22:28 |
| noonedeadpunk | or unless original job does smth stupid that discloses the secret... | 22:28 |
| clarkb | but ya basically your option is to reencrypt it and make new jobs everywhere (possibly just stubs of jobs that pass to parent). Or centralize a job in project-config with a single secret | 22:29 |
| clarkb | we cannot control what the people who know the secret wants to do | 22:29 |
| fungi | for the ansible galaxy openstack region publishing, it seems like 1. need to know what can/should be done to be able to reencrypt the secret, 2. revisit ianw's "rejection" in 850664 since i don't think he was saying not to do what we already do for other secrets and shared jobs in the project-config repo | 22:30 |
| noonedeadpunk | iirc it's sshnaidm who owns that | 22:31 |
| clarkb | noonedeadpunk: owns the secret? | 22:31 |
| noonedeadpunk | yup | 22:31 |
| sshnaidm | secret of ..? | 22:32 |
| fungi | ansible galaxy openstack namespace | 22:32 |
| noonedeadpunk | openstack account in ansible galaxy | 22:32 |
| sshnaidm | ah, yes | 22:32 |
| noonedeadpunk | yes | 22:32 |
| sshnaidm | it's also in publishing job as zuul secret | 22:32 |
| noonedeadpunk | sshnaidm: but it's only in ansible-collection-openstack still, right? | 22:33 |
| sshnaidm | I think so | 22:33 |
| noonedeadpunk | I just returned back to oooold discussion about adding another thing there | 22:33 |
| sshnaidm | why? does someone need it? | 22:33 |
| fungi | the question was about encrypting a version of it for the openstack/project-config repo so that it could be used by the publication job when run for other openstackansible projects | 22:33 |
| noonedeadpunk | And found easier way (for me:)) but seems it makes load for infra :( | 22:34 |
| fungi | rather than only for the ansible-collection-openstack project | 22:34 |
| sshnaidm | yeah, I think we talked about it a few PTGs back :) | 22:34 |
| noonedeadpunk | We didn't have much changes since then in the module, and now it's time for releasing another version :) | 22:35 |
| fungi | noonedeadpunk: alternatively, you could create a new namespace in galaxy with a different account and use that to publish the ansibleopenstack projects | 22:36 |
| noonedeadpunk | yeah, I know. | 22:36 |
| fungi | separate from the one ansible-collection-openstack uses | 22:36 |
| noonedeadpunk | and name it `openstack-alt` to make things confusing... | 22:36 |
| fungi | ansible-collection-openstack is client stuff, not deployment tooling, right? | 22:37 |
| sshnaidm | yes | 22:37 |
| sshnaidm | day-2 | 22:37 |
| noonedeadpunk | so you have like layering there. openstack.cloud are openstack modules | 22:37 |
| noonedeadpunk | but openstack.anything can be `anything` | 22:37 |
| noonedeadpunk | ie openstack.openstack-ansible.some_module | 22:38 |
| sshnaidm | openstack.deploy.xxxx | 22:38 |
| noonedeadpunk | or that | 22:38 |
| noonedeadpunk | (could be tricky with kolla though) | 22:39 |
| noonedeadpunk | but yes, I can create openstack-ansible.deploy as well, dublicate publishing jobs, etc... | 22:40 |
| noonedeadpunk | or jsut disregard that for another couple of years... | 22:40 |
| sshnaidm | btw I use a different user "collectionspublisher" for publishing collection from CI | 22:41 |
| sshnaidm | with his API key | 22:42 |
| sshnaidm | so it's more complicated.. | 22:42 |
| fungi | so to reset for a moment... what we would normally expect here is a generic "publish to galaxy" job in the zuul/zuul-jobs repo, an openstack-specific job in openstack/project-config which inherits from that and does pass-to-parent of a secret also defined in openstack/project-config, then projects adding that job to their individual projects | 22:42 |
| noonedeadpunk | ok, let me follow this way and propose role for zuul-jobs | 22:43 |
| sshnaidm | secret should be different in each repo, isn't it? | 22:43 |
| noonedeadpunk | unless it's a) defined in trusted repo b) job with secret added to another project in trusted repo | 22:44 |
| fungi | sshnaidm: if the secret is in openstack/project-config and used by a job defined in openstack/project-config then it can run in pipelines for individual projects without needing separate copies of the secret | 22:44 |
| sshnaidm | secret is actually API key for Galaxy, firstly it can change, secondly - it's for a specific user only | 22:44 |
| noonedeadpunk | sshnaidm: so I tried to push that, which technically should work https://review.opendev.org/c/openstack/project-config/+/881930 but wil lraise overhead for infra team reviewing such things | 22:44 |
| fungi | care just needs to be taken not to leak the secret to parts of the build which are under control of those projects | 22:45 |
| noonedeadpunk | (also not sure why it failed linters - failure looks weird kinda) | 22:45 |
| fungi | hopefully the "publish to galaxy" step which uses the secret doesn't execute arbitrary/untrusted code from the individual projects | 22:45 |
| sshnaidm | but still, which secret you want to put there - of which user? In galaxy they have specific secrets for each user | 22:46 |
| noonedeadpunk | I think it should be same zuul-ci or smth | 22:46 |
| noonedeadpunk | `collectionspublisher` - whatever I guess? | 22:46 |
| sshnaidm | and login can be via github account only, so this user should have a separate github acc | 22:46 |
| noonedeadpunk | but yes, user will be same for all projects | 22:46 |
| clarkb | so there are two approaches that can be taken. A shared account with jobs constructed such that it relies on info supplied from gerrit/zuul to publish to the correct location. Or individual accounts and you can customise locations and are limited by account access | 22:47 |
| sshnaidm | I had to create a gmail, then to register user in github with this gmail, then to exclude this user from blocked because github thought it's a bot, then to register this user to galaxy :D | 22:47 |
| clarkb | both approaches are taken. The recent quay.io publishing work is an example of the second option | 22:48 |
| clarkb | I think the issue here is that sshnaidm has a dedicated account to this publishing to "openstack" | 22:48 |
| clarkb | "openstack" is a community resource but the galaxy publication is not | 22:49 |
| sshnaidm | it's not an issue, I just didn't want to put my private key | 22:49 |
| clarkb | sshnaidm: and you would add an openstack managed account to be able to publish there? | 22:49 |
| sshnaidm | clarkb, of course | 22:49 |
| sshnaidm | if you have one - I add you to "admins" there and you can publish with it | 22:49 |
| clarkb | I don't and have little interest myself. but noonedeadpunk may want to work that out with interested parties | 22:50 |
| sshnaidm | and I think I was asking for that once :) | 22:50 |
| sshnaidm | yeah, and I don't think noonedeadpunk wants his api key to be for general use.. | 22:50 |
| sshnaidm | better everyone has their own keys in their projects | 22:51 |
| noonedeadpunk | If it requires github account, it might be worth to use openstack org one? | 22:51 |
| sshnaidm | openstack is a user? | 22:51 |
| noonedeadpunk | Well, we have organisation there with some CI-friendly user I believe | 22:52 |
| noonedeadpunk | that's used for mirroring? | 22:52 |
| clarkb | I don't know how that is set up but presumably something exists | 22:54 |
| clarkb | I think the TC manages that? | 22:54 |
| fungi | https://opendev.org/openstack/project-config/src/branch/master/zuul.d/secrets.yaml#L642-L730 | 22:54 |
| fungi | there's an ssh key and an api token defined as part of that secret | 22:55 |
| fungi | one is used for pushing commits to github mirrors, the other for creating new repositories in the openstack org on github | 22:55 |
| fungi | and yes, i believe tc-members are handling the account associated with that secret, but it would be a good thing to confirm | 22:56 |
| * noonedeadpunk not aware of this yet | 22:56 | |
| noonedeadpunk | But will check on that | 22:56 |
| noonedeadpunk | And will also go on with adding role to zuul/zuul-jobs. Even if secrets will be handled by projects individually - that's still worth having same parent job | 22:58 |
| noonedeadpunk | Will abandon https://review.opendev.org/c/openstack/project-config/+/881930 then | 22:58 |
| noonedeadpunk | thanks for the talk and your time! | 22:58 |
| * noonedeadpunk already 1am here so signing out | 22:59 | |
| fungi | you're welcome, and have a good night! | 23:04 |
| ianw | right, just to confirm, my issue with 850664 was what fungi said above -- a generic-ish publishing role should be in zuul-jobs, then we use that from project-config from secrets. rather than putting too much logic into project-config. | 23:22 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!