| opendevreview | Takashi Kajinami proposed openstack/openstack-zuul-jobs master: wip: Drop unused horizon/neutron job template https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/974440 | 15:40 |
|---|---|---|
| opendevreview | James E. Blair proposed openstack/project-config master: Simplify Zuul tenant config for providers https://review.opendev.org/c/openstack/project-config/+/974462 | 16:07 |
| -opendevstatus- NOTICE: The Gerrit service on review.opendev.org will be offline momentarily while we restart it onto new images after an update to the build toolchain | 18:01 | |
| *** gmaan is now known as gmaan_afk | 19:39 | |
| timburke | anyone around to help me figure out what i'm doing wrong when trying to rotate dockerhub secrets? (clarkb maybe?) it seems like the secret is getting decrypted, but login fails -- even though running something like `docker login -u screamingfrenzy -p $( cat /tmp/secret )` locally succeeds | 19:53 |
| timburke | for a little more context (including a comment about how i've been generating the encrypted secret), see https://review.opendev.org/c/openstack/swift/+/973990/2/.zuul.yaml | 19:56 |
| fungi | timburke: when running the zuul-client encrypt command, did you specify --tenant=openstack and --project=openstack/swift | 19:56 |
| timburke | yup | 19:56 |
| fungi | ah, looking | 19:56 |
| timburke | i've tried it both with and without a trailing newline in the --infile; neither works | 19:57 |
| fungi | timburke: it's the swift-upload-image post_failure result you're looking at, i guess? | 20:05 |
| fungi | https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/upload-docker-image/tasks/main.yaml#L12 seems to be the command that's failing there | 20:08 |
| fungi | which looks basically like the command you tested locally | 20:08 |
| timburke | yeah, that's what i was assuming | 20:10 |
| timburke | is there a way for me to test round-tripping the encrypt/decrypt with some locally generated keys? are they... rsa, or...? | 20:11 |
| fungi | timburke: it uses the openssl command-line tool like https://opendev.org/zuul/zuul-client/src/branch/master/zuulclient/utils/__init__.py#L85-L88 | 20:15 |
| fungi | and yes it's basically rsa | 20:15 |
| fungi | though if zuul couldn't decrypt it with that project's private key, it would report a configuration error | 20:16 |
| fungi | https://zuul-ci.org/docs/zuul/latest/project-config.html#encryption talks about the feature at length | 20:18 |
| fungi | and also in https://zuul-ci.org/docs/zuul/latest/config/secret.html | 20:31 |
| fungi | it does look like the job was running successfully prior to when we had to rotate all the project keys, so presumably the prior secret value was working fine | 20:37 |
| timburke | so i got my round-trip with something like `PYTHONPATH=../zuul/ python ../zuul/tools/decrypt_secret.py test.pri <( zuul-client --zuul-url https://zuul.opendev.org encrypt --tenant openstack --project openstack/swift --secret-name swift-dockerhub-2026 --infile /tmp/secret --field-name password --public-key test.pub )` | 20:45 |
| timburke | and it looks good | 20:45 |
| timburke | do we need to worry about shell safety in that task, at https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/upload-docker-image/tasks/main.yaml#L12 ? like, if there were " or $ characters, would that cause trouble? | 20:46 |
| fungi | yeah, but like i said, if there were a problem decrypting it with the project's key zuul would have reported a clear decryption failure error on the change too | 20:46 |
| fungi | mmm, good question | 20:47 |
| fungi | the ansible task uses a command module instead of a shell module, so i don't think it runs under an interpreted shell | 20:48 |
| fungi | clarkb: ^ do you happen to know? | 20:48 |
| timburke | hmmm... https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/command_module.html#parameter-expand_argument_vars | 20:51 |
| timburke | defaults to true | 20:51 |
| timburke | i can try adding some backslashes to my secret file... | 20:52 |
| fungi | ah, so it could be, yep | 20:52 |
| fungi | alternatively, adjust the password to not include things that might get expanded, if possible | 20:53 |
| fungi | also seems unlikely that it would find a match for an actual variable name that way, but not impossible i guess | 20:57 |
| timburke | but not finding a match would drop in an empty string, so no longer the right password. might explain everything! fingers crossed for the new patchset | 20:58 |
| fungi | "If a variable is not matched, it is left unchanged, unlike shell substitution which would remove it." (from the documentation you linked) | 20:59 |
| clarkb | fungi: correct command is like a direct fork and exec without a shell | 21:08 |
| fungi | but apparently it will perform *some* variable substitution | 21:09 |
| fungi | though re-reading the description, it looks like the entire argument would have to match an envvar | 21:09 |
| clarkb | this command is all ansible vars https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/upload-docker-image/tasks/main.yaml#L12 | 21:09 |
| fungi | oh, good point, that's what they're talking about i guess, those aren't jinja2 substitutions, those are the vars being substituted | 21:11 |
| fungi | though the command module documentation talks about also replacing arguments like $HOME | 21:11 |
| clarkb | ah it must do that on the ansible side then | 21:12 |
| fungi | but only starting with ansible 2.16 apparently | 21:13 |
| timburke | it merged! apparently i *did* need to sprinkle in some backslashes! | 23:17 |
| timburke | lol! so *upload* worked, but *promote* now failed! | 23:22 |
| timburke | i guess because *that* job goes and POSTs a json body using a uri module... which will have different quoting rules or something? https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/promote-docker-image/tasks/main.yaml#L13-L18 | 23:25 |
| clarkb | timburke: where di you change the quoting? | 23:34 |
| clarkb | in the secret value itself? | 23:34 |
| timburke | yeah, in the secret file | 23:42 |
| clarkb | hrm I'm not sure I understand why that would help | 23:47 |
| clarkb | the docker secret is just a random string password type value | 23:48 |
| clarkb | but yes I suspect that the escapes you added are now not being interpreted by the http request properly | 23:48 |
| clarkb | timburke: is the problem that the string password include characters that shells want to interpret? | 23:50 |
| clarkb | if so I like to use `pwgen -s 32 1` or similar which should get you a securely generated random string without that problem | 23:50 |
| clarkb | but other password manager tools can usually be configured to avoid (or include) sepcific characters | 23:51 |
| clarkb | another option may be to change the quoting in the docker login task | 23:56 |
| clarkb | so that things are escaped at that level (maybe weith ansible raw tags?) | 23:56 |
| timburke | yeah, it had a few different things -- ", $, ! -- i pinged the guy that owns the email for the dockerhub user to ask him to change the password again, this time to something more ansible/json-friendly | 23:57 |
| clarkb | I think in this case we may need a way to share the secret value (possibly by resetting it to another value so the old one can be exposed) and then we can work through what the best/proper way to make that safe would be. That said I expect with the currnet code using a value from something like `pwgen -s 32 1` would work | 23:57 |
| clarkb | timburke: note you want to rotate the secret anyway as part of the decryption fix | 23:58 |
| timburke | 👍 | 23:58 |
| timburke | yeah, we'd already rotated it once -- i remember you advising we do that *first*, rather than just re-encrypt the old secret with the new key | 23:59 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!