| gouthamr | https://bugs.launchpad.net/keystone/+bug/2153453 is now public | 14:34 |
|---|---|---|
| gouthamr | https://bugs.launchpad.net/keystone/+bug/2158538 is now public | 14:34 |
| xek | o/ | 14:35 |
| gouthamr | hey xek, ty for working on these ^ watching #openstack-keystone for the patches and prepping the OSSA in parallel | 14:39 |
| xek | gouthamr ack | 14:39 |
| fungi | i'm still on hand to review the ossa repo change( for the advisory/advisories | 15:04 |
| opendevreview | Goutham Pacha Ravi proposed openstack/ossa master: Add OSSA-2026-037 https://review.opendev.org/c/openstack/ossa/+/1002324 | 15:22 |
| gouthamr | ty fungi... xek: ^ please fact check | 15:22 |
| fungi | on it | 15:22 |
| gouthamr | gah, noticed something i discussed with xek but didn't add here | 15:25 |
| gouthamr | fixing | 15:25 |
| fungi | just a heads up that the second stable/2025.1 patch is flagged as in merge-conflict with the branch by gerrit | 15:28 |
| gouthamr | yes, xek, that would be something folks would ask for ^ | 15:29 |
| fungi | i was going to bring it up in #openstack-keystone but don't see xek in that channel | 15:29 |
| xek | looking | 15:29 |
| opendevreview | Goutham Pacha Ravi proposed openstack/ossa master: Add OSSA-2026-037 https://review.opendev.org/c/openstack/ossa/+/1002324 | 15:30 |
| fungi | xek: https://review.opendev.org/c/openstack/keystone/+/1002308 specifically | 15:31 |
| fungi | time check, we're an hour past the indicated publication time, so if what's in 1002324 is correct enough to secure systems we can always make further clarifications about which bug was which in a future revision | 16:00 |
| xek | please just add the missing review links | 16:05 |
| gouthamr | ah, | 16:05 |
| fungi | are the bugs not fixed in master without those? one is even wip | 16:06 |
| gouthamr | (yes, i was commenting on gerrit with the same question) | 16:07 |
| fungi | further work to future-proof the software is out of scope for the advisory, which is the list of fixes users need to deploy right now to address the risks described | 16:08 |
| xek | what do you mean by future-proof? | 16:09 |
| fungi | xek: let me rephrase: do users need to apply those patches you listed on master *right now* to avoid the described vulnerabilities being exploited by their users? | 16:10 |
| fungi | that is the purpose of a security advisory | 16:10 |
| xek | yes | 16:11 |
| fungi | so why aren't they needed on stable branches? | 16:11 |
| xek | which ones? they are all needed on stable branches, the only discussion is that those restrict some operations which weren't restricted before and thus may make something that was permitted before not work | 16:12 |
| xek | like using application credentials in one scope to create ec2 credentials in another scope | 16:13 |
| fungi | you listed patches which weren't sent to downstream stakeholders | 16:14 |
| xek | or log in with an ec2 credential meant for swift, have all keystone api access including changing the project_id in the ec2 credential you logged in, changing it's scope, having access to swift objects from another project, etc. | 16:14 |
| fungi | it seems like we're misaligned on how security advisories work and what it is we're publishing | 16:14 |
| xek | yes, because while I was on PTO someone made a decision to remove part of my work | 16:15 |
| fungi | this should have just all been done in public, it's way to complicated to fix in secret, but that's not an argument we have time for right now | 16:15 |
| gouthamr | https://review.opendev.org/c/openstack/keystone/+/1002330 is advisory worthy. If your patches are ready, we can use the public bug workflow to advisory it right after? | 16:16 |
| xek | I agree, it's way easier, I said this before, but it feels like re-developing these features, the bugs go deep | 16:16 |
| fungi | redeveloping features is definitely out of scope for our advisory process. openstack security advisories are strictly for backportable backward-compatible fixes that can be patched entirely in code with no action required by the operator to secure their systems other than applying the patches | 16:20 |
| xek | so advisories would be only for simple changes? is the complication of the remediation the bar on whether we create a security advisory? | 16:21 |
| fungi | that's a very big part of the determination, yes | 16:22 |
| fungi | some things can't be fixed through code alone in a way that's backportable for existing deployments without the operator needing to take additional steps, and we have a separate publication workflow for those more complicated situations | 16:24 |
| xek | I'm trying to determine what's left if we don't include the ec2 ban from keystone API and https://review.opendev.org/c/openstack/keystone/+/1002330 for the current advisory... | 16:24 |
| xek | you don't argue against including https://review.opendev.org/c/openstack/keystone/+/1002289 ? | 16:26 |
| gouthamr | no i'm having second thoughts there | 16:26 |
| gouthamr | explain that to me please, 1002301 makes it so that empty methods are considered "delegated". | 16:27 |
| gouthamr | (1002301 is in the advisory) | 16:27 |
| fungi | what bug does change 1002289 fix? the change doesn't seem to include any bug reference in its commit message | 16:29 |
| gouthamr | i think it fixes "ec2credential" - and that's the OSSN we're referring to | 16:30 |
| gouthamr | i can prep the OSSN in parallel and mark it WIP | 16:30 |
| fungi | reminder, we're now 1.5 hours past when we told downstream stakeholders it would be okay to discuss this in public, and we don't have an advisory published about anything yet | 16:31 |
| gouthamr | yes, i vote on retaining the existing advisory as is. I'll make the small correction on "independently reported". | 16:31 |
| opendevreview | Goutham Pacha Ravi proposed openstack/ossa master: Add OSSA-2026-037 https://review.opendev.org/c/openstack/ossa/+/1002324 | 16:33 |
| fungi | lgtm | 16:34 |
| gouthamr | we're racing in comments | 16:43 |
| gouthamr | if i'm missing something, flag it please | 16:43 |
| gouthamr | i asked if you wanted the "ec2credential" part any more obvious than I already stated.. but, i'd not.. i'm hoping the OSSN for that can go out in a couple days | 16:44 |
| xek | ok, so if we go without https://review.opendev.org/c/openstack/keystone/+/1002289, we also go without https://review.opendev.org/c/openstack/keystone/+/1002082 because it doesn't work without it | 16:45 |
| gouthamr | xek: both of those are precisely the OSSN content? | 16:46 |
| xek | but still, without the 3rd patch, ec2 api can still be used to create long-lived credentials | 16:47 |
| xek | in another project | 16:47 |
| gouthamr | its not me/VMT that are holding the line on that - our opinion is to fix that up publicly (as you're doing) and get reviews and supply an OSSN to operators | 16:47 |
| gouthamr | yes | 16:47 |
| gouthamr | and that's a known gap - we've indicated that to operators | 16:48 |
| xek | right, I'm only saying the OSSA shouldn't say that it fixes that | 16:48 |
| xek | ok, let's go with this, I see your comment | 16:49 |
| fungi | doesn't the notes section of the advisory cover it? | 16:49 |
| gouthamr | it does imo | 16:50 |
| fungi | right, at least that seemed to be the intent of including that note | 16:51 |
| fungi | if it doesn't accurately capture it, we can update via errata later | 16:51 |
| fungi | i've approved the advisory now that it has xek's +1 | 16:52 |
| gouthamr | ty! working on the emails now | 16:52 |
| xek | I +1 | 16:53 |
| xek | we should have really drafted this together on the bug beforehand | 16:54 |
| fungi | i recommend sending those out asap while zuul does its thing to get the site updates in parallel | 16:54 |
| fungi | xek: yes, normally we draft the prose for at least the impact description, affected versions, et cetera in comments on the bug when it's being done under private embargo | 16:55 |
| fungi | to give the project maintainers an opportunity to catch misstatements before publication day | 16:55 |
| gouthamr | omergod, rendering sucks | 16:59 |
| opendevreview | Goutham Pacha Ravi proposed openstack/ossa master: Add OSSA-2026-037 https://review.opendev.org/c/openstack/ossa/+/1002324 | 17:00 |
| gouthamr | my bad on that one, a teeny yaml mistake | 17:00 |
| fungi | k | 17:00 |
| gouthamr | i've the emails prepped if you can push this back to the gate :( | 17:00 |
| fungi | reapproved | 17:00 |
| gouthamr | ++ ty | 17:01 |
| fungi | the e-mails can go out now even if the site isn't updated yet, since they don't refer to one another | 17:01 |
| gouthamr | ++emails sent | 17:06 |
| fungi | accepted the one for openstack-announce | 17:12 |
| gouthamr | thanks fungi | 17:12 |
| gouthamr | https://bugs.launchpad.net/keystone/+bug/2159643 is now public | 17:19 |
| opendevreview | Merged openstack/ossa master: Add OSSA-2026-037 https://review.opendev.org/c/openstack/ossa/+/1002324 | 18:11 |
| xek | fungi, gouthamr, on the complexity thing - to date, we were trying to merge security issues, to release those in larger batches, since it was easier to process and test them together, avoiding merge conflicts, but we could also decide the other way - splitting the issues, like I did with https://bugs.launchpad.net/keystone/+bug/2159643 just now, and even this issue can be split further, since parts of it were discovered independently in | 19:04 |
| xek | https://bugs.launchpad.net/keystone/+bug/2158931 | 19:04 |
| fungi | yes, part of the challenge too is that people kept putting comments in some private bugs about other also private bugs, and we were stuck being able to make any of them public until all of them were public | 19:05 |
| gouthamr | okay, 2158931 is private.. this channel is publicly logged. Are you okay with me opening it up? | 19:05 |
| xek | yes, I already poked you out of band, it contains the same findings | 19:07 |
| xek | and it was already referenced in the opened bug | 19:07 |
| gouthamr | ack | 19:07 |
| gouthamr | https://bugs.launchpad.net/keystone/+bug/2158931 is now public | 19:07 |
| gouthamr | fungi xek: does it make sense to have a new LP for turning off EC2 credentials? | 19:27 |
| gouthamr | it will make it easier to lump all the related work there, and, i can reference all of this in the OSSN to tie the story together | 19:28 |
| xek | yes, maybe not just ec2, also application credentials and maybe others | 19:28 |
| gouthamr | you mean token rescoping with the others? ec2 has a whole host of problems to warrant its own? | 19:29 |
| fungi | as long as it's public, sounds like a fine idea to me, however is needed to best organize what's being done | 19:30 |
| fungi | i think it's up to the keystone maintainers how they want to track that work, whether bug or blueprint or spec or just in the change commit messages | 19:30 |
| gouthamr | yeah, public - folks are already aware now | 19:30 |
| fungi | e.g. with the help of something lightweight like a consistent gerrit change hashtag | 19:31 |
| gouthamr | i agree.. but, i wanted to tie up loose ends here. I informed MITRE about the publication and the gap we've left | 19:31 |
| fungi | thanks! | 19:31 |
| gouthamr | whatever you do to "ec2credential", don't do it silently | 19:31 |
| gouthamr | xek: gtema, keystone-cores.. keep us in the loop, so we'll do the OSSN as promised in the advisory and let operators close the holes we've currently pointed at | 19:32 |
| xek | oh, I already proposed it for appcreds https://review.opendev.org/c/openstack/keystone/+/987160 | 19:32 |
| xek | for the advisory, the policy based approach of disabling ec2 should work | 19:34 |
| xek | https://bugs.launchpad.net/keystone/+bug/2153453/comments/44 | 19:35 |
| xek | the issue is, I see people in the comments asking how to keep using ec2 credentials, while fixing the vulnerability | 19:36 |
| fungi | seems like the answer to that is pretty simple: you can't | 19:39 |
| xek | https://review.opendev.org/c/openstack/keystone/+/1002330 partially disables the problematic APIs, making it safe AFAIK | 19:46 |
| fungi | ah, that. asking in which comments? | 19:50 |
| xek | https://bugs.launchpad.net/keystone/+bug/2153453/comments/49 - the second point at the end | 19:52 |
| gouthamr | i think you responded that "EC2 tokens (for S3 supoprt in Swift)" is unaffected | 19:52 |
| gouthamr | the same tokens shouldn't be used to generate new auth in keystone, and that's the gap to be addressed and recommended to be enforced via an OSSN | 19:53 |
| xek | yes, none of the changes block that use case, the discussion was only around whether there is some potential operator that uses outside this usecase | 19:55 |
| xek | one such use case was ec2 tokens rotation, replacing the blobs with PATCH, that's why it's carved out in the above change | 19:55 |
| gouthamr | good stuff | 19:55 |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0109: EC2-derived tokens retain full privileges https://review.opendev.org/c/openstack/security-doc/+/1002392 | 20:58 |
| xek | gouthamrfungidmendiza ^ | 20:59 |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0109: EC2-derived tokens retain full privileges https://review.opendev.org/c/openstack/security-doc/+/1002392 | 21:25 |
| gouthamr | thanks for working on it xek | 21:26 |
| gouthamr | https://bugs.launchpad.net/keystone/+bug/2153447 is now public | 21:26 |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0109: EC2-derived tokens retain full privileges https://review.opendev.org/c/openstack/security-doc/+/1002392 | 21:26 |
| gouthamr | https://bugs.launchpad.net/keystone/+bug/2154645 is now public | 21:30 |
| opendevreview | Goutham Pacha Ravi proposed openstack/ossa master: OSSA-2026-037 Errata 1 https://review.opendev.org/c/openstack/ossa/+/1002403 | 22:11 |
| gouthamr | fungi: JayF: still around to look at this ^? | 22:14 |
| fungi | yep, reviewing now | 22:20 |
| fungi | gouthamr: quick note, you added the errata history section but not the errata section | 22:22 |
| gouthamr | ! | 22:22 |
| fungi | otherwise lgtm | 22:22 |
| opendevreview | Goutham Pacha Ravi proposed openstack/ossa master: OSSA-2026-037 Errata 1 https://review.opendev.org/c/openstack/ossa/+/1002403 | 22:23 |
| gouthamr | i _think_ Red Hat has managed to assign duplicate CVEs.. i'll confirm with them with less-tired eyes | 22:24 |
| gouthamr | or maybe xek knows what the confusion is there.. we'll get it resolved either case | 22:24 |
| fungi | that does sometimes happen, but also sometimes what you have is a cve for the upstream bug and then a cve for the bug's impact downstream in the distro | 22:25 |
| fungi | though the latter is unusual and only ideally when the downstream impact is not as described upstream for downstream-specific reasons | 22:25 |
| fungi | e.g. additional patches for downstream-only behaviors due to other alterations they're carrying in their fork | 22:26 |
| gouthamr | ah possible | 22:27 |
| gouthamr | ty for flagging that possibility :) i'd've not dug down that road | 22:27 |
| gouthamr | zuul is quite busy, i'll send the emails and watch this like a hawk | 22:32 |
| * gouthamr sent | 22:32 | |
| fungi | accepted to openstack-announce | 22:38 |
| JayF | Late +2 on that | 22:39 |
| opendevreview | Merged openstack/ossa master: OSSA-2026-037 Errata 1 https://review.opendev.org/c/openstack/ossa/+/1002403 | 22:45 |
| gouthamr | \o/ | 22:49 |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0110: Self-service password change does not revoke generators https://review.opendev.org/c/openstack/security-doc/+/1002410 | 23:23 |
| *** mrunge_ is now known as mrunge | 23:24 | |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0110: Self-service password change does not revoke generators https://review.opendev.org/c/openstack/security-doc/+/1002410 | 23:31 |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0110: Self-service password change does not revoke generators https://review.opendev.org/c/openstack/security-doc/+/1002410 | 23:38 |
| opendevreview | Grzegorz Grasza proposed openstack/security-doc master: OSSN-0110: Self-service password change does not revoke generators https://review.opendev.org/c/openstack/security-doc/+/1002410 | 23:41 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!