| stephenfin | fungi: clarkb: If I wanted to do a wholesale import of OSC into SDK, how could I do that while preserving history *through Gerrit*? | 11:08 |
|---|---|---|
| stephenfin | I'm aware I want to use subtree merge (I've done this before, but not in a long time) but presumably that's going to generate a change for every commit ever made to OSC | 11:09 |
| *** tkajinam is now known as Guest9129 | 13:39 | |
| fungi | stephenfin: i think we would use administrative permissions to force push the updated state of the repository, which would then preserve the old change references as-is, they'd just point to the (subsequently retired) old repo | 14:36 |
| jamespage | o/ | 14:37 |
| fungi | what's up jamespage? | 14:37 |
| jamespage | hey fungi | 14:37 |
| jamespage | just prepping downstream in Ubuntu for upcoming milestones - there are some new oslo module versions that don't seem to have published to tarballs.openstack.org (at least the tarball has not been published) | 14:38 |
| jamespage | seem to be present on pypi - https://pypi.org/project/oslo.cache/#files | 14:39 |
| jamespage | but not here - https://tarballs.opendev.org/openstack/oslo.cache | 14:39 |
| fungi | jamespage: probably you're getting caught by latest setuptools enforcing sdist filename normalization | 14:45 |
| fungi | the tarballs do exist, but any . or - in the project name now becomes a _ in the filename | 14:46 |
| jamespage | urgh | 14:46 |
| jamespage | that would explain it | 14:46 |
| fungi | pypi also started complaining about sdist tarballs with non-normalized project name strings in their filenames around november | 14:47 |
| fungi | setuptools actually started applying the normalization in the release from july 1 of last year, but we weren't building sdists with recent setuptools until we switched from invoking setup.py directly to using pyproject-build a few weeks ago | 14:48 |
| fungi | this was part of migrating jobs to pyton 3.12, since venvs created by it no longer preinstall a copy of setuptools | 14:49 |
| fungi | it's all very simple, you see? ;) | 14:50 |
| jamespage | lol | 14:51 |
| fungi | yeah, that's a january i won't be getting back | 15:02 |
| clarkb | fungi: stephenfin: I believe Gerrit changes are project specific now. So I'm not sure you can preserve the change history without recreating all of the changes | 15:07 |
| clarkb | fungi: stephenfin: this change occurred with the transition to notedb as each change is stored as special refs in the repo itself. Previously that info was in the sql db and not namespaced this way | 15:07 |
| clarkb | if you only want to preserve git history that can be done with different shas then you force push as fungi mentions to avoid creating new changes | 15:08 |
| fungi | in this case, by "preserve" i meant they'll still exist in the retired repo | 15:08 |
| clarkb | good point. I just want to call out there are two "histories" in play here and one is more easily preserved by copying a repo into another than the other history is | 15:09 |
| fungi | the commits in the other repo will just be copies, not attached to the original changes except loosely through their change-id commit message footers | 15:09 |
| clarkb | right and maybe that is sufficient | 15:09 |
| clarkb | I do wonder what the value in merging these repos is. | 15:10 |
| fungi | hopefully it's not "monorepo madness" filtering into openstack ;) | 15:10 |
| clarkb | is it jus so that people don't have to install two different tools with pip? or to avoid cloning two repos? Maybe it simplifies needing to make releases for sdk to consume changes within the client (but I would argue that is a good way to ensure the sdk is operating for other consumers in a sane way) | 15:11 |
| fungi | we have a ci system that can speculatively integrate changes between tightly-coupled projects | 15:11 |
| stephenfin | clarkb: fungi: What I'm trying to do is preserve the ability to do a git blame | 15:11 |
| stephenfin | This is what I have at the moment, and we'd lose all history https://review.opendev.org/c/openstack/openstacksdk/+/941637 | 15:11 |
| stephenfin | and no, not monorepo madness. It's to avoid this dance | 15:11 |
| fungi | git blame will still work in the proposed case, yes | 15:11 |
| clarkb | stephenfin: if git blame is the historical context that is important then ya doign a git merge of the repos using filter branch or subtree merge then force pushing the result to gerrit would do the trick | 15:12 |
| clarkb | though everyone (incudling all the CI systems) may have to do a hard reset | 15:12 |
| stephenfin | https://review.opendev.org/c/openstack/python-openstackclient/+/940111 -> https://review.opendev.org/c/openstack/openstacksdk/+/939703 -> https://review.opendev.org/c/openstack/python-openstackclient/+/939706 | 15:12 |
| clarkb | though maybe you can avoid a hard reset here if you have a merge commit that adds in the new tree | 15:13 |
| stephenfin | SDK and OSC are *super* tightly coupled now. If you want to add a new field to a resource, you need to add it to SDK, cut a release of SDK, add it to OSC, and cut a release of that | 15:13 |
| clarkb | basically have a disjoint history until the merge commit. That might avoid the downstream consumption pain | 15:13 |
| fungi | okay, so the sdk repo itself isn't entirely going away, just part of it is moving into osc? | 15:13 |
| clarkb | stephenfin: right but isn't that a signal you need a release anyway for everyone else using sdk? | 15:13 |
| stephenfin | Other way around: I'd like OSC to (mostly) go away and be folded into SDK | 15:13 |
| fungi | or other way around i guess | 15:13 |
| clarkb | my point was that if the cli needs sdk releases to consume things then other people do too. Ya thats annoying but if you merge them now other consumers of the sdk can be left behind more easily | 15:14 |
| stephenfin | clarkb: It probably is, but that's 2x the work for basically no gain | 15:14 |
| fungi | okay, so guts of osc move into sdk, osc itself becomes more of a heavyweight entrypoint wrapper project? | 15:14 |
| stephenfin | exactly | 15:14 |
| clarkb | dogfooding the sdk in this way is a really good way to ensure we're not leaving sdk users in a bind | 15:14 |
| clarkb | but I understand the pain involved | 15:15 |
| clarkb | another thing to keep in mind is that our python package tooling really expects a 1:1 mapping between repo and package | 15:15 |
| clarkb | fixing ^ is likely to be a bigger headache than any git magic | 15:15 |
| clarkb | unelss you move to a package that includes everything and then have some meta packge magic for the package name that no longer gets updated to alias the thing that does | 15:16 |
| clarkb | I think that would work within the existing package publication framework | 15:16 |
| stephenfin | gtema and I discussed this in #openstack-sdks this morning | 15:16 |
| stephenfin | I think the way we'll go for now is to move all the entrypoints and command implementations into SDK, but keep the actual shell code and console_scripts entry point into OSC | 15:17 |
| stephenfin | s/into OSC/in OSC | 15:17 |
| stephenfin | pretty much like how e.g. python-manilaclient contains commands and entrypoints for OSC and OSC simply calls it | 15:18 |
| stephenfin | or them, rather | 15:18 |
| clarkb | I don't like that design for one major reason: It means you're even more dependeny on entrypoints for osc than before. And entrypoints are one of the primary things that makes osc terribly slow | 15:19 |
| clarkb | my personal opinion is that the osc codebase should move towards an entrypointless setup whcih can more than halve its runtime depending on your disk io | 15:20 |
| clarkb | everyone loves to hate on how osc is slow and go off and rewrite a new tool in $otherlanguage saying "look we fixed it" but then we double down on the things that make the python implementation bad. They are inherent to python they are artifacts of our design choices | 15:21 |
| clarkb | *they are not inherent to python; they are artifacts of our design choices | 15:21 |
| clarkb | if osc did two things the runtime would become dominated by network io: don't use entrypoints and cache tokens. Then you're in the same boat as every other client tool and performance should be comparable | 15:22 |
| stephenfin | clarkb: sorry, meetings | 16:18 |
| stephenfin | can't argue with that | 16:18 |
| stephenfin | What I've described above is the (b) option from my discussion with gtema this (EU) morning https://meetings.opendev.org/irclogs/%23openstack-sdks/%23openstack-sdks.2025-02-14.log.html#t2025-02-14T11:13:41 | 16:19 |
| stephenfin | There's no reason we couldn't follow that up with the (a) option, where python-openstackclient becomes a meta package that simply installs e.g. 'openstacksdk[cli]' | 16:20 |
| stephenfin | and eventually get rid of python-openstackclient entirely | 16:21 |
| opendevreview | Elod Illes proposed openstack/openstack-zuul-jobs master: Prepare job templates for stable/2025.1 testing runtime https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/941245 | 16:21 |
| opendevreview | Elod Illes proposed openstack/openstack-zuul-jobs master: Prepare job templates for 2025.2 testing runtimes https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/941246 | 16:21 |
| opendevreview | Elod Illes proposed openstack/openstack-zuul-jobs master: Prepare job templates for stable/2025.1 testing runtime https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/941245 | 18:45 |
| opendevreview | Elod Illes proposed openstack/openstack-zuul-jobs master: Prepare job templates for 2025.2 testing runtimes https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/941246 | 18:45 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!