| @clarkb:matrix.org | corvus: change lgtm. I did leave one minor suggestion though | 00:10 |
|---|---|---|
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [zuul/zuul] 914715: Handle annotated and signed tags when packing refs https://review.opendev.org/c/zuul/zuul/+/914715 | 09:45 | |
| -@gerrit:opendev.org- Nils Gondermann proposed: [zuul/zuul] 915326: Inherit files attribute from abstract parent jobs https://review.opendev.org/c/zuul/zuul/+/915326 | 12:52 | |
| -@gerrit:opendev.org- Nils Gondermann proposed wip: [zuul/zuul] 915326: Inherit files attribute from abstract parent jobs https://review.opendev.org/c/zuul/zuul/+/915326 | 13:02 | |
| -@gerrit:opendev.org- Nils Gondermann marked as active: [zuul/zuul] 915326: Inherit files attribute from abstract parent jobs https://review.opendev.org/c/zuul/zuul/+/915326 | 13:05 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 915300: Split build/buildset list queries into two https://review.opendev.org/c/zuul/zuul/+/915300 | 13:39 | |
| -@gerrit:opendev.org- Joseph Kostreva proposed: [zuul/zuul] 915334: Check gerrit dependencies using isMerged() https://review.opendev.org/c/zuul/zuul/+/915334 | 13:48 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 915366: Add option to set compile flags for pyenv in ensure-python https://review.opendev.org/c/zuul/zuul-jobs/+/915366 | 16:53 | |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 915367: Build python3.12 with optimizations enabled https://review.opendev.org/c/zuul/zuul/+/915367 | 16:55 | |
| @clarkb:matrix.org | That pair of changes is a bit of an experiment. I want to see if we can trade off slower python builds for faster zuul testing and get faster jobs in the end | 16:55 |
| @clarkb:matrix.org | it immediately explodes in ./configure on "check if the compiler works" hrm I wonder if the empty env vars are being set in a way that makes things sad | 16:57 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 915366: Add option to set compile flags for pyenv in ensure-python https://review.opendev.org/c/zuul/zuul-jobs/+/915366 | 16:59 | |
| @clarkb:matrix.org | as promised the build is much slower. It is up to about 10 minutes so far. Not sure we'll make up 10 minutes during the test runtime. But we shall see | 17:17 |
| @clarkb:matrix.org | in the end it took about 18 minutes on this test node | 17:28 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 915300: Split build/buildset list queries into two https://review.opendev.org/c/zuul/zuul/+/915300 | 17:54 | |
| @fungicide:matrix.org | Clark: i compile all the python releases from source on my debian-based workstation, i should probably double-check the optimization options and build dependencies | 19:58 |
| @fungicide:matrix.org | ah, never mind, you've got the same options i pass to configure, and i guess pyenv handles the build dependencies on its own | 20:11 |
| @clarkb:matrix.org | You need extra deps if using clang but I think gcc can do them out of the box | 20:12 |
| @clarkb:matrix.org | There is a new experimental feature to use llvm's BOLT when building with clang that gives another couple of percentage improvements but we use gcc so I ignored that for now | 20:13 |
| @clarkb:matrix.org | fungi: unfortunately it looks like the build costs are greater than any time save but I need to look more closely at the results | 20:14 |
| @fungicide:matrix.org | Clark: however, the stow solution would save us the build time, at the cost of increased complexity | 20:22 |
| @fungicide:matrix.org | oh, speaking of compiling cpython from source, 3.12.3 and 3.13.0a6 were just announced moments ago | 20:27 |
| @clarkb:matrix.org | Ya we may have to dial back some of the options for arch stuff | 20:27 |
| @clarkb:matrix.org | But I suspect we could still get something that performed better and installs quickly that eay | 20:27 |
| @fungicide:matrix.org | probably. the debian packaged cpython does at least use ``--enable-optimizations`` and ``--with-lto`` already | 20:28 |
| @fungicide:matrix.org | but not the ``CFLAGS`` additions | 20:29 |
| @clarkb:matrix.org | Part of this started when comparing 3.11 to 3.12 job runtimes and test timing. 3.12 isn't really bad compared to 3.11 but it isn't better despite improvements that should make it better if not crazily so. Then I realized we were building without a bunch of optimizations | 20:35 |
| @clarkb:matrix.org | The 3.12 job took 18 minutes longer than the 3.11 job. They ran on the same cloud. The build took 18 ish minutes instead of 2 ish minutes | 20:41 |
| @clarkb:matrix.org | The zuul-jobs change is probably worth landing (because it gives people more flexibility), but I don't think we'll be able to take advantage of it the way I had hoped in zuul | 20:41 |
| @fungicide:matrix.org | yeah, if we were doing stow images, we could also include a full `make test` in our image build as a safety, and not worry about the additional time required for that either | 20:52 |
| @fungicide:matrix.org | i've also noticed that there are a lot of fallbacks in cpython builds to account for platforms that lack one c lib or another, so we could also be building in the absence of some libraries and ending up with less performant pure python fallbacks for some parts of stdlib particularly | 20:55 |
| @fungicide:matrix.org | make emits a message with a list of which ones couldn't be built, but it's kinda buried in the other output | 20:56 |
| @clarkb:matrix.org | ya I think we saved about 110 seconds in the stestr runtime or about 3.5% ish | 20:57 |
| @clarkb:matrix.org | which still seems like small improvements compared to what python3.12 should be giving us? But at least it is in the right direction now | 20:58 |
| @clarkb:matrix.org | oh ya that could be too if we're falling back to python implementation of stuff that could be C | 20:58 |
| @fungicide:matrix.org | i've got a 3.13 build underway right now, but it's probably close enough to 3.12's output that i should be able to grab a copy of what the "we managed to build it all" message looks like | 21:00 |
| @clarkb:matrix.org | rereading the python3.12 release notes a lot of the performance improvements have to do with asyncio which zuul is not using so that may explain why they are close together as well | 21:01 |
| @fungicide:matrix.org | as for build times, i also use `make -j32` on a quad-core machine, mainly to parallelize the inbuilt test phase which `--enable-optimizations` relies on | 21:04 |
| @fungicide:matrix.org | i played around with different parallelization amounts to see where the sweet spot on that machine was | 21:05 |
| @clarkb:matrix.org | fungi: the default is probably -jN where N == numcores? | 21:06 |
| @fungicide:matrix.org | maybe, i actually don't know | 21:08 |
| @fungicide:matrix.org | also, for the record, the "we were able to build all the modules" message on 3.13 looks like `Checked 112 modules (33 built-in, 78 shared, 1 n/a on linux-x86_64, 0 disabled, 0 missing, 0 failed on import)` | 21:10 |
| @fungicide:matrix.org | numbers may vary on 3.12 | 21:10 |
| @clarkb:matrix.org | I suspect that message may be hidden away my pyenv but I'll see what the logs say | 21:11 |
| @clarkb:matrix.org | ya we don't get much https://zuul.opendev.org/t/zuul/build/7756d931d7104c09801d48ad645fd831/console#2/0/11/ubuntu-jammy | 21:12 |
| @fungicide:matrix.org | i'm unfortunately unfamiliar with pyenv, my process looks like `export $STUFF && ./configure --options && make && make test && make altinstall ...` | 21:17 |
| @clarkb:matrix.org | I think pyenv is doing all the same stuff under the hood, but the interface it gives is you tell it what version to install and where to install it and then it figures out the rest | 21:19 |
| @clarkb:matrix.org | good when you don't care bout all the small details and honestly it is working pretty well without the tuning | 21:19 |
| @fungicide:matrix.org | yeah, i mean, "what version to install" is just `$1` in my script, and does some quick `git tag` sorting/filtering/stripping to pick the latest patch or prerelease for a given minor version | 21:22 |
| @fungicide:matrix.org | the whole build script is maybe a dozen lines of shell | 21:23 |
| @fungicide:matrix.org | but then again, i'm preinstalling the build deps in the system and only targeting one platform | 21:23 |
| @fungicide:matrix.org | so there's a lot of complexity i'm able to opt out of | 21:24 |
| @mordred:waterwanders.com | I love pyenv | 21:36 |
| @mordred:waterwanders.com | if I ever work for a company that doesn't have its own linux distro, I'm totally going to spin up my own distro that doesn't have any distro packages for python but instead just uses pyenv. and same for node and nvm. something something | 21:37 |
| @clarkb:matrix.org | there is a new package manager manager for nvm you should use that | 21:38 |
| @mordred:waterwanders.com | oh yeah? | 21:38 |
| @clarkb:matrix.org | `pnpm` | 21:38 |
| @clarkb:matrix.org | etherpad uses it now | 21:38 |
| @mordred:waterwanders.com | wait - that's a replacement for npm - not nvm? | 21:39 |
| @clarkb:matrix.org | oh sorry I clearly need a break from staring at screens | 21:40 |
| @clarkb:matrix.org | correct npm not nvm | 21:40 |
| @clarkb:matrix.org | my brain saw node and nvm and immediately thought node and npm | 21:40 |
| @mordred:waterwanders.com | cool. because yarn vs npm wasn't fun enough. but - I'll learn about it anyway :) | 21:40 |
| @fungicide:matrix.org | pnpm seems to want to be a yarn+npm replacement, from what i was able to work out | 21:48 |
| @mordred:waterwanders.com | yeah. of course, yarn wanted to be an npm replacement - so I invoke https://xkcd.com/927/ | 22:38 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!