| mordred | ianw: cool! | 00:00 |
|---|---|---|
| mordred | ianw, Shrews: I think this might be one of these cases of us needing to learn to be systemd native even though systemd is an abomination because the wrapper things cause systemd to get really confused | 00:00 |
| Shrews | mordred: for nodepool, i always just invoke the /etc/init.d scripts directly because of all of the suck | 00:01 |
| mordred | and - by systemd native - I mean I'm looking forward to deploying zuul in docker | 00:01 |
| ianw | 20369 write(2, "Traceback (most recent call last"..., 1469) = 1469 | 00:01 |
| ianw | there's something going on ... | 00:01 |
| *** gfidente|afk has quit IRC | 00:02 | |
| mordred | ianw: journalctl -u zuul-executor | 00:03 |
| mordred | shows a tracevback | 00:03 |
| mordred | ianw: ConnectionRefusedError: [Errno 111] Connection refused | 00:03 |
| mordred | oh - that's for stop | 00:03 |
| ianw | http://paste.openstack.org/show/785707/ | 00:05 |
| ianw | PermissionError: [Errno 13] Permission denined something? | 00:05 |
| clarkb | ianw: that may be when it tries to clean up the old build dirs | 00:06 |
| clarkb | it will do that on startup as any that remain on start were leaked from a previous run, but maybe permissions are breaking that somehow? | 00:07 |
| clarkb | oh wait its in the copy ansible files code path | 00:07 |
| clarkb | likely something else then | 00:07 |
| mordred | yeah | 00:07 |
| mordred | is this a root vs. zuul user issue? ianw - are you running the start as root? | 00:07 |
| ianw | ahh, yes i ran the "-d" start as root | 00:07 |
| mordred | and -d doesn't fork into the background so doesn't downgrade to zuul user right? | 00:08 |
| ianw | no ... ok, let me clear out build dirs | 00:09 |
| mordred | which just means we need to chown that dir I guess | 00:09 |
| *** ociuhandu has joined #openstack-infra | 00:10 | |
| *** michael-beaver has quit IRC | 00:12 | |
| ianw | arrgh, it's running now | 00:12 |
| ianw | no it's not | 00:12 |
| *** ociuhandu has quit IRC | 00:15 | |
| ianw | ok, i chown'd /var/lib/zuul bits and it seems to be working onw | 00:17 |
| ianw | what a nightmare! | 00:17 |
| mordred | ianw: cool. is this with the os.environ removal? | 00:17 |
| mordred | or with the SafeConfigParser? | 00:17 |
| ianw | umm, yeah, it would be | 00:17 |
| ianw | i can stop it, remove that, and see if it fails to start | 00:17 |
| mordred | cool. if so - then let's land the revert and we can figure out the right way to revert-revert - as well as a test case that should be added to show the error | 00:18 |
| ianw | definitely i think it's unsafe from command line | 00:18 |
| ianw | (even modulo me doing it as root) | 00:18 |
| ianw | i've put back in the os.environ, waiting for it to stop | 00:20 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Revert "Revert "Add the process environment to zuul.conf parser"" https://review.opendev.org/692469 | 00:20 |
| *** goldyfruit_ has joined #openstack-infra | 00:20 | |
| mordred | ianw, clarkb: there's a revert revert with the HISTTIMEFORMAT env var added to the test case so we remember what the issue was | 00:21 |
| mordred | Shrews: it's failing build-image because of the issue you were digging in to I suppose? | 00:22 |
| ianw | mordred: it seems to be running | 00:23 |
| ianw | i think the post-mortem here is: i stopped service, but systemd didn't think it was stopped (?) | 00:23 |
| ianw | i tried starting service, systemd didn't report an error, just 0 | 00:23 |
| ianw | i ran executor with -d from root command line | 00:23 |
| ianw | this exposed the os.environ parsing issue ... and screwed up permissions on disk | 00:24 |
| mordred | yeah. I think that's right | 00:24 |
| ianw | mordred pointed out i should try stopping it again, i did, systemd now figured it was stopped and tried to start it | 00:24 |
| Shrews | mordred: probably. the image jobs are totally borked | 00:24 |
| ianw | then we hit permissions errors ... which aren't logged anywhere useful AFAICS (probably the daemon reload stuff i haven't go around to yet) | 00:25 |
| ianw | i cleared out the bad things now written as root, and it seem to be back | 00:25 |
| ianw | ... and that's the story of ze09 | 00:25 |
| mordred | Shrews: zomg. I think that patch is broken because it can't run the non-speculative buildset registry that's needed to test the speculartive buildset registry | 00:26 |
| *** gyee has quit IRC | 00:26 | |
| Shrews | mordred: that digging made my head hurt | 00:26 |
| Shrews | as does your summary | 00:27 |
| mordred | Shrews: so - I think we're going to need to force-merge that patch - but let me depends-on the os.environ revert patch on it and see if that shows that it fixes it | 00:27 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Revert "Add the process environment to zuul.conf parser" https://review.opendev.org/692463 | 00:27 |
| Shrews | mordred: I don’t understand how everything broke all of a sudden | 00:28 |
| mordred | Shrews: me either | 00:29 |
| mordred | Shrews: althoguh to be fair the concept of a buildset registry that handles speculative docker image that is run from docker images that may or may not be speculative docker images is a bit mind-bending | 00:30 |
| Shrews | I might need more alcohol to process that statement | 00:32 |
| mordred | Shrews: nope. depends-on is not showing that the fix works. however - I don't think that job can be *more* broken - so I think force-merging might be in order | 00:35 |
| mordred | ianw, clarkb: change in question is https://review.opendev.org/#/c/692400 - thoughts on force-merge? | 00:36 |
| ianw | LGTM, seems like a better check | 00:37 |
| Shrews | mordred: that patch definitely fixes something (because of the 'auto' interpreter stuff) | 00:37 |
| ianw | i have to run to an appointment, will restart ze12 later for finger | 00:38 |
| mordred | Shrews: oh - yeah - and this broke because the auto change is a change that happened outside of jobs | 00:38 |
| openstackgerrit | Merged zuul/zuul-jobs master: run-buildset-registry: Fix python version check https://review.opendev.org/692400 | 00:39 |
| Shrews | mordred: does the zuul-jobs-test-registry-buildset-registry use some sort of evil recursion that i'm as-of-yet unaware of? because it fixed the other job | 00:39 |
| mordred | Shrews: yeah - I think from reading it it runs a registry non-speculatively so that it can get the images needed to run a speculative registry | 00:39 |
| mordred | Shrews: This runs two registries: a real buildset registry so that we | 00:40 |
| mordred | can receive speculative zuul-registry images, and a fake | 00:40 |
| mordred | buildset registry (running the speculative or latest | 00:40 |
| mordred | zuul-registry) that is used to test using the buildset registry | 00:40 |
| mordred | role. | 00:40 |
| Shrews | oooh, so the 'real' one remained broke, i guess | 00:40 |
| mordred | yeah. that's my theory | 00:41 |
| Shrews | neat | 00:41 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Revert "Add the process environment to zuul.conf parser" https://review.opendev.org/692463 | 00:41 |
| mordred | Shrews: if that's accurate, then this ^^ should work with build-images | 00:41 |
| *** sreejithp has joined #openstack-infra | 00:43 | |
| mordred | Shrews: if not - we should see lovely additional errors | 00:43 |
| Shrews | mordred: so glad you're just starting your day. i expect smooth sailing when i begin mine :) | 00:44 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Plumb through secure.config contents https://review.opendev.org/691800 | 00:47 |
| *** goldyfruit_ has quit IRC | 00:55 | |
| Shrews | zuul-build-image job must be on a fingerless executor :( | 00:56 |
| *** sreejithp has quit IRC | 00:57 | |
| mordred | Shrews: yeah - but ... it hasn't immediately failed and gone into a retry loop | 01:04 |
| mordred | Shrews: so - you know - that's promising | 01:04 |
| *** dave-mccowan has quit IRC | 01:04 | |
| Shrews | mordred: the failure was usually deep into the job | 01:05 |
| *** rlandy|ruck|bbl is now known as rlandy|ruck | 01:07 | |
| *** dave-mccowan has joined #openstack-infra | 01:07 | |
| fungi | i think only ze12 should be lacking a finger server at the moment (unless another has popped in the last few hours) | 01:07 |
| fungi | your luck is terrible ;) | 01:07 |
| mordred | right? | 01:07 |
| Shrews | grrrr | 01:08 |
| Shrews | i've come to expect such things, really | 01:08 |
| mordred | Shrews: it worked | 01:08 |
| Shrews | it paused... i guess that means success | 01:08 |
| mordred | the job is paused now - that means it got through what it needed to for quick-start | 01:08 |
| mordred | yeah | 01:08 |
| mordred | \o/ | 01:08 |
| Shrews | sweet | 01:08 |
| Shrews | i will now away from the computer things to watch football things | 01:09 |
| mordred | Shrews, ianw, fungi: https://review.opendev.org/#/c/692463/ | 01:09 |
| mordred | if you feel like putting a +2 on that - I can get it landed so that ianw can restart executors | 01:09 |
| Shrews | did so | 01:09 |
| mordred | ossum | 01:09 |
| *** rfolco|afk has quit IRC | 01:10 | |
| fungi | lookin | 01:11 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Revert "Revert "Add the process environment to zuul.conf parser"" https://review.opendev.org/692469 | 01:12 |
| mordred | fungi: that's the followup revert-revert that has the relevant failure case added to the test suite | 01:12 |
| *** iurygregory has quit IRC | 01:12 | |
| *** ociuhandu has joined #openstack-infra | 01:12 | |
| fungi | why is there a verified -1 from zuul on it when zuul hasn't commented on the last two patchsets? | 01:14 |
| fungi | shouldn't they have cleared existing verified votes? | 01:14 |
| mordred | well - the two changes were just commit message changes | 01:15 |
| fungi | mmm | 01:15 |
| mordred | so I think it doesn't clear -1s from those? | 01:15 |
| *** dmsimard has quit IRC | 01:15 | |
| fungi | maybe not if we've configured the zuul/zuul repo for that behavior | 01:16 |
| *** dmsimard has joined #openstack-infra | 01:16 | |
| fungi | so anyway, the failure there is the one fixed by 692400? | 01:17 |
| *** ociuhandu has quit IRC | 01:17 | |
| fungi | guess we'll know shortly | 01:17 |
| Shrews | fungi: yes | 01:23 |
| fungi | cool | 01:24 |
| mordred | \o/ | 01:27 |
| mordred | Shrews: we fixed something | 01:27 |
| mordred | Shrews: and by we, I mean you | 01:27 |
| Shrews | mordred: something something blind squirrel something something | 01:30 |
| fungi | a nod's as good as a wink to a blind squirrel? | 01:34 |
| mordred | SQUIRREL | 01:42 |
| *** yamamoto has joined #openstack-infra | 01:47 | |
| *** rlandy|ruck has quit IRC | 01:49 | |
| *** rosmaita has quit IRC | 01:52 | |
| *** dave-mccowan has quit IRC | 01:56 | |
| *** yamamoto has quit IRC | 01:57 | |
| *** ociuhandu has joined #openstack-infra | 01:59 | |
| *** xarses has quit IRC | 02:00 | |
| mordred | ZOMG | 02:07 |
| *** rosmaita has joined #openstack-infra | 02:07 | |
| mordred | the secure.config patch is green | 02:07 |
| *** ociuhandu has quit IRC | 02:07 | |
| mordred | ianw, fungi, Shrews: https://review.opendev.org/#/c/691800 is green - and I've run it against review-dev | 02:07 |
| *** xarses has joined #openstack-infra | 02:14 | |
| *** goldyfruit_ has joined #openstack-infra | 02:17 | |
| *** jerryz has quit IRC | 02:18 | |
| openstackgerrit | Merged zuul/zuul master: Revert "Add the process environment to zuul.conf parser" https://review.opendev.org/692463 | 02:23 |
| *** markvoelker has joined #openstack-infra | 02:26 | |
| *** zhurong has quit IRC | 02:28 | |
| mordred | woot | 02:29 |
| mordred | ianw: zuul executors should be safe to pull and restart as neeeded | 02:29 |
| *** markvoelker has quit IRC | 02:30 | |
| ianw | sorry, back, can do ze12 | 02:52 |
| *** sreejithp has joined #openstack-infra | 02:54 | |
| *** ociuhandu has joined #openstack-infra | 02:55 | |
| *** sreejithp has quit IRC | 02:58 | |
| ianw | ok, that one went a lot smoother! | 03:01 |
| *** ociuhandu has quit IRC | 03:02 | |
| *** licanwei has left #openstack-infra | 03:12 | |
| *** yamamoto has joined #openstack-infra | 03:12 | |
| *** yamamoto has quit IRC | 03:16 | |
| *** mtreinish has quit IRC | 03:28 | |
| mordred | yay! | 03:30 |
| mordred | ianw: sadly, the test change I added in the revert revert did not fail: https://review.opendev.org/#/c/692469/ | 03:31 |
| *** psachin has joined #openstack-infra | 03:35 | |
| *** dave-mccowan has joined #openstack-infra | 03:38 | |
| *** goldyfruit_ has quit IRC | 03:40 | |
| *** dave-mccowan has quit IRC | 03:40 | |
| ianw | hrm, i hate it when failures don't fail | 03:45 |
| mordred | yeah | 03:47 |
| *** factor has joined #openstack-infra | 03:47 | |
| mordred | ianw: I can't get configparser to fail with that error on ze09 in a python repl | 03:47 |
| mordred | with os.environ having all the things | 03:47 |
| mordred | maybe the os.environ bit was a red herring and it was all somehow related to systemd? | 03:48 |
| ianw | hrm, this was pretty repeatable with "zuul-executor -d" on the command line ... | 03:48 |
| ianw | as root ... which i wouldn't recommend trying | 03:48 |
| mordred | ianw: I kind of want to shut down one of the executors and poke some - it's slow at this time so that shouldn't be disruptive, right? | 03:52 |
| *** ociuhandu has joined #openstack-infra | 04:08 | |
| *** ociuhandu has quit IRC | 04:13 | |
| *** ociuhandu has joined #openstack-infra | 04:14 | |
| ianw | mordred: i think that would be ok, especially if it saves someone else hitting a weird corner :) | 04:19 |
| mordred | kk. I'm going to take 09 back out | 04:22 |
| mordred | ianw: neither systemctl stop nor /etc/init.d/zuul-executor seem to have been able to stop it | 04:23 |
| ianw | ok ... maybe i'm not crazy!? | 04:24 |
| mordred | ianw: also - WOW there are a lot of ssh-agent processes | 04:25 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Create a download script https://review.opendev.org/592341 | 04:26 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 04:26 |
| *** ociuhandu has quit IRC | 04:26 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: test-upload-logs-swift: Add missing template for test https://review.opendev.org/692489 | 04:26 |
| ianw | mordred: did you manually clear them or did they disappear? | 04:27 |
| mordred | ianw: I cleared them since z-e was down anyway | 04:29 |
| mordred | ianw: good news is - I have reproduced the issue you were seeing | 04:30 |
| ianw | mordred: excellent! and i like that sentence didn't end with a "but" :) | 04:31 |
| ianw | (so far, anyway :) | 04:31 |
| *** whoami-rajat has joined #openstack-infra | 04:33 | |
| mordred | ianw: I have a reproduction script in /home/zuul/repro.py | 04:33 |
| mordred | now to see if I can get a zuul test case to reproduce properly | 04:34 |
| ianw | there is a related bug https://bugs.python.org/issue26872 | 04:35 |
| ianw | i think it falls in the crack of bug or feature, though | 04:35 |
| ianw | in that case, related to having a % in the virtualenv path | 04:36 |
| ianw | (not sure i want to ask why *that* was a problem ... but anyway) | 04:36 |
| *** udesale has joined #openstack-infra | 04:38 | |
| mordred | ianw: ahhhh - the testcase doesn't work because it's only doing the readConfig set | 04:40 |
| mordred | step | 04:40 |
| mordred | the error isn't triggered until the config gets *used* | 04:40 |
| ianw | ahh, and then expanded, that makes sense i guess | 04:42 |
| *** yamamoto has joined #openstack-infra | 04:43 | |
| *** yamamoto has quit IRC | 04:46 | |
| mordred | yeah | 04:46 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Revert "Revert "Add the process environment to zuul.conf parser"" https://review.opendev.org/692469 | 04:46 |
| mordred | ianw: ^^ that should actually show the failure | 04:46 |
| mordred | and - I've restarted the executor on ze09 | 04:48 |
| *** yamamoto has joined #openstack-infra | 04:51 | |
| openstackgerrit | Merged zuul/zuul-jobs master: test-upload-logs-swift: Add missing template for test https://review.opendev.org/692489 | 04:58 |
| *** raukadah is now known as chandankumar | 05:01 | |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Create a download script https://review.opendev.org/592341 | 05:07 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: upload-logs-swift: Add a unicode file https://review.opendev.org/592853 | 05:07 |
| openstackgerrit | Ian Wienand proposed zuul/zuul-jobs master: test-swift-log-upload: fix dry_run parameter https://review.opendev.org/692490 | 05:07 |
| *** trident has quit IRC | 05:18 | |
| *** dchen has quit IRC | 05:21 | |
| *** yamamoto has quit IRC | 05:25 | |
| *** trident has joined #openstack-infra | 05:25 | |
| *** ociuhandu has joined #openstack-infra | 05:31 | |
| *** surpatil has joined #openstack-infra | 05:31 | |
| *** soniya29 has joined #openstack-infra | 05:32 | |
| *** psachin has quit IRC | 05:35 | |
| *** ociuhandu has quit IRC | 05:35 | |
| *** yamamoto has joined #openstack-infra | 05:36 | |
| *** yamamoto_ has joined #openstack-infra | 05:43 | |
| *** yamamoto has quit IRC | 05:47 | |
| *** kjackal has joined #openstack-infra | 05:47 | |
| *** yamamoto_ has quit IRC | 05:50 | |
| *** jtomasek has joined #openstack-infra | 05:51 | |
| *** yamamoto has joined #openstack-infra | 05:58 | |
| *** ykarel has joined #openstack-infra | 05:59 | |
| *** threestrands has quit IRC | 06:02 | |
| *** yamamoto_ has joined #openstack-infra | 06:15 | |
| *** yamamoto has quit IRC | 06:19 | |
| *** yamamoto_ has quit IRC | 06:21 | |
| *** yamamoto has joined #openstack-infra | 06:22 | |
| *** kjackal has quit IRC | 06:33 | |
| *** SurajPatil has joined #openstack-infra | 06:37 | |
| *** surpatil has quit IRC | 06:40 | |
| *** yamamoto_ has joined #openstack-infra | 06:43 | |
| *** psachin has joined #openstack-infra | 06:45 | |
| *** yamamoto_ has quit IRC | 06:45 | |
| *** yamamoto has quit IRC | 06:46 | |
| *** yamamoto has joined #openstack-infra | 06:48 | |
| *** sreejithp has joined #openstack-infra | 06:54 | |
| *** lpetrut has quit IRC | 06:55 | |
| *** sreejithp has quit IRC | 06:59 | |
| *** ociuhandu has joined #openstack-infra | 07:02 | |
| *** ociuhandu has quit IRC | 07:07 | |
| *** yamamoto_ has joined #openstack-infra | 07:07 | |
| *** yamamoto has quit IRC | 07:10 | |
| *** yamamoto_ has quit IRC | 07:16 | |
| *** dchen has joined #openstack-infra | 07:17 | |
| *** dychen has joined #openstack-infra | 07:21 | |
| *** pcaruana has joined #openstack-infra | 07:27 | |
| *** dychen has quit IRC | 07:29 | |
| jrosser | ianw: there was a config parser / % related bug that cropped up in swift when OSA moved it to py3 | 07:32 |
| jrosser | there was a fix went in there that allowed the old behaviour to continue rather than require the % to be %% | 07:33 |
| AJaeger | ianw: anything new from rax on the quota? | 07:34 |
| *** kjackal has joined #openstack-infra | 07:35 | |
| *** surpatil has joined #openstack-infra | 07:37 | |
| *** SurajPatil has quit IRC | 07:40 | |
| openstackgerrit | Merged zuul/zuul-jobs master: test-swift-log-upload: fix dry_run parameter https://review.opendev.org/692490 | 07:41 |
| *** ccamacho has quit IRC | 07:43 | |
| *** panda|pto has quit IRC | 07:46 | |
| *** panda has joined #openstack-infra | 07:46 | |
| openstackgerrit | Merged zuul/zuul master: Reduce sleep to avoid race conditions https://review.opendev.org/684726 | 08:02 |
| *** jamesden_ has quit IRC | 08:09 | |
| *** jamesdenton has joined #openstack-infra | 08:09 | |
| *** markvoelker has joined #openstack-infra | 08:28 | |
| *** whoami-rajat has quit IRC | 08:33 | |
| *** markvoelker has quit IRC | 08:33 | |
| *** soniya29 has quit IRC | 08:35 | |
| *** tkajinam has quit IRC | 08:53 | |
| *** sreejithp has joined #openstack-infra | 08:55 | |
| *** kopecmartin|off is now known as kopecmartin|pto | 08:57 | |
| *** ykarel is now known as ykarel|lunch | 08:59 | |
| *** sreejithp has quit IRC | 08:59 | |
| frickler | ianw: AJaeger: I tried to look in the infra-root mbox for the rax ticket, but didn't find anything, where is this being handled? | 09:00 |
| *** lucasagomes has joined #openstack-infra | 09:01 | |
| *** kjackal has quit IRC | 09:03 | |
| *** kjackal has joined #openstack-infra | 09:04 | |
| *** ralonsoh has joined #openstack-infra | 09:07 | |
| *** ykarel|lunch is now known as ykarel | 09:24 | |
| *** iurygregory has joined #openstack-infra | 09:27 | |
| *** trident has quit IRC | 09:27 | |
| *** ykarel is now known as ykarel|mtg | 09:30 | |
| *** trident has joined #openstack-infra | 09:34 | |
| *** soniya29 has joined #openstack-infra | 09:36 | |
| ianw | AJaeger/fricker: "I have adjusted your server limits you should be able to build again. " | 09:37 |
| *** ykarel|mtg is now known as ykarel | 09:38 | |
| ianw | jhesketh/AJaeger: thanks for reviews on the script stuff; seems to work https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_413/692457/2/check/openstack-infra-extra-integration-bionic/4135367/ | 09:40 |
| ianw | can do some more testing but i think the basics are there | 09:40 |
| *** ociuhandu has joined #openstack-infra | 09:40 | |
| ianw | the official line also seems to be that the supported way to build images is with xenserver, so i don't think we can expect much input on the qemu-img side of things :/ | 09:42 |
| *** sreejithp has joined #openstack-infra | 09:58 | |
| AJaeger | ianw, frickler: Indeed http://grafana.openstack.org/d/8wFIHcSiz/nodepool-rackspace?orgId=1 show we use it heavily agan. thanks | 10:01 |
| *** sreejithp has quit IRC | 10:02 | |
| AJaeger | ianw: regarding download script: What about adding to devstack (or wherever) a script that downloads everything? Give it a review and a job name as parameter... | 10:03 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remove opensuse-150 nodeset https://review.opendev.org/691966 | 10:13 |
| AJaeger | clarkb: I added some depends-on, and think topic:use-opensuse-15 has all needed patches up now to remove jobs and node definitions | 10:15 |
| AJaeger | clarkb: 6 changes to go - if you abandon your two that are duplicates. | 10:17 |
| *** trident has quit IRC | 10:30 | |
| *** ociuhandu has quit IRC | 10:34 | |
| *** trident has joined #openstack-infra | 10:36 | |
| *** aedc has joined #openstack-infra | 10:38 | |
| *** surpatil has quit IRC | 10:47 | |
| *** lucasagomes has quit IRC | 10:56 | |
| *** sreejithp has joined #openstack-infra | 10:58 | |
| *** sreejithp has quit IRC | 10:58 | |
| *** surpatil has joined #openstack-infra | 10:58 | |
| *** dmsimard1 has joined #openstack-infra | 11:07 | |
| *** dmsimard has quit IRC | 11:08 | |
| fungi | frickler: yeah, for whatever reason, ticket updates for that account don't go to an alias of the infra-root address (or i would have likely spotted the outage there fairly early) | 11:08 |
| *** kjackal has quit IRC | 11:13 | |
| *** kjackal has joined #openstack-infra | 11:17 | |
| *** liuyulong has joined #openstack-infra | 11:22 | |
| *** pcaruana has quit IRC | 11:26 | |
| *** pcaruana has joined #openstack-infra | 11:41 | |
| *** auristor has quit IRC | 11:44 | |
| *** dmsimard1 is now known as dmsimard | 11:46 | |
| *** auristor has joined #openstack-infra | 11:48 | |
| *** ramishra has quit IRC | 11:53 | |
| *** rh-jelabarre has quit IRC | 11:54 | |
| *** rh-jelabarre has joined #openstack-infra | 11:54 | |
| *** rfolco|afk has joined #openstack-infra | 11:57 | |
| *** apetrich has quit IRC | 12:05 | |
| *** apetrich has joined #openstack-infra | 12:06 | |
| *** rfolco|afk is now known as rfolco | 12:12 | |
| *** rlandy has joined #openstack-infra | 12:16 | |
| *** rlandy is now known as rlandy|ruck | 12:17 | |
| *** kjackal has quit IRC | 12:18 | |
| *** kjackal has joined #openstack-infra | 12:19 | |
| *** dkehn_ has quit IRC | 12:20 | |
| *** markvoelker has joined #openstack-infra | 12:24 | |
| *** Erifkard has joined #openstack-infra | 12:30 | |
| *** lucasagomes has joined #openstack-infra | 12:35 | |
| *** udesale has quit IRC | 12:38 | |
| *** udesale has joined #openstack-infra | 12:38 | |
| *** psachin has quit IRC | 12:38 | |
| *** SpamapS has joined #openstack-infra | 12:43 | |
| *** ykarel is now known as ykarel|afk | 12:51 | |
| *** udesale has quit IRC | 12:59 | |
| tdasilva | clarkb, corvus: hello! any idea why the 0.0.5 release of the nosehtmloutput in pypi dates back to 2013, but on github it says 2017? https://pypi.org/project/nosehtmloutput/#history https://github.com/openstack/nose-html-output/releases | 13:05 |
| tdasilva | I'm running into an issue where doing a pip install is installing some older version and not the one with this fix: https://github.com/openstack/nose-html-output/commit/71d12999b06908bbb019f69c89361bd44bec316c | 13:07 |
| *** mriedem has joined #openstack-infra | 13:08 | |
| *** trident has quit IRC | 13:09 | |
| *** trident has joined #openstack-infra | 13:15 | |
| *** goldyfruit_ has joined #openstack-infra | 13:15 | |
| AJaeger | tdasilva: indeed, git code shows for 0.0.5 as 2017 tag - see https://opendev.org/openstack/nose-html-output/commit/71d12999b06908bbb019f69c89361bd44bec316c . Something looks wrong here. | 13:21 |
| AJaeger | mordred: the tarball on pypi has ownership "mordred/mordred" and was created 2013-08-08 | 13:23 |
| AJaeger | tdasilva: most folks are travelling right now to the summit, if nobody answers, best to write an email to openstack-infra mailing list with your findings. | 13:24 |
| tdasilva | AJaeger: oh yeah, good point about OS summit, thanks for the heads up. | 13:26 |
| *** ykarel|afk has quit IRC | 13:30 | |
| *** Goneri has joined #openstack-infra | 13:34 | |
| fungi | tdasilva: it does indeed look like the sdist which was uploaded in 2013 was created at the point where this change merged https://review.openstack.org/40903 | 13:41 |
| fungi | we weren't generating releases of projects automatically on tagging back then, so i'm willing to bet tagging it got overlooked at the time. then years later some fixes merged and we picked tagged the next release in sequence, not realizing an equivalent version had been uploaded to pypi years before | 13:43 |
| tdasilva | fungi, AJaeger: for full context, i'm trying to fix swift's lower-constraints jobs that have been failing this week. Looks like the parent job was changed to run with py3 recently and that started failing our tests. | 13:43 |
| fungi | we should probably just re-tag it as 0.0.6 or something | 13:43 |
| tdasilva | so I proposed this patch https://review.opendev.org/#/c/692124/ | 13:43 |
| tdasilva | but i think it is now failing with that warning from nosehtmloutput | 13:44 |
| AJaeger | fungi: I agree with tagging a 0.0.6 | 13:45 |
| tdasilva | yep, makes sense | 13:45 |
| AJaeger | pabelanger: are you interested in https://review.opendev.org/685601 and friends for windmill or should I abandon? I'll abandon in a few days if I don't hear anything from you... | 13:47 |
| fungi | yeah, here's the discussion when 0.0.5 was tagged: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-08-22.log.html#t2017-08-22T17:45:47 | 13:47 |
| fungi | so almost certainly didn't notice there was already a years-old 0.0.5 on pypi with no corresponding tag in the repo | 13:48 |
| fungi | and subsequently didn't notice when the push to pypi failed (our pypi upload jobs weren't as robust a couple years ago as they are now, and sometimes pypi upload failures went unnoticed) | 13:49 |
| fungi | there's been a new commit for opendev migration merged since the 0.0.5 tag anyway, so i think we should tag the master branch tip instead of re-tagging the same commit as 0.0.5 | 13:50 |
| fungi | alternatively i could re-tag that commit as 0.0.6 to signal there was something screwy there (brief note in the tag message) and then tag master as 0.0.7 for good measure | 13:51 |
| fungi | amotoki: you indicated back in 2017 that you were volunteering to maintain nose-html-output, if you still care about it are you okay with that plan? (re-tag the erroneous 0.0.5 as 0.0.6 with a tag message explaining what happened, and then tag the tip of master as 0.0.7)? | 13:52 |
| tdasilva | fungi: that irc log reminds me we should look into moving to stestr :/ | 13:55 |
| fungi | heh | 13:56 |
| amotoki | fungi: I didn't care nose-html-output recently as horizon and pluigns succeeded to drop nose usage. I can check if needed. | 13:56 |
| fungi | amotoki: no worries, i suspect swift is the only user now. thanks for responding! | 13:57 |
| amotoki | horizon and plugins now use django test runner as django test cases conflict with testtools | 13:57 |
| AJaeger | fungi: there's no merge since the last tag, so no need for a 0.0.7... | 13:59 |
| fungi | AJaeger: there's one | 13:59 |
| AJaeger | Oh, missed it | 13:59 |
| fungi | the opendev migration change is newer than the 0.0.5 tag | 13:59 |
| AJaeger | indeed... | 13:59 |
| fungi | so if i'm tagging i may as well tag the branch tip too | 13:59 |
| AJaeger | yep | 14:00 |
| fungi | infra-root: if any of you are around, any objections to this plan for nose-html-output (re-tag the erroneous 0.0.5 as 0.0.6 with a tag message explaining what happened, and then tag the tip of master as 0.0.7)? | 14:00 |
| fungi | i'd like to get swift's changes un-stuck for them | 14:01 |
| fungi | though also if they're the only remaining stakeholders for that repo it may make sense for them to adopt it from us | 14:01 |
| frickler | fungi: +1 to both ... or +2 even ;) | 14:03 |
| tdasilva | maybe starlingx is also using? http://codesearch.openstack.org/?q=nosehtmloutput&i=nope&files=&repos= | 14:03 |
| fungi | possible | 14:03 |
| fungi | i wonder if they're using it for an old horizon build or something | 14:03 |
| tdasilva | either way, i guess we should really move off of it, i'll put that on swift's todo list | 14:04 |
| fungi | my network access is really flaky from this airport | 14:04 |
| Shrews | fungi: safe flight | 14:08 |
| fungi | thanks! | 14:10 |
| *** eharney has joined #openstack-infra | 14:11 | |
| *** jaosorior has joined #openstack-infra | 14:13 | |
| *** surpatil has quit IRC | 14:24 | |
| *** mtreinish has joined #openstack-infra | 14:28 | |
| fungi | infra-root: tdasilva: digging deeper into this, i'm not sure the original plan will work. nose-html-output doesn't use pbr so we need to push changes through review bumping the version in htmloutput/version.py | 14:29 |
| fungi | and then push tags of those once they merge | 14:29 |
| fungi | so i think i'll bump that variable from 0.0.5 to 0.0.7 | 14:30 |
| fungi | and the 0.0.6 tag can still be a re-tagging of the 0.0.5 tag | 14:30 |
| fungi | but there won't be a 0.0.6 sdist on pypi unless we manually make one | 14:30 |
| tdasilva | fungi: oic, maybe just tag master to 0.0.6? | 14:31 |
| fungi | well, we still need to merge a change for that first | 14:31 |
| fungi | versioning is done in repository file content there, not with tags | 14:31 |
| *** ociuhandu has joined #openstack-infra | 14:31 | |
| tdasilva | right, i meant as opposed to not having a 0.0.6 version on pypi | 14:32 |
| fungi | i'm not really worried about 0.0.6 missing from pypi as long as we get something newer than 0.0.5 published | 14:32 |
| tdasilva | ok | 14:32 |
| tdasilva | would you like me to propose the patch and you can approve? | 14:32 |
| tdasilva | i'm assuming infra is core? | 14:32 |
| fungi | yeah, i think we should burn 0.0.6 since the 0.0.5 tag was really for 0.0.6, and jump straight to 0.0.7. and yes happy to approve that change if you propose it, i think infra-core is still part of the core reviewer group on that repo | 14:34 |
| fungi | confirmed, infra-core has approval and tagging access for that repo | 14:36 |
| *** ociuhandu has quit IRC | 14:36 | |
| *** EmilienM is now known as EvilienM | 14:44 | |
| openstackgerrit | Thiago da Silva proposed openstack/nose-html-output master: Bump version to 0.0.7 https://review.opendev.org/692555 | 14:44 |
| tdasilva | fungi: ^^ | 14:45 |
| fungi | #status log re-tagged openstack/nose-html-output 0.0.5 as 0.0.6 with detailed tag message | 14:49 |
| openstackstatus | fungi: finished logging | 14:49 |
| openstackgerrit | Merged openstack/nose-html-output master: Bump version to 0.0.7 https://review.opendev.org/692555 | 14:50 |
| sgw | Morning folks, not sure who is around and who is traveling, I had a question about PBR and tagging yesterday evening. | 14:52 |
| sgw | AJaeger: smcginnis: Monty suggested I check with you guys maybe | 14:53 |
| *** bnemec is now known as beekneemech | 14:56 | |
| *** soniya29 has quit IRC | 14:58 | |
| fungi | #status log tagged openstack/nose-html-output 0.0.7 release | 15:01 |
| openstackstatus | fungi: finished logging | 15:01 |
| fungi | tdasilva: infra-root: ^ | 15:01 |
| clarkb | re nose-html-output we tried really really hard back in ~2013 to get everyone off nose | 15:01 |
| fungi | i'll check back in on my layover and see if we need a manually-built sdist for that uploaded | 15:01 |
| clarkb | and we basically succeeded with the exception of horizon and swift | 15:02 |
| fungi | but boarding now | 15:02 |
| clarkb | I personally have no interest in supporting the tooling since the whole reason for not supporting it was the pain | 15:02 |
| clarkb | and why we invested in not using nose | 15:02 |
| clarkb | but if horizon and swift are still using nose I think we can happily give them control of the tool | 15:02 |
| tdasilva | fungi: thank you!!! have a safe flight | 15:02 |
| *** kjackal has quit IRC | 15:03 | |
| clarkb | fungi: I'll see you in ~18 hours? | 15:04 |
| tdasilva | clarkb: IMO it makes more sense to have swift move to stestr, i'll bring it up to the rest of the community | 15:04 |
| clarkb | tdasilva: that works too. Mostly wanted to point out that I have no objections to letting other people mangae that toolchain if they rely on it | 15:05 |
| clarkb | and now I've got to start heading towards the airport myself | 15:07 |
| tdasilva | clarkb: gotcha, have a safe flight too, hope you all enjoy the summit | 15:08 |
| fungi | clarkb: yep, waiting to take off for atlanta and then have a couple hours there before my shanghai connection | 15:09 |
| fungi | safe travels! | 15:09 |
| fungi | thanks tdasilva! | 15:10 |
| mriedem | clarkb: i must be an idiot but i don't understand the interface to https://lamada.eu/dstat-graph/ | 15:13 |
| mriedem | i have https://13cf3dd11b8f009809dc-97cb3b32849366f5bed744685e46b266.ssl.cf5.rackcdn.com/692206/3/check/tempest-integrated-compute/35ecb4a/controller/logs/dstat-csv_log.txt.gz | 15:13 |
| mriedem | but what do i drag and drop ? | 15:13 |
| fungi | tdasilva: https://pypi.org/project/nosehtmloutput/#files shows a 0.0.7 sdist and wheel, so probably all set but let us know if that breaks | 15:13 |
| tdasilva | fungi: yep, testing that now | 15:14 |
| tdasilva | fungi: many thanks | 15:14 |
| *** ykarel has joined #openstack-infra | 15:14 | |
| fungi | yw! | 15:14 |
| fungi | sorry for the 6-year old and 2-year-old oversights which led to that | 15:15 |
| mriedem | clarkb: nevermind, i figured it out | 15:16 |
| mriedem | i'm a super smart professional | 15:16 |
| fungi | sgw: i may be able to dig into your question once i get to my next layover. saw it in scrollback yesterday, not sure i'll have an answer but can at least try to reproduce and see if i can identify the behavior differences | 15:17 |
| *** ykarel has quit IRC | 15:20 | |
| *** gyee has joined #openstack-infra | 15:20 | |
| fungi | sgw: reviewing your problem statement, you don't seem to specify which versions of pbr were had differing behaviors | 15:20 |
| sgw | fungi: thanks, I tried digging into it a little that's why I ended up asking. I have a feeling it has something to do with branching and where the tags reside on which branch | 15:21 |
| sgw | Sorry, the datever is from 3.1.1 and the semver is 4.2.0 (1.999.999.rc1.dev5). | 15:23 |
| fungi | and this is with the master branch checked out, or another branch? | 15:23 |
| fungi | sgw: oh! i see it | 15:24 |
| fungi | your newer tag has a "v" prefix | 15:24 |
| fungi | pbr only fairly recently grew the ability to treat vX.Y.Z... tags as the same as X.Y.Z... | 15:25 |
| sgw | yes master of fault, yes 'v', although the 2.0.1 is the latest but it's on the r/stx.2.0 branch | 15:25 |
| fungi | so i'm willing to bet 3.1.1 ignores v-tags as not being version numbers | 15:25 |
| fungi | anyway, taking off, i have to disconnect. anyway i'd put money on that being the difference | 15:25 |
| fungi | up, up and /away! | 15:25 |
| sgw | fair enough, we can use the constraints to get 4.2, safe travels! | 15:26 |
| *** lucasagomes has quit IRC | 15:27 | |
| *** prometheanfire has quit IRC | 15:28 | |
| *** prometheanfire has joined #openstack-infra | 15:29 | |
| *** igordc has joined #openstack-infra | 15:36 | |
| *** igordc has quit IRC | 15:47 | |
| AJaeger | sgw: sorry, cannot help with pbr | 15:47 |
| sgw | AJaeger: thanks, other folks are looking, I am going to continue to dig. | 15:47 |
| *** Erifkard has quit IRC | 15:51 | |
| *** rkukura has joined #openstack-infra | 16:05 | |
| *** chandankumar is now known as raukadah | 16:12 | |
| *** markvoelker has quit IRC | 16:12 | |
| *** raukadah is now known as raukadah|ptg | 16:12 | |
| *** mriedem is now known as mriedem_feeds | 16:16 | |
| clarkb | mriedem_feeds: ya the ui is bad :( | 16:21 |
| *** mattw4 has joined #openstack-infra | 16:22 | |
| *** markvoelker has joined #openstack-infra | 16:23 | |
| *** EvilienM is now known as EmilienM | 16:35 | |
| *** iurygregory has quit IRC | 16:41 | |
| *** goldyfruit___ has joined #openstack-infra | 16:45 | |
| *** goldyfruit_ has quit IRC | 16:47 | |
| *** antonym has quit IRC | 16:48 | |
| *** ociuhandu has joined #openstack-infra | 16:52 | |
| *** jaosorior has quit IRC | 17:01 | |
| *** ykarel has joined #openstack-infra | 17:06 | |
| fungi | made it to atlanta. boarding shanghai flight in an hour if there's anything else which needs looking at | 17:09 |
| *** igordc has joined #openstack-infra | 17:13 | |
| *** antonym has joined #openstack-infra | 17:22 | |
| *** ociuhandu has quit IRC | 17:27 | |
| *** ociuhandu has joined #openstack-infra | 17:29 | |
| *** ykarel is now known as ykarel|away | 17:31 | |
| *** ociuhandu has quit IRC | 17:33 | |
| *** mriedem_feeds is now known as mriedem | 17:34 | |
| mriedem | fungi: i have this one mole that could be a problem... | 17:34 |
| fungi | probably best if you just get rid of all your skin | 17:36 |
| *** ykarel|away has quit IRC | 17:39 | |
| Shrews | fungi: my favorite part about ATL is I always have to traverse the entire airport to get to my next gate | 17:41 |
| Shrews | and by "my favorite" i mean "the worst" | 17:41 |
| fungi | yep, a->f this layover, so... *almost* the entire airport | 17:43 |
| fungi | only t->f would have been farther | 17:43 |
| *** eharney has quit IRC | 17:44 | |
| fungi | still, it's got nothing on a gatwick->heathrow layover. those get miserable | 17:45 |
| Shrews | did that once too. i think the bus ride between the two caused enough misery to force me to forget most of it | 17:46 |
| fungi | yeah, i've learned to watch out for those and pick a different itinerary | 17:48 |
| Shrews | fungi: how long is the next leg of the journey? | 17:50 |
| fungi | i'm afraid to look | 17:55 |
| fungi | thankfully my boarding pass doesn't say | 17:56 |
| fungi | quick timezone math says 16.5 hours in the air | 17:59 |
| fungi | based on their arrival estimate anyway | 17:59 |
| Shrews | ouch | 18:00 |
| fungi | infra-root: if anyone gets a chance to dig into it, i think we might be failing to renew the letsencrypt cert for opendev.org... certcheck complained it expires in 29 days, but i haven't managed to track down where we're logging those renewals | 18:02 |
| fungi | my flight's boarding in a few moments and then i'll be going dark for... well, far too long | 18:02 |
| *** ralonsoh has quit IRC | 18:06 | |
| *** jerryz has joined #openstack-infra | 18:10 | |
| fungi | okay, the announcements say my tz math is wrong, only 15.5hrs in the air | 18:17 |
| rosmaita | fungi: safe travels, see you in shanghai ... enjoy your 15.5hrs of airline comfort | 18:18 |
| *** ociuhandu has joined #openstack-infra | 18:20 | |
| *** jaosorior has joined #openstack-infra | 18:24 | |
| fungi | thanks! | 18:24 |
| fungi | i'll do my best to just be unconscious as much of that as i can manage | 18:26 |
| *** ociuhandu has quit IRC | 18:28 | |
| *** aedc has quit IRC | 19:01 | |
| *** jaosorior has quit IRC | 19:25 | |
| *** whoami-rajat has joined #openstack-infra | 19:34 | |
| *** eharney has joined #openstack-infra | 19:35 | |
| *** markvoelker has quit IRC | 19:36 | |
| *** rkukura has quit IRC | 19:43 | |
| *** dklyle has quit IRC | 19:44 | |
| *** dklyle has joined #openstack-infra | 19:51 | |
| *** markvoelker has joined #openstack-infra | 19:53 | |
| *** jamesdenton has quit IRC | 19:57 | |
| *** dklyle has quit IRC | 20:00 | |
| *** dklyle has joined #openstack-infra | 20:00 | |
| *** ociuhandu has joined #openstack-infra | 20:30 | |
| *** ociuhandu has quit IRC | 20:36 | |
| *** ociuhandu has joined #openstack-infra | 20:39 | |
| *** ociuhandu has quit IRC | 20:43 | |
| *** slaweq has joined #openstack-infra | 20:46 | |
| *** ociuhandu has joined #openstack-infra | 20:47 | |
| *** igordc has quit IRC | 20:50 | |
| *** sreejithp has joined #openstack-infra | 20:53 | |
| *** haleyb has quit IRC | 20:57 | |
| *** ociuhandu has quit IRC | 20:57 | |
| *** haleyb has joined #openstack-infra | 21:00 | |
| *** ociuhandu has joined #openstack-infra | 21:01 | |
| *** slaweq has quit IRC | 21:05 | |
| *** ociuhandu has quit IRC | 21:06 | |
| *** goldyfruit_ has joined #openstack-infra | 21:08 | |
| *** dklyle has quit IRC | 21:08 | |
| *** goldyfruit___ has quit IRC | 21:10 | |
| *** dklyle has joined #openstack-infra | 21:14 | |
| *** Buggys has quit IRC | 21:21 | |
| *** dklyle has quit IRC | 21:22 | |
| *** sreejithp has quit IRC | 21:22 | |
| *** dklyle has joined #openstack-infra | 21:23 | |
| *** goldyfruit___ has joined #openstack-infra | 21:32 | |
| *** goldyfruit_ has quit IRC | 21:34 | |
| *** rlandy|ruck has quit IRC | 21:34 | |
| *** dklyle has quit IRC | 21:38 | |
| *** mriedem has quit IRC | 21:39 | |
| *** dklyle has joined #openstack-infra | 21:44 | |
| *** diablo_rojo has joined #openstack-infra | 21:47 | |
| *** diablo_rojo has quit IRC | 21:48 | |
| *** diablo_rojo has joined #openstack-infra | 21:49 | |
| *** diablo_rojo has quit IRC | 21:51 | |
| *** dklyle has quit IRC | 21:51 | |
| *** diablo_rojo has joined #openstack-infra | 21:51 | |
| *** dklyle has joined #openstack-infra | 21:52 | |
| *** dklyle has quit IRC | 21:53 | |
| *** dklyle has joined #openstack-infra | 21:53 | |
| *** ociuhandu has joined #openstack-infra | 21:53 | |
| *** diablo_rojo__ has joined #openstack-infra | 21:59 | |
| *** diablo_rojo has quit IRC | 22:01 | |
| *** diablo_rojo__ has quit IRC | 22:02 | |
| *** diablo_rojo__ has joined #openstack-infra | 22:02 | |
| *** whoami-rajat has quit IRC | 22:04 | |
| *** ociuhandu has quit IRC | 22:04 | |
| *** ociuhandu has joined #openstack-infra | 22:11 | |
| *** aedc has joined #openstack-infra | 22:13 | |
| *** ociuhandu has quit IRC | 22:19 | |
| *** diablo_rojo__ has quit IRC | 22:19 | |
| *** dklyle has quit IRC | 22:22 | |
| *** jamesdenton has joined #openstack-infra | 22:35 | |
| *** Goneri has quit IRC | 22:35 | |
| *** mattw4 has quit IRC | 22:36 | |
| *** dklyle has joined #openstack-infra | 22:37 | |
| *** gyee has quit IRC | 22:39 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Revert "Revert "Add the process environment to zuul.conf parser"" https://review.opendev.org/692469 | 22:41 |
| *** pcaruana has quit IRC | 22:44 | |
| *** goldyfruit___ has quit IRC | 22:51 | |
| *** rosmaita has left #openstack-infra | 23:03 | |
| *** ociuhandu has joined #openstack-infra | 23:11 | |
| *** ociuhandu has quit IRC | 23:16 | |
| *** rfolco has quit IRC | 23:18 | |
| *** ociuhandu has joined #openstack-infra | 23:38 | |
| *** rh-jelabarre has quit IRC | 23:41 | |
| *** ociuhandu has quit IRC | 23:46 | |
| *** ociuhandu has joined #openstack-infra | 23:47 | |
| *** ociuhandu has quit IRC | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!