| tonyb | clarkb: the reviews seem good. I have an idea to make things a little clearer. | 00:40 |
|---|---|---|
| tonyb | clarkb: I'll look over the whole redirection process as it's somewhat confusing at best | 00:41 |
| frickler | clarkb: I already deleted my autohold, so no blocker from my side | 07:43 |
| frickler | if seems there are lots of stuck instances and images in that cloud, since it is going to be redeployed anyway, going ahead with the next cleanup changes should be fine I think | 07:52 |
| opendevreview | Bernhard Berg proposed zuul/zuul-jobs master: prepare-workspace-git: Add allow deleting current branch https://review.opendev.org/c/zuul/zuul-jobs/+/917539 | 09:40 |
| opendevreview | Jens Harbott proposed opendev/system-config master: Increase the number of mailman3 outgoing runners to 4 https://review.opendev.org/c/opendev/system-config/+/920765 | 09:41 |
| oschwart | hey folks, good day. Can I ask zuul admins for an autohold, for this change https://review.opendev.org/c/openstack/designate-tempest-plugin/+/920653, this job designate-bind9 | 09:47 |
| oschwart | reason: I would like to debug this failure https://zuul.opendev.org/t/openstack/build/100d3262cda94ed2947435a353e5cf7e/console, I just got a 255 after I ran subprocess.CalledProcessError, but I don't really know what happened there | 09:48 |
| frickler | oschwart: hold set up, we'll need your ssh public key later to give you access | 10:12 |
| oschwart | frickler: sorry, I only see now that you recheck to hold the node https://review.opendev.org/c/openstack/designate-tempest-plugin/+/920653/comments/a7aacd7e_d4c26fac | 10:29 |
| oschwart | I uploaded another patch-set that prints additional exception details (maybe something will appear) | 10:30 |
| oschwart | frickler: so as far as I understand, my new patch-set will trigger the same event as the recheck for the autohold, right? like we will still get it | 10:32 |
| frickler | oschwart: yes, the filter will match any patchset, and the next failing job will be held, regardless of how it was triggered | 10:39 |
| oschwart | frickler: ack, thanks | 10:40 |
| frickler | oschwart: ssh root@104.239.136.190 should work for you now, let us know if you need more help or once you are done with debugging | 11:29 |
| oschwart | frickler: I could ssh the host, thanks | 11:33 |
| oschwart | frickler: how much time can I have that autohold? I would like to consult with a colleague that will start working soon | 12:38 |
| frickler | oschwart: any time you need, we'd likely start asking whether you stiil need to node after a week or so, but no real pressure anywhere | 12:49 |
| oschwart | great, thanks | 12:56 |
| yoctozepto | nebulous will be moving out of opendev because of the in-consortium pushback of the technology stack (gerrit, zuul, launchpad) being too different to the well-established, more-integrated, de-facto standard of github; fungi, clarkb, corvus, frickler and others - thank you for the common journey so far and here's hoping I get to play with the | 13:54 |
| yoctozepto | opendev community in the future :-) | 13:54 |
| Clark[m] | frickler: yes, we can land the extra cleanup changes and then use nodepool erase when done if anything remains | 14:18 |
| clarkb | yoctozepto: thank you for the heads up. Maybe let us know when you aren't using zuul anymore (basically no more merging of stuff in gerrit) and we can wind down the zuul tenant | 14:40 |
| clarkb | I've approved https://review.opendev.org/c/openstack/project-config/+/921073 as the next step in inmotion cleanup | 14:46 |
| opendevreview | Merged openstack/project-config master: Clean up diskimages from inmotion cloud https://review.opendev.org/c/openstack/project-config/+/921073 | 15:25 |
| clarkb | I'll check on that in a bit and continue to approve the other two changes if that looks good | 15:25 |
| clarkb | looking at the gerrit 3.9 to 3.10 upgrade logs it does look like the reindexing is done online. That should make the next upgrade straightforward from a reindexing standpoint | 15:33 |
| clarkb | the only inmotion images left are in a deleting state. I guess we can proceed with the next change now | 15:51 |
| clarkb | I need to finish up some morning thigns then I'll approve that and get ready to run the erase command | 15:52 |
| opendevreview | Monty Taylor proposed zuul/zuul-jobs master: Remove default value from ensure_nodejs https://review.opendev.org/c/zuul/zuul-jobs/+/921066 | 16:13 |
| opendevreview | Monty Taylor proposed zuul/zuul-jobs master: Remove deprecated javascript jobs https://review.opendev.org/c/zuul/zuul-jobs/+/921067 | 16:13 |
| clarkb | I have approved https://review.opendev.org/c/openstack/project-config/+/921074 and will check the nodepool state after that applies | 16:40 |
| clarkb | tonyb: any cahnce you can review https://review.opendev.org/c/opendev/system-config/+/921075 it is the other step in inmotion cleanup | 16:40 |
| opendevreview | Merged openstack/project-config master: Remove the inmotion cloud entirely https://review.opendev.org/c/openstack/project-config/+/921074 | 17:01 |
| clarkb | I think that applied cleanly. At least the nl02 logs lookabout how I woudl expect them to | 17:14 |
| clarkb | I've run the nodepool erase command which cleaned up the deleting nodes and images as expected. I did have to docker exec -it the command because it asks for confirmation. Took me half a second to realize that was why it threw an exception first | 17:17 |
| clarkb | tonyb: https://review.opendev.org/c/opendev/system-config/+/921075 is next up if you have time to review that | 17:17 |
| tonyb | clarkb: done | 17:37 |
| clarkb | thanks I think that gets us in a state where I can tell yuriys we're ready for them to do the clodu side changes when I talk to them later today | 17:44 |
| fungi | yoctozepto: good luck! i just count myself lucky i get to work on projects who value open source services and aren't lured by the siren song of proprietary commercial services who leech off open source community efforts | 17:45 |
| fungi | not everyone sees free/libre open source as a goal though, some only consider it a means to an end | 17:46 |
| tonyb | clarkb: when you have the call details please pass them on. | 17:48 |
| tonyb | clarkb: Are you okay with me +A'ing Fixups: for jitsi-meet | https://review.opendev.org/c/opendev/system-config/+/921320 ? | 18:08 |
| clarkb | tonyb: yup I don't expect that to have any issues | 18:18 |
| opendevreview | Merged opendev/system-config master: Remove inmotion cloud management https://review.opendev.org/c/opendev/system-config/+/921075 | 18:56 |
| opendevreview | Merged opendev/system-config master: Increase the number of mailman3 outgoing runners to 4 https://review.opendev.org/c/opendev/system-config/+/920765 | 18:56 |
| clarkb | the complaints from mailman are due to the database image being updated and the service losing connectivity to it. I don't think we've applied the new extra config data yet and that is unrelated. Working to confirm this | 19:55 |
| clarkb | nevermind looks like services all restarted just now and the new instances count is there | 19:55 |
| clarkb | it must've updated the db first before stopping the other services, then noticed config updated and restarted everything? | 19:56 |
| clarkb | those restarts are not fast, but web is responding for me again now | 19:58 |
| fungi | in the past we saw it send similar errors during a db container restart | 20:17 |
| fungi | though they were being sent to the wrong address previously and ended up kicked back as undeliverable a week later | 20:17 |
| fungi | hopefully now they come through ~immediately instead | 20:18 |
| *** dasm is now known as Guest8716 | 20:46 | |
| Clark[m] | Yes I think it was immediate based on docker timestamps | 21:36 |
| clarkb | infra-root I followed up to the thread with yuriys with the two things they wanted from us to start setting up the new cloud. The first thign is ssh keys which I pointed to in our config management. The other is a list of email addrs to create dashboard accounts in the system that manages the openstack stuff (so a level above openstack not horizon). I asked that each of us that wants | 23:05 |
| clarkb | one explicitly opt into that which you can do replying to my email in the thread | 23:05 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!