| opendevreview | Stephen Finucane proposed openstack/project-config master: Add ironic-core, nova-core to osc/sdk repo config https://review.opendev.org/c/openstack/project-config/+/924048 | 09:25 |
|---|---|---|
| opendevreview | Radosław Piliszek proposed openstack/project-config master: Mark NebulOuS as moved (retire on OpenDev) https://review.opendev.org/c/openstack/project-config/+/921725 | 09:54 |
| opendevreview | Takashi Kajinami proposed zuul/zuul-jobs master: Explicitly install public_suffix 5.1.1 https://review.opendev.org/c/zuul/zuul-jobs/+/924057 | 10:24 |
| opendevreview | Merged openstack/project-config master: Add ironic-core, nova-core to osc/sdk repo config https://review.opendev.org/c/openstack/project-config/+/924048 | 12:15 |
| mnaser | clarkb: sorry yesterday was a busy day, thanks for sending the email | 13:00 |
| fungi | also looks like all the core reviewers replied in favor | 13:09 |
| fungi | #status log Added Mohammed Naser to inactive x/ospurge reviewer groups following positive feedback from existing group members | 13:13 |
| opendevstatus | fungi: finished logging | 13:13 |
| opendevreview | Curt Moore proposed opendev/glean master: Add ability to set MTU for RH based systems https://review.opendev.org/c/opendev/glean/+/924066 | 13:31 |
| opendevreview | Curt Moore proposed opendev/glean master: Fix issue when adding bonded interfaces via udev https://review.opendev.org/c/opendev/glean/+/924068 | 13:35 |
| opendevreview | Harry Kominos proposed openstack/diskimage-builder master: feat: Add new autoupdates elemenent https://review.opendev.org/c/openstack/diskimage-builder/+/924069 | 13:36 |
| opendevreview | Curt Moore proposed opendev/glean master: Specify proper interface type https://review.opendev.org/c/opendev/glean/+/924070 | 13:40 |
| opendevreview | Curt Moore proposed opendev/glean master: Configure proper interface type https://review.opendev.org/c/opendev/glean/+/924070 | 13:49 |
| opendevreview | Curt Moore proposed opendev/glean master: Add ability to parse DNS configuration from all potential locations https://review.opendev.org/c/opendev/glean/+/924071 | 13:49 |
| opendevreview | Harry Kominos proposed openstack/diskimage-builder master: feat: Add new autoupdates element https://review.opendev.org/c/openstack/diskimage-builder/+/924069 | 13:53 |
| opendevreview | Manuel CREACH proposed openstack/diskimage-builder master: On nodes with Fiber Channel multipath, , the growvols command is failing sometimes with the following stack trace : https://review.opendev.org/c/openstack/diskimage-builder/+/924074 | 14:36 |
| opendevreview | Manuel CREACH proposed openstack/diskimage-builder master: On nodes with Fiber Channel multipath, , the growvols command is failing sometimes with the following stack trace : https://review.opendev.org/c/openstack/diskimage-builder/+/924074 | 14:39 |
| opendevreview | Harry Kominos proposed openstack/diskimage-builder master: feat: Add new autoupdates element https://review.opendev.org/c/openstack/diskimage-builder/+/924069 | 15:25 |
| clarkb | fwiw I tried to review ^ those chanegs above as I think for the most part they represent some good improvements to those tools. But I'll be out the next few business days so may be worth keeping an eye on if there is forward progress on them | 16:43 |
| fungi | yeah, i was sort of watching the glean changes as well | 16:45 |
| clarkb | I'm noticing that arm64 pypi issues seem to persist. It looks like /var/cache/apache2 may be full there and htcacheclean isn't keeping up. It looks like the limit for htcacheclean is 60000M but df -h shows the partition is only 49GB large? | 18:22 |
| clarkb | anyway I suspect that may be impacing responses for content. I'll work on a patch to dial that limit down a bit and see if it clears itself up | 18:22 |
| clarkb | oh 60000M is our global value and I guess we didn't realize this arm mirror had a smaller disk? | 18:23 |
| fungi | i don't think it was always that high. maybe it got raised without checking all the volumes | 18:28 |
| fungi | huh, actually the most recent change was to lower it | 18:29 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Limit linaro mirror Apache cache size https://review.opendev.org/c/opendev/system-config/+/924088 | 18:29 |
| clarkb | fungi: I suspect it was just missed when we deployed that new mirror node on the new cloud | 18:30 |
| clarkb | we may need to manually clear the cache too if htcacheclean is unable to catch back up again with the new limit, but we can figure that out after we land the change to reduce the limit | 18:31 |
| fungi | yeah, looks like our other arm mirror has 100gb volumes instead of 50gb | 18:32 |
| fungi | the linaro mirror has a single 100gb cinder volume attached as vdb, and that's carved up into two logical volumes | 18:34 |
| fungi | looks like we have sufficient quota for more | 18:35 |
| clarkb | fungi: I think that is misleading though | 18:35 |
| clarkb | we need the extra disk there for the test nodes | 18:35 |
| fungi | | maxTotalVolumeGigabytes | 1000 | | 18:35 |
| fungi | | totalGigabytesUsed | 200 | | 18:35 |
| fungi | oh | 18:35 |
| clarkb | it is an extremely space constrained environment due to how the works on arm donation ended up | 18:36 |
| fungi | though our other project does say it's not using any, maybe it's not reflected in quotas at all | 18:36 |
| clarkb | correct, the quotas are just defaults set up by kolla or cinder or whatever. But they don't necessarily reflect the reality of how much disk space is available on the hosts | 18:37 |
| fungi | makes sense. too bad | 18:37 |
| clarkb | https://review.opendev.org/c/opendev/system-config/+/924088 passes testing if we think the new numbers in there for the cache size make sense | 19:09 |
| fungi | yep, approved | 19:14 |
| opendevreview | Merged opendev/system-config master: Limit linaro mirror Apache cache size https://review.opendev.org/c/opendev/system-config/+/924088 | 19:49 |
| clarkb | that has applied but it applied 14 minutse after the current cron run started so we won't see it try to reduce the size just yet | 20:23 |
| * yoctozepto is reminding about https://review.opendev.org/c/openstack/project-config/+/921725 before going to sleep o/ | 20:28 | |
| clarkb | the last 60000M run has ended so at 2100UTC we should start a 40000M run | 20:28 |
| clarkb | yoctozepto: fungi: I left a note on there so didn't approve it. I'm also out the next few days so didn't want to approve it if we decide to make that edit | 20:31 |
| clarkb | my neighbor has a truck parked just perfect so the sun reflects off the windshield through the tiny gap in my blinds and into by eyeballs as I sit here typing this | 20:43 |
| fungi | aluminum foil your windows, problem solved | 21:08 |
| clarkb | I flipped the blinds over and that did it. | 21:09 |
| clarkb | the new htcacheclean limit is running now. I'm waiting for it to actually delete stuff so I can recheck a change | 21:09 |
| clarkb | I suspect that it must crawl the tree and make an accounting first then do deletions rather than deleting all at once? | 21:09 |
| clarkb | er rather than deleting as it goes | 21:09 |
| clarkb | there are 9.4GB free now | 21:25 |
| clarkb | so ya it seemed to do the accounting then unlink all at once | 21:26 |
| clarkb | https://review.opendev.org/c/openstack/diskimage-builder/+/924019 is the change I rechecked to see if arm is happier now | 21:33 |
| clarkb | woo the rpm functest job is still failing on arm64 but the deb functests pass now so ya I think this was at least part of the issue | 21:49 |
| clarkb | disk usage is remaining stable on the mirror as well | 23:18 |
| clarkb | I'm going to consider that done for now | 23:18 |
| fungi | thanks for (finding and) fixing it! | 23:51 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!