| *** cloudnull has quit IRC | 00:57 | |
| *** Dmitrii-Sh has quit IRC | 01:00 | |
| *** cloudnull has joined #opendev | 01:01 | |
| *** Dmitrii-Sh has joined #opendev | 01:02 | |
| ianw | OSError: [Errno 28] No space left on device | 01:13 |
|---|---|---|
| ianw | this isn't great, that was on review with the borg run | 01:13 |
| clarkb | it looks like there is plenty of disk on the various fs's there now | 01:15 |
| clarkb | ~/.cache appears to be where borg spools to | 01:16 |
| ianw | http://paste.openstack.org/show/799900/ | 01:16 |
| clarkb | https://borgbackup.readthedocs.io/en/stable/quickstart.html#important-note-about-free-space | 01:16 |
| ianw | it completed to vexxhost but not rax | 01:16 |
| clarkb | oh was it hte remote that ran out of disk? | 01:16 |
| ianw | unclear | 01:16 |
| ianw | they both would have been running at the same time, i wonder if that's a problem | 01:17 |
| ianw | they're separated by about 40 minutes in cron, but that's not enough for the initial backup | 01:17 |
| clarkb | oh ya I wonder if it spools twice in that instance | 01:17 |
| clarkb | maybe we should offset by like 12 hours instead? | 01:17 |
| ianw | yeah, it would probably be better to split the day in half | 01:17 |
| ianw | This archive: 104.73 GB 83.52 GB 60.39 GB | 01:18 |
| *** mlavalle has quit IRC | 01:18 | |
| clarkb | ianw: I think we should consider setting borg config /path/to/repo additional_free_space 2G too (from that doc I linked) | 01:19 |
| clarkb | basically borg will police itself on using up too much disk | 01:20 |
| clarkb | though its still not clear to me if that is the target or the source | 01:21 |
| clarkb | ah ok that is on the target side so not related I don't think | 01:21 |
| clarkb | unless it was the target that filled | 01:22 |
| ianw | i wouldn't think so, that shoul dhave been on the 3tb rax volume | 01:22 |
| ianw | i might try a manual run of rax backup now and see what happens with space | 01:22 |
| clarkb | k, I guess monitor both sides? | 01:23 |
| clarkb | but that seems reasonable | 01:23 |
| ianw | it's running and review is @ /dev/xvda1 39G 24G 15G 62% / | 01:23 |
| ianw | it might have run for long enough for cacti to catch it | 01:24 |
| clarkb | looks like it is running against vexxhost not rax | 01:24 |
| clarkb | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=32&rra_id=all looks fine | 01:25 |
| clarkb | ok I need to figure out dinner now | 01:25 |
| clarkb | but your debugging plan seemsreaosnable | 01:25 |
| ianw | oops pasted the wrong one | 01:25 |
| ianw | do we need /home/gerrit2/.launchpadlib/cache? | 01:27 |
| clarkb | fungi: ^ ? I don't know | 01:29 |
| ianw | it's 658848 files, which is probably better if not being tracked | 01:34 |
| clarkb | ++ | 01:35 |
| ianw | it's probably jeepyb behind that i guess | 01:35 |
| clarkb | ya | 01:35 |
| clarkb | my hunch is we probably don't need to back it up | 01:35 |
| clarkb | and its just keeping group membership type deals in sync | 01:35 |
| clarkb | and we can always rebuild it from lp directly if necessary | 01:36 |
| fungi | clarkb: yes, /home/gerrit2/.launchpadlib/cache is the openid session key for jeepyb gerrit hooks which do bug and bp updating | 01:45 |
| fungi | we have instructions for regenerating it, i think | 01:45 |
| clarkb | huh any idea why there are so many files? | 01:45 |
| fungi | i think it's also caching api calls? | 01:47 |
| fungi | that is rather a lot | 01:47 |
| fungi | oh, it's /home/gerrit2/.launchpadlib/creds we need to preserve | 01:47 |
| fungi | cache will probably repopulate on its own | 01:47 |
| fungi | sorry for the mixup | 01:47 |
| ianw | /home/gerrit2/review_site/tmp is another one | 02:02 |
| ianw | that's another ~90,000 files | 02:04 |
| ianw | cd review_site/cache | 02:06 |
| *** cloudnull has quit IRC | 02:10 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: borg-backup: space out cron jobs evenly https://review.opendev.org/762270 | 02:14 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: review: trim backups https://review.opendev.org/762271 | 02:14 |
| *** cloudnull has joined #opendev | 02:14 | |
| *** stevebaker has quit IRC | 02:20 | |
| ianw | i think we can skip the indexes too. if we're in a situation of restoring, i can't imagine the indexes would be valid, and it seems we have errors due to things coming and going during the backup process | 02:22 |
| fungi | makes sense | 02:23 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: review: trim backups https://review.opendev.org/762271 | 02:25 |
| ianw | ok, doh, it looks like i forgot to symlink /opt/backups to /opt/backups-202010 on the rax ord server (that's the 3tb volume), so the disk really has filled up | 02:33 |
| ianw | i should maybe put in an assert that's a symlink; though it's not in testing | 02:34 |
| ianw | ... i guess if i just move it it should work | 02:35 |
| *** hamalq has quit IRC | 02:39 | |
| *** Dmitrii-Sh has quit IRC | 03:21 | |
| *** Dmitrii-Sh has joined #opendev | 03:24 | |
| *** d34dh0r53 has quit IRC | 03:34 | |
| *** ykarel has joined #opendev | 05:18 | |
| *** ykarel_ has joined #opendev | 05:43 | |
| *** ykarel has quit IRC | 05:46 | |
| *** ysandeep|away is now known as ysandeep|ruck | 06:00 | |
| *** ykarel__ has joined #opendev | 06:14 | |
| *** ykarel_ has quit IRC | 06:17 | |
| *** bhagyashris is now known as bhagyashris|off | 06:17 | |
| *** ykarel__ has quit IRC | 06:26 | |
| *** ykarel__ has joined #opendev | 06:26 | |
| openstackgerrit | Merged opendev/system-config master: Use sudo to move applytest results https://review.opendev.org/762181 | 06:36 |
| *** marios has joined #opendev | 06:40 | |
| openstackgerrit | Merged opendev/ansible-role-puppet master: Add delete to synchronize https://review.opendev.org/762078 | 07:01 |
| brinzhang_ | fungi, ttx: thanks, it's ok now :D | 07:20 |
| *** tosky has joined #opendev | 07:23 | |
| *** ykarel__ is now known as ykarel | 07:24 | |
| *** ysandeep|ruck is now known as ysandeep|lunch | 07:32 | |
| *** eolivare has joined #opendev | 07:34 | |
| *** hjensas has quit IRC | 07:38 | |
| *** ralonsoh has joined #opendev | 07:44 | |
| openstackgerrit | Brin Zhang proposed openstack/project-config master: Create inspur/python-venusclient repository https://review.opendev.org/762092 | 07:59 |
| openstackgerrit | Brin Zhang proposed openstack/project-config master: Create inspur/venus-specs repository https://review.opendev.org/762095 | 07:59 |
| openstackgerrit | Brin Zhang proposed openstack/project-config master: Create inspur/venus-tempest-plugin repository https://review.opendev.org/762309 | 07:59 |
| *** cloudnull has quit IRC | 08:00 | |
| *** cloudnull has joined #opendev | 08:00 | |
| *** andrewbonney has joined #opendev | 08:02 | |
| *** lpetrut has joined #opendev | 08:13 | |
| *** fressi has joined #opendev | 08:14 | |
| *** yoctozepto has quit IRC | 08:29 | |
| *** yoctozepto has joined #opendev | 08:30 | |
| *** roman_g has joined #opendev | 08:45 | |
| *** ysandeep|lunch is now known as ysandeep|ruck | 09:07 | |
| openstackgerrit | Brin Zhang proposed openstack/project-config master: Create inspur/venus-tempest-plugin repository https://review.opendev.org/762309 | 09:20 |
| *** sboyron has joined #opendev | 09:30 | |
| *** brinzhang0 has joined #opendev | 09:33 | |
| *** brinzhang_ has quit IRC | 09:36 | |
| *** mgoddard has quit IRC | 09:39 | |
| *** dtantsur|afk is now known as dtantsur | 09:45 | |
| *** slaweq has joined #opendev | 09:57 | |
| *** fressi has quit IRC | 10:12 | |
| *** iurygregory has quit IRC | 10:19 | |
| *** DSpider has joined #opendev | 10:29 | |
| *** fressi has joined #opendev | 10:30 | |
| *** iurygregory has joined #opendev | 10:36 | |
| *** iurygregory has quit IRC | 11:11 | |
| *** slaweq has quit IRC | 11:32 | |
| *** iurygregory_ has joined #opendev | 11:35 | |
| *** iurygregory_ is now known as iurygregory | 11:35 | |
| *** fressi has quit IRC | 11:44 | |
| *** fressi has joined #opendev | 11:53 | |
| *** fressi has quit IRC | 11:59 | |
| *** slaweq has joined #opendev | 12:25 | |
| *** lourot has quit IRC | 12:37 | |
| *** lourot has joined #opendev | 12:37 | |
| openstackgerrit | Dmitry Tantsur proposed opendev/glean master: Do not fail if routes are missing from a network https://review.opendev.org/762339 | 12:50 |
| *** d34dh0r53 has joined #opendev | 13:05 | |
| *** fressi has joined #opendev | 13:10 | |
| dtantsur | heya, folks! has anyone *actually* tried glean with NM? | 13:53 |
| dtantsur | it seems to me that it fails to change interfaces to static configuration | 13:53 |
| fungi | nm support got added to glean because we couldn't boot at least some distro without it, but now i can't recall which one | 14:33 |
| fungi | it's also possible that it only happens to work in that combination in the specific network environments our providers have, there are almost certainly corner cases | 14:34 |
| *** fressi has quit IRC | 15:00 | |
| openstackgerrit | zbr proposed zuul/zuul-jobs master: DNM: trigger ensure-docker https://review.opendev.org/762352 | 15:03 |
| *** lpetrut has quit IRC | 15:08 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Include docker_registry when tagging docker-image https://review.opendev.org/762353 | 15:17 |
| *** sboyron has quit IRC | 15:53 | |
| *** mlavalle has joined #opendev | 16:00 | |
| clarkb | dtantsur: fungi we boot centos 7 and 8 on all pur test nodes with glean and network manager | 16:03 |
| clarkb | on rax these are statically configured interfaces, everywhere else it is dhcp and router advertisements iirc | 16:03 |
| fungi | "statically configured" via configdrive, to be precise | 16:04 |
| clarkb | dtantsur: glean doesnt change configs fwiw, if there is preexisting config it ignores it iirc | 16:04 |
| clarkb | it ignores the interface I mean and assumes you know better than it | 16:04 |
| dtantsur | clarkb: not exactly. it ignores the existing config for DHCP case, it does write new configs for static configuration. | 16:06 |
| dtantsur | the problem is: NM ignores the new config files, and so far I've been unable to make NM re-read them even by restarting it | 16:06 |
| clarkb | dtantsur: well it checks for config files existing and foesnt modify them if they exost | 16:06 |
| dtantsur | it does, I can show you | 16:06 |
| clarkb | hrm I seem to recall the eay to rerun glean is to delete the files first but ok | 16:07 |
| dtantsur | clarkb: see, https://opendev.org/opendev/glean/src/branch/master/glean/cmd.py#L316-L319 happens after the static types are processed | 16:07 |
| dtantsur | which may be an oversight, but I find it useful :) | 16:07 |
| clarkb | but ya all our rax centos nodes are booted then statically co figured with network mamager | 16:07 |
| dtantsur | or rather: with NM the file *will* exist at least for the primary interface | 16:07 |
| dtantsur | okay, I tested on fedora, maybe it has some behavior change.. I'll test centos 8 now | 16:08 |
| dtantsur | while I have your attention, please check https://review.opendev.org/762339 | 16:08 |
| openstackgerrit | Paul Belanger proposed zuul/zuul-jobs master: Include docker_registry when tagging docker-image https://review.opendev.org/762353 | 16:15 |
| *** jhesketh has quit IRC | 16:26 | |
| *** ykarel has quit IRC | 16:28 | |
| *** ysandeep|ruck is now known as ysandeep|away | 16:37 | |
| roman_g | Hello team, | 16:41 |
| roman_g | May I ask for a few lines of fresh Zuul logs for NODE_FAILURE, please? | 16:41 |
| roman_g | From this job: https://zuul.opendev.org/t/openstack/builds?job_name=airship-airshipctl-gate-test&project=airship/airshipctl | 16:42 |
| roman_g | That's for us to be able to show those logs to the cloud provider. | 16:42 |
| fungi | starting to skim logs on the launcher for that provider, it looks like we're limited to 150 vcpus and sometimes performing boot calls which would put us over that | 16:47 |
| fungi | roman_g: but most of the errors we're getting are like: | 16:48 |
| fungi | 2020-11-11 12:20:09,175 ERROR nodepool.NodeLauncher: [e: 60fe5dbc8e62440daa6098c0d0f86262] [node_request: 300-0011656199] [node: 0021720399] Detailed node error: No valid host was found. There are not enough hosts available. | 16:48 |
| fungi | so nova's replying back to us that it doesn't have a hypervisor host to place the instance on | 16:49 |
| roman_g | fungi thank you. Is that Nova API message? | 16:49 |
| fungi | "No valid host was found. There are not enough hosts available." is what openstacksdk says the detailed error response from the nova api was | 16:49 |
| fungi | roman_g: we've seen that error from them 687 times so far today since utc midnight | 16:51 |
| roman_g | Oh, wow. | 16:51 |
| roman_g | Thank you, fungi | 16:52 |
| clarkb | thats the same issue we've had with that cloud since the beginning fwiw | 16:53 |
| roman_g | I know. | 16:53 |
| clarkb | (also note we try 3 times before giving up and providing the node failure) | 16:53 |
| *** marios is now known as marios|out | 16:58 | |
| *** marios|out has quit IRC | 17:02 | |
| *** eolivare has quit IRC | 17:19 | |
| clarkb | melwitt: all 80 log processing daemons are still alive which is a good sign (implies oomkiller hasn't run and culled any of them) | 17:30 |
| fungi | that's swell news indeed | 17:37 |
| *** fressi has joined #opendev | 17:45 | |
| avass | zbr: as long as they're easily accessible I wouldn't complain. but they're invaluabe for searching logs | 17:49 |
| avass | woops | 17:49 |
| melwitt | clarkb: noice, that is good news | 18:07 |
| *** hamalq has joined #opendev | 18:09 | |
| *** hamalq has quit IRC | 18:09 | |
| *** hamalq has joined #opendev | 18:10 | |
| openstackgerrit | Merged zuul/zuul-jobs master: More E208 (final) https://review.opendev.org/761297 | 18:24 |
| * zbr is wondering how final is really that final. | 18:32 | |
| *** stevebaker has joined #opendev | 18:34 | |
| *** dtantsur is now known as dtantsur|afk | 18:43 | |
| *** roman_g has quit IRC | 18:59 | |
| *** stevebaker has quit IRC | 19:01 | |
| *** ralonsoh has quit IRC | 19:01 | |
| *** stevebaker has joined #opendev | 19:03 | |
| *** sboyron has joined #opendev | 19:27 | |
| *** Dmitrii-Sh has quit IRC | 19:41 | |
| *** slaweq has quit IRC | 20:50 | |
| ianw | clarkb / fungi: would you mind a poke @ https://review.opendev.org/#/c/762271/ and https://review.opendev.org/#/c/762270/1 just to try getting the backups under control a bit more. trims some bits of the review backup | 20:52 |
| clarkb | ianw: I like what the commit message says on https://review.opendev.org/#/c/762270/1/playbooks/roles/borg-backup/tasks/main.yaml but that ansible is wow :) | 20:54 |
| clarkb | I've got a meeting in a couple minutes but I'll do some math on paper to understand that then give it a proper review | 20:55 |
| ianw | pretty sure it works; with 2 backup hosts it works out to running at (5 + ((24 / 2) * 0)) and (5 + ((24 / 2) * 1)) ... :) | 20:55 |
| clarkb | ianw: minor note on https://review.opendev.org/#/c/762271/2 | 20:56 |
| *** sboyron has quit IRC | 21:37 | |
| *** Green_Bird has quit IRC | 21:37 | |
| *** sshnaidm|off has quit IRC | 21:37 | |
| *** bhagyashris|off has quit IRC | 21:37 | |
| *** chandankumar has quit IRC | 21:37 | |
| *** osmanlicilegi has quit IRC | 21:37 | |
| *** timburke has quit IRC | 21:37 | |
| *** tbarron has quit IRC | 21:37 | |
| *** sboyron has joined #opendev | 21:37 | |
| *** Green_Bird has joined #opendev | 21:37 | |
| *** sshnaidm|off has joined #opendev | 21:37 | |
| *** bhagyashris|off has joined #opendev | 21:37 | |
| *** timburke has joined #opendev | 21:37 | |
| *** chandankumar has joined #opendev | 21:37 | |
| *** osmanlicilegi has joined #opendev | 21:37 | |
| *** tbarron has joined #opendev | 21:37 | |
| *** stevebaker has quit IRC | 21:40 | |
| *** tosky has quit IRC | 21:40 | |
| *** toma4 has quit IRC | 21:40 | |
| *** amotoki has quit IRC | 21:40 | |
| *** ianw has quit IRC | 21:40 | |
| *** mordred has quit IRC | 21:40 | |
| *** persia has quit IRC | 21:40 | |
| *** priteau has quit IRC | 21:40 | |
| *** mrunge has quit IRC | 21:40 | |
| *** brinzhang0 has quit IRC | 21:40 | |
| *** elod has quit IRC | 21:40 | |
| *** hemanth_n has quit IRC | 21:40 | |
| *** zer0c00l has quit IRC | 21:40 | |
| *** sean-k-mooney has quit IRC | 21:40 | |
| *** priteau has joined #opendev | 21:41 | |
| *** mordred has joined #opendev | 21:41 | |
| *** persia has joined #opendev | 21:41 | |
| *** mrunge has joined #opendev | 21:41 | |
| *** brinzhang0 has joined #opendev | 21:41 | |
| *** elod has joined #opendev | 21:41 | |
| *** hemanth_n has joined #opendev | 21:41 | |
| *** zer0c00l has joined #opendev | 21:41 | |
| *** sean-k-mooney has joined #opendev | 21:41 | |
| *** mtreinish has quit IRC | 21:41 | |
| *** xavpaice has quit IRC | 21:41 | |
| *** owalsh has quit IRC | 21:41 | |
| *** gouthamr_ has quit IRC | 21:41 | |
| *** stevebaker has joined #opendev | 21:41 | |
| *** tosky has joined #opendev | 21:41 | |
| *** toma4 has joined #opendev | 21:41 | |
| *** amotoki has joined #opendev | 21:41 | |
| *** ianw has joined #opendev | 21:41 | |
| *** gnuoy has quit IRC | 21:41 | |
| *** calcmandan has quit IRC | 21:41 | |
| *** valleedelisle has quit IRC | 21:41 | |
| *** mhu has quit IRC | 21:41 | |
| *** sgw has quit IRC | 21:41 | |
| *** stephenfin has quit IRC | 21:41 | |
| *** kopecmartin has quit IRC | 21:41 | |
| *** gnuoy has joined #opendev | 21:42 | |
| *** calcmandan has joined #opendev | 21:42 | |
| *** valleedelisle has joined #opendev | 21:42 | |
| *** sgw has joined #opendev | 21:42 | |
| *** mhu has joined #opendev | 21:42 | |
| *** kopecmartin has joined #opendev | 21:42 | |
| *** stephenfin has joined #opendev | 21:42 | |
| *** mtreinish has joined #opendev | 21:42 | |
| *** xavpaice has joined #opendev | 21:42 | |
| *** owalsh has joined #opendev | 21:42 | |
| *** gouthamr_ has joined #opendev | 21:42 | |
| *** mordred has quit IRC | 21:45 | |
| *** iurygregory has quit IRC | 21:46 | |
| *** yoctozepto has quit IRC | 21:46 | |
| *** cloudnull has quit IRC | 21:46 | |
| *** janders has quit IRC | 21:46 | |
| *** dmsimard has quit IRC | 21:46 | |
| *** ttx has quit IRC | 21:46 | |
| *** frickler has quit IRC | 21:46 | |
| *** icey has quit IRC | 21:46 | |
| *** guillaumec has quit IRC | 21:46 | |
| *** weshay|ruck has quit IRC | 21:46 | |
| *** dtroyer has quit IRC | 21:46 | |
| *** spotz has quit IRC | 21:46 | |
| *** iurygregory has joined #opendev | 21:47 | |
| *** yoctozepto has joined #opendev | 21:47 | |
| *** cloudnull has joined #opendev | 21:47 | |
| *** janders has joined #opendev | 21:47 | |
| *** dmsimard has joined #opendev | 21:47 | |
| *** ttx has joined #opendev | 21:47 | |
| *** frickler has joined #opendev | 21:47 | |
| *** Eighth_Doctor has quit IRC | 21:47 | |
| *** slaweq has joined #opendev | 21:47 | |
| *** sboyron has quit IRC | 21:48 | |
| *** mlavalle has quit IRC | 21:48 | |
| *** johnsom has quit IRC | 21:48 | |
| *** donnyd has quit IRC | 21:48 | |
| *** hillpd has quit IRC | 21:48 | |
| *** dviroel has quit IRC | 21:48 | |
| *** clayg has quit IRC | 21:48 | |
| *** dtantsur|afk has quit IRC | 21:48 | |
| *** cmurphy has quit IRC | 21:48 | |
| *** mgagne has quit IRC | 21:48 | |
| *** fdegir has quit IRC | 21:48 | |
| *** fungi has quit IRC | 21:48 | |
| *** icey has joined #opendev | 21:50 | |
| *** guillaumec has joined #opendev | 21:50 | |
| *** weshay|ruck has joined #opendev | 21:50 | |
| *** dtroyer has joined #opendev | 21:50 | |
| *** spotz has joined #opendev | 21:50 | |
| *** mlavalle has joined #opendev | 21:51 | |
| *** johnsom has joined #opendev | 21:51 | |
| *** donnyd has joined #opendev | 21:51 | |
| *** hillpd has joined #opendev | 21:51 | |
| *** dviroel has joined #opendev | 21:51 | |
| *** clayg has joined #opendev | 21:51 | |
| *** dtantsur|afk has joined #opendev | 21:51 | |
| *** cmurphy has joined #opendev | 21:51 | |
| *** mgagne has joined #opendev | 21:51 | |
| *** fdegir has joined #opendev | 21:51 | |
| *** fungi has joined #opendev | 21:51 | |
| *** SotK has quit IRC | 21:51 | |
| *** tristanC has quit IRC | 21:51 | |
| *** dirk has quit IRC | 21:51 | |
| *** cgoncalves has quit IRC | 21:51 | |
| *** prometheanfire has quit IRC | 21:51 | |
| *** noonedeadpunk has quit IRC | 21:51 | |
| *** avass has quit IRC | 21:51 | |
| *** dulek has quit IRC | 21:51 | |
| *** odyssey4me has quit IRC | 21:51 | |
| *** danpawlik has quit IRC | 21:51 | |
| *** clarkb has quit IRC | 21:51 | |
| *** dmellado has quit IRC | 21:51 | |
| *** smcginnis has quit IRC | 21:51 | |
| *** tobias-urdin has quit IRC | 21:51 | |
| *** JayF has quit IRC | 21:51 | |
| *** andrewbonney has quit IRC | 21:52 | |
| *** dmellado has joined #opendev | 21:53 | |
| *** tobias-urdin has joined #opendev | 21:53 | |
| *** smcginnis has joined #opendev | 21:53 | |
| *** JayF has joined #opendev | 21:53 | |
| *** SotK has joined #opendev | 21:53 | |
| *** tristanC has joined #opendev | 21:53 | |
| *** dirk has joined #opendev | 21:53 | |
| *** cgoncalves has joined #opendev | 21:53 | |
| *** prometheanfire has joined #opendev | 21:53 | |
| *** noonedeadpunk has joined #opendev | 21:53 | |
| *** avass has joined #opendev | 21:53 | |
| *** dulek has joined #opendev | 21:53 | |
| *** odyssey4me has joined #opendev | 21:53 | |
| *** danpawlik has joined #opendev | 21:53 | |
| *** clarkb has joined #opendev | 21:53 | |
| *** slaweq has quit IRC | 21:53 | |
| *** d34dh0r53 has quit IRC | 21:53 | |
| *** whoami-rajat__ has quit IRC | 21:53 | |
| *** bbezak has quit IRC | 21:53 | |
| *** ildikov has quit IRC | 21:53 | |
| *** slaweq has joined #opendev | 21:53 | |
| *** d34dh0r53 has joined #opendev | 21:53 | |
| *** whoami-rajat__ has joined #opendev | 21:53 | |
| *** bbezak has joined #opendev | 21:53 | |
| *** ildikov has joined #opendev | 21:53 | |
| *** slaweq has quit IRC | 21:53 | |
| *** d34dh0r53 has quit IRC | 21:53 | |
| *** whoami-rajat__ has quit IRC | 21:53 | |
| *** bbezak has quit IRC | 21:53 | |
| *** ildikov has quit IRC | 21:53 | |
| *** SotK has quit IRC | 21:53 | |
| *** tristanC has quit IRC | 21:53 | |
| *** dirk has quit IRC | 21:53 | |
| *** cgoncalves has quit IRC | 21:53 | |
| *** prometheanfire has quit IRC | 21:53 | |
| *** noonedeadpunk has quit IRC | 21:53 | |
| *** avass has quit IRC | 21:53 | |
| *** dulek has quit IRC | 21:53 | |
| *** odyssey4me has quit IRC | 21:53 | |
| *** danpawlik has quit IRC | 21:53 | |
| *** clarkb has quit IRC | 21:53 | |
| *** dmellado has quit IRC | 21:53 | |
| *** smcginnis has quit IRC | 21:53 | |
| *** tobias-urdin has quit IRC | 21:53 | |
| *** JayF has quit IRC | 21:53 | |
| *** icey has quit IRC | 21:53 | |
| *** guillaumec has quit IRC | 21:53 | |
| *** weshay|ruck has quit IRC | 21:53 | |
| *** dtroyer has quit IRC | 21:53 | |
| *** spotz has quit IRC | 21:53 | |
| *** mlavalle has quit IRC | 21:53 | |
| *** johnsom has quit IRC | 21:53 | |
| *** donnyd has quit IRC | 21:53 | |
| *** hillpd has quit IRC | 21:53 | |
| *** dviroel has quit IRC | 21:53 | |
| *** clayg has quit IRC | 21:53 | |
| *** dtantsur|afk has quit IRC | 21:53 | |
| *** cmurphy has quit IRC | 21:53 | |
| *** mgagne has quit IRC | 21:53 | |
| *** fdegir has quit IRC | 21:53 | |
| *** fungi has quit IRC | 21:53 | |
| *** iurygregory has quit IRC | 21:53 | |
| *** yoctozepto has quit IRC | 21:53 | |
| *** cloudnull has quit IRC | 21:53 | |
| *** janders has quit IRC | 21:53 | |
| *** dmsimard has quit IRC | 21:53 | |
| *** ttx has quit IRC | 21:53 | |
| *** frickler has quit IRC | 21:53 | |
| *** gnuoy has quit IRC | 21:53 | |
| *** calcmandan has quit IRC | 21:53 | |
| *** valleedelisle has quit IRC | 21:53 | |
| *** mhu has quit IRC | 21:53 | |
| *** sgw has quit IRC | 21:53 | |
| *** stephenfin has quit IRC | 21:53 | |
| *** kopecmartin has quit IRC | 21:53 | |
| *** mtreinish has quit IRC | 21:53 | |
| *** xavpaice has quit IRC | 21:53 | |
| *** owalsh has quit IRC | 21:53 | |
| *** gouthamr_ has quit IRC | 21:53 | |
| *** brinzhang0 has quit IRC | 21:53 | |
| *** elod has quit IRC | 21:53 | |
| *** hemanth_n has quit IRC | 21:53 | |
| *** zer0c00l has quit IRC | 21:53 | |
| *** sean-k-mooney has quit IRC | 21:53 | |
| *** persia has quit IRC | 21:53 | |
| *** priteau has quit IRC | 21:53 | |
| *** mrunge has quit IRC | 21:53 | |
| *** stevebaker has quit IRC | 21:53 | |
| *** tosky has quit IRC | 21:53 | |
| *** toma4 has quit IRC | 21:53 | |
| *** amotoki has quit IRC | 21:53 | |
| *** ianw has quit IRC | 21:53 | |
| *** Green_Bird has quit IRC | 21:53 | |
| *** sshnaidm|off has quit IRC | 21:53 | |
| *** bhagyashris|off has quit IRC | 21:53 | |
| *** chandankumar has quit IRC | 21:53 | |
| *** osmanlicilegi has quit IRC | 21:53 | |
| *** timburke has quit IRC | 21:53 | |
| *** tbarron has quit IRC | 21:53 | |
| *** ChanServ has quit IRC | 21:53 | |
| *** rpittau|afk has quit IRC | 21:53 | |
| *** mnaser has quit IRC | 21:53 | |
| *** ShadowJonathan has quit IRC | 21:53 | |
| *** jentoio has quit IRC | 21:53 | |
| *** DSpider has quit IRC | 21:53 | |
| *** auristor has quit IRC | 21:53 | |
| *** seongsoocho has quit IRC | 21:53 | |
| *** jrosser has quit IRC | 21:53 | |
| *** mattmceuen has quit IRC | 21:53 | |
| *** portdirect has quit IRC | 21:53 | |
| *** mwhahaha has quit IRC | 21:53 | |
| *** gmann has quit IRC | 21:53 | |
| *** melwitt has quit IRC | 21:53 | |
| *** tkajinam has quit IRC | 21:53 | |
| *** zbr has quit IRC | 21:53 | |
| *** openstackgerrit has quit IRC | 21:53 | |
| *** TheJulia has quit IRC | 21:53 | |
| *** hamalq has quit IRC | 21:53 | |
| *** guilhermesp has quit IRC | 21:53 | |
| *** gouthamr has quit IRC | 21:53 | |
| *** mugsie has quit IRC | 21:53 | |
| *** paladox has quit IRC | 21:53 | |
| *** mnasiadka has quit IRC | 21:53 | |
| *** rm_work has quit IRC | 21:53 | |
| *** aprice has quit IRC | 21:53 | |
| *** knikolla has quit IRC | 21:53 | |
| *** erbarr has quit IRC | 21:53 | |
| *** Open10K8S has quit IRC | 21:53 | |
| *** jbryce has quit IRC | 21:53 | |
| *** diablo_rojo_phon has quit IRC | 21:53 | |
| *** logan- has quit IRC | 21:53 | |
| *** cap has quit IRC | 21:53 | |
| *** jroll has quit IRC | 21:53 | |
| *** fnordahl has quit IRC | 21:53 | |
| *** nautik has quit IRC | 21:53 | |
| *** Tengu has quit IRC | 21:53 | |
| *** tobiash has quit IRC | 21:53 | |
| *** zigo has quit IRC | 21:53 | |
| *** redrobot has quit IRC | 21:53 | |
| *** corvus has quit IRC | 21:53 | |
| *** ildikov has joined #opendev | 21:54 | |
| *** bbezak has joined #opendev | 21:54 | |
| *** whoami-rajat__ has joined #opendev | 21:54 | |
| *** d34dh0r53 has joined #opendev | 21:54 | |
| *** slaweq has joined #opendev | 21:54 | |
| *** clarkb has joined #opendev | 21:54 | |
| *** danpawlik has joined #opendev | 21:54 | |
| *** odyssey4me has joined #opendev | 21:54 | |
| *** dulek has joined #opendev | 21:54 | |
| *** avass has joined #opendev | 21:54 | |
| *** noonedeadpunk has joined #opendev | 21:54 | |
| *** prometheanfire has joined #opendev | 21:54 | |
| *** cgoncalves has joined #opendev | 21:54 | |
| *** dirk has joined #opendev | 21:54 | |
| *** tristanC has joined #opendev | 21:54 | |
| *** SotK has joined #opendev | 21:54 | |
| *** JayF has joined #opendev | 21:54 | |
| *** smcginnis has joined #opendev | 21:54 | |
| *** tobias-urdin has joined #opendev | 21:54 | |
| *** dmellado has joined #opendev | 21:54 | |
| *** fungi has joined #opendev | 21:54 | |
| *** fdegir has joined #opendev | 21:54 | |
| *** mgagne has joined #opendev | 21:54 | |
| *** cmurphy has joined #opendev | 21:54 | |
| *** dtantsur|afk has joined #opendev | 21:54 | |
| *** clayg has joined #opendev | 21:54 | |
| *** dviroel has joined #opendev | 21:54 | |
| *** hillpd has joined #opendev | 21:54 | |
| *** donnyd has joined #opendev | 21:54 | |
| *** johnsom has joined #opendev | 21:54 | |
| *** mlavalle has joined #opendev | 21:54 | |
| *** spotz has joined #opendev | 21:54 | |
| *** dtroyer has joined #opendev | 21:54 | |
| *** weshay|ruck has joined #opendev | 21:54 | |
| *** guillaumec has joined #opendev | 21:54 | |
| *** icey has joined #opendev | 21:54 | |
| *** frickler has joined #opendev | 21:54 | |
| *** ttx has joined #opendev | 21:54 | |
| *** dmsimard has joined #opendev | 21:54 | |
| *** janders has joined #opendev | 21:54 | |
| *** cloudnull has joined #opendev | 21:54 | |
| *** yoctozepto has joined #opendev | 21:54 | |
| *** iurygregory has joined #opendev | 21:54 | |
| *** gouthamr_ has joined #opendev | 21:54 | |
| *** owalsh has joined #opendev | 21:54 | |
| *** xavpaice has joined #opendev | 21:54 | |
| *** mtreinish has joined #opendev | 21:54 | |
| *** stephenfin has joined #opendev | 21:54 | |
| *** kopecmartin has joined #opendev | 21:54 | |
| *** mhu has joined #opendev | 21:54 | |
| *** sgw has joined #opendev | 21:54 | |
| *** valleedelisle has joined #opendev | 21:54 | |
| *** calcmandan has joined #opendev | 21:54 | |
| *** gnuoy has joined #opendev | 21:54 | |
| *** ianw has joined #opendev | 21:54 | |
| *** amotoki has joined #opendev | 21:54 | |
| *** toma4 has joined #opendev | 21:54 | |
| *** tosky has joined #opendev | 21:54 | |
| *** stevebaker has joined #opendev | 21:54 | |
| *** sean-k-mooney has joined #opendev | 21:54 | |
| *** zer0c00l has joined #opendev | 21:54 | |
| *** hemanth_n has joined #opendev | 21:54 | |
| *** elod has joined #opendev | 21:54 | |
| *** brinzhang0 has joined #opendev | 21:54 | |
| *** mrunge has joined #opendev | 21:54 | |
| *** persia has joined #opendev | 21:54 | |
| *** priteau has joined #opendev | 21:54 | |
| *** tbarron has joined #opendev | 21:54 | |
| *** osmanlicilegi has joined #opendev | 21:54 | |
| *** chandankumar has joined #opendev | 21:54 | |
| *** timburke has joined #opendev | 21:54 | |
| *** bhagyashris|off has joined #opendev | 21:54 | |
| *** sshnaidm|off has joined #opendev | 21:54 | |
| *** Green_Bird has joined #opendev | 21:54 | |
| *** hamalq has joined #opendev | 21:54 | |
| *** DSpider has joined #opendev | 21:54 | |
| *** paladox has joined #opendev | 21:54 | |
| *** TheJulia has joined #opendev | 21:54 | |
| *** rpittau|afk has joined #opendev | 21:54 | |
| *** auristor has joined #opendev | 21:54 | |
| *** diablo_rojo_phon has joined #opendev | 21:54 | |
| *** mnaser has joined #opendev | 21:54 | |
| *** gouthamr has joined #opendev | 21:54 | |
| *** guilhermesp has joined #opendev | 21:54 | |
| *** ShadowJonathan has joined #opendev | 21:54 | |
| *** jentoio has joined #opendev | 21:54 | |
| *** seongsoocho has joined #opendev | 21:54 | |
| *** rm_work has joined #opendev | 21:54 | |
| *** mnasiadka has joined #opendev | 21:54 | |
| *** aprice has joined #opendev | 21:54 | |
| *** knikolla has joined #opendev | 21:54 | |
| *** jrosser has joined #opendev | 21:54 | |
| *** erbarr has joined #opendev | 21:54 | |
| *** Open10K8S has joined #opendev | 21:54 | |
| *** portdirect has joined #opendev | 21:54 | |
| *** jbryce has joined #opendev | 21:54 | |
| *** mattmceuen has joined #opendev | 21:54 | |
| *** mwhahaha has joined #opendev | 21:54 | |
| *** gmann has joined #opendev | 21:54 | |
| *** Tengu has joined #opendev | 21:54 | |
| *** melwitt has joined #opendev | 21:54 | |
| *** mugsie has joined #opendev | 21:54 | |
| *** tkajinam has joined #opendev | 21:54 | |
| *** ChanServ has joined #opendev | 21:54 | |
| *** zbr has joined #opendev | 21:54 | |
| *** logan- has joined #opendev | 21:54 | |
| *** openstackgerrit has joined #opendev | 21:54 | |
| *** tobiash has joined #opendev | 21:54 | |
| *** cap has joined #opendev | 21:54 | |
| *** nautik has joined #opendev | 21:54 | |
| *** fnordahl has joined #opendev | 21:54 | |
| *** zigo has joined #opendev | 21:54 | |
| *** redrobot has joined #opendev | 21:54 | |
| *** jroll has joined #opendev | 21:54 | |
| *** corvus has joined #opendev | 21:54 | |
| *** tepper.freenode.net sets mode: +o ChanServ | 21:54 | |
| *** paladox has quit IRC | 21:54 | |
| *** mnasiadka has quit IRC | 21:54 | |
| *** rm_work has quit IRC | 21:54 | |
| *** aprice has quit IRC | 21:54 | |
| *** knikolla has quit IRC | 21:54 | |
| *** erbarr has quit IRC | 21:54 | |
| *** Open10K8S has quit IRC | 21:54 | |
| *** jbryce has quit IRC | 21:54 | |
| *** diablo_rojo_phon has quit IRC | 21:54 | |
| *** logan- has quit IRC | 21:54 | |
| *** cap has quit IRC | 21:54 | |
| *** jroll has quit IRC | 21:54 | |
| *** fnordahl has quit IRC | 21:54 | |
| *** nautik has quit IRC | 21:54 | |
| *** Tengu has quit IRC | 21:54 | |
| *** tobiash has quit IRC | 21:54 | |
| *** zigo has quit IRC | 21:54 | |
| *** redrobot has quit IRC | 21:54 | |
| *** corvus has quit IRC | 21:54 | |
| *** paladox has joined #opendev | 21:54 | |
| *** diablo_rojo_phon has joined #opendev | 21:54 | |
| *** rm_work has joined #opendev | 21:54 | |
| *** mnasiadka has joined #opendev | 21:54 | |
| *** aprice has joined #opendev | 21:54 | |
| *** knikolla has joined #opendev | 21:54 | |
| *** erbarr has joined #opendev | 21:54 | |
| *** Open10K8S has joined #opendev | 21:54 | |
| *** jbryce has joined #opendev | 21:54 | |
| *** logan- has joined #opendev | 21:54 | |
| *** cap has joined #opendev | 21:54 | |
| *** nautik has joined #opendev | 21:54 | |
| *** fnordahl has joined #opendev | 21:54 | |
| *** jroll has joined #opendev | 21:54 | |
| *** logan- has quit IRC | 21:54 | |
| *** Tengu has joined #opendev | 21:55 | |
| *** tobiash has joined #opendev | 21:55 | |
| *** zigo has joined #opendev | 21:55 | |
| *** redrobot has joined #opendev | 21:55 | |
| *** corvus has joined #opendev | 21:55 | |
| *** logan- has joined #opendev | 21:57 | |
| *** mnasiadka has quit IRC | 21:57 | |
| *** mnasiadka has joined #opendev | 21:57 | |
| *** erbarr has quit IRC | 21:58 | |
| *** erbarr has joined #opendev | 21:59 | |
| *** slaweq has quit IRC | 22:11 | |
| *** Eighth_Doctor has joined #opendev | 22:15 | |
| clarkb | ah we loop over the backup servers there. Ya I think that math works | 22:20 |
| *** mordred has joined #opendev | 22:25 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: WIP: boot test of containerfile image https://review.opendev.org/722148 | 22:57 |
| *** tosky has quit IRC | 23:08 | |
| clarkb | ianw: I've approved the cron spacing change, I think the child that trims the gerrit list is probably safe to land too ? | 23:13 |
| ianw | clarkb: yeah, should be safe. once it lands, i'd actually like to try pruning out the old backups on the servers | 23:21 |
| ianw | it would be a good exercise at any rate | 23:21 |
| clarkb | ok I'm trying to pretend today is a holiday since the kids are out of school, maybe you want to approve it? | 23:21 |
| ianw | no worries, i can babysit it | 23:22 |
| fungi | 762271 seems like it would be better to get rid of the wildcards entirely? | 23:26 |
| ianw | fungi: i'm not sure it's a prefix match? | 23:29 |
| fungi | ianw: in that case why is one cache* and another cache/*? | 23:30 |
| clarkb | I left that note but I think its fine for the path | 23:30 |
| fungi | but yeah, it's just an assumption on my part that telling borg not to backup .../cache also causes it to not back up .../cache/* | 23:31 |
| ianw | i think i was thinking that anything starting with cache in that directory is fine to avoid, while i wanted to be more specific on the gerrit2/review_site match | 23:31 |
| clarkb | it will just match cache and not cache/foo cache/bar etc | 23:31 |
| ianw | but, i don't know if ihad a great reason for that | 23:31 |
| clarkb | but ya not necesary to have the * in that case | 23:31 |
| fungi | ianw: looking at the server there is just one directory there named "cache" | 23:31 |
| fungi | unless i'm looking at the wrong thing | 23:32 |
| fungi | did you find other directories starting with the string "cache"? | 23:32 |
| ianw | no but i think i was thinking there might be. i'm not tied to it, it's probably better to change to /* since clearly that would be less confusing | 23:32 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: review: trim backups https://review.opendev.org/762271 | 23:36 |
| fungi | the other things which concerns me though is that if it actually tries to expand that glob and there are thousands of files matching it, that could lead to inefficiencies/errors. but probably not. that was my only real reason for suggesting we just specify the directories themselves | 23:37 |
| ianw | yeah you can regex etc there too. i'd say it's going to walk the files anyway, but ignore them on match | 23:42 |
| openstackgerrit | Merged opendev/system-config master: borg-backup: space out cron jobs evenly https://review.opendev.org/762270 | 23:44 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!