| *** dviroel|out is now known as dviroel | 00:21 | |
| *** dviroel is now known as dviroel|out | 01:24 | |
| *** ysandeep|out is now known as ysandeep|rover | 04:35 | |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 05:25 | |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 05:32 | |
| noonedeadpunk | huh. And on Pypi package not available anymore https://pypi.org/project/ctx/ | 07:11 |
|---|---|---|
| noonedeadpunk | I _think_ nothing from openstack should use that as it's not in allowable requirements https://opendev.org/openstack/requirements/src/branch/master/global-requirements.txt | 07:14 |
| noonedeadpunk | but that's crazy... | 07:14 |
| noonedeadpunk | jrosser: I bet we were landing some patch to fix package pinning for ceph_client lately? | 07:14 |
| jrosser | we did on master, but I think what was wrong there was a little different | 07:15 |
| noonedeadpunk | it wasn't ceph likely, it was smth else... | 07:16 |
| jrosser | I think I put a link up there ^^^^ | 07:17 |
| noonedeadpunk | ah | 07:18 |
| noonedeadpunk | yes, fair | 07:18 |
| noonedeadpunk | so basically when we get to ceph_client role indeed we already have thing installed | 07:19 |
| noonedeadpunk | I _guess_ we can backport that? | 07:21 |
| jrosser | maybe :/ | 07:22 |
| jrosser | it was pretty late yesterday and i wasnt sure if this was the same situation as we had on master | 07:26 |
| jrosser | and also why we are not failing in the wallaby CI job like spatel saw locally | 07:26 |
| jrosser | but i could believe that being because in the AIO everything is collapsed on one node | 07:27 |
| jrosser | yes, so backporting that patch would stop UCA being the installed in place of the distro package | 07:33 |
| jrosser | so we would end up with a (install from distro during nova) -> (upgrade to whatever is in ceph.com with ceph_client) | 07:34 |
| jrosser | rather than (install from UCA during nova) -> (downgrade to whatever is in ceph.com with ceph_client) | 07:34 |
| *** ysandeep|rover is now known as ysandeep|rover|lunch | 07:44 | |
| noonedeadpunk | I can't recall how much disabling recommends was breaking stuff | 07:48 |
| noonedeadpunk | Not very, right? | 07:48 |
| kleini | I am on W. https://jira.mariadb.org/browse/MDEV-27459 that bug seems to hit me more and more. If some Galera nodes crashes, systemd restarts it, it joins and the donor for the sync remains in donor state. first node out of the cluster. and this goes on, until quorum is not possible any more. Manually restarting then is really hard. Bug is fixed with Galera 26.4.11 but W runs 26.4.9. What would be best option to | 08:02 |
| kleini | mitigate that problem? | 08:02 |
| noonedeadpunk | one option is to override mariadb version in variables. I think we stopped hitting it when set galera_wsrep_slave_threads: 1, but I won't put money that it solved this issue as well | 08:07 |
| noonedeadpunk | At least I can't recall hitting it afterwards | 08:08 |
| kleini | Will have an eye on that. | 08:13 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow lxc container backing store to be defined as part of the scenario https://review.opendev.org/c/openstack/openstack-ansible/+/843011 | 08:31 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add zuul jobs and template for testing different LXC backing stores https://review.opendev.org/c/openstack/openstack-ansible/+/843027 | 08:31 |
| *** ysandeep|rover|lunch is now known as ysandeep|rover | 08:33 | |
| noonedeadpunk | btw I'd say nginx uid/gid are quite consistant from what I see in regions | 08:49 |
| jrosser | yeah, i was just starting to look at making it a fixed value | 08:51 |
| jrosser | it's probably variable between distros though? | 08:51 |
| jrosser | i'm expecting it to be in 100-499 range according to LSB | 08:52 |
| noonedeadpunk | I wonder if we should just set owner/group here https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/repo_all.yml#L28-L33 | 08:56 |
| noonedeadpunk | I checked several regions, and we have 998:33 everywhere. Could be cross-distro issue, yes, but in CI we totally use same one... | 08:58 |
| noonedeadpunk | ok, I will leave it if you're already watching. | 08:58 |
| noonedeadpunk | upgrade jobs always fail just in case - it's not accidental failures just in case | 09:02 |
| noonedeadpunk | (for infra) | 09:02 |
| jrosser | do we know why? /me not looked yet | 09:03 |
| jrosser | also i'm not sure we can set the user/group on a mount? | 09:03 |
| noonedeadpunk | Yeah, we can | 09:05 |
| noonedeadpunk | or well | 09:05 |
| noonedeadpunk | https://opendev.org/openstack/ansible-role-systemd_mount/src/branch/master/tasks/systemd_mounts.yml#L31-L32 | 09:05 |
| noonedeadpunk | but directory in upgrade should already exist... | 09:06 |
| jrosser | isnt this a bit like nfs where the mount is pretty much 'by root' and the permissions are dealt with on each client | 09:06 |
| jrosser | unlike smb for example where the mount is 'by user' | 09:06 |
| noonedeadpunk | tbh don't know a lot about gluster | 09:12 |
| noonedeadpunk | ut yes, I think you're right and it's alike nfs | 09:13 |
| noonedeadpunk | but on NFS you need to set bunch of options for mount to get things working properly | 09:14 |
| noonedeadpunk | and eventually things fail when we're trying to override files that were created on another host | 09:16 |
| *** arxcruz is now known as arxcruz|off | 10:52 | |
| *** dviroel|out is now known as dviroel | 11:30 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-repo_server master: Run content restore only on first host https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843267 | 11:58 |
| jrosser | there are quite some patches to merge in the repo_server if anyone has time https://review.opendev.org/q/project:openstack%252Fopenstack-ansible-repo_server | 12:19 |
| jrosser | it would be nice to get those in as I think there is some more refactoring to do to make it simpler | 12:19 |
| mgariepy | i'll take a look. | 12:19 |
| jrosser | i would like to combine tasks/repo_pre_install.yml and tasks/repo_install.yml in particular | 12:20 |
| mgariepy | rocky8 still failing :( | 12:22 |
| mgariepy | https://zuul.opendev.org/t/openstack/build/005ef281d2fb478ebf1acf1142196888/log/job-output.txt#9599 | 12:22 |
| mgariepy | jrosser, commented on : https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/841619 | 12:29 |
| jrosser | did you press reply? not sure i see it? | 12:30 |
| mgariepy | LOL | 12:30 |
| mgariepy | code 18.. haha | 12:30 |
| jrosser | ah yes, we should remove that | 12:32 |
| *** ysandeep|rover is now known as ysandeep|rover|afk | 12:41 | |
| mgariepy | noonedeadpunk, why did you remove W on :https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837588 ? | 12:43 |
| noonedeadpunk | because of dependant patch not merged yet | 12:44 |
| noonedeadpunk | and then zuul will invalidate V+1 | 12:44 |
| mgariepy | ok | 12:44 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Use distro packages for nginx on centos. https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/841619 | 12:48 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Use distro packages for nginx on centos. https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/841619 | 12:55 |
| spatel | morning folks | 12:58 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow lxc container backing store to be defined as part of the scenario https://review.opendev.org/c/openstack/openstack-ansible/+/843011 | 13:04 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add zuul jobs and template for testing different LXC backing stores https://review.opendev.org/c/openstack/openstack-ansible/+/843027 | 13:04 |
| jrosser | hmmm no ARA report on an infra job | 13:06 |
| jrosser | damiandabrowski[m]: did you look at the infra jobs upgrade errors at all? (stale file handle whilst untar the old repo data) | 13:10 |
| spatel | jrosser i have added my comments here - https://bugs.launchpad.net/openstack-ansible/+bug/1975643 | 13:15 |
| damiandabrowski[m] | jrosser: this one? https://zuul.opendev.org/t/openstack/build/7e33eaf8cfb5445c92bcbe38647b792d/log/job-output.txt#19556-19558 | 13:16 |
| damiandabrowski[m] | i saw this error in zuul but didn't have to to look more into this :/ | 13:16 |
| damiandabrowski[m] | but I'm curious about `Cannot utime: Stale file handle` | 13:17 |
| jrosser | yeah thats the one | 13:17 |
| damiandabrowski[m] | maybe this one will solve it? https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843267 | 13:17 |
| jrosser | i have a bunch of meetings shortly but might be able to look at it later | 13:17 |
| mgariepy | gluster mirror on rocky doens't work because of the relase and something else.. | 13:17 |
| jrosser | it is rather unfortunate that https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843267 patch does not run the infra upgrade job | 13:18 |
| jrosser | i expect that is how we have merged a broken thing | 13:18 |
| jrosser | mgariepy: i think i fixed the gluster/rocky thing https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/842999 | 13:20 |
| jrosser | more test coverage gaps there though :/ | 13:20 |
| mgariepy | https://paste.openstack.org/show/b5iG0k49RRDi5qLYVFNg/ | 13:23 |
| jrosser | that looks like you don't have 842999 applied | 13:25 |
| jrosser | the mirrorlist should be "release=8-stream" | 13:26 |
| jrosser | *should contain | 13:26 |
| mgariepy | ok. | 13:26 |
| jrosser | just for added awkwardness if you're applying that by hand it needs to be done in /etc/ansible/ansible_collections/...... | 13:27 |
| jrosser | which isn't a git repo due to `ansible-galaxy` $reasons | 13:27 |
| mgariepy | i was testing in a container with dnf command directly . | 13:27 |
| mgariepy | ;) | 13:27 |
| jrosser | ahha | 13:28 |
| jrosser | if you hack the repo file and change the `release` string it should work | 13:28 |
| mgariepy | i was looing at 842571 .. now it has 2 recheck :) | 13:28 |
| *** ysandeep|rover|afk is now known as ysandeep|rover | 13:31 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Add infra upgrade jobs https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843284 | 13:37 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-plugins master: Use the integrated repo infra jobs for testing https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/843286 | 13:42 |
| Adri2000 | hello noonedeadpunk - I'm trying to debug some issues I have with Swift playbooks and ended up on this old patch from you: https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/765354 - I see a a change in that patch that I'm not sure was intended... before, it copied rings to both /etc/swift/ and /etc/swift/ring_build_files/; now it only copies rings to | 13:42 |
| Adri2000 | /etc/swift/; was that the intention? | 13:42 |
| noonedeadpunk | Adri2000: I guess I just missed that. It wasn't intention | 13:45 |
| Adri2000 | ok, I *think* that because of that we get md5sum mismatches when we add/remove nodes to the rings... but I'll need a few days before I can confirm this and test a fix. unless you or someone else is confident about a fix and wants to push it, I'll come back later and certainly push a patch. thanks for your reply :) | 13:54 |
| noonedeadpunk | Adri2000: but I'm not sure why they're needed there? | 13:56 |
| noonedeadpunk | ` md5sum mismatches` ok, yes, that would explain | 13:56 |
| noonedeadpunk | I bet there was some fresh bug report related to that as well | 13:56 |
| noonedeadpunk | https://bugs.launchpad.net/openstack-ansible/+bug/1973045 | 13:57 |
| noonedeadpunk | I feel we're doing smth quite weird about ring files verification... | 13:59 |
| noonedeadpunk | anyway | 13:59 |
| noonedeadpunk | Will just fix what I broke | 14:00 |
| Adri2000 | this bug report looks similar indeed | 14:02 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Sync rings to all swift hosts to build_files https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/843289 | 14:04 |
| noonedeadpunk | Adri2000: you should check if that fix ^ | 14:10 |
| noonedeadpunk | but not sure about that.... huh | 14:12 |
| Adri2000 | noonedeadpunk: unfortunately can't test right now, will but share some feedback asap. thanks for pushing this. | 14:22 |
| noonedeadpunk | the logic we're verifying rings is really weird and makes limited sense to me. Need to look for some better solution. As verifying against local files makes more sense to me... | 14:31 |
| noonedeadpunk | But need to reproduce env first | 14:32 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Add infra upgrade jobs https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843284 | 14:47 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-repo_server master: Add infra upgrade jobs https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843284 | 15:03 |
| *** dviroel is now known as dviroel|lunch | 15:09 | |
| opendevreview | Merged openstack/openstack-ansible-repo_server master: Run content restore only on first host https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843267 | 15:58 |
| *** dviroel|lunch is now known as dviroel | 16:16 | |
| *** ysandeep|rover is now known as ysandeep|out | 16:59 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: WIP - Centos-9 Stream support https://review.opendev.org/c/openstack/openstack-ansible/+/823417 | 17:38 |
| Praveenkakkera | Hello Every one | 17:55 |
| Praveenkakkera | Greeting for the day | 17:56 |
| Praveenkakkera | Hello Any one is avalible to Help me | 17:56 |
| Praveenkakkera | I Am trying to Deploy the openstack wallaby on centos 8 with help on openstack Ansible | 17:56 |
| Praveenkakkera | I am copleated till openstack-ansible setup-hosts.yml | 17:56 |
| Praveenkakkera | when i am running openstack-ansible setup-infrastructure.yml deployment is getting failed on the galara cluster setuo | 17:56 |
| Praveenkakkera | when i tried deep my lxc container in infra 1 is unable to communicate to the Infra 2 and infra3 containers | 17:56 |
| Praveenkakkera | any one can help me on this please | 17:56 |
| noonedeadpunk | Praveenkakkera didn't give us any chance to help... | 18:24 |
| jrosser | centos-9 unblocker https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/842988 | 18:28 |
| jrosser | hmm these infra upgrade jobs are still failing https://zuul.opendev.org/t/openstack/build/13c907fac7c6440a9d5131aa306ef8cd/log/job-output.txt#19287 | 18:29 |
| opendevreview | Merged openstack/openstack-ansible master: Update repo verification file URI https://review.opendev.org/c/openstack/openstack-ansible/+/842571 | 18:36 |
| jrosser | stale file handle means that the inode is no longer there | 18:37 |
| jrosser | i really wonder if there is a race condition, it's via fuse..... | 18:37 |
| opendevreview | Merged openstack/openstack-ansible-repo_server master: Add infra upgrade jobs https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/843284 | 18:39 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Mark Bionic and Buster as unsupported for Y https://review.opendev.org/c/openstack/openstack-ansible/+/843340 | 19:06 |
| jrosser | cool \o/ https://review.opendev.org/c/openstack/openstack-ansible/+/823417 | 19:41 |
| jrosser | how much of centos-9 do we dare add to * CI jobs this close to release..... i guess we need to support it for Y? | 19:42 |
| mgariepy | Good job ! | 19:45 |
| noonedeadpunk | jrosser: yep, we need it for Y as in Z we should drop C8 I believe | 19:52 |
| noonedeadpunk | As I don't see how we can have libvirt binding for py38 there | 19:53 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add Centos-9 Stream support https://review.opendev.org/c/openstack/openstack-ansible/+/823417 | 19:54 |
| jrosser | theres a couple of outstanding depends-on https://review.opendev.org/q/topic:osa%252Fcentos-9 | 19:55 |
| noonedeadpunk | "couple" :D | 19:56 |
| noonedeadpunk | I think upgrade jobs should be fixed now, so we should re-check some | 19:56 |
| noonedeadpunk | fwiw I was pretty annoyed by answers in #maria for CentOS 9. Was abount to ping monty directly.... | 19:57 |
| jrosser | right - that was pretty uncool | 19:59 |
| jrosser | i've seen some unpleasantness in #maria as well when people turn up asking beginner questions | 20:00 |
| mgariepy | Just read the channel and yep it's not cool .. | 20:01 |
| opendevreview | Merged openstack/openstack-ansible-tests stable/wallaby: Fix rich version for repos that don't have test-requirements https://review.opendev.org/c/openstack/openstack-ansible-tests/+/843175 | 20:01 |
| *** dviroel is now known as dviroel|afk | 20:07 | |
| jrosser | i expect we still have a big bunch of work to do on the minor roles once we merge centos-9 | 20:14 |
| jrosser | there is still something wrong with the infra upgrade jobs https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837588/28 | 21:22 |
| *** tosky_ is now known as tosky | 21:49 | |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts master: Add centos-9 support https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/842236 | 22:51 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Add support for centos-9 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/823983 | 23:35 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!