| opendevreview | Merged openstack/openstack-ansible stable/yoga: Unset OSA-defined variables for bootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/868273 | 00:47 |
|---|---|---|
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/869656 | 08:35 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Bump ansible version to 2.14.1 https://review.opendev.org/c/openstack/openstack-ansible/+/869599 | 08:38 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible/+/869657 | 08:38 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Remove "warn" paramter from command module https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/869658 | 08:39 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ops master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-ops/+/869660 | 08:40 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/869661 | 08:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/869662 | 08:42 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/869663 | 08:43 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Bump ansible version to 2.14.1 https://review.opendev.org/c/openstack/openstack-ansible/+/869599 | 08:46 |
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-pki master: Update variables gathering to use vars/varnames lookups https://review.opendev.org/c/openstack/ansible-role-pki/+/869664 | 09:21 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/869663 | 09:24 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible/+/869657 | 09:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible version to 2.14.1 https://review.opendev.org/c/openstack/openstack-ansible/+/869599 | 09:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/869661 | 09:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/869656 | 09:39 |
| jrosser | oh my - i think i need more coffee | 09:41 |
| jrosser | noonedeadpunk: ^ thanks for fixing those :) | 09:41 |
| noonedeadpunk | no worries :-) | 09:42 |
| jrosser | noonedeadpunk: i was wondering about making a variable for the server URL in here https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml | 09:43 |
| jrosser | like one single place to switch over all the repos to point at a mirror, or github instead of opendev for example | 09:43 |
| jrosser | but i think this may interfere with the sha bump tool? | 09:43 |
| noonedeadpunk | well, I can patch the tool - it's not a problem | 09:44 |
| noonedeadpunk | I can't recall if we can do same in a-r-r though.... | 09:45 |
| jrosser | we had some wierd network outage yesterday, like broken route/transit to opendev.org and i was thinking how nice it would be to be able to easily switch the whole thing to use github | 09:45 |
| noonedeadpunk | well, it would be also quite sweet for me as well to switch to internal git mirror :) | 09:46 |
| noonedeadpunk | rather then override each one just define single variable | 09:47 |
| jrosser | yeah | 09:47 |
| jrosser | bootstrap is a bit different but i expect that can be done too | 09:47 |
| noonedeadpunk | well, bootstrap is maybe a bit less of an issue... maybe... | 09:48 |
| noonedeadpunk | as you don't need to pull things that frequently | 09:48 |
| noonedeadpunk | and we have a mix of github/opendev there already | 09:49 |
| noonedeadpunk | though I kind of wonder where such variable should be defined then | 09:49 |
| noonedeadpunk | given we also have openstack_testing.yml | 09:50 |
| jrosser | we can put one in each of openstack_services / openstack_testing as defaults as a starting point, they will be very low priority | 09:52 |
| jrosser | also the 2.14 patch failed oddly on focal - need to look if we even have a valid python version there | 09:53 |
| jrosser | and i don't recall if we discussed how long to keep focal support either | 09:53 |
| noonedeadpunk | I haven't checked 2.14 release notes yet... In case it requires >3.8 then we have an issue | 09:53 |
| noonedeadpunk | we must keep it for Antelope | 09:53 |
| noonedeadpunk | and drop right after | 09:53 |
| noonedeadpunk | as it's part of PTI for SLURP upgrades | 09:54 |
| jrosser | ah `ansible - Increase minimum Python requirement to Python 3.9 for CLI utilities and controller code` | 09:54 |
| noonedeadpunk | tbh I'd propose then to stay on 2.13 for AA | 09:55 |
| opendevreview | Merged openstack/openstack-ansible-os_ironic master: Update IPA image for the Zed release https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/869570 | 09:56 |
| opendevreview | Merged openstack/openstack-ansible-plugins stable/zed: Limit maximum number of threads for parallel git clone https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/869463 | 10:06 |
| opendevreview | Merged openstack/openstack-ansible-plugins stable/zed: Update TOX_CONSTRAINTS_FILE for stable/zed https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/867498 | 10:06 |
| opendevreview | Merged openstack/ansible-role-pki master: Allow to define mode and ownership for CA private keys https://review.opendev.org/c/openstack/ansible-role-pki/+/867553 | 10:15 |
| opendevreview | Merged openstack/openstack-ansible-os_nova stable/zed: Enable rbd download when nova_glance_rbd is in use https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/869466 | 10:15 |
| jrosser | noonedeadpunk: do you want to backport https://review.opendev.org/c/openstack/ansible-role-pki/+/867553 ? | 10:15 |
| noonedeadpunk | well, might be nice to, but https://review.opendev.org/c/openstack/ansible-role-pki/+/867555 also works | 10:17 |
| opendevreview | Andrew Bonney proposed openstack/openstack-ansible master: Correct series names for documentation https://review.opendev.org/c/openstack/openstack-ansible/+/869670 | 10:18 |
| opendevreview | Merged openstack/openstack-ansible-openstack_hosts master: Allow to manage extra services, mounts and networks https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/868534 | 10:25 |
| opendevreview | Merged openstack/ansible-role-systemd_networkd master: Allow to provide multiple VLANs https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/868500 | 10:28 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Unify vars for glusterfs RHEL variants and remove rocky-8 workaround. https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/866116 | 10:32 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Add variable to control no_log in db_setup role https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/869546 | 10:32 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/zed: Sync ZFS pool names https://review.opendev.org/c/openstack/openstack-ansible/+/869633 | 10:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Sync ZFS pool names https://review.opendev.org/c/openstack/openstack-ansible/+/869634 | 10:36 |
| opendevreview | Merged openstack/openstack-ansible-ops master: Remove "warn" parameter from command module https://review.opendev.org/c/openstack/openstack-ansible-ops/+/869660 | 10:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts stable/zed: Ensure tar is installed on LXC host https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/868176 | 10:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts stable/yoga: Ensure tar is installed on LXC host https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/868177 | 10:38 |
| opendevreview | Merged openstack/openstack-ansible stable/zed: Define name for all collections in a-r-r https://review.opendev.org/c/openstack/openstack-ansible/+/869460 | 11:21 |
| opendevreview | Merged openstack/openstack-ansible stable/zed: Prevent bootstrap failure when all roles/collections are overriden https://review.opendev.org/c/openstack/openstack-ansible/+/869458 | 11:21 |
| opendevreview | Merged openstack/openstack-ansible stable/xena: Unset OSA-defined variables for bootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/868271 | 11:21 |
| opendevreview | Merged openstack/openstack-ansible master: [doc] Update repositiories for mirroring https://review.opendev.org/c/openstack/openstack-ansible/+/868506 | 11:21 |
| opendevreview | Merged openstack/openstack-ansible master: Add reminder to contributor docs to update amphora/IPA images https://review.opendev.org/c/openstack/openstack-ansible/+/869574 | 11:21 |
| opendevreview | Merged openstack/openstack-ansible master: Set defaults for octavia-ovn-provider driver https://review.opendev.org/c/openstack/openstack-ansible/+/868461 | 11:37 |
| opendevreview | Merged openstack/openstack-ansible master: Sync ZFS pool names https://review.opendev.org/c/openstack/openstack-ansible/+/869575 | 11:37 |
| opendevreview | Merged openstack/openstack-ansible master: Correct series names for documentation https://review.opendev.org/c/openstack/openstack-ansible/+/869670 | 11:37 |
| opendevreview | Merged openstack/openstack-ansible master: [doc] Add example on how to provision LXC bridges with OSA https://review.opendev.org/c/openstack/openstack-ansible/+/868507 | 11:37 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/zed: Correct series names for documentation https://review.opendev.org/c/openstack/openstack-ansible/+/869636 | 11:39 |
| opendevreview | Merged openstack/openstack-ansible-os_nova stable/yoga: Enable rbd download when nova_glance_rbd is in use https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/869467 | 11:47 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Fix comment typo in nova install playbook https://review.opendev.org/c/openstack/openstack-ansible/+/869686 | 11:49 |
| moha7 | Because of this error: http://ix.io/4kNT I think it's needed to add 'RETRY' to the step "Distribute the fernet key repository" for `os-keystone-install.yml`. I passed that failure after 3 attempts of re-deploying that yml script. | 11:49 |
| opendevreview | Merged openstack/ansible-role-systemd_mount master: Fix mount's systemd unit dependency logic https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/868511 | 12:13 |
| jrosser | moha7: we need to know why it failed - adding retries there is wrong, and also is not the same as running the playbook multiple times | 12:15 |
| jrosser | moha7: and fundamentally it is network related "ssh: connect to host 172.17.246.173 port 22: No route to host" | 12:16 |
| opendevreview | Merged openstack/openstack-ansible-os_ironic stable/zed: Update IPA image for the Zed release https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/869465 | 12:17 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/zed: [doc] Update repositiories for mirroring https://review.opendev.org/c/openstack/openstack-ansible/+/869637 | 12:17 |
| opendevreview | Merged openstack/openstack-ansible stable/zed: Correct series names for documentation https://review.opendev.org/c/openstack/openstack-ansible/+/869636 | 12:39 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Deploy 3 keystone containers for infra CI jobs https://review.opendev.org/c/openstack/openstack-ansible/+/869711 | 12:41 |
| jrosser | moha7: i added CI coverage for that case of fernet key synchronisation ^ | 12:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-plugins master: [DNM] test 869711 https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/869713 | 12:55 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Bump pip and wheel to latest versions https://review.opendev.org/c/openstack/openstack-ansible/+/869715 | 13:07 |
| noonedeadpunk | I'm a bit confused about what we're doing in cinder role.... | 13:11 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-os_cinder/src/branch/master/tasks/cinder_install_source.yml#L40-L58 won't rsync drop symlink we create? | 13:11 |
| noonedeadpunk | and then we have https://opendev.org/openstack/openstack-ansible-os_cinder/src/branch/master/vars/main.yml#L69-L75 /o\ | 13:14 |
| opendevreview | Merged openstack/openstack-ansible stable/zed: [doc] Update repositiories for mirroring https://review.opendev.org/c/openstack/openstack-ansible/+/869637 | 13:15 |
| jrosser | oh the complexity :/ | 13:15 |
| noonedeadpunk | I assume we should have 2 files there tops - one https://opendev.org/openstack/cinder/src/branch/master/etc/cinder/rootwrap.d/volume.filters and second https://opendev.org/openstack/os-brick/src/branch/master/etc/os-brick/rootwrap.d/os-brick.filters | 13:17 |
| noonedeadpunk | Eventually if we need os-brick is a question.... | 13:24 |
| jrosser | it's a bit ugly because our code doesnt quite manage to say "all of these and all of those, delete everything else" | 13:25 |
| jrosser | when in fact there is only one file at each location | 13:26 |
| noonedeadpunk | Ok, os-brick likely needed. But then we can fully drop rsync I assume | 13:28 |
| noonedeadpunk | ah, ok. now I got it. we assume there can be more files and in order not to maintain the list, we just rsync.... | 13:31 |
| jrosser | yes | 13:32 |
| jrosser | i was wondering if rsync can accept two source places | 13:32 |
| jrosser | rsync <these> <those> <to-there> | 13:32 |
| noonedeadpunk | I kind of wonder why --delete doesn't touch symlink... I thought that some flag should be used for that... | 13:35 |
| noonedeadpunk | Like `--no-links` or smth | 13:36 |
| noonedeadpunk | it's even more weird.... /etc/cinder is a symlink to /openstack/venvs/cinder-26.1.0.dev29/etc/cinder so commenting out rsync change nothing | 14:02 |
| noonedeadpunk | so seems like un-needed leftover after smart sources change | 14:02 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Remove rsync requirement for cinder https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/869725 | 14:10 |
| noonedeadpunk | and basically that's the reason why it didn't delete symlink - it was synchronizing directory inside itself | 14:11 |
| opendevreview | Andrew Bonney proposed openstack/ansible-role-systemd_networkd master: Fix static routes to use Destination rather than Source key https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/869733 | 14:58 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:01 |
| opendevmeet | Meeting started Tue Jan 10 15:01:24 2023 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:01 |
| noonedeadpunk | #topic rollcall | 15:01 |
| jamesdenton | o/ | 15:02 |
| noonedeadpunk | o/ | 15:02 |
| NeilHanlon | o/ | 15:03 |
| NeilHanlon | \o, even | 15:03 |
| noonedeadpunk | :D | 15:03 |
| damiandabrowski | hi! | 15:03 |
| noonedeadpunk | #topic office hours | 15:06 |
| noonedeadpunk | Tbh I don't really have an agenda for todays meeting :-) | 15:07 |
| opendevreview | Andrew Bonney proposed openstack/ansible-role-systemd_networkd master: Handle omitted variables which appear as empty strings https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/869736 | 15:07 |
| noonedeadpunk | I'm waiting for https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/868177 and https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/868176 to issue role/services bump to tag new releases | 15:08 |
| damiandabrowski | and I was disrupted by some internal things, but this week i resumed work on internal TLS | 15:09 |
| noonedeadpunk | And I didn't have a chance to look at PKI role regarding usage of pipes | 15:09 |
| jamesdenton | last day to get submissions in for Vancouver | 15:10 |
| noonedeadpunk | yes good point ^ | 15:10 |
| noonedeadpunk | I've sumbitted osa onboarding at very least and hope to be there | 15:11 |
| jamesdenton | i saw that, thanks | 15:11 |
| noonedeadpunk | But I might have tricky situation with travels, or it might get sorted out by summit | 15:12 |
| jamesdenton | so, based on recent activity in the channel it's prob a good idea for me to put together an OVN Quick Start guide, or at a minimum improve whatever docs we have | 15:12 |
| jamesdenton | let's hope you get sorted | 15:12 |
| noonedeadpunk | yeah, I saw docs you pushed, but didn't finish reviewing | 15:13 |
| noonedeadpunk | and yep, it's quite some activity regarding OVN happening | 15:13 |
| jamesdenton | after mgariepy comments yesterday, i may have some more tweaks | 15:13 |
| noonedeadpunk | And I think it's mostly due to breaking changes we made for Zed, so spatel's blog post not valid for Z+ | 15:13 |
| jamesdenton | right. his blog is quite popular | 15:14 |
| spatel | noonedeadpunk i validated in my lab and fixing my blog for zed :) | 15:14 |
| jamesdenton | nice | 15:14 |
| noonedeadpunk | maybe they can reflact state somewhere there ^_^ | 15:14 |
| noonedeadpunk | ah, awesome! | 15:14 |
| noonedeadpunk | jamesdenton: you should get new revision of your book to beat spatel's blog success :p | 15:15 |
| jamesdenton | authoring is a young mans game | 15:15 |
| jamesdenton | s/mans/persons | 15:15 |
| spatel | Me and james should come up with new book.. OVN on your way :) | 15:15 |
| jamesdenton | :) | 15:15 |
| jamesdenton | i have much to learn | 15:16 |
| NeilHanlon | I was hoping i could make it to vancouver this year.. but I don't think it'll end up happening :( | 15:19 |
| noonedeadpunk | sad news :( | 15:19 |
| NeilHanlon | we will see. if I can get work to pay that might happen | 15:20 |
| spatel | who else going to Vancouver? | 15:20 |
| NeilHanlon | I will be at FOSDEM next month, if anyone is around :) | 15:20 |
| noonedeadpunk | it's quite close to where am I, but a bit tired of traveling at the moment, so was going to skip fosdem tbh | 15:22 |
| NeilHanlon | that's fair. it's quite busy | 15:22 |
| mgariepy | hey i'm late. | 15:24 |
| jamesdenton | there's no large trout button in this irc client | 15:25 |
| noonedeadpunk | but will see actually... | 15:25 |
| mgariepy | i probably won't be in vancouver i got some major home renovation during this time. | 15:26 |
| noonedeadpunk | There's one small thing. ansible-core 2.14 requires python >=3.9. And Ubuntu 20.04 does have 3.8 out of the box | 15:27 |
| noonedeadpunk | And since we should keep 20.04 support for Antelope for upgrade path from Y, my proposal would be to stay on 2.13 for now | 15:27 |
| spatel | what are we going to get with 2.14? | 15:28 |
| jrosser | o/ sorry late | 15:32 |
| noonedeadpunk | not much I guess - plenty of changes but it's not we want smth specific | 15:33 |
| noonedeadpunk | except will to keep closer to latest versions of used software | 15:33 |
| jrosser | andrewbonney: do we need https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/868177 in Zed? Like it's broken without? | 15:34 |
| jrosser | argh | 15:35 |
| jrosser | i mean https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/869736 | 15:35 |
| andrewbonney | No I don't think it's broken, you just get errors in the log. The static route one does cause brokenness if it's required by a deployment though | 15:35 |
| noonedeadpunk | So you're trying to override _lxc_container_systemd_networks? | 15:36 |
| opendevreview | Merged openstack/openstack-ansible master: Block unauthenticated Ironic API endpoints from untrusted networks https://review.opendev.org/c/openstack/openstack-ansible/+/868075 | 15:37 |
| jrosser | no we have a static route defined in provider_networks and it results in broken config | 15:37 |
| noonedeadpunk | ah | 15:37 |
| noonedeadpunk | yeah, fair... We should backport that then as well | 15:37 |
| jrosser | andrewbonney is doing a multinode Zed upgrade this week so we will find some bugs i expect | 15:38 |
| * noonedeadpunk crosses fingers | 15:38 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible stable/zed: Block unauthenticated Ironic API endpoints from untrusted networks https://review.opendev.org/c/openstack/openstack-ansible/+/869641 | 15:38 |
| noonedeadpunk | we're going to upgrade straight to AA | 15:38 |
| jrosser | moha7: i have a passing test with multiple keystone / rsync https://zuul.opendev.org/t/openstack/build/992d02393eac48faa2ef13d180949eb8/log/job-output.txt#13455-13457 | 15:40 |
| spatel | noonedeadpunk I have question related shared keystone deployment with OSA. I have openstack RegionOne up and running and i wants to add new cloud RegionTwo | 15:41 |
| spatel | what i should tell new Openstack that use old openstack for keystone? | 15:42 |
| noonedeadpunk | I think we should also backport AIO fix for keystone - when we randomly were failing temepst | 15:43 |
| spatel | I added this in user_var* file on new openstack - https://paste.opendev.org/show/bimxztCDsaMGpXVj1yxY/ | 15:43 |
| noonedeadpunk | we merged 2 things on master and seems it doesn't happen anymore | 15:43 |
| spatel | if we are in meeting then i will talk later...sorry | 15:44 |
| noonedeadpunk | one was https://opendev.org/openstack/openstack-ansible/commit/078c82b03456d46641a3ec05e3d14bd3ac6d1cd5 | 15:44 |
| opendevreview | Andrew Bonney proposed openstack/ansible-role-systemd_networkd master: Handle omitted variables which appear as empty strings https://review.opendev.org/c/openstack/ansible-role-systemd_networkd/+/869736 | 15:45 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Increase thread/process to 2 for keystone https://review.opendev.org/c/openstack/openstack-ansible/+/869642 | 15:46 |
| noonedeadpunk | And I think we changed smth for tempest as well.... | 15:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Increase thread/process to 2 for keystone https://review.opendev.org/c/openstack/openstack-ansible/+/869642 | 15:47 |
| noonedeadpunk | As I see random failures of tempest for Y | 15:48 |
| jrosser | there were a bunch of other places we reduced threads/workers in roles where that was forgotton | 15:49 |
| jrosser | i think mgariepy made a lot of patches like that | 15:49 |
| jrosser | would be worth at some point deciding what we want to implement this cycle | 15:51 |
| jamesdenton | Refresher: https://etherpad.opendev.org/p/osa-antelope-ptg | 15:52 |
| jrosser | https://etherpad.opendev.org/p/osa-antelope-ptg | 15:52 |
| jrosser | oh snap :) | 15:52 |
| jamesdenton | mind meld | 15:52 |
| jrosser | the only thing i have to add to that is checking we are doing the right thing with whatever system/reader scope stuff is now | 15:53 |
| jrosser | as we are trying to use the ironic ansible modules here and failing pretty badly | 15:54 |
| jamesdenton | not familiar, sorry | 15:54 |
| jrosser | all to do with system / not system scope tokens needed for that service somehow differently to other services | 15:54 |
| jamesdenton | ahh | 15:54 |
| noonedeadpunk | I _think_ we should be doing it quite right. or well, except we're not enforcing usage of system scopes for services. But I'm not sure we should, given that for services separate "service" role is needed. | 15:55 |
| noonedeadpunk | But eventually there's quite a mess in this topic right now and it's not really aligned | 15:55 |
| jrosser | right | 15:55 |
| jrosser | perhaps need to look at what the default setup in openrc is as well | 15:56 |
| noonedeadpunk | so eventually, we should end up not giving service users admin role at all | 15:56 |
| noonedeadpunk | it should be service role, but likely system scoped - but service role was just a discussion point last time I checked | 15:57 |
| noonedeadpunk | for openrc we have a way to enable system scope iirc | 15:57 |
| noonedeadpunk | but from what I recall - system scopes should not be enforced, unless I missed smth | 15:58 |
| mgariepy | jrosser, noonedeadpunk threads.. https://review.opendev.org/c/openstack/openstack-ansible/+/850942 | 15:58 |
| noonedeadpunk | so I wonder if that could be an issue with just ansible modules | 15:59 |
| mgariepy | arf, | 15:59 |
| mgariepy | no comments.. again :S | 15:59 |
| opendevreview | Merged openstack/openstack-ansible stable/zed: Sync ZFS pool names https://review.opendev.org/c/openstack/openstack-ansible/+/869633 | 15:59 |
| jrosser | i took a look and think it is policy in ironic | 15:59 |
| jrosser | thinks like "list nodes" was only available with a system scoped token | 15:59 |
| jrosser | and it was very confusing how this was all changed between yoga/zed.. | 16:00 |
| noonedeadpunk | the problem is thta system scopes were not implemented in cinder if I'm right. So enforcing them could be not safe for other services | 16:00 |
| noonedeadpunk | well, yeah, there was a plan to enforce them in Z, but as it was not aligned I can recall postponing this | 16:01 |
| noonedeadpunk | but again, I could miss smth | 16:01 |
| noonedeadpunk | #endmeeting | 16:01 |
| opendevmeet | Meeting ended Tue Jan 10 16:01:36 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:01 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-01-10-15.01.html | 16:01 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-01-10-15.01.txt | 16:01 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-01-10-15.01.log.html | 16:01 |
| noonedeadpunk | spatel: so, you have keystone in regionA and you're deploying regionB? And you have independant deploy hosts for regionA and regionB? | 16:02 |
| jrosser | it was this sort of thing `"baremetal:node:create": "role:admin and system_scope:all"` | 16:02 |
| spatel | Yes.. I have i want to shared keystone between region | 16:03 |
| spatel | I have added this snippet in RegionB to point ReagionA - https://paste.opendev.org/show/bimxztCDsaMGpXVj1yxY/ | 16:04 |
| spatel | Question how does RegionB will add endpoint in RegionA without auth? | 16:05 |
| noonedeadpunk | jrosser: ah, I pushed that some time ago to address system scopes https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/832837 | 16:07 |
| noonedeadpunk | that's why I thought that we should be doing what's needed | 16:08 |
| noonedeadpunk | likely worth rebasing and checking the result.... | 16:09 |
| jrosser | i think this is with our admin user | 16:09 |
| jrosser | trying to issue a system scoped token for that user to then use the ironic module | 16:09 |
| jrosser | what i mean is that what i've been trying would have authed as admin | 16:10 |
| noonedeadpunk | well, our admin user should be system scoped - I think it's default behaviour for keystone now. So should jsut depend on openrc iirc | 16:10 |
| jrosser | hmm | 16:10 |
| noonedeadpunk | and we have that https://opendev.org/openstack/openstack-ansible-openstack_openrc/src/branch/master/tasks/main.yml#L29-L32 | 16:11 |
| noonedeadpunk | though, `openrc_system_scope` will also affect clouds.yaml https://opendev.org/openstack/openstack-ansible-openstack_openrc/src/branch/master/templates/clouds.yaml.j2#L6-L12 | 16:12 |
| opendevreview | Jonathan Rosser proposed openstack/ansible-role-pki master: Update variables gathering to use vars/varnames lookups https://review.opendev.org/c/openstack/ansible-role-pki/+/869664 | 16:37 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow git servers for openstack services and tempest to be overridden https://review.opendev.org/c/openstack/openstack-ansible/+/869748 | 17:20 |
| spatel | noonedeadpunk around.. did you see my ask :) | 17:31 |
| noonedeadpunk | spatel: sorry, I missed that - was side-pinged | 17:32 |
| spatel | no worry, just making sure you saw or i should repost | 17:32 |
| noonedeadpunk | spatel: um, and deploy hosts are independant, right? | 17:38 |
| spatel | This is AIO box | 17:38 |
| noonedeadpunk | Ah, yes, I think you can't play with env.d for regions anyway | 17:38 |
| spatel | Does it matter ? | 17:39 |
| spatel | why env.d coming in picture here? | 17:39 |
| noonedeadpunk | nah, it's not, forget it :D | 17:39 |
| spatel | +1 | 17:39 |
| spatel | All i am doing is telling new openstack go and use keystone on old openstack :) | 17:40 |
| spatel | I did that but i encounter error saying RegionTwo doesn't found, ofc because nobody created yet. | 17:40 |
| noonedeadpunk | Regarding https://paste.opendev.org/show/bimxztCDsaMGpXVj1yxY/ - I don't think you need keystone_service_adminurl but what you need is keystone_service_publicuri | 17:41 |
| spatel | copy that.. | 17:41 |
| noonedeadpunk | Also, you should avoid defining identity_hosts in openstack_user_config | 17:41 |
| spatel | but still i don't understand logic here | 17:42 |
| noonedeadpunk | and keystone_auth_admin_password in user_secrets should be exactly the same as for first region | 17:42 |
| spatel | But that doesn't matter if i have keystone installed or not. right? | 17:42 |
| noonedeadpunk | You don't need to get second keystone installed I assume? | 17:43 |
| noonedeadpunk | As you want to use keystone in regionB from regionA? | 17:43 |
| spatel | I did copy user_secret.yml and sync between both cloud so i have same creds for everything | 17:43 |
| spatel | Yes, but i can install and leave it there and not use. I don't think that will cause any issue, correct? | 17:43 |
| noonedeadpunk | Nah, it shouldn't cause issues, just might give some confusion. | 17:46 |
| spatel | 100% agreed with you. I am just trying to understand. | 17:46 |
| noonedeadpunk | I kind of wonder if you really want to use same creds for everything. As database/rabbitmq users likely worth to differ | 17:47 |
| spatel | I don't know how OSA handle this condition when we point it to other cloud and it will do magic behind the shell | 17:47 |
| noonedeadpunk | or well, doesn't matter much either | 17:47 |
| noonedeadpunk | osa shouldn't care much about it | 17:47 |
| spatel | This is my lab to understand stuff before i rollout to production | 17:47 |
| noonedeadpunk | as long as cloud.yaml and openrc files on utility are correct and pointing to the valid keystone - it should be fine | 17:48 |
| spatel | Do i need to create or copy openrc file to new cloud manually? | 17:48 |
| noonedeadpunk | And it's taken from keystone_service_internalurl | 17:48 |
| noonedeadpunk | nope, it will be provisioned with utility setip | 17:49 |
| spatel | ? | 17:49 |
| noonedeadpunk | So defining keystone_service_*uri and taking care of secrets is close to only thing you should need | 17:49 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow git servers for openstack services and tempest to be overridden https://review.opendev.org/c/openstack/openstack-ansible/+/869748 | 17:49 |
| spatel | You are saying all i need keystone_service_*uri and password and just run setup-everything.yml ? | 17:51 |
| noonedeadpunk | But I personally would think about separating keystone service users per regions. | 17:51 |
| noonedeadpunk | As then password rotation might become a nightmare in day2 | 17:51 |
| noonedeadpunk | but for POC - yeah, I'd say it should just work. except you will need some extra config for horizon | 17:52 |
| spatel | we don't do password rotate | 17:52 |
| noonedeadpunk | Nobody ever quit your company alive ? :D | 17:52 |
| spatel | That is me.. I am the only one here doing this shit :) | 17:53 |
| noonedeadpunk | Ah, then fair :D | 17:53 |
| spatel | Even if they take password nothing going to happened :D everything is under private net (nothing public) | 17:53 |
| spatel | We run private cloud.. hehe | 17:54 |
| spatel | Still i have question, lets say if i define two variable keystone_auth_admin_password & keystone_service_*uri but how does other service register or create endpoint in RegionTwo? | 17:55 |
| spatel | How does new openstack auth using RegionOne and create RegionTwo? | 17:56 |
| spatel | Yesterday i was getting error related RegionTwo didn't find so i created by hand on old openstack and everything start working. | 17:56 |
| spatel | That is what trying to understand how... We don't have any document related that in osa offical doc | 17:57 |
| spatel | i can give it a try again - keystone_service_publicuri: https://openstack.example.com:5000 is this correct way to define it? | 17:59 |
| noonedeadpunk | yup | 17:59 |
| noonedeadpunk | Oh, well... | 17:59 |
| noonedeadpunk | Maybe we indeed don't create regions properly.... | 17:59 |
| spatel | I didn't find anything in code who does, may be i missed something. | 18:00 |
| spatel | only solution i found was create RegionTwo identity endpoint by hand and after that everything started flowing smoothly | 18:01 |
| noonedeadpunk | yeah, true | 18:01 |
| noonedeadpunk | We do create region only during keystone bootstrap | 18:01 |
| noonedeadpunk | And since you don't run keystone role - it simply doesn't happen | 18:01 |
| spatel | We don't do bootstrap in this case correct? | 18:01 |
| noonedeadpunk | Yup | 18:02 |
| spatel | I am not crazy here :D | 18:02 |
| noonedeadpunk | So yeah, you need to create region by hand as of today. I assume that might be worth fixing though... Or just write to docs :D | 18:02 |
| spatel | Let me play enough to find bugs and then we can fix all at once. | 18:03 |
| spatel | Then i don't need keystone_service_publicuri: https://openstack.example.com:5000 because my previous solution working as i mention if i create by hand | 18:04 |
| noonedeadpunk | well, it will create wrong public endpoints then? | 18:08 |
| spatel | but it works for me with this snippet - https://paste.opendev.org/show/bimxztCDsaMGpXVj1yxY/ | 18:09 |
| noonedeadpunk | ah, yeah, you;re right probably | 18:09 |
| noonedeadpunk | keystone_service_publicuri in fact is used only for keystone role | 18:09 |
| noonedeadpunk | and well. I found some nasty thing in placement I think | 18:10 |
| spatel | hmm? | 18:12 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-os_placement/src/branch/master/templates/placement.conf.j2#L10 | 18:16 |
| spatel | Ooo | 18:17 |
| spatel | oh wait.. i am not seeing anything wrong here.. why do you think it will break? | 18:19 |
| spatel | you are saying keystone_service_publicuri need to specified ? | 18:19 |
| noonedeadpunk | Well, I'm not convinced it's needed there. | 18:21 |
| noonedeadpunk | but I see couple of services that use it in keystone_authtoken | 18:22 |
| noonedeadpunk | So yeah, might be worth to define it after all | 18:22 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow git servers for openstack services and tempest to be overridden https://review.opendev.org/c/openstack/openstack-ansible/+/869748 | 18:50 |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts stable/zed: Ensure tar is installed on LXC host https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/868176 | 18:58 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow git servers for openstack services and tempest to be overridden https://review.opendev.org/c/openstack/openstack-ansible/+/869748 | 19:02 |
| opendevreview | Merged openstack/openstack-ansible-plugins stable/yoga: Limit maximum number of threads for parallel git clone https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/869464 | 19:21 |
| BobZAnnapolis | hey folks - we rebooted the controllers, system came back up . . sort of...looks like the REST API calls are being sent to everyone but nothing is being written into Galera - can't create new users, instances, etc ? Any ideas on what to run after the controllers get rebooted to get full functionality back ? tia | 19:32 |
| jrosser | BobZAnnapolis: you could try come if this https://docs.openstack.org/openstack-ansible/latest/admin/maintenance-tasks.html | 19:42 |
| jrosser | *some of… | 19:42 |
| BobZAnnapolis | jrosser: Thanks, we've started running thru those. . . .almost looks like the db is in read-only mode. Components are rcvng the API "create" commands, but then timing out w/o being able to complete the create requests :-( | 19:54 |
| moha7 | A channel gathering tweets about OpenStack: https://t.me/opstweets | 20:00 |
| opendevreview | Merged openstack/openstack-ansible master: Deploy 3 keystone containers for infra CI jobs https://review.opendev.org/c/openstack/openstack-ansible/+/869711 | 20:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Allow empty records for container_skel https://review.opendev.org/c/openstack/openstack-ansible/+/869762 | 20:22 |
| noonedeadpunk | that is quite contraversial change ^, I'm quite open for suggestions on how to do that better/in more clear way. I was thinking on implementing some new property, like "virtual" or smth like that, but once realized that item can be simply empty - decided to simplify that | 20:27 |
| noonedeadpunk | But I kind of need some functionality like that - to define a group that will not create any containers, but will include all existing containers of host that part of this group | 20:28 |
| noonedeadpunk | Will mark it as WIP for now as want to play a bit more and see if it doesn't have unexpected flaws | 20:46 |
| mgariepy | no comments.. again :S | 21:08 |
| mgariepy | oops. | 21:08 |
| BobZAnnapolis | ok, new problem....got the Galera DB cluster back working & synch'd up but now. . . .after the the 3-controller reboot - we can't create new instances or volumes - we can delete ones that error out & don't finish getting created, so we can write to disk & start the process on a compute node - but it never finishes :-( - logs indicate a 2-minute timeout from getting a response and then the scheduler attempts to | 21:09 |
| noonedeadpunk | I assume this might be rabbitmq thing | 21:13 |
| noonedeadpunk | try out re-running `openstack-ansible playbooks/rabbitmq-install -e rabbitmq_upgrade=true` | 21:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Restore dynamic_inventory unit testing https://review.opendev.org/c/openstack/openstack-ansible/+/869776 | 22:43 |
| -opendevstatus- NOTICE: One of our CI job log storage providers appears to be having trouble with log uploads and retrievals. We are in the process of removing that provider from the pool. | 22:43 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Allow empty records for container_skel https://review.opendev.org/c/openstack/openstack-ansible/+/869762 | 22:45 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Allow empty records for container_skel https://review.opendev.org/c/openstack/openstack-ansible/+/869762 | 22:45 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!