| grauzikas | Hello everyone. I spent a lot of time by trying to setup openstack by using openstack ansible and i cant make ansible to install ovn northd… i have been following a lot of manuals, examples from google, but nothing works… it creates container for that, but nothing inside off it. Not sure may be something changed and manuals not updated or im missing something. | 06:53 |
|---|---|---|
| grauzikas | my configs looks like this https://pastebin.com/cDBSNRbm | 06:54 |
| grauzikas | may be im missing something, may be it reads some different configs or it should be in different folders… dont understand why and cant find anything in manuals | 06:54 |
| grauzikas | will try now go thrue bugs may be there some one will be shared them configs and then i will try to follow it… already spent so much time that i could install everything manually (was doing that previously, but to have posibility easier upgrade everything wanted to give a try to ansible) | 06:56 |
| grauzikas | https://www.irccloud.com/pastebin/5CoN2Jsa | 07:09 |
| noonedeadpunk | platta: at very least we need to know what the task has failed to be able to trace it down | 07:26 |
| noonedeadpunk | we do have some variables in code that can disable no_log, but they are different per usecase | 07:27 |
| noonedeadpunk | grauzikas: hey | 07:27 |
| jrosser | good morning o/ | 07:27 |
| noonedeadpunk | so if you have container created - I'm really very surprised that northd not getting installed inside it | 07:29 |
| noonedeadpunk | defining network-northd_hosts as you did should be really enough for it to get it | 07:29 |
| jrosser | noonedeadpunk: i see overrides of env.d for nova and neutron there which i would be suspicious of? | 07:30 |
| jrosser | (i think?) | 07:30 |
| noonedeadpunk | ah, I missed that | 07:31 |
| noonedeadpunk | grauzikas: I assume you've found some old blog post designed to pre-Zed setup | 07:32 |
| noonedeadpunk | you should not need any of these env.d overrides as of today | 07:32 |
| noonedeadpunk | also I usually suggest using `network-infra_hosts` instead of `network_hosts` to avoid potential confusion | 07:34 |
| jrosser | grauzikas: i would *highly* recommend building an all-in-one following the quickstart guide to get you a reference | 07:34 |
| jrosser | this is very useful even if you intend to have a multinode deployment in order to have a comparison of how all the parts fit together | 07:35 |
| noonedeadpunk | (or just as `git clone https://opendev.org/openstack/openstack-ansible; cd openstack-ansible; ./scripts/gate-check-commit.sh`) | 07:36 |
| noonedeadpunk | but do that only on some clean system you ready to re-setup afterwards | 07:36 |
| jrosser | or in a vm | 07:37 |
| noonedeadpunk | you can also inspect inventory using a script /opt/openstack-ansible/scripts/inventory-manage.py -g | 07:41 |
| grauzikas | ok thank you… will try now your sugestions. | 08:35 |
| grauzikas | is it enough to retry deploy everything again? | 08:35 |
| grauzikas | apt-get --purge remove haproxy keepalived openvswitch-common openvswitch-switch ovn-host ovn-common -y && rm -rf /etc/haproxy/ /etc/keepalived/ && rm -rf /openstack && for container in $(lxc-ls -1); do lxc-stop -n $container; lxc-destroy -n $container; done | 08:35 |
| grauzikas | on all nodes | 08:36 |
| grauzikas | also im missing loggings in some of servers, for example in neutron i must go inside lxc container, edit neutron configuration and define log path and file and restart to get logings and i found that a lot of servers has same issue. should i some how define in ansible to enable loging for servives or this is manual job after installation? | 09:28 |
| jrosser | grauzikas: the logging should all be going to the systemd journal | 10:25 |
| jrosser | do you have anything thats not doing that? | 10:26 |
| jrosser | you should not have to configure anything manually after installation | 10:26 |
| jrosser | there is information here about the different ways that you can customised the configuration where needed https://docs.openstack.org/openstack-ansible/latest/reference/configuration/using-overrides.html | 10:28 |
| jrosser | the `config_template` mechanism we use is especially powerful | 10:28 |
| grauzikas | ok trying to rerun playbooks after removing all containers and openstack_ansible will see what will be :) | 10:36 |
| jrosser | there is also a playbook to destroy containers if you need it | 10:37 |
| grauzikas | is it correct binding for vxlan in ovn scenario? - neutron_ovn_controller and for vlan (used as exit to router where we have announced ips by bgp) neutron_ovn_gateway ? | 11:50 |
| grauzikas | btw thanks for playbook what destroy containers, previusly was removing by hand and removing keys and so on :) | 11:54 |
| noonedeadpunk | no, vxlan is not a thing in OVN | 12:01 |
| noonedeadpunk | or well, I _think_ it can be used as external networks, but never tried that | 12:02 |
| gillesMo | Hello Stackers ! | 12:53 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron stable/2023.2: Correct 'neutron-policy-override' tag https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/925734 | 12:54 |
| gillesMo | I'm trying to upgrade a Lab cluster which is OpenStack Wallaby on Ubuntu 18.04 to first, Ubuntu 20.04, and I fail already. | 12:54 |
| gillesMo | I'm following the doc distribution upgrade, but I had some failures (rabbitMQ, fernet tokens and creds renew...) | 12:55 |
| gillesMo | But, for now, the main problem during setup-openstack (for keystone and all other projects), is that it does not build or find builds on the right repo container. | 12:57 |
| gillesMo | I don't understand how it's supposed to work. It seems replication via lsynd is only from the "primary" container to the others, but as I have upraded a non primary, some files have been put in the new secondary repo container, but not replicated | 12:58 |
| jrosser | gillesMo: if you make the old repo containers be "down" for haproxy then you will only use the upgraded one when deploying the services | 13:02 |
| jrosser | you can do that by putting the backends in maintenance mode in haproxy | 13:03 |
| jrosser | or more brute force by shutting down the web servers or whole repo server containers | 13:03 |
| jrosser | in later releases a shared filesystem is used to make that whole trouble with lsycnd go away | 13:03 |
| mgariepy | gillesMo, it's a bit outdated but there are a few tips here: https://etherpad.opendev.org/p/osa-newton-xenial-upgrade | 13:06 |
| gillesMo | Oh, yes. Thank you ! In the doc, it said (optional) for putting in MAINT mode backends, I'll check | 13:09 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron stable/2024.1: Correct 'neutron-policy-override' tag https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/925733 | 13:10 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Remove deprecated http-use-htx option https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/925877 | 14:05 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Remove deprecated 'stats bind-process' directive https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/925881 | 14:17 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Remove deprecated 'stats bind-process' directive https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/925881 | 14:20 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Remove the deprecated 'nbproc' config option from the example settings https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/925884 | 14:25 |
| gillesMo | It seems that this commit solves my constraint problem, I'v cherrypicked it : | 14:35 |
| gillesMo | https://opendev.org/openstack/ansible-role-python_venv_build/commit/57a2f226ebca7a2ec21920d4fd2b7ccf45490684 | 14:36 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Ensure Octavia communicates with Neutron through internal URL https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/925770 | 15:22 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Ensure Octavia communicates with Neutron through internal URL https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/925770 | 15:23 |
| platta | I'm stuck in the setup-openstack.yml playbook, getting a censored error for the task "open stack.osa.db_setup: Create database for service". HAProxy shows Galera as UP, and Keystone backend as DOWN. I can verify Galera is up. I do see that in my user_secrets.yml file there is no keystone_galera_password like there is for some of the other services. | 16:10 |
| platta | Not sure if that's supposed to be there, but I generated the secrets file using the script provided. Any direction on what to check next? | 16:10 |
| noonedeadpunk | you can run a playbook with `-e _oslodb_setup_nolog=False` to uncensor the output | 16:12 |
| noonedeadpunk | but yes, these are supposed to be there | 16:12 |
| noonedeadpunk | or well, it's a keystone_container_mysql_password | 16:13 |
| noonedeadpunk | platta: but run with -e _oslodb_setup_nolog=False first to see what's actually an issue is | 16:14 |
| platta | keystone_container_mysql_password does exist in my secrets file. I'll re-run the playbook with that setting to see what's happening. | 16:17 |
| noonedeadpunk | so the script would populate passwords for everything that's in empty user_secrets.yml is present | 16:18 |
| noonedeadpunk | and then you're expected to use https://opendev.org/openstack/openstack-ansible/src/branch/master/etc/openstack_deploy/user_secrets.yml as a base for generation | 16:18 |
| noonedeadpunk | it contains all required passwords for all services | 16:18 |
| noonedeadpunk | but you can also pouplate/generate them manually as well | 16:18 |
| platta | failed: [ark-keystone-container-76e19f64 -> ark-utility-container-e8c00157(172.29.239.82)] (item={'name': 'keystone', 'users': [{'username': 'keystone', 'password': 'xxx'}]}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "keystone", "users": [{"password": "xxx", "username": "keystone"}]}, "msg": "unable to connect to database, | 16:30 |
| platta | check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (2006, \"MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))\")"} | 16:30 |
| platta | I tried `ansible galera_container -m shell -a "mysql -h localhost -e 'SELECT user FROM mysql.user;'"` and it doesn't show any service-specific users. I don't know if that's supposed to happen for all of them earlier on. | 16:36 |
| platta | Just realized I didn't include the task on that: TASK [openstack.osa.db_setup : Create database for service] | 16:46 |
| platta | Looks like what's failing is here https://opendev.org/openstack/openstack-ansible-plugins/src/commit/5b8a1d9be03146ffac8e91e92a044429e9286dbd/roles/db_setup/tasks/main.yml and I think it's expecting the keystone user to already exist. I'm doing a little searching in the opendev repositories, but I'm not sure at what point in the process those users | 16:56 |
| platta | are supposed to get created. | 16:56 |
| platta | Last piece before I have to step away for a while. If I attach to the utility container and run `mysql -h 172.29.236.101 -e 'show databases;'` I get the same error as shown by Ansible (that IP is the internal load balancer). I either get ERROR 2026 (HY000): TLS/SSL error: unexpected eof while reading or ERROR 2026 (HY000): TLS/SSL error: Broken | 17:06 |
| platta | pipe (32) if I run the command multiple times. Potentially a network configuration issue? | 17:06 |
| opendevreview | Merged openstack/openstack-ansible master: Use unbound_clients role from plugins collection https://review.opendev.org/c/openstack/openstack-ansible/+/923409 | 17:58 |
| opendevreview | Merged openstack/openstack-ansible master: Enchance reference_group logic for inventory https://review.opendev.org/c/openstack/openstack-ansible/+/923596 | 18:03 |
| opendevreview | Merged openstack/openstack-ansible master: Permit Ubuntu Noble for deploy host and targets in requirements checks https://review.opendev.org/c/openstack/openstack-ansible/+/924311 | 21:05 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!