| moha7 | On the infra1-ovn-northd-container, I have a problem with openvswitch-switch with this error: Dependency failed | 05:25 |
|---|---|---|
| moha7 | It needs to install openvswitch-switch on the hosts itself! But why the deploying tool didn't install it?! | 05:42 |
| moha7 | > I have OpenStack Victoria with Ubuntu 20.04. Is it possible to upgrade it to Yoga in one go? | 06:02 |
| moha7 | It's my question too. What would be the procedure if I want to upgrade in the future (for example from Yoga with OVS to Zed with OVN)? | 06:03 |
| moha7 | Generally, Is it safe to upgrade openstack at all? or you prefer to deploy a new environment (named B) and migrate all VMs from env A to env B? | 06:05 |
| moha7 | jrosser: Days ago you mentioned that `br-vlan` bridge is depricated. Someone, here, told: "Since OSA don't do agents in containers anymore, `br-vlan` has sort of lost its purpose and is just an unnecessary hop."; Is it the reason that it's no longer recommended? | 06:09 |
| jrosser | moha7: do you need br-vlan in a production deployment, no - you don't because it is possible to give an interface directly to neutron like a bond or ethX and it will work fine. | 11:04 |
| jrosser | moha7: does copying the AIO example and making br-vlan and a veth and a new interface add an extra hop, yes it does | 11:05 |
| jrosser | moha7: does the AIO work without br-vlan? no, it doesnt, because the AIO is designed to be in a single host/VM with just one interface some "extras" are required make the networking work in that special case | 11:06 |
| jrosser | so the answer you want is that you should set up the networking to suit your environment, do not simply copy the AIO setup assuming that it is the only possibility | 11:07 |
| jrosser | moha7: regarding upgrades, we do not test automatic OVS->OVN or linuxbridge->OVN upgrades although jamesdenton might have some notes to help do it manually. expect to have downtime whilst doing that sort of upgrade | 11:08 |
| jrosser | moha7: it is safe to upgrade openstack-ansible, if you look at our CI jobs then every patch that is merged must pass a N-1 to N version upgrade followed by tempest test suite. Look at the instructions here https://docs.openstack.org/openstack-ansible/latest/admin/upgrades/major-upgrades.html | 11:10 |
| jrosser | moha7: regarding V->Y upgrades in one step that is not officially supported by OSA and especially not by the Nova project though having said that some people here like noonedeadpunk do upgrades successfully in their environments which *do* skip releases so it can be done. | 11:11 |
| jrosser | moha7: for your infra1-ovn-northd-container problem please always say which branch you are using and which OS and perhaps jamesdenton will have some insight for you. | 11:13 |
| moha7 | Thank you for your complete answers. Now I'm on Yoga (25.2.0). Hosts are Ubuntu 22.04. I installed the package `openvswitch-switch` manually on controller hosts to pass that error. Finally I have a WORKING OSA (: | 11:34 |
| jrosser | ok so just bear in mind that as we've said before the OVN deployment in Yoga was "Experimental" and a bunch of stuff was re-organised for the Zed release | 11:37 |
| moha7 | But I can not create provider (external) networks. Is the netplan config for OVS the same as what I'm using now: http://ix.io/4k5U ? | 11:37 |
| jrosser | we will not be testing the upgrade path with OVN for Y->Z | 11:38 |
| jrosser | sorry i don't really use netplan nor OVS | 11:38 |
| jrosser | but regardless of that you should have some vlan type networks defined somewhere for neutron? | 11:39 |
| jrosser | also i do not think that the netplan config really matters? you will define an interface to use in openstack_user_config so it is independant of netplan so long as it exists | 11:40 |
| jrosser | moha7: from the example here is how the external vlan type networking is defined for the OVS scenario https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio.j2#L140-L150 | 11:42 |
| jrosser | in this case it uses "eth12" but you would substitute whatever interface actually is your provider networks there | 11:42 |
| moha7 | jrosser> just curios what OS do you usually go to install OSA on? | 11:44 |
| jrosser | currently ubuntu focal but soon we will upgrade to jammy | 11:44 |
| jrosser | but really i don't think this is a big factor in if things work or not | 11:44 |
| moha7 | Ah, so you don't use netplan on Ubuntu, right? | 11:45 |
| jrosser | for historical reasons i still use ifupdown | 11:45 |
| jrosser | but it doesnt matter | 11:46 |
| jrosser | it doesnt matter becasue the things that go into openstack_user_config and the neutron config files are interface names | 11:46 |
| jrosser | how those were configured netplan / ifupdown / NetworkManager / systemd_networkd is really irellevant, and OSA lets you use whichever tool you like to do that | 11:47 |
| jrosser | some openstack deployment tools will take complete ownership of all of that, but OSA is designed more like a toolbox and you have a large amount of freedom to integrate however you want | 11:48 |
| jrosser | but that does make it more difficult to get started as there is not "one correct way" to do any of it | 11:49 |
| jrosser | moha7: i should give you an example of why the AIO is so important, please look here https://review.opendev.org/c/openstack/openstack-ansible/+/867954 | 11:50 |
| jrosser | this is just one patch to openstack-ansible.... press "Zuul Summary", sort of in the middle | 11:51 |
| jrosser | you see there that there are perhaps 20 different AIO setups have been run for that patch, and all have to succeed for it to merge | 11:52 |
| jrosser | they include with/without LXC containers, upgrades from the N-1 release, ceph storage.... | 11:52 |
| jrosser | that is why we have relatively high confidence in the AIO and say it is always a reference for you to try first | 11:53 |
| moha7 | At this time, The AIO config examples (user_variables and user_config files) are not out of the box. I couldn't dominant issues, So came back to work on the multinode. Finally deployed. | 11:53 |
| jrosser | hmm ok | 11:54 |
| moha7 | But since you emphasize, I'll it again | 11:55 |
| jrosser | i also should show you this https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/868415 | 11:55 |
| jrosser | you see there a patch to neutron which tests with/without LXC, OVS, OVN, linuxbridge and upgrades | 11:55 |
| jrosser | all must pass | 11:55 |
| moha7 | https://www.cnblogs.com/liujitao79/p/15251805.html --> the guy is defining bridges directly by OVS. He also installs OVS manually on hosts as encountered with it as a dependency for contaoners | 11:56 |
| jrosser | that is not a scenario we have ever run in our testing | 11:56 |
| jrosser | just to be clear - that is using OVS for providing things like br-mgmt | 11:57 |
| jrosser | that is not something we ever do with standard OSA even when OVS is used for openstack networking | 11:58 |
| jrosser | moha7> At this time, The AIO config examples (user_variables and user_config files) are not out of the box. | 12:04 |
| jrosser | ^ you should let the AIO generate it's own config, not put in your own | 12:04 |
| moha7 | +1 | 12:22 |
| ThiagoCMC | Hey folks, the following task is faiiling: | 13:42 |
| ThiagoCMC | TASK [python_venv_build : Fail if wheels are not built for multiple hosts] | 13:42 |
| ThiagoCMC | Error: "You are not building wheels while running role against multiple hosts. This might result in DOS-ing OpenDev infrustructure servers. In order to proceed, please ensure that you have repo servers for selected OS version and architecture. If you want to avoid building wheel on purpose, ensure that you run playbook in serial manner. In case of causing unreasonable load on the opendev.org git | 13:43 |
| ThiagoCMC | servers, your access may be blocked to protect other users and the OpenDev CI infrastructure which are reliant on this service." | 13:43 |
| ThiagoCMC | I tried yesterday, and today, a few times, any idea? | 13:43 |
| ThiagoCMC | On Utility container. | 13:43 |
| ThiagoCMC | It was running with Victoria 22.1, now I'm trying a minor upgrade to `stable/victoria`. | 13:46 |
| jrosser | ThiagoCMC: it is becasue there is no valid host available to build wheels (repo server) for some reason | 13:47 |
| jrosser | the architecture and OS have to match the targets, and also ansible facts have to be present in order for the right build host to be chosen | 13:48 |
| jrosser | it should print the targets found here https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/stable/victoria/tasks/main.yml#L39 | 13:50 |
| jrosser | the right thing to do is to find why venv_build_targets is not populated | 13:50 |
| jrosser | ThiagoCMC: unfortunately we had to add this condition as a hard failure, partly because no-one in this situation was prepared to do sufficient debugging to work out why it happens. | 13:53 |
| jrosser | when you get to nova specifically, without fixing this you will git clone the pretty huge nova repo with pip in parallel for each of your compute nodes and this has a pretty bad effect on the opendev git server | 13:55 |
| ThiagoCMC | I don't wanna cause any troubles! And I only have 3 compute nodes. | 14:00 |
| ThiagoCMC | Does it worth trying to debug/fix this for old Victoria? | 14:00 |
| ThiagoCMC | jrosser, I didn't copy/paste the whole message, here's the end of it: | 14:01 |
| ThiagoCMC | "Found venv_build_targets: {'20.04': {'x86_64': 'os-control-1_repo_container-XXXX'}}"}" | 14:02 |
| jrosser | well 3 is probably ok tbh | 14:02 |
| ThiagoCMC | It's just a small deployment at home. ^_^ | 14:03 |
| jrosser | but this logic is still present in master today, and we still don't understand what happens to get to where you are now | 14:03 |
| ThiagoCMC | Hmmm... I'd like to help more then! | 14:03 |
| jrosser | https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/defaults/main.yml#L129 | 14:04 |
| jrosser | thats where it selects the build host for whatever host you're deploying to | 14:04 |
| ThiagoCMC | Would help to destroy/re-create at least one of the Utility containers? I have 5. | 14:05 |
| ThiagoCMC | It happens in all of them at the same time. | 14:05 |
| jrosser | not really | 14:06 |
| jrosser | better to understand why the conditional fails that makes that error by working back through the vars | 14:06 |
| jrosser | my guess is that this is perhaps as simple as missing facts somewhere | 14:06 |
| jrosser | anyway i have to head out now - would be great if you can find a root cause | 14:07 |
| ThiagoCMC | Ok, I'll try! But I'm no expert in Ansible. lol | 14:07 |
| ThiagoCMC | I can give you access to a jumpbox via TeamViewer later on! =P | 14:08 |
| cloudnull | Happy new year all | 16:01 |
| ThiagoCMC | Happy New Year! | 16:11 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!