| *** manuvakery1 is now known as manumoh | 02:19 | |
| user090 | Hi i like to get some guide to set up octavia with OVN networking , I am not able to find proper docs for it, using rocky9 for deployment I have created bridge br-lbaas on compute nodes, but ovs system complaning that bridge exists , | 05:57 |
|---|---|---|
| user090 | https://pastebin.com/xFAqv7u4 | 05:57 |
| user090 | This ismy user_config.yml file snippet | 05:57 |
| noonedeadpunk | hey | 06:01 |
| noonedeadpunk | user090: so eventually for any octavia setup you need to meet 2 conditions: 1. have a lbaas network inside octavia lxc container 2. have the same network in Neutron available for VMS | 06:05 |
| noonedeadpunk | as basically you need to have access from API to VM via internal link | 06:05 |
| user090 | @noonedeadpunk that means I need bridge with static ip addresses across physical nodes, I was running like that with old clusters with linux bridge I configured bridge interface on each node, same setup seems not working with ovn as ovs creating a bridge and there is duplicate error | 06:08 |
| noonedeadpunk | So with ovs/ovn you need a bridge only on control plane | 06:19 |
| noonedeadpunk | and you never needed an IP on the bridge either | 06:20 |
| user090 | noonedeadpunk: the amphora vm is created on hypervisor nodes right? so how can they communicate from hypervisor without defining network i dont get it | 06:23 |
| noonedeadpunk | so if that network is a VLAN - you create a VLAN internal network in neutron | 06:26 |
| noonedeadpunk | it really kinda depends on how your cluster is setup and if you split interfaces | 06:26 |
| noonedeadpunk | but if you expect same vlan ID for lbaas to be present on your current network mapping - you just create a neutron network and define it's uuid for octavia | 06:28 |
| noonedeadpunk | and then on control plane you configure the same vlan and add to the br-lbaas there | 06:29 |
| noonedeadpunk | so that's why I said you don't need to create brdiges on computes | 06:38 |
| noonedeadpunk | user090: but if you have hyperconverged setup - it's a little bit more tricky - as you need to have both ovs and lxb bridge | 06:38 |
| user090 | My control plane services are running on lxc but hypervisors are using metal install. | 06:39 |
| noonedeadpunk | yeah, sure | 06:40 |
| user090 | So neutron and nova compute services are metal install on compute nodes | 06:40 |
| noonedeadpunk | right | 06:41 |
| user090 | So i have to delete all compute nodes bridges and keep br-lb only on control plane that should work right , octavia_provider_network_type: vlan and octavia_provider_segmentation_id: 6 do i have to declare them or I just manually create br-lb and connect to bond0.6 | 06:42 |
| user090 | Thanks let me try again | 06:51 |
| noonedeadpunk | user090: sorry, was a bit afk. so octavia_provider_network_type and octavia_provider_segmentation_id are controlling neutron network | 07:19 |
| noonedeadpunk | you can create network in neutron manually and define `octavia_service_net_setup: false octavia_neutron_management_network_uuid: $UUID` instead of these | 07:20 |
| noonedeadpunk | br-lb on control plane is independent process from this | 07:20 |
| noonedeadpunk | and yes, you either need to create the bridge manually, or you can leverage systemd_networkd role for it's configuration | 07:22 |
| noonedeadpunk | ie https://docs.openstack.org/openstack-ansible/latest/user/network-arch/example.html#configuring-network-interfaces | 07:22 |
| user090 | noonedeadpunk: Well i successfully completed playbook but loadbalancer is in Error state probably some config issue management network created automatically, octavia-worker complaining : Failure: octavia.common.exceptions.ComputeWaitTimeoutException: | 07:45 |
| noonedeadpunk | well, that sounds like connectivity issue between controller and amphora... | 07:47 |
| noonedeadpunk | I'd suggest to create some cirros VM on the network for debugging first | 07:47 |
| user090 | Well here is my doubt as by the conf I dont have vlan 6 on hypervisors so how can control plane communicate with amphora vms , I dont understand the flow | 07:58 |
| user090 | br-lb bridge currently only on control plane | 07:58 |
| derekokeeffe85 | Morning all, just wondering if someone could give me a pointer as where to start looking to solve this https://paste.openstack.org/show/bUzHTJxoiKgbi5F96PsB/ setup_infrastructure completed with no errors | 07:59 |
| f0o | derekokeeffe85: the command is no longer 'mysql' but 'mariadb' | 08:09 |
| derekokeeffe85 | Ah thank you | 08:09 |
| f0o | this changed in upstream packaging a few versions ago, could be that the docs didnt reflect this.. could also be that some OS have the change and others still use mysql for compatibility as symlink... it's a bit of a mess when upstream yolo-change binaries like that | 08:10 |
| derekokeeffe85 | Just looking at the docs now to see if it was my mistake probably is) Just replace mysql with mariadb | 08:11 |
| derekokeeffe85 | for the command to run? | 08:11 |
| f0o | yeah | 08:12 |
| f0o | funny enough the mariadb binary still uses .mysql_history for CLI history... shows you how that rename still drags legacy | 08:14 |
| derekokeeffe85 | Perfect, thanks f0o. That ran. Time to run setup_openstack now, fingers crossed :) | 08:15 |
| f0o | :) | 08:20 |
| noonedeadpunk | I think I proposed doc updates recently to cover that | 08:27 |
| derekokeeffe85 | getting this error now f0o https://paste.openstack.org/show/bCmfrevQ1jARL8IyMSWJ/ something to do with mariadb over mysql? I cloned 30.0.1 following 2024.2 deployment guide | 08:27 |
| noonedeadpunk | derekokeeffe85: try running the same with `-e _oslodb_setup_nolog=False` | 08:28 |
| noonedeadpunk | yeah, patch to docs landed now: https://review.opendev.org/c/openstack/openstack-ansible/+/942569 | 08:29 |
| f0o | derekokeeffe85: no that reads more like the containers arent running or there's some other issue with SSH'ing into them | 08:29 |
| noonedeadpunk | probably worth backporting to 2024.2... | 08:29 |
| derekokeeffe85 | So like this noonedeadpunk openstack-ansible openstack.osa.setup_openstack -e _oslodb_setup_nolog=False -vvv | 08:29 |
| derekokeeffe85 | Having issues with MTU and ssh as ansible but thought I had those sorted | 08:29 |
| noonedeadpunk | yeah | 08:30 |
| derekokeeffe85 | Will do | 08:30 |
| noonedeadpunk | as this will unveal the real issue which is hiding behind no_log | 08:30 |
| noonedeadpunk | but also that will log your passwords in plain text to the output log | 08:30 |
| derekokeeffe85 | Perfect | 08:30 |
| Core9525 | any possible way to remove/change overcommitment ratio from 3.1 to 2.1? | 08:31 |
| Core9525 | or to 1.2:1 | 08:31 |
| Core9525 | without impact on running env? | 08:32 |
| Core9525 | team any suggestions | 08:32 |
| derekokeeffe85 | This is the output https://paste.openstack.org/show/bkRhe3n8TGOYtUXkjeeZ/ | 08:32 |
| f0o | Core9525: https://github.com/openstack/openstack-ansible-os_nova/blob/master/defaults/main.yml#L345-L350 | 08:35 |
| f0o | do you mean those? | 08:35 |
| Core9525 | f0o yeah, right now my allocation is 3:1 for both ram and cpu, the workloads are running already, can we change this to lower valur withot impacting the workload. | 08:38 |
| f0o | yeah you can change that at will, it is only considered on scheduling and not for preexisting workload | 08:38 |
| f0o | if you issue a migration of an instance then it will be taken into account | 08:39 |
| Core9525 | f0o++ | 08:41 |
| Core9525 | thanks | 08:41 |
| noonedeadpunk | derekokeeffe85: and are you able to use `mysql` from this utility container? | 08:44 |
| noonedeadpunk | As still somehow looks like mtu stuff | 08:44 |
| noonedeadpunk | or well. `mariadb` :D | 08:44 |
| Core9525 | any one using or tried ironic service implemented? | 08:46 |
| noonedeadpunk | Core9525: I know that jrosser using it, so yes | 08:47 |
| Core9525 | noonedeadpunk++ | 08:48 |
| jrosser | hello o/ | 08:48 |
| f0o | while we talk about compute stuff; why was the watcher role abandoned? | 08:49 |
| f0o | just a matter of too little time and nobody using it? | 08:49 |
| noonedeadpunk | f0o: back in the days there we talks about deperecation of watcher as a whole, and nobody was interested in maintaing the role | 08:49 |
| noonedeadpunk | last PTG we discussed about reviving the role, but pretty much lacked time | 08:49 |
| noonedeadpunk | so it's on the table to bring it back | 08:49 |
| f0o | and then broadcom came around and revived it indirectly haha | 08:50 |
| noonedeadpunk | yeah.... | 08:50 |
| noonedeadpunk | So if you want/have time to look into it - I am happy to sort out administrative burden of getting repo back and adding it back to ci/governance | 08:51 |
| f0o | unfortunately my ansible skill is far below usable here :/ | 08:52 |
| f0o | I can do small patches here and there but ansible as a whole is still black magic to me | 08:52 |
| noonedeadpunk | are you interested to use/test watcher if we'd bring it back? | 08:52 |
| f0o | yeah that's a no-brainer for me/us | 08:53 |
| noonedeadpunk | ++ | 08:53 |
| jrosser | Core9525: ironic via openstack-ansible is usable, but ironic itself is very flexible so you have to understand and design what you want, then use openstaack-ansible to deploy that | 08:53 |
| jrosser | it's certainly not "deploy the defaults and it all just works" | 08:54 |
| derekokeeffe85 | Sorry noonedeadpunk got called away, willbe back in a bit to look into what you said | 08:54 |
| Core9525 | jrosser i am going through the documts openstack-ansible role for ironic, looks straigh, let me start work this, if stuck ill ask for the help. | 08:56 |
| Core9525 | jrosser++ | 08:56 |
| Core9525 | thanks | 08:56 |
| derekokeeffe85 | noonedeadpunk I can scp a 3GB file between nodes (my test for MTU issues) so I don't think it's MTU. Also, I can't run mysql on the utility container. bash: mysql: command not found | 10:03 |
| noonedeadpunk | mariadb | 10:22 |
| derekokeeffe85 | Nope | 10:23 |
| derekokeeffe85 | ERROR 2013 (HY000): Lost connection to server at 'handshake: reading initial communication packet', system error: 11 | 10:26 |
| noonedeadpunk | derekokeeffe85: have you checked the ping? | 10:51 |
| noonedeadpunk | with big packet size? | 10:51 |
| noonedeadpunk | ie `ping -s 1472 -M do`? | 10:53 |
| noonedeadpunk | basically - 1472 is expected MTU - 28 | 10:53 |
| derekokeeffe85 | Yep that ping works across all the nodes to each other noonedeadpunk | 11:01 |
| noonedeadpunk | and you have MTU 1500 set universally in containers? | 11:06 |
| noonedeadpunk | I need to work on publishing network testing script.... | 11:07 |
| derekokeeffe85 | MTU is 9000 across the board on all interfaces - ping -s 8472 -M do works from all the hosts to each other. Oh I didn't check the MTU on the containers | 11:07 |
| noonedeadpunk | As it does quite nice job, except taking eternity to finish | 11:07 |
| noonedeadpunk | so containers likely have 1500 out of the box | 11:07 |
| noonedeadpunk | you'd need to set a different MTU explicitly there | 11:07 |
| derekokeeffe85 | so on my mgmt subnet interfaces that are on the containers it's set at 9000. The second interface om the 10.0.3.0 subnet are 1500, I didn't check all of them but the main services have them at those settings | 11:11 |
| derekokeeffe85 | That would be a handy script to have :) | 11:12 |
| noonedeadpunk | let's me publish it on gist at least.... | 11:17 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-haproxy_server master: Add custom errorfiles https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/946617 | 11:18 |
| f0o | jrosser: this would be my very naive approach to adding custom errorfile entries into the global portion | 11:18 |
| noonedeadpunk | derekokeeffe85: https://gist.github.com/noonedeadpunk/b9eee2331a3c732e4def0b97530940ba | 11:19 |
| noonedeadpunk | you'd need to launch test_networks.yml | 11:19 |
| noonedeadpunk | you can set `-e openstack_tested_networks_throughput=false` to skip iperf | 11:21 |
| derekokeeffe85 | So use openstack-ansible test_networks.yml -e openstack_tested_networks_throughput=false` to skip iperf is it noonedeadpunk | 11:26 |
| noonedeadpunk | yeah | 11:33 |
| derekokeeffe85 | Running now | 11:34 |
| f0o | revisiting #939601 - is there any action required from me? I see it wants a rebase | 12:20 |
| f0o | (https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/939601) | 12:21 |
| derekokeeffe85 | noonedeadpunk I have to run again but when I get back (maybe tomorrow) I'll let you know. Think I may have found an issue with my setup that your script highlighted | 12:46 |
| mgariepy | not sure what can be wrong but i can't find the reason why i cannot schedule a vm in nova after an upgrade to Xena. | 17:58 |
| mgariepy | NoValidHost: No valid host was found. | 17:58 |
| *** jamesdenton_ is now known as jamesdenton | 18:52 | |
| kstev | @noonedeadpunk i am preparing a Yoga to Zed upgrade and notice the release notes mention "you must ensure that following variables are defined explicitly to ensure parity with existing functionality" including neutron_plugin_base: - metering ... i have the 'router' and 'neutron_dynamic_routing.services.bgp.bgp_plugin.BgpPlugin' plugins defined currently. i noticed define-neutron-plugin.yml appears to add the metering plugin. must meteri | 19:04 |
| kstev | ng be added? i'd rather not add it if we do not have to | 19:04 |
| kstev | https://docs.openstack.org/releasenotes/openstack-ansible/zed.html | 19:07 |
| noonedeadpunk | mgariepy: any logs from scheduler? | 19:41 |
| mgariepy | some issue with placement aggregate | 19:42 |
| mgariepy | compute host in aggrete have : `compute.cloud.domain.com` but hypervisor list does only have `compute` | 19:43 |
| mgariepy | new deployment not upgraded from ubuntu 14.04 since 2014.. | 19:46 |
| mgariepy | also the placement request do have : `member_of=in:d586eea7-878e-4131-b96f-d3af2a0c29b1,e48a8749-4a4e-47c2-a80d-0646363ab9ad` | 19:49 |
| mgariepy | which is 2 aggrate that i have. | 19:49 |
| mgariepy | but it's not from the flavor. | 19:50 |
| noonedeadpunk | well, aggregates in placement and in nova not necessarily synced | 19:52 |
| noonedeadpunk | especially in older releases | 19:52 |
| mgariepy | yeah. | 19:52 |
| noonedeadpunk | I do recall times when I had to manage them independently | 19:52 |
| jamesdenton | @kstev i'm not aware of metering being a requirement, but not sure why it would've been included other than it being defined in an AIO | 19:52 |
| jamesdenton | https://github.com/openstack/openstack-ansible/blob/a1f47e174343573efb17ab9e56082faade55dee4/tests/roles/bootstrap-host/templates/user_variables.aio.yml.j2#L148 | 19:53 |
| noonedeadpunk | jamesdenton: I think it used to be the default | 19:53 |
| jamesdenton | ahh maybe | 19:53 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-os_neutron/src/tag/yoga-eol/defaults/main.yml#L348-L350 | 19:53 |
| jamesdenton | oh, well there you go | 19:54 |
| noonedeadpunk | so in release note and in upgrade script I just wrote defaults if variable is not defined | 19:54 |
| jamesdenton | makes sense! | 19:54 |
| noonedeadpunk | if neutron_plugin_base is defined for group/host vars - upgrade script will do nothing | 19:54 |
| noonedeadpunk | (at least it should) | 19:54 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible/src/tag/zed-eom/scripts/upgrade-utilities/define-neutron-plugin.yml#L48 | 19:55 |
| jamesdenton | ♥ | 19:56 |
| jamesdenton | thank you noonedeadpunk | 19:56 |
| noonedeadpunk | thanks for stepping in with the reply :) long time no see :) | 19:57 |
| noonedeadpunk | was missing you around (▰˘◡˘▰) | 19:57 |
| jamesdenton | :) | 19:58 |
| jamesdenton | it has been a minute! i blame my bouncer | 19:58 |
| jamesdenton | and my job :P | 19:58 |
| mgariepy | is there a tool to sync nova and placement ? | 19:59 |
| mgariepy | or i need to rename the hosts... to get the automatic sync.. | 20:00 |
| mgariepy | i suspect it's because of the fqdn issue for some/most of the node. | 20:00 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!