| jrosser | good morning | 07:48 |
|---|---|---|
| hamidlotfi_ | morning | 07:49 |
| ayush | hi team | 09:33 |
| ayush | i am facing one issue, which i am trying to diagnose | 09:33 |
| opendevreview | Merged openstack/openstack-ansible-os_adjutant master: Install pkg-config package https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/888985 | 09:47 |
| opendevreview | Merged openstack/openstack-ansible-os_adjutant master: Use proper galera port in configuration https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/890092 | 09:48 |
| opendevreview | Merged openstack/openstack-ansible-os_adjutant master: Fix linters and metadata https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/888469 | 09:48 |
| opendevreview | Merged openstack/openstack-ansible-os_adjutant master: Stop reffering _member_ role https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/891462 | 09:50 |
| opendevreview | Merged openstack/openstack-ansible-os_adjutant master: Replace deprecated UPPER_CONSTRAINTS_FILE variable https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/847721 | 09:50 |
| opendevreview | David Hitze proposed openstack/openstack-ansible-galera_server master: Added vars to override systemd for mariabackup https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894530 | 11:25 |
| farbod_ | hi guys | 12:35 |
| farbod_ | i am configuring my cluster to be able to access to public IPs on the main interface. i made a bridge to my main NIC and assigned the public IP to it and used this bridge as the network_interface in the br-ex network, but i lost my connection to the server from its own public IP address. i have access to only one NIC and there is some public IP addresses on the main NIC of each server. is there a way to use this as a | 12:38 |
| farbod_ | bridge and still be able to access to the server from its main IP? | 12:38 |
| farbod_ | here is my infra node /etc/network/interfaces file : https://paste.opendev.org/show/bc40BANAEEPFQCFeP9qj/ and here is the compute node network configuration: https://paste.opendev.org/show/bxcA6Bw1YhZCCKuwdOTe/ and here is the user config .yml file: https://paste.opendev.org/show/bqHLLAd1FkU3ainKdloO/ and here is the user variable .yml file: https://paste.opendev.org/show/bJsuF9eGLZiq6w721xmp/ | 12:41 |
| jrosser | farbod_: do you have a range of public IP allocated, or do you just have one for each server? | 13:04 |
| farbod_ | one public IP associated to each server for public access and some additional IPs, can be a subnet to bridge. i was able to set this additional IPs with a bridge to Proxmox LXC containers. Right now i have two servers, each one have a public IP for Public access and one of the has another public IP which is accessible by bridging. | 13:21 |
| jrosser | i'm not really following - you have a layer 2 CIDR of some size allocated to the interface by the provider? | 13:23 |
| jrosser | that might / might not be shared across your two servers? | 13:24 |
| farbod_ | yes the additional IPs or subnets are allocated by the provider and they are not shared. only accessible by the desired server. | 13:25 |
| jrosser | and one of those servers has 65.21.28.0/26, like 64 addresses? | 13:27 |
| jrosser | thats what corresponds to what i assume your network address is and the netmask from your paste 255.255.255.192 | 13:28 |
| farbod_ | it has access to only one IP on that subnet: | 13:29 |
| farbod_ | IP: 65.21.28.3 | 13:29 |
| farbod_ | Gateway: 65.21.28.1 | 13:29 |
| farbod_ | Netmask: 255.255.255.192 | 13:29 |
| farbod_ | Broadcast: 65.21.28.63 | 13:29 |
| jrosser | so that subnet is shared with other users? sorry this is pretty confusing | 13:32 |
| farbod_ | yes | 13:32 |
| opendevreview | David Hitze proposed openstack/openstack-ansible-galera_server master: Added vars to override systemd for mariabackup https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894530 | 13:33 |
| jrosser | farbod_: what do you want to do with these external IP? | 13:33 |
| jrosser | just lets really get back to basics | 13:33 |
| farbod_ | assign them to VMs | 13:34 |
| jrosser | so you really need a range of IP assigned to you i think | 13:34 |
| opendevreview | David Hitze proposed openstack/openstack-ansible-galera_server master: Added vars to override systemd for mariabackup https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/894530 | 13:34 |
| farbod_ | maybe want to have one VM for test :) | 13:34 |
| jrosser | from what you have said you have one IP in some subnet that also has other users in it | 13:34 |
| farbod_ | yes | 13:35 |
| jrosser | usually, you tell neutron "this is my external network, you can allocate router IP from this range and floating IP from this other range" <- hand waving explanation but mostly right | 13:35 |
| jrosser | and then neutron itself allocates an IP from the range you give it to the router | 13:36 |
| jrosser | but you keep talking about wanting to assign the IP to a VM :/ | 13:36 |
| jrosser | which would be one of 1) allowing the VM to attach directly to the external network or 2) using a floating IP and a neutron router | 13:37 |
| jrosser | if you want to use the external IP that you get from your server provider i think that you are making like pretty difficult for a first go with openstack | 13:38 |
| jrosser | we have a special configuration here https://docs.openstack.org/openstack-ansible/2023.1/user/aio/quickstart.html | 13:38 |
| jrosser | this is specifically designed to be deployed on a single server behind a single IP for test/evaulation purposes | 13:39 |
| farbod_ | my last attempt to made this IP assignable to a VM is in my configuration which i tried to connect br-ex to br-flat which its port is main NIC. and also i assigned the servers public IP to the bridge to be accessible. but i lost my connection to the server while setup. I don't understand the disconnection reason. | 13:39 |
| jrosser | with the catch being that it can't do "proper" external networking, because there is only assumed to be one external IP | 13:39 |
| jrosser | farbod_: but which IP? | 13:39 |
| jrosser | you need a fixed IP on an interface somewhere to SSH/deploy with, and also that haproxy makes it's endpoint | 13:40 |
| jrosser | this is not at all the same as the external network used by neutron | 13:40 |
| farbod_ | i understand | 13:40 |
| farbod_ | can we combine these two because of limitation on number of NICs? :) | 13:41 |
| jrosser | perhaps but i never did anthing like this | 13:42 |
| farbod_ | what about a virtual ethernet on the main NIC? | 13:42 |
| jrosser | what you are doing is very very similar to the all-in-one that i just linked to | 13:43 |
| jrosser | there are other people here who have built deployments like this in hosting providers, like admin1 | 13:44 |
| jrosser | and you'd have to get advice from them if it is possible to make a production type environment with real external IP for the workload] | 13:45 |
| jrosser | or if it's best to stick to some more artifical testing | 13:45 |
| farbod_ | One working approach is that i am able to make a vlan and have subnets on that VLAN. which i tried that out and worked. My VLAN with public subnet is .4040 on main NIC and there are public IPs on it that i can make a network in dashboard and assign that IPs to VMs. But there is a trade of. The IPs on a VLAN have additional cost per traffic. but the IPs associated to the main NIC of the server don't have additional cost. | 13:48 |
| farbod_ | on another hand i can order much more bigger subnets on main NIC than the .4040 VLAN that i mentioned. | 13:48 |
| jrosser | it sounds like you do not have sufficient separation between the bridge/interface that you give to neutron / OVS | 13:49 |
| jrosser | and the one that you put an IP on to deploy/manage the server | 13:49 |
| farbod_ | Another solution that my provider provides is that i can use routed IPs through my main IP on the server. These IPs gateway is the main IP on the server. any solution for this? | 13:50 |
| jrosser | the thing is that openstack-ansible is really pretty agnostic to all this | 13:51 |
| jrosser | so long as the host kernel has the appropriate L2/L3 config and you're trying to do something actually supported by neutron, it should work | 13:52 |
| jrosser | the example networking configs given in the openstack-ansible documentation are really just starting points and you can do whatever you like | 13:52 |
| farbod_ | i read all of them many times for better understanding but i am stuck now :) | 13:53 |
| jrosser | if you look at some of the other deployment tools (maybe commercially supported ones) you might find they have very immovable requirements for hosts / interfaces etc | 13:53 |
| jrosser | and if you don't have that then it just doesnt work | 13:53 |
| jrosser | that is not where you are with openstack-ansible, either follow the reference architecture which we document and test in CI | 13:54 |
| jrosser | or with enough knowedge of neutron you can configure pretty much anything beyond that as needed | 13:54 |
| farbod_ | I understand | 13:54 |
| farbod_ | i need to learn more | 13:55 |
| jrosser | did you build an all-in-one first? | 13:55 |
| jrosser | following the quickstart guide? | 13:55 |
| farbod_ | couple of months ago, yes. but didn't try public access like one i want now. | 13:56 |
| jrosser | then perhaps review the AIO config | 13:56 |
| jrosser | see how eth12 is connected to the br-vlan bridge | 13:56 |
| jrosser | and becomes a flat network type for neutron | 13:57 |
| jrosser | eth12 is the interface that neutron uses for it's exernal network in that case | 13:57 |
| farbod_ | eth12 is a physical interface? | 13:57 |
| jrosser | no, it's just an arbitrary IP interface with that name | 13:57 |
| farbod_ | you mean this: | 13:58 |
| farbod_ | auto br-vlan | 13:58 |
| farbod_ | iface br-vlan inet static | 13:58 |
| farbod_ | bridge_stp off | 13:58 |
| farbod_ | bridge_waitport 0 | 13:58 |
| farbod_ | bridge_fd 0 | 13:58 |
| farbod_ | address 172.29.248.100 | 13:58 |
| farbod_ | netmask 255.255.252.0 | 13:58 |
| farbod_ | offload-sg off | 13:58 |
| farbod_ | # Create veth pair, don't bomb if already exists | 13:58 |
| farbod_ | pre-up ip link add br-vlan-veth type veth peer name eth12 || true | 13:58 |
| farbod_ | # Set both ends UP | 13:58 |
| farbod_ | pre-up ip link set br-vlan-veth up | 13:58 |
| farbod_ | pre-up ip link set eth12 up | 13:58 |
| farbod_ | # Delete veth pair on DOWN | 13:58 |
| farbod_ | post-down ip link del br-vlan-veth || true | 13:58 |
| farbod_ | bridge_ports br-vlan-veth | 13:58 |
| jrosser | paste.opendev.org :( | 13:58 |
| mgariepy | !pastebin ! | 13:58 |
| opendevmeet | mgariepy: Error: "pastebin" is not a valid command. | 13:58 |
| farbod_ | sorry | 13:58 |
| jrosser | if you follow the pattern in the AIO you have an interface name that you understand is the one you give to neutron | 13:59 |
| jrosser | it's up to you to make sure it is connected to something useful | 13:59 |
| jrosser | in the AIO case a veth is used to connect it to br-vlan | 13:59 |
| farbod_ | sorry but what is ? | 14:00 |
| farbod_ | br-vlan-veth | 14:00 |
| jrosser | like an ethernet cable connected between br-vlan and eth12 | 14:00 |
| farbod_ | it will use br-vlan as the network_interface? | 14:01 |
| jrosser | it? | 14:01 |
| farbod_ | i mean i have to pass br-vlan or br-vlan-veth or eth12 to network_interface? | 14:02 |
| jrosser | look at the AIO config :) | 14:02 |
| jrosser | it is there as a reference for getting started | 14:02 |
| farbod_ | host_bind_override uses eth12 | 14:02 |
| farbod_ | and another question | 14:02 |
| jrosser | just to be clear this whole eth12 business is a hack for making things work on a server with not enough interfaces | 14:03 |
| farbod_ | i really didn't understand the difference between host_bind_override and network_interface. but the second one worked for OVS setup | 14:03 |
| jamesdenton | host_bind_override is irrelevant for OVS-based setups | 14:04 |
| farbod_ | OK | 14:05 |
| jamesdenton | network_interface is the way to go for OVS-based setups when you want the playbooks to manage the physical interface connected to br-ex or whatever the provider bridge is | 14:05 |
| jrosser | it is described here https://github.com/openstack/openstack-ansible/blob/master/doc/source/reference/architecture/metal-networking.rst#L109 | 14:06 |
| jamesdenton | if left out, you have to perform the 'ovs-vsctl add-port' command yourself | 14:06 |
| jrosser | so what happens in the AIO is we make a pseudo-physical interface eth12 becasue there is not a real one to use | 14:07 |
| farbod_ | As i understand eth12 will be a connection between br-vlan and neutron for public access? | 14:08 |
| jrosser | then arrange for it to be hooked up to OVS here https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.aio.j2#L145 | 14:08 |
| farbod_ | br-ex <-> eth12 <-> br-vlan? | 14:08 |
| farbod_ | which br-vlan is my way to public? | 14:09 |
| jrosser | well again in the AIO we connect it to br-vlan really out of convenience, you connect it to where is needed on your actual server | 14:09 |
| jrosser | in the AIO we define the flat network as being untagged traffic on br-vlan | 14:09 |
| jrosser | you might want to make that be untagged traffic on your actual physical interface | 14:09 |
| farbod_ | 👍 | 14:10 |
| jrosser | and it sounds like you need a bridge there of some kind anyways | 14:10 |
| jrosser | becasue you want somewhere to put an IP | 14:10 |
| jrosser | for management / horizon / whatever | 14:10 |
| farbod_ | a basic question :) can a bridge have access to public without a port to a physical NIC? | 14:11 |
| jrosser | i am going to say that depends if you put an ip on it or not, (ip forwarding etc) but jamesdenton will correct me here if i'm wrong | 14:12 |
| farbod_ | Let me test the AIO veth solution. | 14:12 |
| jrosser | if you want L2 inbound to work to something connected to a bridge then i think you do need a port to the physical nic | 14:13 |
| jrosser | as L2 things like arp have to work | 14:14 |
| jamesdenton | i think the answer is "probably" but the gymnastics involved is not what i would recommend to someone new to this | 14:16 |
| jamesdenton | there are linux networking and openstack networtking fundamentals that cannot be ignored | 14:16 |
| jrosser | unfortunately getting from zero to openstack involves a ton of both of those | 14:20 |
| farbod_ | I understand | 14:21 |
| farbod_ | take a look at this: https://paste.opendev.org/show/bi6KDV7AvI7yMtrjIx8R/ | 14:26 |
| jamesdenton | and the plan is to use eth12 for the ovs bridge? or? | 14:29 |
| farbod_ | yes | 14:30 |
| farbod_ | pass it ti networ_interface | 14:30 |
| jrosser | jamesdenton: ^ this is a reasonable approach for giving a specific interface to neutron when there actually is only one physical one? | 14:31 |
| farbod_ | With above configuration server is not accessible from the Public IP | 14:32 |
| jrosser | should enp8s0 be connected to the br-vlan bridge? | 14:34 |
| farbod_ | i think so | 14:36 |
| jrosser | from #ansible `sdoran> "Legacy roles" are roles outside of collections. They still work fine, but new role features won't be added to standalone roles.` | 14:55 |
| farbod_ | ? | 14:55 |
| jrosser | thats just interesting for the ansible people here | 14:56 |
| jrosser | farbod_: did you get your server accessible? | 14:56 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Add libpython mapping for debian bookworm https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894554 | 16:14 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Ensure systemd-resolved is present in debian container images https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894555 | 16:14 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Sync additional apt config from the host to the container base image https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894556 | 16:14 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Add ca-certificates into debian base image during debootstrap. https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894557 | 16:14 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Switch to native systemd-resolved from resolv.conf https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894558 | 16:14 |
| farbod_ | jrosser: yes i did | 16:19 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Allow deployment on debian bookworm hosts https://review.opendev.org/c/openstack/openstack-ansible/+/894560 | 16:25 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add CI jobs for debian bookworm https://review.opendev.org/c/openstack/openstack-ansible/+/894561 | 16:25 |
| farbod_ | https://paste.opendev.org/show/bJ9wvn4SbxwNp88QFABJ/ In this configuration for AIO why should we assign another ip to br-vlan in second part? i don't understand it. | 16:56 |
| farbod_ | I thought this configuration should work: https://paste.opendev.org/show/b1NDwUlMzA1kFZ4IXsrC/ but server is not accessible on its public IP. | 16:58 |
| jamesdenton | to define the gateway address for the flat neutron network/subnet that you'd setup | 16:58 |
| farbod_ | Could you please check my configuration? | 17:00 |
| jamesdenton | what does a working configuration look like? straight from your provider without any modifications? | 17:00 |
| farbod_ | No, I manipulate it. | 17:01 |
| jamesdenton | right, but what did the working configuration look like? | 17:02 |
| farbod_ | Assigning IP directly to main NIC with no bridges. On enp8s | 17:02 |
| jamesdenton | enp8s0 with no vlan tag? | 17:03 |
| farbod_ | yes | 17:03 |
| jamesdenton | hwaddress 58:11:22:c4:54:2c is the MAC of enp8s0? | 17:04 |
| farbod_ | yes | 17:04 |
| jamesdenton | does 'ip link show' reflect that? | 17:05 |
| farbod_ | My provider doesn't let me set a seprate mac address for this IP | 17:05 |
| farbod_ | yes ip link show reflects that. | 17:05 |
| jamesdenton | and br-vlan is UP? | 17:06 |
| jamesdenton | you could try restarting the instance with this config in place, if you have console access | 17:06 |
| farbod_ | no | 17:06 |
| farbod_ | i restarted it | 17:06 |
| jamesdenton | ip link set br-vlan up | 17:06 |
| farbod_ | I don't have console access. I have rescue mode access. | 17:08 |
| supamatt | jamesdenton: you guys ever seen this issue? https://bugs.launchpad.net/bugs/2033193 | 17:09 |
| supamatt | maybe not, bc I suspect you have moved over to Q35 machine type VMs | 17:09 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Ensure systemd-resolved is present in debian container images https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894555 | 18:30 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Sync additional apt config from the host to the container base image https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894556 | 18:30 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Add ca-certificates into debian base image during debootstrap. https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894557 | 18:30 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Switch to native systemd-resolved from resolv.conf https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/894558 | 18:30 |
| jamesdenton | supamatt can't say i have | 18:35 |
| farbod_ | Can i set specific mac address for an instance? | 19:38 |
| jamesdenton | openstack port list --device-id <instance uuid> | 19:44 |
| farbod_ | although i added my pub key to instance it refuses it. why? | 19:58 |
| jrosser | are you connecting as the right user? | 20:01 |
| farbod_ | yes | 20:01 |
| farbod_ | debian | 20:01 |
| farbod_ | on debian image | 20:01 |
| jrosser | you can look in the instance log to check that cloud-init was able to find it's datasource | 20:02 |
| farbod_ | i didn't specify a cloud init | 20:07 |
| farbod_ | how to create a user and set password for it in cloud init? | 20:09 |
| jrosser | it is automatic through the openstack metadata service | 20:09 |
| jrosser | what do you see in the instance log | 20:09 |
| farbod_ | https://paste.opendev.org/show/bphMmT4qa7OiT4PAEvX1/ | 20:10 |
| jrosser | here is the trouble | 20:12 |
| jrosser | [ 30.546171] cloud-init[452]: 2023-09-11 20:06:58,342 - url_helper.py[ERROR]: Timed out, no response from urls: ['http://169.254.169.254/openstack'] | 20:13 |
| jrosser | [ 30.549671] cloud-init[452]: 2023-09-11 20:06:58,342 - util.py[WARNING]: No active metadata service found | 20:13 |
| jrosser | are you connecting the instance directly to the external network? | 20:13 |
| farbod_ | yes | 20:13 |
| jrosser | ok then i dont think you will have the metadata service available there (though my OVN understanding is sketchy) | 20:14 |
| jrosser | i would try using config-drive instead to pass the metadata to the instance | 20:14 |
| jamesdenton | ehhhh there IS a metadata service, and it ought to work even for that scenario | 20:14 |
| jamesdenton | but config-drive is prob the easiest at this point | 20:14 |
| jrosser | right - in my stuff i don;t think it would work without a neutron router | 20:15 |
| jrosser | (linuxbridge) | 20:15 |
| jrosser | but i expect OVN handles it differently | 20:15 |
| jamesdenton | there's a way to get it to push a route via dhcp namespace | 20:15 |
| jamesdenton | can't recall the option offhand | 20:16 |
| jamesdenton | oh yeah, irrelevant with OVN lol | 20:16 |
| farbod_ | i just added it to a private network and it worked ! | 20:17 |
| farbod_ | And also with the help of you guys i am now able to assign public IPs with all its limitations! | 20:18 |
| farbod_ | Thanks a lot jamesdenton & jrosser. | 20:18 |
| jamesdenton | good luck! | 20:19 |
| jrosser | awesome, glad its working | 20:19 |
| farbod_ | yes | 20:19 |
| jrosser | is this a test for an actual deployment, or what you'll end up with? | 20:19 |
| farbod_ | jamesdenton: in the last configuration i provided to you it was a syntax error lol | 20:19 |
| farbod_ | i am just learning | 20:20 |
| farbod_ | the veth pair worked pefectlly | 20:20 |
| jamesdenton | good deal | 20:20 |
| farbod_ | every thing but dns doesn't work. even when i changed it in resolve.conf | 20:28 |
| farbod_ | Configured it in Subnet :) | 20:34 |
| farbod_ | But guys, are my questions annoying? :) I think i talked a lot these days and made you uncomfortable. | 20:49 |
| jamesdenton | not at all | 20:53 |
| jamesdenton | i imagine most of us are busy with our own things, too. Happy to help where we can | 20:53 |
| farbod_ | jamesdenton: I am spending time with your books these days, They are amazing, Thanks a lot | 20:54 |
| jamesdenton | cool! glad you found them | 20:55 |
| * farbod_ 🙏 | 20:55 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!