| *** thuvh1 is now known as thuvh | 03:47 | |
| mnasiadka | mmalchuk: no problem | 05:13 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: Fix etc hosts generation for interfaces with dashes https://review.opendev.org/c/openstack/ansible-collection-kolla/+/923544 | 05:24 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Fix prechecks for interfaces with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/922478 | 05:24 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Fix prechecks for interfaces with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/922478 | 05:25 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Bump Ansible versions to 2.16 and 2.17 https://review.opendev.org/c/openstack/kolla-ansible/+/921743 | 05:27 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: Fix etc hosts generation for interfaces with dashes https://review.opendev.org/c/openstack/ansible-collection-kolla/+/923544 | 07:00 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: test interface names with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/794545 | 07:01 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Bump Ansible versions to 2.16 and 2.17 https://review.opendev.org/c/openstack/kolla-ansible/+/921743 | 07:32 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Fix prechecks for interfaces with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/922478 | 07:33 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: test interface names with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/794545 | 07:33 |
| PrzemekK | Guys any fiixes in kolla-ansible for https://bugs.launchpad.net/nova/+bug/2059809 ? | 07:52 |
| mnasiadka | No, that's not kolla-ansible bug | 08:00 |
| sylvr | Good morning #openstack-kolla | 08:28 |
| jovial | Hey sylvr | 08:28 |
| sylvr | jovial: did you get my message about the possible bug of cloud-utils-growpart missing ? | 08:31 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Support running without root privileges https://review.opendev.org/c/openstack/kayobe/+/923430 | 08:35 |
| jovial | slyvr: Sorry, I missed that one. Was it a case of missing that package? | 08:45 |
| sylvr | jovial: yes ! do you want me to create an issue/fill a but report on it ? I'm currently starting from scratch on an improved infrastructure and v2024.1, I can wait to encounter the bug again before submitting anything if you'd like | 08:57 |
| opendevreview | Sylvère Kanapa proposed openstack/kayobe master: Add missing "cloud-init" as "cloud-init-datasource" don't have dependency on it. https://review.opendev.org/c/openstack/kayobe/+/922795 | 08:59 |
| jovial | slyvr: Good call with the updated patch - thanks for that. Reminds me that we need to build and test the images in CI :) | 09:50 |
| jovial | slyvr: For the extra package, you could include it in your current patch if you liked or feel free to submit a new one | 09:53 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Add missing "cloud-init" as "cloud-init-datasource" don't have dependency on it. https://review.opendev.org/c/openstack/kayobe/+/922795 | 09:54 |
| sylvr | jovial: yeah I think I answered but never sent the answer (it stayed in draft), I think I'm starting to understand how to use git-review | 09:55 |
| sylvr | well, I think I'll do another patch, as I'm not sure it wasn't cause by some of my misconfiguration ^^ | 09:56 |
| sylvr | caused* | 09:56 |
| jovial | Sure - really appreciate the contributions - thanks :) | 09:58 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Bump up Ansible supported versions to 9.x/10.x https://review.opendev.org/c/openstack/kayobe/+/923556 | 10:22 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: arista-switch: Fix eos_config call https://review.opendev.org/c/openstack/kayobe/+/922631 | 11:44 |
| opendevreview | Mark Goddard proposed openstack/kolla-ansible master: Add precheck for Horizon config file renames in Caracal https://review.opendev.org/c/openstack/kolla-ansible/+/923558 | 11:44 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Fix prechecks for interfaces with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/922478 | 11:56 |
| mnasiadka | frickler: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/923544 and https://review.opendev.org/c/openstack/kolla-ansible/+/922478 - can you have a look? We ned those backported in for making the CI patch pass https://review.opendev.org/c/openstack/kolla-ansible/+/794545 | 11:58 |
| sylvr | quick question : when configuring a seed VM (with the hypervisor being the kayobe-control-host) do I need to manually create the libvirt network or is it part of the kayobe configuration ? | 12:41 |
| sylvr | I added mgmt_libvirt_network_name: mgmt in networks.yml as https://docs.openstack.org/kayobe/latest/configuration/reference/network.html | 12:42 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: test interface names with dashes https://review.opendev.org/c/openstack/kolla-ansible/+/794545 | 12:58 |
| jovial | sylvr: kayobe would normally create the libvirt networks for you. IIRC it is all determined by which networks you map to the seed; then you need to make sure the matching interfaces on the hypervisor use linux bridges | 13:16 |
| sylvr | jovial: so the bridges have to be created manually ? | 13:17 |
| jovial | sylvr: kayobe can do that for you. You need something like mgmt_bridge_ports and mgt_interface: brmgmt to be defined on the seed hypervisor | 13:18 |
| sylvr | so inventory/group_vars/seed-hypervisor/network-interfaces ? | 13:19 |
| jovial | mgmt_interface* | 13:19 |
| jovial | ^ prefix needs to match | 13:19 |
| jovial | yes, in inventory/group_vars/seed-hypervisor/network-interfaces | 13:20 |
| opendevreview | Michal Nasiadka proposed openstack/kayobe master: arista-switch: Fix eos_config call https://review.opendev.org/c/openstack/kayobe/+/922631 | 13:21 |
| sylvr | jovial: Thanks ! I'm going to look into that | 13:21 |
| jovial | slyvr: A useful command to debug what it will configure is: `kayobe configuration dump --limit seed-hypervisor --var-name seed_hypervisor_libvirt_networks` | 13:23 |
| sylvr | so "mgmt_bridge_ports: brmgmt" means create a bridge named 'brmgmt' on interface 'mgmt' right ? | 13:23 |
| jovial | so I think you want something like: `mgmt_bridge_ports: ["eth0"]` | 13:24 |
| jovial | but yeah, `mgmt_interface: brmgmt` defines the name of the linux bridge that it will create | 13:25 |
| sylvr | oooh okay | 13:25 |
| sylvr | I'm starting to get it | 13:25 |
| jovial | Took me a little while to grasp at first too, but the network configuration is quite powerful :) | 13:26 |
| sylvr | yeah, I see that ^^ | 13:30 |
| sylvr | I might start to contribute to the doc, I found it a bit hard to follow/understand sometimes, and I think some exemple would be great | 13:33 |
| jovial | That would be very much appreciated. Sometimes you need someone who has never used it before to point out the confusing bits. | 13:34 |
| sylvr | actually, I need to write a memoir so maybe if I write it good enough it might give an exemple of deployment? | 13:34 |
| jovial | sounds like a great idea - a kind of how to deploy a cloud from scratch kind of guide | 13:36 |
| sylvr | yes | 13:36 |
| sylvr | I'll send you an email with more info (where to see my memoir, kayobe conf etc etc) if you'd like | 13:40 |
| jovial | I'd certainly welcome something like that as an addition in: https://docs.openstack.org/kayobe/latest/resources.html or as a guide in the kayobe docs itself if you don't mind making it public. | 13:45 |
| sylvr | jovial: yes that's one of the exemple I check from time to times, but it differs a bit too much from my deployment (I remember another exemple of deployment, but I think it was removed from the doc with 2023.2 or 2024.1) | 13:49 |
| sylvr | and yes I don't mind making it public as it already the case : https://forge.univ-lyon1.fr/p1919734/kayobe-config (the branch I'm currently working on is "deployment", I didn't push the diagrams yet) | 13:51 |
| sylvr | okay, I'm still having issue with my config as it fail at "bounce networkmanager device" : https://pastebin.com/W2gGmRRT | 13:53 |
| jovial | And is the code pushed to some branch in that repo? | 13:55 |
| jovial | Are you trying to use eno3 as a bridge port in brmgmt? Kind of looks like the link is down. | 13:56 |
| sylvr | I'm going to push right now | 13:59 |
| sylvr | well, I'm not sure of what I'm doing through kayobe, but I think I know what I'm supposed to be doing in my config | 14:02 |
| sylvr | and yes eno3 on the seed-hypervisor is the interface of the out-of-band network | 14:03 |
| sylvr | jovial: the error is different now | 14:06 |
| jovial | First thing I'd check is what it has templated out to /etc/NetworkManager/system-connections. Do you see type bridge in brmgmt.nmconnection? | 14:07 |
| jovial | slyvr: What are you seeing now? | 14:07 |
| sylvr | jovial : https://pastebin.com/tbN4GxDx | 14:09 |
| sylvr | jovial : the error is similar, but I changed the content of networks.yml after the first error I send and before pushing | 14:10 |
| sylvr | "mgmt_libvirt_network_name: brmgmt" was "mgmt_libvirt_network_name: mgmt" I'm not sure it changed anything | 14:11 |
| jovial | sylvr: I wonder if that eno1-161b7efa-1183-4385-8955-349d81326ebb.nmconnection is conflicting with eno1.nmconnection | 14:12 |
| sylvr | jovial : same | 14:12 |
| jovial | Could you try moving that file out of that directory? | 14:13 |
| sylvr | yup, I rm'ed it | 14:13 |
| jovial | Also could you show the contents of brmgmt.nmconnection and eno1.nmconnection | 14:13 |
| sylvr | jovial : here you go ! https://pastebin.com/NyiB1BpC | 14:16 |
| jovial | mgmt_libvirt_network_name is going to be what appears in virsh net-list by the way | 14:17 |
| sylvr | yeah okay, I'll revert it to before then, wasn't sure it didn't cause any misconfiguration before ^^ thansk | 14:18 |
| jovial | OK, that all looks good. Could you try `kayobe seed hypervisor host configure -t network` again? | 14:18 |
| jovial | Also, do we know where eno1-161b7efa-1183-4385-8955-349d81326ebb.nmconnection came from? Was that configured by the installer maybe? | 14:20 |
| sylvr | I think it was me, I had an issue where it looked for 192.168.1.1 on the external network ^^ I remember fixing it by manually configuring it... | 14:21 |
| sylvr | jovial : still the same error https://pastebin.com/tUHJiAFB | 14:23 |
| jovial | and what do you get for `ip link show eno1`? | 14:25 |
| sylvr | 4: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000 | 14:27 |
| sylvr | link/ether b8:ca:3a:5b:f6:70 brd ff:ff:ff:ff:ff:ff | 14:27 |
| sylvr | altname enp1s0f0 | 14:27 |
| jovial | Weird - it doesn't have a master. I'm wondering if we need to remove eno1.nmconnection and eno3.nmconnection to get it to bounce those interfaces too | 14:31 |
| sylvr | well, I'm going to try | 14:31 |
| sylvr | seems it worked I'm at stackhpc.timezone | 14:34 |
| jovial | Nice :) | 14:35 |
| sylvr | yup, now I have this new error : https://pastebin.com/izewPPsm | 14:35 |
| sylvr | and I'm not sure how or why, as kayobe should have created the missing libvirt network? | 14:35 |
| jovial | Kind of looking like it was expecting a dict and we gave it a string some how | 14:36 |
| sylvr | oh yeah that's my type of mistake | 14:37 |
| jovial | I think the mistake is here: https://forge.univ-lyon1.fr/p1919734/kayobe-config/-/blob/deployment/etc/kayobe/seed-hypervisor.yml?ref_type=heads#L98 | 14:41 |
| jovial | Try just not defining that variable | 14:42 |
| jovial | If you do manually define it, then it needs be a list of dicts with the keys: name, mode, bridge | 14:42 |
| sylvr | jovial : thanks ! | 14:43 |
| sylvr | you saved me a looot of time | 14:44 |
| jovial | Also, could you create a bug report (https://bugs.launchpad.net/kayobe) for the interface bouncing? Seems like it could be made to be more robust. | 14:45 |
| sylvr | yes! | 14:45 |
| jovial | No worries, happy to help :) | 14:45 |
| sylvr | jovial : it's working now, I'm going to stop for the week-end and leave soon, what title should I put for the bug report ? I'm not even sure I understand why it failed ^^ | 14:50 |
| jovial | slyvr: Something like: "Kayobe fails to bounce slave interfaces when configuring a bridge" | 14:53 |
| jovial | and have a good weekend :) | 14:54 |
| sylvr | jovial: Done! I added you to the bug subscribe list. Again, thanks a lot and see you next week ! Good weekend to you too ! | 14:56 |
| jovial | marvelous, thanks again, and see you next week | 14:57 |
| sylvr | Hi, I'm back sooner I wanted to get going and I'm having issue provisioning the seed vm | 18:15 |
| sylvr | I think I misconfigured something | 18:15 |
| sylvr | it fail while trying to ssh the vm, I checked with virsh and the vm is running, looking at logs in /var/log/libvirt-consoles/seed-bifrost-console.log I see what looks like a iPXE boot loop | 18:17 |
| jovial | sylvr: Can you try and apply this patch: https://review.opendev.org/c/openstack/kayobe/+/921660 | 19:12 |
| jovial | sylvr, In your kayobe checkout: git fetch https://review.opendev.org/openstack/kayobe refs/changes/60/921660/23 && git cherry-pick FETCH_HEAD | 19:13 |
| jovial | then with you kayobe virtualenv activated: `pip install -e .` | 19:13 |
| jovial | You then need, `kayobe control host upgrade` and to set `seed_vm_boot_firmware: efi`. Then deprovision and reprovision the seed. | 19:14 |
| sylvr | jovial: I'm on it | 19:17 |
| sylvr | I guess "seed_vm_boot_firmware: efi" was supposed to be in my seed-vm.yml file ? | 19:18 |
| sylvr | jovial: yep, that fixed it (at least this time I'm booting rockylinux) | 19:20 |
| sylvr | jovial: thanks! :D | 19:23 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!