Monday, 2024-06-24

opendevreviewMichal Nasiadka proposed openstack/kayobe master: Stop replacing dashes with underscores for Kolla interfaces  https://review.opendev.org/c/openstack/kayobe/+/92248007:00
opendevreviewMichal Nasiadka proposed openstack/kayobe master: Stop replacing dashes with underscores for Kolla interfaces  https://review.opendev.org/c/openstack/kayobe/+/92248007:16
opendevreviewMichal Nasiadka proposed openstack/kayobe master: Stop replacing dashes with underscores for Kolla interfaces  https://review.opendev.org/c/openstack/kayobe/+/92248007:17
mnasiadkagood morning07:29
mnasiadkafrickler: https://review.opendev.org/c/openstack/kolla-ansible/+/922478 - if you could have a look - I stumbled upon that in my $downstream deployment :)07:29
SvenKieskemorning07:34
SvenKieskemhm, doesn't ansible replace dashes anymore with underscores or is it just different in the facts module? https://bugs.launchpad.net/kolla-ansible/+bug/192735707:36
SvenKieskeah, a bug is linked, should read that first I guess07:36
SvenKieskemhm, this sounds like a good workaround, but also like a bug in upstream ansible, no?07:37
mnasiadkano07:38
mnasiadkathey claim it's how python works, and they don't give a shit07:38
mnasiadka:)07:38
SvenKieskelol, have a link for my entertainment? :)07:38
mnasiadkacan't find it right now07:41
mnasiadkabut it's been like this for years07:41
SvenKieskeI know about the general issue with dashes, but it seems new to me that this differs across ansible. I thought it was the same everywhere.07:48
mnasiadkaI just filled out the PTG form, it's going to take place on October 21-25, 202408:08
opendevreviewMark Goddard proposed openstack/ansible-collection-kolla master: Add stats callback plugin  https://review.opendev.org/c/openstack/ansible-collection-kolla/+/91034708:24
SvenKieskeI think frickler spoilered the date already some weeks ago (the meeting before the last meeting afaik) :D08:25
opendevreviewMerged openstack/kolla-ansible stable/2023.1: Skyline: Fix incorrect keystone port  https://review.opendev.org/c/openstack/kolla-ansible/+/92230508:28
opendevreviewMerged openstack/kolla-ansible stable/2024.1: Skyline: Fix incorrect keystone port  https://review.opendev.org/c/openstack/kolla-ansible/+/92230308:35
opendevreviewMerged openstack/kolla-ansible stable/2023.2: Skyline: Fix incorrect keystone port  https://review.opendev.org/c/openstack/kolla-ansible/+/92230408:52
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: Fix prechecks for interfaces with dashes  https://review.opendev.org/c/openstack/kolla-ansible/+/92247809:01
opendevreviewMichal Nasiadka proposed openstack/kayobe master: Stop replacing dashes with underscores for Kolla interfaces  https://review.opendev.org/c/openstack/kayobe/+/92248009:05
opendevreviewGaël THEROND proposed openstack/kolla-ansible master: Add new hardware prometheus exporter:  https://review.opendev.org/c/openstack/kolla-ansible/+/92221109:22
opendevreviewGaël THEROND proposed openstack/kolla-ansible master: Add new hardware prometheus exporter:  https://review.opendev.org/c/openstack/kolla-ansible/+/92221109:23
opendevreviewDoug Szumski proposed openstack/kolla-ansible master: CI: test interface names with dashes  https://review.opendev.org/c/openstack/kolla-ansible/+/79454509:45
opendevreviewBartosz Bezak proposed openstack/kolla master: add kayobe to release management doc's Launchpad admin section  https://review.opendev.org/c/openstack/kolla/+/92259309:49
opendevreviewMatt Crees proposed openstack/kolla stable/2023.2: Move apt_prefs pins to respective images  https://review.opendev.org/c/openstack/kolla/+/92260510:22
opendevreviewMatt Crees proposed openstack/kolla stable/2023.2: Move apt_prefs pins to respective images  https://review.opendev.org/c/openstack/kolla/+/92260510:23
srdemircanHi everyone,10:25
srdemircanI’m trying to deploy baremetal nodes with ironic. I have configured ironic by following kolla-ansible's ironic guide. Somehow I could not deploy a node with default configurations. 10:25
srdemircanI can successfully register and inspect(with both redfish and inspector interfaces) my baremetal node. But when I try to deploy, ironic_python_agent try to send collected node information to inspector service(to /v1/continue endpoint) and it fails because inspector’s node_cache is empty. 10:25
srdemircanI can not find what I am missing. Do you have any idea?10:25
opendevreviewMatt Crees proposed openstack/kolla-ansible stable/2023.2: RMQ: enable all stable feature flags at once  https://review.opendev.org/c/openstack/kolla-ansible/+/92260610:26
opendevreviewMatt Crees proposed openstack/kolla-ansible stable/2023.2: RMQ: enable all stable feature flags at once  https://review.opendev.org/c/openstack/kolla-ansible/+/92260610:27
opendevreviewMatt Crees proposed openstack/kolla-ansible stable/2024.1: RMQ: enable all stable feature flags at once  https://review.opendev.org/c/openstack/kolla-ansible/+/92260710:27
jheikkinHi, I am upgrading all-in-one kolla-ansible installation in a VM. I am getting strange dependency failure on TASK [prechecks : Checking dbus-python package] I have updated Ansible version and run kolla-ansible install-deps. So I do not understand why dbus-python is missing.10:57
fricklerjheikkin: can you show the full error on paste.opendev.org? what versions of kolla-ansible are you upgrading from and to? on which OS+version?10:59
opendevreviewVictor Chembaev proposed openstack/kolla-ansible master: Fix kolla systemd unit template to prevent restart all kolla services with docker.service restart  https://review.opendev.org/c/openstack/kolla-ansible/+/91863911:03
jheikkinHere is a paste https://paste.opendev.org/show/bkmr34sJNDUCpBUKvRou/ I do not recall which version I have, is there way to check it? the base OS is debian-bookworm and I run on podman containers 11:04
jheikkinI am upgrading to latest release using the instructions on kolla-ansible official documentation11:05
kevkojheikkin: it should fail if your don't have dbus on remote managing node ...11:13
kevkojheikkin: i would say that python3-dbus and dbus is installed in OS by default ...can u verify with dpkg -l | grep -i dbus  .... ? 11:14
jheikkinYes, dbus is installed in os11:18
jheikkinbut this is a failing python module. Anyway I can also manually install the module in virtualenv and see if this allows me to move on11:19
PrzemekKjheikkin: did You run precheck ? openstack versions show You cant skip more than 2 version in SLURP11:20
SvenKieskejheikkin: did you verify python3-dbus, or just dbus? these are not the same thing!11:22
SvenKieskehttps://packages.debian.org/bookworm/dbus vs https://packages.debian.org/bookworm/python3-dbus11:22
jheikkinI do not have python3-dbus installed11:22
jheikkinit was not in the requirements on the all-in-one installation either11:22
SvenKieskeyou need python3-dbus, apparently, I wonder though why it's not in our requirements.txt if we need it?11:22
SvenKieskeok, it's "only" needed for baremetal deployment, it seems.11:23
SvenKieskehttps://github.com/openstack/kolla-ansible/blob/master/ansible/roles/prechecks/tasks/package_checks.yml#L1211:24
jheikkinSo I edit the inventory and remove everything from baremetal group?11:24
SvenKieskeno, if you want to deploy baremetal you need that group I guess. you just need to install python3-dbus on your hosts.11:24
SvenKieskeI'm just wondering if we can't fix this on the kolla side, this seems rather unfortunate.11:25
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: [CI] Force config drive for ipv6 jobs  https://review.opendev.org/c/openstack/kolla-ansible/+/91418511:26
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: [CI] Trust kolla generated CA cert  https://review.opendev.org/c/openstack/kolla-ansible/+/92255911:26
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: [CI] Fix swift disk preparation  https://review.opendev.org/c/openstack/kolla-ansible/+/91394211:26
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Bash replacement  https://review.opendev.org/c/openstack/kolla-ansible/+/92036811:26
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add testing project - rally  https://review.opendev.org/c/openstack/kolla-ansible/+/92246211:26
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: [CI] Rally tests and tasks  https://review.opendev.org/c/openstack/kolla-ansible/+/91372811:26
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: [DNM] Try to turn on swift tempest smoke tests  https://review.opendev.org/c/openstack/kolla-ansible/+/91441111:26
jheikkinI am just deploying a PoC in a VM. But when I do this following the instructions on documentation, it should mention that baremetal group needs to be left empty. Now it is not mentioned11:26
kevkojheikkin: python3-dbus should be requirements of dbus 11:26
SvenKieskekevko: it's not, at least not in bookworm: https://packages.debian.org/bookworm/dbus11:27
SvenKieskeI also don't see why it should, it's just python3 language bindings for dbus, to make interaction with dbus easier from python3, not needed for dbus itself at all?11:28
PrzemekKkolla-ansible-stable-2023.2\ansible\roles\prechecks\tasks\package_checks.yml (4 hits) Line 12: - name: Checking dbus-python package Line 13:   command: "{{ ansible_facts.python.executable }} -c \"import dbus\"" Line 14:   register: dbus_present Line 17:   failed_when: dbus_present is failed11:29
SvenKieskePrzemekK: yeah I already linked to that line above, the github link :)11:29
kevkoSvenKieske: well, but this is bug of kolla-ansible or ansible-collections-kolla which should install this package ... as it's installing python3-docker11:31
kevkofor example 11:31
jheikkinhmm11:31
jheikkinI think this could be I need to pass right python interpreter11:31
jheikkinIn the error it says "cmd": ["/usr/bin/python3", "-c", "import dbus"11:32
SvenKieskekevko: but this is a package only needed for certain roles, so I can see why it was not added, then again I would also say it would be nice to include it11:32
jheikkinso this package is actually present in my kolla-ansible venv11:32
PrzemekKthere is bug for that https://bugs.launchpad.net/kolla-ansible/+bug/202158511:32
SvenKieskeah11:32
kevkoSvenKieske: joking ? :D 11:33
kevkoSvenKieske: ./ansible/module_utils/kolla_systemd_worker.py:import dbus <<<< 11:33
kevkoSvenKieske: it's kolla_systemd_worker  ...so it's the main brick of kolla-ansible ..crucial dependency needed :D 11:33
SvenKieskekevko: I was just going by the precheck, which only runs if you have hosts in "baremetal" group11:33
SvenKieskeok, then we need to fix that, agreed11:33
SvenKieskedon't we even have simple import test that catches this? something feels off about this..11:34
SvenKieskeit's at least not in requirements.txt, is it maybe in global requirements? or just in our CI runners already installed?11:35
PrzemekKBug #2021585 reported by Stefan on 2023-05-30 :D11:36
kevkoSvenKieske: w8 a minute11:42
kevkoSvenKieske: BUT - it's actually egg or chicken  question :) 11:43
fricklerjheikkin: just being curious: how did you install your bookworm vm, from the official debian cloud image or some other way?11:43
kevkoSvenKieske: does precheck report that i am missing python3-dbus or should bootstrap via ansible-collections-kolla install it 11:44
kevkoI think it should be installed in bootstrap - so add to ansible-collections-kolla to docker role11:46
jheikkinfrickler: I think I used an ISO image from debian web site. I did not use a cloud image for the VM.11:50
opendevreviewMichal Arbet proposed openstack/ansible-collection-kolla master: Add installation of dbus package  https://review.opendev.org/c/openstack/ansible-collection-kolla/+/92261411:52
kevkojheikkin: go to your home and apply patch for ansible-collection-kolla ^^  ...run bootstrap ...run prechecks again 11:58
SvenKieskekevko: but why to docker role? now you need to install it in both podman and docker role. well better then nothing I guess11:58
kevkoSvenKieske: i agree that it's maybe not ideal placement for installation as it has connection with systemd not docker/podman .... yeah you are right ... but i know/rememeber that there is mechanism for package installation  ...i mean that dance with venv or not venv ...bookworm or others ... etc ... and on the end ..it's dependency for kolla12:03
kevkosystemd worker ..which has connection to kolla docker/podman worker .. so i feel that it is not ideal ...but actually not bad placement on the end .....12:03
tis_at_teutoHi there, i have a question regarding a change-request. I made a git review https://review.opendev.org/c/openstack/kolla-ansible/+/922391 and the change was verified last week. But now i was wondering what comes next. Is there anything i should do to get progress  in there? Thanks and regards.12:03
jheikkinI will likely reinstall the VM from scratch. I wanted to test the upgrade process but I think I was too many versions behind. I originally created the VM and did all-in-one in March this year. But I did not have this issue then - otherwise I'd have the dbus-python installed I guess.12:03
kevkotis_at_teuto: please, user uppercase for commit message ...and then i am ok with that review12:06
kevkojheikkin: yes, but we added systemd wrapper ... so it's new dependency 12:07
kevkojheikkin: or ...two years :D 12:07
jheikkinkevko: I installed the vm and pulled latest kolla code somewhere in March 2024. Anyhow I will do fresh install to another VM and see if I hit the same issue. 12:09
opendevreviewTino Schmeier proposed openstack/kolla-ansible master: Use template instead of copy for prometheus-alert-rules  https://review.opendev.org/c/openstack/kolla-ansible/+/92239112:12
tis_at_teutokevko: Thanks, i've updated the commit-message.12:16
kevkojheikkin: you will hit the same issue :) 12:19
kevkotis_at_teuto: +2 from me as it's trivial 12:21
kevkohmm, we should clean kolla reviews :) 12:23
kevkoi mean, abandon what should be abandoned ...review what should be reviewed ..etc 12:24
SvenKieskekevko: I made a scan with a tool called deptry (https://github.com/fpgmaas/deptry); I also made a shot at integrating it with tox tests, but there are some challenges, regardless, found some more missing imports: https://paste.opendev.org/show/byH6Il1bC1dfT61Bh4D1/12:45
opendevreviewSven Kieske proposed openstack/kolla-ansible master: refactor requirements.txt  https://review.opendev.org/c/openstack/kolla-ansible/+/92261712:52
opendevreviewSven Kieske proposed openstack/kolla-ansible master: add missing license for hvac library  https://review.opendev.org/c/openstack/kolla-ansible/+/92261812:55
kevkoSvenKieske: libvirt dependency is only for vagrant ... (honestly i really don't know if anyone is using vagrant :D )12:56
SvenKieskesure, but distutils is there12:57
opendevreviewSven Kieske proposed openstack/kolla-ansible master: add missing distutils requirements  https://review.opendev.org/c/openstack/kolla-ansible/+/92261912:59
kevkoSvenKieske: yes, but distutils is installed in every linux distro as part of standard preinstalled system-wide python installation 13:00
kevkoSvenKieske: so you can be sure that distutils is installed and you want to have distro packaged version ..not the pip one 13:01
SvenKieskemhm, are you sure?13:03
kevkoSvenKieske: 100 % 13:10
kevkoSvenKieske: but, distutils is deprecated and will be removed 13:10
sylvrHello! I'm trying to provision my overcloud, but I got an error saying the user defined deployment image is missing, I didn't remembering putting that in my config, do I need to create my own qcow2 files to provision my overcloud or kayobe can create those files ?13:16
sylvrMy understanding is that `kayobe overcloud deployment image build` should do the trick ?13:19
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Remove usage of distutils  https://review.opendev.org/c/openstack/kolla-ansible/+/92262113:19
kevkoSvenKieske: ^^ 13:19
sylvrsylvr: uuh it was `kayobe overcloud host image build`13:23
SvenKieskekevko: just wanted to ask how we handle the deprecation of distutils :D13:35
kevkoSvenKieske: ^^ :) ...there is also need to rewrite easy_install stuff in pbr ...but it's not kolla 13:38
kevkonot our responsibility 13:40
sylvrHello ! How can I stop the following error from occuring while running `kayobe overcloud provision` : Deploy step deploy.prepare_instance_boot failed: Failed to install a bootloader when deploying node ffcc09d8-ede5-4c2a-8e2b-c0056d22e70f: No EFI14:09
sylvr partition could be detected on device /dev/sda and EFI partition UUID has not been recorded during deployment (which is often the case for whole disk images). Are you usi14:09
sylvrng a UEFI-compatible image? (logs taken from the node using Ironic)14:09
jovialsylvr - If building with DIB, you need an element like block-device-efi to have an EFI compatible image14:10
jovialits either that or boot in legacy mode ;-)14:10
sylvrjovial: in bifrost.yml the parameter "kolla_bifrost_dib_elements_default" ?14:12
jovialslyvr: I think you want it in here: https://github.com/openstack/kayobe-config/blob/master/etc/kayobe/overcloud-dib.yml#L3714:17
jovialSee: https://docs.openstack.org/kayobe/latest/configuration/reference/overcloud-dib.html14:18
sylvroh yes that makes much more sense ! thanks14:18
jovialWorth knowing that there is also an LVM variant: https://docs.openstack.org/diskimage-builder/latest/elements/block-device-efi-lvm/README.html14:22
opendevreviewVictor Chembaev proposed openstack/kolla-ansible master: Fix octavia-interface timeout  https://review.opendev.org/c/openstack/kolla-ansible/+/92037714:23
opendevreviewVictor Chembaev proposed openstack/kolla-ansible master: Fix kolla-ansible systemd restart behaviour  https://review.opendev.org/c/openstack/kolla-ansible/+/91863914:25
kevkoSvenKieske: my first custom defined rally task ...just create volume ... check it ... save as html >> https://paste.openstack.org/show/bdkDEdDQEqch9Y829udQ/14:30
kevkoSvenKieske: this will simple replace all openstack calls and tests in bashes....very elegant and simple :) 14:30
SvenKieskekevko: looks nice :)14:33
kevkoSvenKieske: cool right ? 14:40
kevkoSvenKieske: cool is that you can for example start vm ..then restart libvirt ..restart mariadb ..restart rabbit ...whatever ..and then run another VM ..as a test ... described in simple yaml ...and you will get result :) 14:40
SvenKieskeand you can use all those preexisting tempest scenario tests, no? :)14:41
kevkoSvenKieske: you can use rally as wrapper to tempest ..you can exec smoke tests only ..or regexp .. so there is a question if we want to run all thousands of tests which tempest has ... Or use rally to define our own yamls to test our specific scenarios 14:46
kevkoThis report is report for my own task I defined to yaml 14:47
kevkoOr combination 14:48
SvenKieskeI don't think it's reasonable to run all those thousands of tempest tests, but there will be a good subset that extends our test coverage for important things.14:49
kevkoYep14:50
opendevreviewMichal Nasiadka proposed openstack/kayobe master: arista-switch: Fix eos_config call  https://review.opendev.org/c/openstack/kayobe/+/92263115:00
opendevreviewMichal Nasiadka proposed openstack/kayobe master: arista-switch: Fix eos_config call  https://review.opendev.org/c/openstack/kayobe/+/92263115:02
sylvrjovial: do you know if having the management network and the introspection network on the same network might cause an issue ? my nodes are active, but it doesn't seems like they're available through ssh..15:08
jovialsylvr: Should be OK. Could be that cloud init failed to run for some reason? You really need something like a devuser to debug this kind of issue: https://docs.openstack.org/diskimage-builder/latest/elements/devuser/README.html15:11
sylvrokay ! will do15:13
jovialslyvr: You might need to explitly enable ConfigDrive as a datasource. Like this: https://github.com/stackhpc/stackhpc-kayobe-config/blob/stackhpc/2023.1/etc/kayobe/stackhpc-overcloud-dib.yml#L4115:19
jovialWith the cloud-init-datasources element15:20
jovialIf you find that is true, could you open a bug against kayobe? Really the default config should create a working image out of the box.15:24
sylvrjovial: got access to the nodes trhough IPMI and logged in as devuser, no IP are configured by default on the node15:45
sylvrjovial: I think it is by default but I'll add it to the extras just to be sure, going to rebuild and re-provision to check if it changes something15:46
jovialslyvr: Oh yeah, I see it in the defaults. I'd recommend checking the cloud init logs in /var/log/cloud-init.log15:49
sylvrjovial: I checked for this file on kayobe-control-host/seed-bifrost and on one of the node and I couldn't find it16:06
jovialsylvr: You need to check on the host you are provisioning via the devuser16:10
sylvrjovial: I checked, nothing16:11
sylvrwell, not nothing, but no cloud-init.log in /var/log16:12
jovialslyvr: Anything in the journal? journalctl -u cloud-init or similar?16:16
jovialAnd what is the status of cloud-init.service?16:16
sylvrjovial: no entries for journalctl -u cloud-init and no cloud-init.service16:18
jovialIs the cloud-init package installed? 16:18
sylvron the node I try to provide ? no16:20
jovialOh... Try adding the cloud-init element16:20
sylvrrebuilding the images, did I miss to add it or is it supposed to be by default ?16:23
sylvrI may leave soon, I'll probably send an update tomorrow16:23
jovialsylvr: Well it seems like cloud-init-datasources doesn't create a dependency on cloud-init. I guess we should add cloud-init to the default elements in kayobe.16:24
sylvrjovial: okok, thanks for you help and I'll keep you in touch16:25
jovialslyvr: No worries, this is all useful feedback :)16:27
sylvrjovial: yep that fixed the issue ! now running kayobe overcloud host configure16:39
jovialsylvr: Glad to hear it. Fancy submitting a patch to upstream kayobe to add it to the defaults: https://github.com/openstack/kayobe/blob/master/ansible/inventory/group_vars/all/overcloud-dib#L38?16:41
sylvrjovial: yep, I'd like that ! is it okay if I do that tomorrow ?16:43
jovialslyvr: Yeah, no problem - thanks :)16:44
sylvrjovial: awesome, I took notes, I'll keep you in touch tomorrow ! thanks again !16:45
opendevreviewVictor Chembaev proposed openstack/kolla-ansible master: Fix octavia-interface timeout  https://review.opendev.org/c/openstack/kolla-ansible/+/92037718:05

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!