| *** ysandeep|PTO is now known as ysandeep | 05:17 | |
| *** prometheanfire is now known as Guest0 | 06:40 | |
| *** ysandeep is now known as ysandeep|lunch | 08:56 | |
| *** ysandeep|lunch is now known as ysandeep | 10:02 | |
| *** ysandeep is now known as ysandeep|afk | 10:20 | |
| *** ysandeep|afk is now known as ysandeep | 10:59 | |
| *** dviroel|out is now known as dviroel|rover | 11:20 | |
| anskiy | noonedeadpunk: hello! I've been trying to update Nova to Zed release on focal and hit the `No module named 'bmemcached'` bug, which is supposed to be fixed by https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/860540 this thing, but it isn't merge yet :( | 11:32 |
|---|---|---|
| noonedeadpunk | anskiy: that's absolutely true. | 11:32 |
| noonedeadpunk | feel free to vote on it as well if it solves the issue | 11:33 |
| noonedeadpunk | As even +1 does matter much | 11:33 |
| anskiy | noonedeadpunk: done. Now according to this thing: https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/855829/: it works fine, openvswitch-switch is masked on controller node and I can reach newly created VMs on compute node. Tested with OVN on metal. | 11:57 |
| noonedeadpunk | Awesome! Thanks for taking time and testing this out | 12:16 |
| noonedeadpunk | eventually, would be also great to merge https://review.opendev.org/c/openstack/openstack-ansible/+/862499/2 and backport to Y | 12:17 |
| jamesdenton_ | noonedeadpunk thanks for the review | 12:24 |
| *** jamesdenton_ is now known as jamesdenton | 12:24 | |
| jamesdenton | i'm a bit uncertain how to handle an upgrade scenario for LXB. The default is specified here: https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/all/neutron.yml#L17 | 12:25 |
| jamesdenton | i don't think we can trust the inventory, since children/hosts are generated for all neutron agents despite their (lack of) deployment | 12:28 |
| jrosser | jamesdenton: does moving that var from group_vars to user_variables help? | 12:41 |
| jrosser | perhaps thats also aligned with needing the deployer to now more specifically choose the network driver | 12:41 |
| jamesdenton | i haven't confirmed directly, but i would think it would, since that's where the user would normally override to ml2.ovs or ml2.ovn. | 12:43 |
| jrosser | or maybe things are orthoganal, you can create host groups for OVN but still have the backend set to LXB | 12:43 |
| jamesdenton | but then it's on the deployer to explicitly set that, and i'm not sure we can easily set a default moving forward if the existing ml2.lxb don't set that | 12:44 |
| jamesdenton | you might confirm, but in my environment, all groups (neutron_*_agent) and now neutron_ovn_* are populated | 12:45 |
| noonedeadpunk | regarding upgrade - we can run task against neutron_all and see if `neutron_plugin_type` defined. If it's not - create /etc/openstack_deploy/group_vars/neutron_all and place there neutron_plugin_type:ml2.lxb | 12:53 |
| noonedeadpunk | or some user_zed_upgrade | 12:53 |
| jamesdenton | but how would you couple that with a new default? having a hard time with that logic | 12:55 |
| noonedeadpunk | I think group_vars under /etc/openstack_deploy have prescedence over ones in /opt/openstack-ansible | 12:56 |
| noonedeadpunk | because it depends on inventory order that's being loaded | 12:57 |
| noonedeadpunk | and inventory from openstack_deploy is loaded after dynamic_inventory | 12:57 |
| jamesdenton | ok, so as of now the default is coming from /opt/openstack-ansible/inventory/group_vars/all/neutron.yml. If I delete or comment out "neutron_plugin_type: ml2.lxb" there, and the test for neutron_plugin_type fails, i'll know that environment is using ml2.lxb via default, and can create a file at /etc/openstack_deploy/group_vars/neutron_all.yml with ml2.lxb set. But as-is, this would impact greenfield (i would think) | 13:28 |
| jamesdenton | unless we force operators to specify a default | 13:28 |
| noonedeadpunk | but we can change default here? https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/defaults/main.yml#L363 | 13:51 |
| noonedeadpunk | I'm not just sure what to do with that though... https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/horizon_all.yml#L17 | 13:52 |
| noonedeadpunk | Likely we can check just for neutron_l3_agent length | 13:52 |
| jamesdenton | well, first of all, i didn't know that default was there :D | 13:53 |
| jamesdenton | but it's not considered with the following: ansible neutron_all -m debug -a "var={{neutron_plugin_type}}" - so maybe there's a better test | 13:54 |
| jamesdenton | maybe it's different for others, but in the inventory neutron_l3_agent is populated whether or not it's used. same with neutron_linuxbridge_agent and neutron_openvswitch_agent groups | 13:55 |
| noonedeadpunk | Maybe we can create smth like that https://paste.openstack.org/show/bltwlhpqLiQeUejsiy5k/ in https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/upgrade-utilities ? | 13:57 |
| noonedeadpunk | ah, no, not exactly | 13:58 |
| noonedeadpunk | code is completely wrong there... | 13:59 |
| jamesdenton | :| | 14:00 |
| noonedeadpunk | but we can play against neutron_all and delegate_to localhost and create hostvars for each neutron host.... | 14:01 |
| noonedeadpunk | quite suboptimal, but still | 14:02 |
| jamesdenton | FYI: i am not convinced that the neutron_plugin_type in defaults/main.yml is effective, testing now | 14:03 |
| noonedeadpunk | if it's not defined anywhere else it must be | 14:04 |
| jamesdenton | well, it is also defined in /opt/openstack-ansible/inventory/group_vars/all/neutron.yml. When i deleted that file yesterday, i had issues in the playbooks with "neutron_plugin_type not defined" | 14:05 |
| jamesdenton | but verifying now | 14:05 |
| opendevreview | Merged openstack/openstack-ansible-os_nova master: Improve way of cache backend selection https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/860540 | 14:13 |
| jamesdenton | it was the haproxy playbooks that had issues, due to references: https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/haproxy/haproxy.yml#L358-L359 | 14:14 |
| jamesdenton | and others in that file | 14:15 |
| jamesdenton | so i guess we can't rely on the role default | 14:15 |
| jrosser | each role should have a set of defaults to define it's "external api" | 14:16 |
| jrosser | even if this is just for documenation-as-code | 14:16 |
| jrosser | where that key/value is needed across roles then it must be also in group/host vars or user_variables as "wiring" between roles | 14:17 |
| jamesdenton | right now it is a group var explicitly set in main openstack-ansible | 14:18 |
| *** Guest0 is now known as prometheanfire | 14:18 | |
| jamesdenton | but user_variables override in many cases | 14:18 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_neutron master: [WIP] add ovn ssl config https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/862403 | 14:22 |
| noonedeadpunk | I think for haproxy usecase we can say haproxy_ssl: "{{ (neutron_plugin_type == 'ml2.calico' | default(False)) | ternary(False, haproxy_ssl_all_vips) }}" ? | 14:25 |
| jrosser | ^ just delete calico support :) | 14:26 |
| jamesdenton | or that :D | 14:26 |
| jrosser | sorry i've not got round to that yet | 14:26 |
| jamesdenton | but noonedeadpunk i think that could work | 14:26 |
| jrosser | theres just stupidity there with the metadata service | 14:26 |
| noonedeadpunk | oh, yes, actually we can just drop calico support :D | 14:26 |
| jamesdenton | i think opendaylight is another in there | 14:27 |
| jamesdenton | would be nice to know if anyone is using that | 14:27 |
| jamesdenton | and i suspect the NSX-T stuff is likely not used. Last I heard, vmware is only supporting it in VIO | 14:27 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/ussuri: Set erlang version to 23.3.4.12-1 ofr Ubuntu https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/862564 | 14:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/ussuri: Set erlang version to 23.3.4.12-1 for Ubuntu https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/862564 | 14:40 |
| jamesdenton | did we deprecate ml2.calico already? | 14:43 |
| noonedeadpunk | Nope, we didn't | 14:44 |
| noonedeadpunk | I think we just agreed to | 14:44 |
| jamesdenton | fair enough. shall we consider others, too? | 14:44 |
| jamesdenton | nuage, opendaylight, etc.... maybe with a mailing list callout? | 14:45 |
| noonedeadpunk | odl plugin for neutron seems quite maintained | 14:46 |
| noonedeadpunk | Btw about deprecations - let's drop rsyslog :) https://review.opendev.org/c/openstack/openstack-ansible/+/861886 | 14:47 |
| jamesdenton | is that the determination? that's fine | 14:47 |
| noonedeadpunk | Nah, I don't know. We can do a ML callout indeed | 14:47 |
| jamesdenton | may need to try and deploy it one day to see what breaks | 14:48 |
| jamesdenton | not sure what state it's in | 14:48 |
| cloudnull | happy halloween! | 14:52 |
| jamesdenton | a wild cloudnull appears! | 14:53 |
| * cloudnull vanishes in a puff of smoke | 14:53 | |
| cloudnull | how's it? | 14:54 |
| noonedeadpunk | hm, btw for calico we just need etcd3gw.... | 14:54 |
| noonedeadpunk | So might be worth fixing it finally :D | 14:54 |
| jamesdenton | i was just reading on some thread somewhere.. that maybe they're also recommending calico core plugin vs ml2? | 14:55 |
| jamesdenton | would need to confirm | 14:55 |
| jamesdenton | things be good, cloudnull. ready to get my candy on | 14:55 |
| cloudnull | hell yeah! | 14:56 |
| cloudnull | I'm ready to turn off the lights and hide inside :D | 14:56 |
| jamesdenton | https://media.giphy.com/media/3oeSAF90T9N04MyefS/giphy.gif | 14:57 |
| cloudnull | https://i.imgur.com/xBbwN5o.gif | 14:57 |
| jamesdenton | :D | 14:58 |
| cloudnull | 😆 | 14:58 |
| cloudnull | anyone do work with the Nvidia grid driver? | 14:58 |
| cloudnull | RE: https://docs.openstack.org/nova/queens/admin/virtual-gpu.html | 14:59 |
| jrosser | cloudnull: yes we use that here | 14:59 |
| cloudnull | queens pfft... /me is old | 14:59 |
| cloudnull | better link https://docs.openstack.org/nova/latest/admin/virtual-gpu.html | 14:59 |
| cloudnull | jrosser works well ? | 15:00 |
| jamesdenton | just had a colleague set it up, with Yoga i think | 15:00 |
| cloudnull | if I'm groking things right, we have to use the grid driver both in the instance and on the server to make it go, right? | 15:01 |
| jrosser | yes | 15:01 |
| cloudnull | 👍 | 15:01 |
| noonedeadpunk | we do as well | 15:01 |
| cloudnull | OK I did the 90 day eval for the license I'll give this a go | 15:02 |
| noonedeadpunk | cloudnull: but drivers are different just in case. And all licensing only on instances | 15:02 |
| cloudnull | right its a license per node right? | 15:02 |
| noonedeadpunk | nope | 15:02 |
| noonedeadpunk | It's license per VM | 15:02 |
| cloudnull | ++ | 15:02 |
| cloudnull | I think we can make that work | 15:03 |
| noonedeadpunk | And when they're talking about vCS per node - they mean bundle of 8 licenses IIRC | 15:03 |
| jrosser | we make a volume that folk can mount that contains the grid driver and some scripts that wire in the URL of the licence server | 15:03 |
| noonedeadpunk | so technically you can cover more nodes with such license depending on how small vgpus you want | 15:03 |
| jrosser | but that only works becasue it's users and licences all for a single org | 15:03 |
| noonedeadpunk | we do build images with driver/license in it and it's nasty | 15:04 |
| cloudnull | ++ | 15:04 |
| noonedeadpunk | ugh | 15:04 |
| cloudnull | it sounds like a nightmare :D | 15:04 |
| * cloudnull just getting up to speed on grid | 15:04 | |
| noonedeadpunk | you're quite lucky to start now I would say.... | 15:05 |
| noonedeadpunk | as missed all fun with licensing servers | 15:05 |
| cloudnull | I kinda wish I could run AMD GPUs with MxGPU - which is all exposed via SR-IOV. sadly the workloads I want to run all require Cuda. | 15:06 |
| noonedeadpunk | +1 | 15:06 |
| noonedeadpunk | same here | 15:06 |
| noonedeadpunk | stupid devs :D | 15:06 |
| noonedeadpunk | When they will learn OpenCL | 15:07 |
| cloudnull | they won't | 15:07 |
| noonedeadpunk | I kind of had some hopes that Intel coming to the GPU market with OpenCL as well will somehow move weights from Cuda... | 15:08 |
| noonedeadpunk | But they seem to far from being ready | 15:08 |
| cloudnull | new ARC gpus have the AV1 encoder | 15:08 |
| cloudnull | which is super neat | 15:08 |
| opendevreview | Merged openstack/openstack-ansible master: Switch Ceph for Ubuntu Jammy to distro https://review.opendev.org/c/openstack/openstack-ansible/+/862499 | 15:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Switch Ceph for Ubuntu Jammy to distro https://review.opendev.org/c/openstack/openstack-ansible/+/862994 | 15:09 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: [doc] Better document integrated usage of ceph-ansible https://review.opendev.org/c/openstack/openstack-ansible/+/862676 | 15:22 |
| opendevreview | Merged openstack/openstack-ansible-os_neutron master: Disable/stop/mask Open vSwitch on ovn-northd nodes https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/855829 | 15:25 |
| noonedeadpunk | tbh I still like https://review.opendev.org/c/openstack/openstack-ansible/+/862508 more but seems there're quite strong opinions against it. | 15:37 |
| noonedeadpunk | thanks for editing doc! | 15:38 |
| *** dviroel|rover is now known as dviroel|rover|lunch | 15:38 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_client master: Deprecate rsyslog_client role https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_client/+/863085 | 15:56 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rsyslog_server master: Deprecate rsyslog_server role https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_server/+/863086 | 15:59 |
| *** dviroel|rover|lunch is now known as dviroel|rover | 16:46 | |
| noonedeadpunk | ok, I played with etcd for calico aiming to fix it, and I'm fed enough with that | 17:02 |
| noonedeadpunk | Seems like etcd3gw driver is jsut can't work with modern etcd | 17:03 |
| opendevreview | Merged openstack/openstack-ansible master: Remove usage of rsyslog roles https://review.opendev.org/c/openstack/openstack-ansible/+/861886 | 17:13 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ops master: Add support for enabling ELK stack security https://review.opendev.org/c/openstack/openstack-ansible-ops/+/862873 | 17:21 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ops master: Enable cluster ID monitoring setting for beats using logstash https://review.opendev.org/c/openstack/openstack-ansible-ops/+/862874 | 17:21 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ops master: Set permissions on elastic data directories https://review.opendev.org/c/openstack/openstack-ansible-ops/+/862875 | 17:21 |
| opendevreview | Merged openstack/openstack-ansible-ops master: Fix role installation in tests https://review.opendev.org/c/openstack/openstack-ansible-ops/+/862915 | 18:11 |
| *** dviroel|rover is now known as dviroel|rover|brb | 21:11 | |
| *** dviroel|rover|brb is now known as dviroel|rover | 22:58 | |
| *** dviroel|rover is now known as dviroel|rover|dinner | 23:17 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
