| *** rpittau|afk is now known as rpittau | 06:59 | |
| *** jhorstmann2 is now known as jhorstmann | 07:40 | |
| ignaziocassano | Hello, please help me on manila, What is changed on generic driver with DHSS True from stein to wallaby ? On wallaby the service instance is not created | 08:30 |
|---|---|---|
| ignaziocassano | kolla create several service network, if I create my share network, share log reports Error starting thread.: manila.exception.ServiceInstanceException: Ambiguous service networks. | 08:31 |
| mgoddard | priteau: have you seen https://review.opendev.org/c/openstack/tenks/+/799901 ? | 08:48 |
| Fl1nt | Hi everyone! | 09:22 |
| Fl1nt | mgoddard, How do we target a specific branch when contributing? | 09:23 |
| mgoddard | Fl1nt: what are you trying to contribute? | 09:23 |
| Fl1nt | the bump version you told me we don't know what is the rule ^^ | 09:23 |
| mgoddard | Fl1nt: git fetch gerrit; git checkout gerrit/stable/train | 09:54 |
| mgoddard | Fl1nt: make and commit change, or cherry-pick | 09:54 |
| mgoddard | Fl1nt: git review | 09:54 |
| Fl1nt | oh ok cool thx! | 09:54 |
| shyamb | Hi | 10:01 |
| shyamb | Any document to use local registry for kolla openstack deployment? | 10:01 |
| shyamb | can we pull and push openstack images to local registry using kolla-ansible | 10:02 |
| opendevreview | Gaël THEROND proposed openstack/kolla stable/train: Bump prometheus elasticsearch exporter version. * Fix few metrics. * Introduce posix based cli flags. https://review.opendev.org/c/openstack/kolla/+/800002 | 10:02 |
| Fl1nt | shyamb, you can use local registry with kolla yes | 10:02 |
| Fl1nt | you just need to set the appropriate docker registry directives within your globals.yml | 10:03 |
| Fl1nt | such as | 10:03 |
| Fl1nt | docker_registry: "<REGISTRY URL>" | 10:04 |
| Fl1nt | docker_namespace: "<IMAGES BUILD NAMESPACE>" | 10:04 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: Support Ansible diff mode https://review.opendev.org/c/openstack/kayobe/+/800003 | 10:04 |
| Fl1nt | docker_registry_username: "<Username>" | 10:04 |
| shyamb | Fl1nt: Yes, but in that case we need to make sure images are pushed to local registry already | 10:04 |
| shyamb | right? | 10:04 |
| Fl1nt | yes, you do it either on your own, or by using kolla-build | 10:05 |
| Fl1nt | with your local registry as a kolla-build target | 10:05 |
| shyamb | Fl1nt: I want to pull images from dockerhub and push to local registry | 10:06 |
| shyamb | I think kolla-build builts the images again | 10:06 |
| Fl1nt | you'll have to do it on your own | 10:06 |
| Fl1nt | yes kolla-build as it name suggest is used to build your own set of images. | 10:06 |
| shyamb | Fl1nt: Okay, thanks | 10:06 |
| Fl1nt | sure you're welcome | 10:07 |
| Fl1nt | mgoddard, any ongoing works around collectd and libvirt/prometheus to your knowledge? It's one of our downstream patch that I still need to create upstream, but as I remind the team had a discussion around this topic, I prefer to ask. | 10:11 |
| mgoddard | shyamb: you can pull, then retag for your registry, then push | 10:11 |
| mgoddard | Fl1nt: I don't see much activity on collectd | 10:12 |
| mgoddard | Fl1nt: there are a few open patches for adding a libvirt prometheus exporter | 10:12 |
| Fl1nt | ok, gonna add support for libvirt metrics introspection and push to prometheus using the write_prometheus plugin. | 10:12 |
| Fl1nt | I didn't find a really supported prometheus exporter out there, any suggestion? | 10:13 |
| Fl1nt | the only one that seems to have traction to my knowledge is this one: https://github.com/zhangjianweibj/prometheus-libvirt-exporter | 10:14 |
| shyamb | mgoddard: How can I use kolla-ansible tool to pull the images? | 10:14 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: Fix --check argument for overcloud host configure https://review.opendev.org/c/openstack/kayobe/+/800006 | 10:14 |
| mgoddard | shyamb: you can't really, use docker on one host | 10:14 |
| shyamb | mgoddard: Okay | 10:15 |
| mgoddard | shyamb: an example here, if you can make sense of it https://github.com/stackhpc/a-universe-from-nothing/blob/master/etc/kayobe/ansible/pull-retag-push.yml#L84 | 10:15 |
| jingvar | from default multiverse deployment | 10:18 |
| jingvar | [root@compute1 ~]# ls /etc/sysconfig/network-scripts/ | grep eth1 | 10:19 |
| mgoddard | shyamb: there was a proposal to add this functionality to kolla-build. You are welcome to work on it | 10:19 |
| jingvar | ifcfg-eth1.101 | 10:19 |
| jingvar | ifcfg-eth1.104 | 10:19 |
| jingvar | ifcfg-eth1.105 | 10:19 |
| jingvar | commit fd81c754648a9f2b57da2f6d88cf1221b21604d6 (HEAD -> a-multiverse-from-nothing-master, origin/a-multiverse-from-nothing-master) | 10:20 |
| jingvar | compute nodes don't have configuration for eth1 | 10:20 |
| jingvar | can comeone to check an env | 10:22 |
| mgoddard | jingvar: centos or ubuntu? | 10:22 |
| jingvar | centos | 10:22 |
| mgoddard | jingvar: and the problem is mtu related? | 10:23 |
| jingvar | yes | 10:23 |
| jingvar | can't bring up vlans because they have bigger mtu | 10:24 |
| mgoddard | jingvar: where have you set the mtu? | 10:24 |
| mgoddard | on the vlan interfaces? | 10:24 |
| jingvar | networks | 10:24 |
| mgoddard | ok | 10:25 |
| mgoddard | jingvar: try adding a network that maps to eth1 | 10:25 |
| jingvar | https://github.com/jingvar/a-universe-from-nothing -b test | 10:25 |
| jingvar | dummy network& | 10:25 |
| jingvar | ? | 10:25 |
| mgoddard | yeah, sometimes it's required if you want a bridge or bond | 10:26 |
| mgoddard | networks.yml: | 10:27 |
| jingvar | sorry I little bit cut my finger | 10:27 |
| jingvar | in this case i got your model | 10:27 |
| mgoddard | cloud_mtu: 9000 | 10:27 |
| mgoddard | etc/kayobe/inventory/group_vars/compute/network-interfaces: | 10:28 |
| mgoddard | cloud_interface: eth1 | 10:28 |
| mgoddard | compute.yml: | 10:28 |
| mgoddard | compute_extra_network_interfaces: [cloud] | 10:28 |
| mgoddard | hopefully that makes sense | 10:28 |
| Fl1nt | those MTU issues on infrastructure networks are really crazy, like, c'mon we're 2021 who the hell doesn't get end to end 9000+ ready network equipments? I mean, except on internet access (and just because of the RFC), your network should be using MTU 9000 everywhere ^^ | 10:30 |
| jingvar | yep, but usaly external and public 1500 | 10:31 |
| Fl1nt | yep, exact, my precise point ^^ | 10:31 |
| Fl1nt | wasn't ranting on you jingvar, it's just a general remarks regarding networks all over that are poorly administred ^^ | 10:32 |
| Fl1nt | administrated. | 10:32 |
| jingvar | I undertand | 10:33 |
| jingvar | but what intresting | 10:34 |
| jingvar | [centos@controller1 ~]$ ip a | grep eth2 | 10:34 |
| jingvar | 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc fq_codel master breth2 state UP group default qlen 1000 | 10:34 |
| jingvar | some how it has right mtu | 10:35 |
| jingvar | no ip, and I think no network binded for this interface | 10:35 |
| mgoddard | jingvar: because it's listed explicitly: https://github.com/jingvar/a-universe-from-nothing/blob/test/etc/kayobe/inventory/group_vars/controllers/network-interfaces#L13 | 10:36 |
| jingvar | maybe do the same for computes? | 10:38 |
| mgoddard | jingvar: could do. The network is a bit overloaded though - really the workload provisioning network should not be available to computes. Ironic isn't enabled, so it's not really even used on controllers | 10:46 |
| jingvar | I think in real world we will use bond | 10:51 |
| jingvar | and may be dev model should be a little bit closer to prod | 10:52 |
| jingvar | but it depend on hardware, I will use blades with vnics | 10:54 |
| mgoddard | jingvar: sure, it would be good to have an example with bonds | 10:56 |
| mgoddard | we could have single member bonds | 10:57 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix crmadmin not working on masakari Debuntu images https://review.opendev.org/c/openstack/kolla/+/799787 | 11:23 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix missing pacemaker-cli-utils in debuntu hacluster images https://review.opendev.org/c/openstack/kolla/+/799782 | 12:26 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix crmadmin not working on masakari Debuntu images https://review.opendev.org/c/openstack/kolla/+/799787 | 12:26 |
| ignaziocassano | Hello, probabluy we discoverd a bug in wallaby manila generic driver. Starting by Victoria manila share conge requires the glance section for creating the service instance | 12:35 |
| ignaziocassano | kolla does not insert the glance section | 12:35 |
| *** rpittau is now known as rpittau|afk | 12:45 | |
| shyamb | ykarel: Any other approach to push image to undercloud from remote node? | 13:10 |
| shyamb | one way is to copy image from remote node to undercloud using podman and then running tripleo command pon undercloud | 13:11 |
| shyamb | But I have not seen any option of podman command to copy container to remote node's ppodman space | 13:12 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: docs: Improve all-in-one scenario https://review.opendev.org/c/openstack/kayobe/+/797003 | 13:34 |
| opendevreview | Gaël THEROND proposed openstack/kolla stable/train: Bump prometheus elasticsearch exporter version. https://review.opendev.org/c/openstack/kolla/+/800002 | 13:42 |
| Fl1nt | mgoddard, just for you to know, as collectd write_prometheus plugin is 1:1 compatible with prometheus own collectd_exporter, I plan to only implement it once and so it just scrap config activate depending on a enable switch on collectd or prometheus_collectd_exporter. We already have everything else available. | 14:03 |
| sean-k-mooney | mgoddard: i may have asked this before but ye dont currently plan to supprot centos 9 in xena do ye? that woudl be in Yoga | 14:18 |
| priteau | I think we understood that xena would be centos stream 9 only? | 14:20 |
| priteau | With Wallaby also supporting it, so it could be used a release to switch OS | 14:21 |
| sean-k-mooney | i was just having the same converation internally i did not think that aligned with offial supported runtimes for xena | 14:22 |
| sean-k-mooney | i was suggestign that rdo shoudl supprot centos 8 in xena too | 14:23 |
| sean-k-mooney | which they would like to avoid | 14:23 |
| sean-k-mooney | but nova will not be gating on python 3.9 or centos 9 in tempest for xena for example | 14:23 |
| sean-k-mooney | https://github.com/openstack/governance/blob/master/reference/runtimes/xena.rst | 14:24 |
| sean-k-mooney | from a down stream perspective wee need to supprot rhel 9 and python 3.9 with wallaby so we will be fixing any issue we find and backporting them | 14:24 |
| sean-k-mooney | but i had not expected any dpeloyment project ot ofcialy suprpot centos 9 at xena ga | 14:25 |
| sean-k-mooney | i had assumed it woudl be backported like ooo are going to do if it was added | 14:25 |
| mgoddard | sean-k-mooney: we are at the mercy of RDO :) | 14:26 |
| mgoddard | if they provide only CS9 packages then we are stuck | 14:27 |
| sean-k-mooney | or centos source only i guess | 14:27 |
| sean-k-mooney | was this discussed previously at the PTG for example | 14:28 |
| sean-k-mooney | i have not been keeping track | 14:28 |
| sean-k-mooney | i jsut was not expecting support in this cycle for centos 9 hosts? and contianer? | 14:28 |
| priteau | Even with source containers I believe we still rely on a few packages built by RDO. OVS maybe? | 14:29 |
| sean-k-mooney | priteau: i was thinking c8 with source packages | 14:30 |
| priteau | what do you call source packages? | 14:30 |
| sean-k-mooney | tars for tarballs.openstack.org | 14:31 |
| sean-k-mooney | or git repos | 14:31 |
| priteau | Yes, kolla source images (as opposed to binary) | 14:31 |
| sean-k-mooney | just the source build option in kolla | 14:31 |
| priteau | We are on the same page. I was pointing out that even with source images, I think we still install a few RPMs from RDO | 14:31 |
| priteau | Not for OpenStack itself, but for dependencies | 14:32 |
| sean-k-mooney | perhaps although you coudl use the deps form wallaby rdo repos | 14:32 |
| opendevreview | Piotr Parczewski proposed openstack/kolla-ansible master: Reduce container metrics cardinality https://review.opendev.org/c/openstack/kolla-ansible/+/800068 | 14:32 |
| sean-k-mooney | anyway if ye plan to suport c9 then that shoudl not be an issue | 14:32 |
| sean-k-mooney | context of all this is i was grying to gauge the proably of getting a nova tempest devstack job running on c9 at some point later this cycle | 14:33 |
| sean-k-mooney | so i was talking to the rdo folks about packaging and eventlets and python 3.9 | 14:33 |
| sean-k-mooney | i was just surpirsed that they were considerign goign c9 only for xena | 14:34 |
| sean-k-mooney | espceilly assuming there master branch is currenly centos 8 stream | 14:34 |
| priteau | I think we'll be happy if we can use c8s for another release. We don't really want to switch OS every 6 months | 14:35 |
| Fl1nt | priteau, good luck with that :p RH is pushing hard to accelerate release cycle everywhere, would it be on OS or Openstack itself, once at berlin they even did a keynote expecting Openstack release to happens every 3 months. | 14:37 |
| Fl1nt | which is unrealistic as fuck | 14:37 |
| priteau | I was there too ;-) | 14:38 |
| Fl1nt | oh ^^ cool :D | 14:39 |
| Fl1nt | honestly when you reach that level of release cycle, you should probably just stream everything at some point, but doing that isn't doing any good to the community as much of companies using Openstack are just unable to keep up with lifecycle | 14:40 |
| Fl1nt | even when doing it the full CICD way | 14:40 |
| Fl1nt | as there are still HW issue | 14:40 |
| Fl1nt | Let's take an exemple | 14:41 |
| Fl1nt | today, we're doing the full platform lifecycle the CICD way, but even with such platform | 14:41 |
| Fl1nt | sometimes I have to delay a bit the upgrade of some platforms because they don't get enough HW in order to do the appropriate no outage, no interruption upgrade dance. | 14:42 |
| sean-k-mooney | Fl1nt: working at redhat im not sure that is acurate | 14:46 |
| sean-k-mooney | Fl1nt: from my perspective the openstack product lifecyle is slowign down not speedign up | 14:46 |
| sean-k-mooney | Fl1nt: i think your confusint redhat with soemone else otn the 3 month release of openstack | 14:47 |
| sean-k-mooney | Fl1nt: our current release process moved form release evey upstream release (until train) to release ever 3 upstream reelases after trian | 14:48 |
| sean-k-mooney | Fl1nt: so currently redhats product release interval for different opentack versions is ever 18-24 months | 14:48 |
| sean-k-mooney | Fl1nt: vs when i joing wehre it was every upstream release | 14:49 |
| sean-k-mooney | i know form an engeienring point of vew we would liek to speed up our product release cycle but there is busness pressure form customer to slow it down more and do more feature backports | 14:50 |
| sean-k-mooney | so its a blancing act of keeping custoemr happy and minimising technical debt | 14:50 |
| sean-k-mooney | Fl1nt: for context the last prodctied release of openstack that redhat has is based on train, the next one will be based on wallaby and the one after that is still undefiend but likely a similar gap of upstream releases | 14:52 |
| Fl1nt | sean-k-mooney, I'm talking about upstream, not RH services like RHOS etc :D that could explain the confusion ^^ | 14:53 |
| Fl1nt | There was one of you boss at Berlin explaining RH was looking to increase upstream release cycle to one release every 3 months. | 14:53 |
| sean-k-mooney | there was talk about either shortinging it to release every milestone or extending to once a year | 14:55 |
| Fl1nt | I know there should have lot of RH customers using RHOS/RH Specific platforms, but all in all there are way much more companies using vanilla upstream Openstack based on RDO/CentOS and I was refering of those companies ^^ | 14:55 |
| sean-k-mooney | but chanaging the lifecycle has alwasy come up evey few release and nver changed so i doubt it will happen in the short term | 14:56 |
| Fl1nt | First hand exemple: Ubisoft was 100% using CentOS/RDO for their platform, they're currently planning to switch to Ubuntu since the CentOS Stream annoucement, even when I still worked there, their were already discussion taking place about either or not they should change. | 14:56 |
| sean-k-mooney | ya im not surprised | 14:57 |
| sean-k-mooney | rdo in thoey i think plan to have some level of supprot for rocky linux | 14:57 |
| Fl1nt | oh sure for now having a release cycle of 1 release every 6 months is quite ok as it allow companies to plan and act even by lagging two release beyond. | 14:57 |
| sean-k-mooney | i like canonicals approch of release a new lts OS very 2 years and support it for 5 then supprot opnetack on both the point release and the latest lts | 14:59 |
| sean-k-mooney | with one version to cross over | 14:59 |
| Fl1nt | On my side I'm not trusting Rocky more than that, because why wouldn't someone that already built and sell a community based initiative do it again? I mean, they even are already well suited to such scenario as they're well backed by big GAFAM. | 14:59 |
| sean-k-mooney | so you can upgrade openstack then your distro seperatly | 14:59 |
| Fl1nt | exactly, we're having the same approach internally with our customers, 5 years of support guarantee plus a EOL/Maintenance period. | 15:00 |
| Fl1nt | however, for legacy/contractual reason we sometimes support decades old softwares/platforms ^^ | 15:00 |
| Fl1nt | Software lifecycle is really hard to do right honestly, I do understand devs eager to fix things, and in the meantime I do understand enterprises willing to make money out of their investment and not constantly being in a WIP status. | 15:02 |
| *** frickler is now known as frickler_pto | 15:03 | |
| cndxt | hello everyone ! engineer here.. with comprehension problems regarding kolla-ansible and its networking layout. anyone feel like helping? :-) | 15:05 |
| Fl1nt | go ahead ^^ | 15:05 |
| Fl1nt | mgoddard, do you want that we activate collectd virt plugin based on some specific enable_collectd_plugin_virt var or can we activate it as soon as you enable_collectd yes ? :D | 15:08 |
| mgoddard | Fl1nt: I guess just follow existing patterns | 15:09 |
| Fl1nt | well, there is nothing for now, we just activate the network plugin which is the binary collectd protocol used to communicate so... I suppose I'll do it my way and improve on others suggestions I guess :D | 15:10 |
| cndxt | Fl1nt: 5 bare metal machines, each with 3 ifaces connected, one has physical access to a /27 wan network, without an ip set. the other two have access to separate local networks, each a /24 with two ip's set. What i can't wrap my head around is the neutron network thing. Any hint on how to configure the interfaces in globals.yml? | 15:16 |
| Fl1nt | yes | 15:18 |
| Fl1nt | so, here is how I usually do it: | 15:19 |
| Fl1nt | TBN: I only use OpenVswitch and no linuxbridge in between. | 15:19 |
| Fl1nt | so | 15:19 |
| Fl1nt | it depends on wether or not you want to use CEPH first. | 15:20 |
| Fl1nt | as a storage backend. | 15:20 |
| Fl1nt | as CEPH would perform way better using a properly segmented frontend and backend network, that would require you to create macvlan interfaces on top of your interfaces or to deal with SR-IOV VF PF | 15:21 |
| Fl1nt | so basically, for your neutron being able to use your wan interface as an external network, you would need to set it using neutron_external_interface: <wan_/27_interface> | 15:23 |
| cndxt | that is currently the case. neutron_external_interface = eno1, network_interface=eno2, storage_interface=eno3. where eno1 = wan (but no ip assigned as per documentation), eno2 = local (with ip) and eno3 = local (with ip) | 15:30 |
| cndxt | not going down the ceph route in this project :-) | 15:32 |
| cndxt | so the question is, when having 2 interfaces for local traffic and one for external traffic (wan), and working wan ip is prohibited as per docs, how are the machines supposed to fetch apt packages during bootstrapping? | 15:34 |
| Fl1nt | using your local ip | 15:48 |
| Fl1nt | in our own case, our platforms are all behind a HA Bastion that is used as the default route. | 15:48 |
| cndxt | i see.. thanks for sharing. i guess theres missing a piece | 16:09 |
| opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add support for virtual machine metrics collect. * Activate collectd virt plugin. * Activate collectd write_prometheus plugin https://review.opendev.org/c/openstack/kolla-ansible/+/800080 | 16:17 |
| opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add support for VMs metrics collect. * Activate collectd virt plugin. * Activate collectd write_prometheus plugin https://review.opendev.org/c/openstack/kolla-ansible/+/800080 | 16:18 |
| opendevreview | Gaël THEROND proposed openstack/kolla-ansible master: Add support for VMs metrics collect. https://review.opendev.org/c/openstack/kolla-ansible/+/800080 | 16:19 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Make kolla custom config globs configurable https://review.opendev.org/c/openstack/kayobe/+/792834 | 16:53 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Import merge_configs and merge_yaml from Kolla Ansible https://review.opendev.org/c/openstack/kayobe/+/778994 | 16:57 |
| opendevreview | Will Szumski proposed openstack/kayobe master: WIP: Use merge_configs and merge_yaml to generate custom config https://review.opendev.org/c/openstack/kayobe/+/782749 | 16:57 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Make kolla custom config globs configurable https://review.opendev.org/c/openstack/kayobe/+/792834 | 16:57 |
| *** mgoddard- is now known as mgoddard | 20:50 | |
| opendevreview | Ghanshyam proposed openstack/kolla-cli master: Moving IRC network reference to OFTC https://review.opendev.org/c/openstack/kolla-cli/+/800136 | 23:40 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!