| *** k_mouza has joined #openstack-kolla | 00:00 | |
| *** k_mouza has quit IRC | 00:05 | |
| *** k_mouza has joined #openstack-kolla | 00:16 | |
| *** k_mouza has quit IRC | 00:21 | |
| *** recyclehero has quit IRC | 00:26 | |
| *** recyclehero has joined #openstack-kolla | 00:27 | |
| *** k_mouza has joined #openstack-kolla | 00:31 | |
| *** k_mouza has quit IRC | 00:36 | |
| *** k_mouza has joined #openstack-kolla | 00:56 | |
| *** Tony31 has quit IRC | 00:58 | |
| *** k_mouza has quit IRC | 01:00 | |
| *** ricolin_ has joined #openstack-kolla | 01:14 | |
| *** also_stingrayza has joined #openstack-kolla | 02:26 | |
| *** stingrayza has quit IRC | 02:29 | |
| *** brinzhang_ has joined #openstack-kolla | 02:31 | |
| *** brinzhang0 has quit IRC | 02:34 | |
| *** ricolin_ has quit IRC | 03:10 | |
| *** skramaja has joined #openstack-kolla | 03:27 | |
| *** cah_link has joined #openstack-kolla | 04:13 | |
| *** cah_link has quit IRC | 04:21 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-kolla | 04:33 | |
| *** shyamb has joined #openstack-kolla | 04:43 | |
| *** shyam89 has joined #openstack-kolla | 04:43 | |
| oklhost | good morning | 05:00 |
|---|---|---|
| *** brinzhang0 has joined #openstack-kolla | 05:14 | |
| *** brinzhang_ has quit IRC | 05:17 | |
| *** shyam89 has quit IRC | 05:19 | |
| *** shyamb has quit IRC | 05:19 | |
| *** shyamb has joined #openstack-kolla | 05:20 | |
| *** shyam89 has joined #openstack-kolla | 05:20 | |
| recyclehero | morning | 05:34 |
| recyclehero | I deployed to centos source container on centos | 05:34 |
| recyclehero | still I see one of the logs that I saw on debian | 05:34 |
| recyclehero | I think I am doing some config wrong | 05:35 |
| recyclehero | 2020-10-01 09:02:33.396 6 ERROR nova.servicegroup.drivers.db [-] The services table record for the nova-compute service on host openstackCompute1 is gone. You either need to stop this service if it should be deleted or restart it to recreate the record in the database.: nova.exception_Remote.ServiceNotFound_Remote: Service 4 could not be found. | 05:35 |
| recyclehero | this is nova-compute.log | 05:35 |
| recyclehero | I ahve two host now with no netwrok node in multinode with linuxbrdige and provide network | 05:35 |
| recyclehero | what comes to ur mind? I really appreciate ur help I am running out of time | 05:36 |
| recyclehero | kolla_internal_vip_address: "172.16.0.1" | 05:37 |
| recyclehero | if I have enable_haproxy no this should be commented out? | 05:38 |
| *** cah_link has joined #openstack-kolla | 05:45 | |
| *** bhagyashri is now known as bhagyashris | 06:02 | |
| *** ricolin_ has joined #openstack-kolla | 06:07 | |
| *** Tony31 has joined #openstack-kolla | 06:09 | |
| Tony31 | Hi guys | 06:09 |
| recyclehero | hi | 06:13 |
| *** brinzhang0 has quit IRC | 06:15 | |
| recyclehero | could this not being able to lunch instance be related to me not actually introducing the nfs share which I should have? | 06:20 |
| recyclehero | this is my global.yml BTW | 06:20 |
| recyclehero | https://dpaste.com/BC39SW353 | 06:20 |
| recyclehero | though kolla_internal_vip_address: "172.16.0.1" | 06:24 |
| recyclehero | was not commented | 06:24 |
| *** sean-k-mooney1 has joined #openstack-kolla | 06:26 | |
| *** shyam89 has quit IRC | 06:27 | |
| *** shyamb has quit IRC | 06:27 | |
| *** sean-k-mooney has quit IRC | 06:29 | |
| *** wuchunyang has joined #openstack-kolla | 06:31 | |
| *** shyamb has joined #openstack-kolla | 06:44 | |
| *** shyam89 has joined #openstack-kolla | 06:44 | |
| *** shyam89 has quit IRC | 06:48 | |
| *** shyamb has quit IRC | 06:48 | |
| *** shyamb has joined #openstack-kolla | 06:48 | |
| *** shyam89 has joined #openstack-kolla | 06:49 | |
| Tony31 | I have configured a new network for octavia. The network is currently configured on the controller and compute node. I'm unsure how to add this network to neutron. I tried adding it to "external_net_names:" list. | 07:04 |
| Tony31 | Is there a page that explains what I need to do here? | 07:04 |
| *** ricolin_ has quit IRC | 07:10 | |
| *** bengates has joined #openstack-kolla | 07:18 | |
| *** shyam89 has quit IRC | 07:29 | |
| *** shyamb has quit IRC | 07:29 | |
| *** wuchunyang has quit IRC | 07:35 | |
| *** e0ne has joined #openstack-kolla | 07:42 | |
| *** jbadiapa has joined #openstack-kolla | 07:47 | |
| recyclehero | I cant become root in the nova_compute container to inspect logs. what kind of system is kolla. its only bringing me pain and suffering | 07:47 |
| *** gfidente has joined #openstack-kolla | 07:48 | |
| recyclehero | and there is no passwd in passwords.yml | 07:48 |
| recyclehero | i want to inspect journalctl ... | 07:48 |
| recyclehero | and there are no logs in log/libvirt on the hosts. in debian containers I saw some logs there | 07:50 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Revert "CI: workaround overcloud upgrade issue with ironic" https://review.opendev.org/755437 | 07:52 |
| *** born2bake has joined #openstack-kolla | 07:53 | |
| mnasiadka | morning | 07:54 |
| Tony31 | morning | 07:59 |
| Tony31 | @recyclehere have you tried: `docker exec -it -u root [container]` ? | 08:00 |
| Tony31 | @mnasiadka - would you be able to help me? I am working on installing a network to run octavia. I have the network now provisioned on the controller and compute. I'm unsure how to add this network to neutron. I am using kayobe. I tried adding this network to "external_net_names:" but after deployment I checked brctl show and it doesnt list the bridge. Do you have any tips or pointers on how this is supposed to be done? I wasnt | 08:02 |
| Tony31 | able to figure it out | 08:02 |
| mnasiadka | Tony31: well, external_net_names won't setup the bridge for you, you need a proper mapping in network-interfaces (controllers group_vars) as a starter? And I can't know your setup (if it's a vlan on existing interface, etc) :) | 08:12 |
| recyclehero | Tony31: thank you | 08:13 |
| Tony31 | @mnasiadka I have this network config but I had believed I need more than this to set up the neutron side? | 08:13 |
| recyclehero | could it be possible that I didnt introduve the nfs share and now the placement server cant find any suitable hypervisor? | 08:13 |
| Tony31 | octav_interface: broct | 08:14 |
| Tony31 | octav_bridge_ports: | 08:14 |
| Tony31 | - eth6 | 08:14 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/ussuri: Make no_proxy handling more robust https://review.opendev.org/755441 | 08:19 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Control Neutron migrations https://review.opendev.org/755328 | 08:20 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: [DNM] Add NFV upgrade job https://review.opendev.org/755331 | 08:22 |
| recyclehero | if I am changing the /etc/kolla/config/nfs_share I should deploy again? | 08:25 |
| recyclehero | whats the process? | 08:25 |
| recyclehero | destory - use tools clean-* - bootstarp - deploy | 08:25 |
| recyclehero | or just deploy | 08:25 |
| Tony31 | I think deploy, because if you destroy then you remove the database with your openstack config, right? | 08:26 |
| recyclehero | Tony31: i think so. just needed affirmation | 08:27 |
| Tony31 | i am using kayobe. IF I make any config change to the networks / nodes then I use host configure, then deploy. IF I just make a config change, like changing .conf then I use deploy. | 08:29 |
| Tony31 | "destroy" is destructive, i dont think you want to do that | 08:29 |
| hrw | mnasiadka: https://review.opendev.org/#/c/747534/2 | 08:30 |
| patchbot | patch 747534 - kolla - Debian: use QEMU 5.0 from buster-backports - 2 patch sets | 08:30 |
| hrw | https://review.opendev.org/#/c/754355/ is also easy one | 08:30 |
| patchbot | patch 754355 - kolla - convert STATUS_* consts into Enum - 3 patch sets | 08:30 |
| *** shyamb has joined #openstack-kolla | 08:36 | |
| *** shyam89 has joined #openstack-kolla | 08:36 | |
| Tony31 | Please could you help me understand what this means? 'reason': u'Bridge interface broct has missing ports: eth6'}\n" http://paste.openstack.org/show/798591/ | 08:37 |
| Tony31 | What does it mean by port in this context? I have specified the interface name to bridge but I'm getting an error with it | 08:37 |
| *** shyamb has quit IRC | 08:41 | |
| *** shyam89 has quit IRC | 08:41 | |
| *** shyamb has joined #openstack-kolla | 08:42 | |
| *** shyam89 has joined #openstack-kolla | 08:42 | |
| mnasiadka | Tony31: https://docs.openstack.org/kayobe/latest/configuration/reference/network.html#configuring-ethernet-interfaces | 08:49 |
| mgoddard | morning | 08:49 |
| Tony31 | morning mgoddard | 08:50 |
| *** strigazi has quit IRC | 08:51 | |
| Tony31 | @mnasiadka, thanks for the direct link - as far as I can see, I have this configured. But the deployment is throwing an error and I dont understand why | 08:51 |
| *** strigazi has joined #openstack-kolla | 08:51 | |
| Tony31 | just trying something - penny may have dropped | 08:54 |
| *** devfaz has joined #openstack-kolla | 08:58 | |
| *** fbermel has joined #openstack-kolla | 08:59 | |
| fbermel | hello | 08:59 |
| fbermel | can someone here tell me why this pipeline if failing? | 09:01 |
| fbermel | https://zuul.opendev.org/t/openstack/build/edc39dcddced45d5941d8eeb1f5c79f2/console | 09:01 |
| fbermel | *is | 09:01 |
| *** e0ne_ has joined #openstack-kolla | 09:02 | |
| *** born2bake has quit IRC | 09:03 | |
| *** born2bake has joined #openstack-kolla | 09:03 | |
| Tony31 | nope - still same issue with config and confusion on this. The error is: "Bridge interface broct has missing ports: eth6"}". Why would it say the port is missing and then list the port? It's not missing if it can list it, right? Would you be able to help me understand what am I missing? | 09:04 |
| hrw | fbermel: check https://489fff0f2f979379ccac-edfb11d7c283f3d767689d2badcfe4a2.ssl.cf1.rackcdn.com/711923/12/check/kolla-ansible-centos8-source/edc39dc/primary/logs/ansible/deploy | 09:05 |
| *** e0ne has quit IRC | 09:05 | |
| * hrw out | 09:06 | |
| mgoddard | Tony31: I think it is referring to the running state rather than config. You've configured the bridge with eth6 as a port, but after bouncing the interfaces it is not an active member of the bridge | 09:06 |
| Tony31 | ah I didnt think of that, thanks. I'll try removing and re-applying this config in that case. Thank you | 09:07 |
| Tony31 | maybe previous config is conflicting | 09:07 |
| recyclehero | guys nfs in successfuly introduced to the infra. whats wrong that I cant lunch an instance? | 09:08 |
| recyclehero | in the horizon in admin I see a compute host but no hypervisors | 09:09 |
| *** jonaspaulo has joined #openstack-kolla | 09:10 | |
| recyclehero | 2020-10-01 12:40:26.132 6 ERROR nova.servicegroup.drivers.db [-] The services table record for the nova-compute service on host openstackCompute1 is gone. You either need to stop this service if it should be deleted or restart it to recreate the record in the database.: nova.exception_Remote.ServiceNotFound_Remote: Service 4 could not be found. | 09:11 |
| Tony31 | compute node = gone looks like another problem @recyclehero | 09:15 |
| Tony31 | compute node = hypervisor, right? | 09:15 |
| Tony31 | which service are you adding NFS to? Does it support instance volumes? | 09:16 |
| recyclehero | Tony31: I added nfs as the docs suggest in the deploy host /etc/kolla/config/nfs_share | 09:16 |
| recyclehero | Tony31: I didnt understand the part about compute | 09:17 |
| recyclehero | I tested with openstack cli and horizon and I was able to make isntances | 09:17 |
| *** Tony31 has quit IRC | 09:21 | |
| *** kevko_ has joined #openstack-kolla | 09:29 | |
| recyclehero | 2020-10-01 12:59:16.420 6 INFO nova.servicegroup.drivers.db [-] Recovered from being unable to report status. | 09:30 |
| recyclehero | nova-compute.log | 09:30 |
| *** shyam89 has quit IRC | 09:30 | |
| *** shyamb has quit IRC | 09:30 | |
| *** shyamb has joined #openstack-kolla | 09:30 | |
| *** shyam89 has joined #openstack-kolla | 09:30 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add support to kolla_docker for tmpfs mounts https://review.opendev.org/754377 | 09:33 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add support for using a tmpfs for Cinder image conversion https://review.opendev.org/754378 | 09:33 |
| recyclehero | ERROR nova.compute.resource_tracker [req-7950b41d-7e25-4b0a-beea-1522a756ca23 - - - - -] Unable to find services table record for nova-compute host openstackCompute1: nova.exception_Remote.Compute | 09:40 |
| recyclehero | how does the nova-compute on the compute host talks with the control host | 09:40 |
| recyclehero | ? | 09:40 |
| recyclehero | https://dpaste.com/5ULDPLFPQ | 09:43 |
| recyclehero | this is the output of docker ps on the compute node | 09:43 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add support to kolla_docker for tmpfs mounts https://review.opendev.org/754377 | 09:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add support for using a tmpfs for Cinder image conversion https://review.opendev.org/754378 | 09:46 |
| *** priteau has joined #openstack-kolla | 09:48 | |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe master: CI: Add a CentOS 8 overcloud job with TLS enabled https://review.opendev.org/736497 | 09:49 |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Performance: avoid set_fact in Kolla Ansible host vars https://review.opendev.org/745569 | 09:49 |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Bump Ansible version range to 2.9 - 2.9 https://review.opendev.org/754054 | 09:49 |
| *** k_mouza has joined #openstack-kolla | 09:51 | |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Disable ironic in the overcloud by default https://review.opendev.org/755008 | 09:51 |
| *** k_mouza has quit IRC | 09:55 | |
| *** shyam89 has quit IRC | 09:57 | |
| *** shyamb has quit IRC | 09:57 | |
| *** k_mouza has joined #openstack-kolla | 09:59 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Switch to Victoria stable branches https://review.opendev.org/755339 | 10:01 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Fix version-check.py idempotency https://review.opendev.org/755470 | 10:01 |
| *** k_mouza has quit IRC | 10:04 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: When EPEL install flag is not set https://review.opendev.org/755350 | 10:05 |
| *** k_mouza has joined #openstack-kolla | 10:08 | |
| yoctozepto | ^^ sounds like a song title | 10:12 |
| yoctozepto | "When EPEL install flag is not set... lalalala" | 10:13 |
| yoctozepto | :-) | 10:13 |
| openstackgerrit | Fabian Bermel proposed openstack/kolla master: Checks for heat_user_domain explicitly https://review.opendev.org/711923 | 10:13 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services https://review.opendev.org/676389 | 10:15 |
| mnasiadka | yoctozepto: can you reply on that change ^^ - and let's get over it? I don't understand "mixed in", or maybe I'm blind :) | 10:16 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Disable ironic in the overcloud by default https://review.opendev.org/755008 | 10:17 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Disable overcloud networks for Ironic when unused https://review.opendev.org/755015 | 10:17 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: dev: fix test scripts when ironic is disabled https://review.opendev.org/755474 | 10:17 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Add support for keystone auth_mellon config https://review.opendev.org/755475 | 10:24 |
| *** abdysn has joined #openstack-kolla | 10:24 | |
| yoctozepto | mnasiadka: replied | 10:28 |
| *** kevko_ has quit IRC | 10:29 | |
| recyclehero | guys does nova compute communicate with other nova services which are on control host(scheduler, api and ...) over http or message queue? | 10:29 |
| mnasiadka | yoctozepto: yeah well, it's mixed in - in that sense, we needed to somehow to glance health checks - since we already had a stats port in haproxy - I just added it in glance's variant :) | 10:30 |
| mnasiadka | yoctozepto: but I think we can live with that | 10:30 |
| recyclehero | as I enable_haproxy no I see no rabbitmq or any way of communicating from the compute host to the control host other than API | 10:31 |
| recyclehero | https://dpaste.com/5ULDPLFPQ | 10:31 |
| recyclehero | docker ps on compute host | 10:31 |
| yoctozepto | mnasiadka: ok | 10:32 |
| * recyclehero feels ignored | 10:32 | |
| recyclehero | I mean does centos-source-nova-compute:ussuri has its own oslo messaging rabbit mq inside the container? | 10:35 |
| yoctozepto | no, it uses the common one | 10:36 |
| recyclehero | yoctozepto: so each deploy host must have it own rabbitmq container? | 10:37 |
| yoctozepto | no, each controller has | 10:37 |
| recyclehero | can u please see the containers on my compute host and explain how they connect with the controller ? | 10:37 |
| recyclehero | yoctozepto: https://dpaste.com/5ULDPLFPQ | 10:38 |
| yoctozepto | they speak over rabbitmq and apis | 10:38 |
| yoctozepto | it means you should see traffic from nova on controller to nova on compute via rabbitmq | 10:38 |
| yoctozepto | when you e.g. try to launch a new instance | 10:39 |
| recyclehero | when I try to launch a new instance I get this at nova-compute.log from the compute host | 10:39 |
| recyclehero | 2020-10-01 09:02:33.396 6 ERROR nova.servicegroup.drivers.db [-] The services table record for the nova-compute service on host openstackCompute1 is gone. You either need to stop this service if it should be deleted or restart it to recreate the record in the database.: nova.exception_Remote.ServiceNotFound_Remote: Service 4 could not be found. | 10:39 |
| recyclehero | I am wondering enabling_ha proxy and removing network node in favour of provider networks in global.yml could be the cause of my problem | 10:42 |
| recyclehero | enable_haproxy false | 10:42 |
| recyclehero | no | 10:42 |
| recyclehero | I am doing this cuz I dont have vlan | 10:42 |
| openstackgerrit | Merged openstack/kolla master: Debian: use QEMU 5.0 from buster-backports https://review.opendev.org/747534 | 10:46 |
| hrw | yay | 10:59 |
| yoctozepto | recyclehero: it sounds like you did not finish cleaning up | 11:07 |
| recyclehero | cleaning up what? | 11:07 |
| recyclehero | two things that I am chaning I will be thankful if u would consider. disabling ha proxy and removing netowrk node from inventory so I can have a simple flat provider network | 11:09 |
| recyclehero | if these have effect on my situation | 11:09 |
| *** kevko_ has joined #openstack-kolla | 11:09 | |
| yoctozepto | well, they should not | 11:11 |
| recyclehero | https://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration.html says disabling ha proxy has not been tested by kolla community | 11:11 |
| recyclehero | ok | 11:11 |
| recyclehero | I will enable it | 11:12 |
| yoctozepto | the error message is about the entry being removed while it was running | 11:12 |
| yoctozepto | i would try restarting it and seeing what it complains about then | 11:12 |
| recyclehero | restarting docker containers or services? | 11:12 |
| yoctozepto | recyclehero: go to that host and restart nova_compute container | 11:13 |
| yoctozepto | then check its logs | 11:13 |
| recyclehero | aha ok I just restarted the hosts completly | 11:13 |
| recyclehero | another thing, If I dont want tenant network could I have no inventory for [network] | 11:14 |
| *** abdysn has quit IRC | 11:19 | |
| *** abdysn has joined #openstack-kolla | 11:19 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/ussuri: Make no_proxy handling more robust https://review.opendev.org/755441 | 11:28 |
| recyclehero | after restart libvirt log : uable to bund to port | 11:29 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services https://review.opendev.org/676389 | 11:31 |
| r3ap3r | mnasiadka: I forgot to mention, the network diagram I posted yesterday, all of the VLANs can currently talk to eachother. I haven't started restricting access yet. | 11:36 |
| *** hjensas is now known as hjensas|afk | 11:38 | |
| openstackgerrit | Merged openstack/kayobe stable/ussuri: Performance: Parallelise Kolla Ansible host vars generation https://review.opendev.org/754053 | 11:40 |
| openstackgerrit | Merged openstack/kayobe master: Revert "CI: workaround overcloud upgrade issue with ironic" https://review.opendev.org/755437 | 11:43 |
| *** e0ne has joined #openstack-kolla | 11:45 | |
| *** e0ne_ has quit IRC | 11:48 | |
| mnasiadka | r3ap3r: so all mons and osds should be listening on the storage network, I don't really understand why you have that issue :) | 11:50 |
| r3ap3r | mnasiadka: I know right? lol | 11:51 |
| r3ap3r | mnasiadka: it makes total sense to me networking wise but something just isn't happy? | 11:52 |
| mnasiadka | r3ap3r: seems it's not, you would need to run some debug from the glance container | 11:56 |
| mnasiadka | r3ap3r: probably the best is to write a small python script that uses python-rbd to create a volume and list contents | 11:56 |
| mnasiadka | r3ap3r: you can use this guide: https://webcache.googleusercontent.com/search?q=cache:CvyNYhmsvJQJ:https://docs.ceph.com/docs/mimic/rbd/api/librbdpy/+&cd=1&hl=pl&ct=clnk&gl=pl&client=safari | 11:58 |
| *** k_mouza has quit IRC | 12:01 | |
| *** kevko_ has quit IRC | 12:02 | |
| *** k_mouza has joined #openstack-kolla | 12:07 | |
| *** Fl1nt has joined #openstack-kolla | 12:08 | |
| Fl1nt | Hi everyone, I hope you're all doing it good! | 12:08 |
| recyclehero | the nova-compute container (centos source) fails to start when tring to start. no space left when mounting /var/lib/nova/mnt | 12:09 |
| recyclehero | the nfs share is empty and would it mattered anyway | 12:09 |
| *** ozzzo has quit IRC | 12:10 | |
| recyclehero | openstack on docker is either the stupidest idea I have ever seen or it should be the most rewarding one | 12:11 |
| Fl1nt | recyclehero, or you don't get enough knowledge of both yet to understand the underlying mechanism. | 12:12 |
| *** k_mouza has quit IRC | 12:12 | |
| *** kevko_ has joined #openstack-kolla | 12:12 | |
| Fl1nt | it works perfectly on my own and with a large and I mean, A LARGE multicontinental/regions infrastructure and it helped a lot to unclutter all those lifecycles issues. | 12:12 |
| recyclehero | Fl1nt: I dont say I have great knowledge about docker and ansible but I knew and worked my way through openstack | 12:12 |
| recyclehero | Fl1nt: how much time did it take to setup? | 12:13 |
| Fl1nt | 1 day for the Kolla-ansible configuration, 1 day to deploy. | 12:13 |
| Fl1nt | 1 day to get the proper informations about network and hosts. | 12:14 |
| Fl1nt | so 3 days in a row. | 12:14 |
| recyclehero | may I ask whats ur storage and network architeture? | 12:14 |
| recyclehero | looks like | 12:15 |
| Fl1nt | CEPH for the storage, and for our network it's using cumulus network (OVS/VXLAN). | 12:15 |
| recyclehero | and one last thing host machines and ur container choice? | 12:15 |
| Fl1nt | HPE | 12:16 |
| Fl1nt | Computes are AMD | 12:16 |
| Fl1nt | Controllers are INTEL | 12:16 |
| Fl1nt | we've got 3 controllers per regions | 12:16 |
| recyclehero | I meant distro choice but its great to know these 2 | 12:16 |
| Fl1nt | CentOS | 12:17 |
| recyclehero | centos on centos, great | 12:17 |
| priteau | Fl1nt: I thought you advocated using RHEL? | 12:17 |
| Fl1nt | priteau, I try but you know ^^ money ^^ | 12:17 |
| Fl1nt | As we've been acquired, we're trying to see how we can better push RHEL, but it's a contracting hell for now ^^ | 12:19 |
| Fl1nt | recyclehero, are you using ubuntu? | 12:20 |
| *** k_mouza has joined #openstack-kolla | 12:21 | |
| hrw | "openstack on docker is either the stupidest idea" - quote of the week ;D | 12:23 |
| Fl1nt | priteau, btw, working on Cloudkitty right now, is there anything required for the cloudkitty bootstrap container to not fail while trying to do SQL related operations? I mean, it's the first deployment, except for DB creation and population it shouldn't fail right? | 12:23 |
| Fl1nt | hrw, I had my cup of coffee in hands, hopefully I didn't yet started to sip it ^^ | 12:24 |
| priteau | I don't remember seeing it failing, do you have errors in logs? | 12:24 |
| Fl1nt | yep, I'll extract them for sure | 12:24 |
| hrw | recyclehero: remember that you do not have to go kolla way. CentOS, Debian, Ubuntu have binary packages for OpenStack which you can use directly. | 12:24 |
| hrw | recyclehero: you can even go devstack way | 12:25 |
| r3ap3r | mnasiadka: Thanks, I'll give it a shot. :-) | 12:25 |
| *** k_mouza has quit IRC | 12:25 | |
| *** k_mouza has joined #openstack-kolla | 12:31 | |
| *** ozzzo has joined #openstack-kolla | 12:31 | |
| r3ap3r | mnasiadka: if it is an consolation, I have ProxMox connected to the same Ceph server and it works absolutely fine however it is using the `admin` user to connect. That is literally the only difference at this point. This leads me to believe that it may be with the setup I used from here: https://docs.ceph.com/en/latest/rbd/rbd-openstack/ ? I pretty much followed it to the T until I got to the `Configure | 12:36 |
| r3ap3r | Openstack to use Ceph` part. Instead of copying the keyrings to the nodes, I sent them to the kolla-ansible deployed so they can be put into their respective containers. I setup `client.glance`, `client.cinder` and `client.cinder-backup` to start off with, I may work on swift later but trying to keep things simple for the moment. I still have to try "regenerating" the keyrings, putting them in the kolla-ansible | 12:36 |
| r3ap3r | deployer and then `reconfiguring`, I have been super busy with other things so I haven't had a chance to test it out yet. | 12:36 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: dev: fix test scripts when ironic is disabled https://review.opendev.org/755474 | 12:39 |
| *** k_mouza has quit IRC | 12:44 | |
| recyclehero | hrw: Yeah, I am very near that point. this problem with the nova_compute container that wont run after restart came out of no where. so space left on the device when mounting /var/lib/nova/mnt which is my configured nfs | 12:48 |
| openstackgerrit | Merged openstack/kolla-ansible master: Change the default haproxy template to split variant https://review.opendev.org/753369 | 12:49 |
| hrw | recyclehero: I did not know that we support nfs as shared storage | 12:49 |
| Fl1nt | recyclehero, I'll suggest you look at your NFS access rights/policies. | 12:49 |
| hrw | recyclehero: as everyone goes either for all-in-one to test sth or for ceph as storage | 12:50 |
| recyclehero | https://docs.openstack.org/kolla-ansible/ussuri/reference/storage/cinder-guide.html | 12:50 |
| Fl1nt | priteau, here is the log:http://paste.openstack.org/show/YmT893X9G2TNulbKb5vf/ FYI I'm using a ES storage backend and a prometheus collector backend, but as the container fails with the dbsync upgrade, it shouldn't be impacting it. | 12:51 |
| *** k_mouza has joined #openstack-kolla | 12:53 | |
| *** k_mouza has quit IRC | 12:58 | |
| *** cah_link has quit IRC | 13:02 | |
| *** k_mouza has joined #openstack-kolla | 13:03 | |
| *** k_mouza has quit IRC | 13:05 | |
| *** k_mouza has joined #openstack-kolla | 13:05 | |
| *** wuchunyang has joined #openstack-kolla | 13:07 | |
| *** cah_link has joined #openstack-kolla | 13:09 | |
| *** ricolin_ has joined #openstack-kolla | 13:11 | |
| *** shyam89 has joined #openstack-kolla | 13:18 | |
| *** shyamb has joined #openstack-kolla | 13:18 | |
| *** sean-k-mooney1 is now known as sean-k-mooney | 13:19 | |
| priteau | Fl1nt: Check under /var/log/kolla/cloudkitty | 13:20 |
| priteau | Looking at your paste, it might be the cloudkitty-storage-init command failing | 13:20 |
| priteau | which *is* storage-related | 13:20 |
| *** ricolin_ has quit IRC | 13:21 | |
| Fl1nt | yes, I've seen what's going on. | 13:21 |
| Fl1nt | it's the elastisearch index which is missing, I thought (wrongly) that storage-init would have created if not found out as it's the default and whatever required. | 13:22 |
| Fl1nt | I'll so have to create a task about that on the role too ^^ | 13:23 |
| *** shyam89 has quit IRC | 13:53 | |
| *** shyamb has quit IRC | 13:53 | |
| *** shyamb has joined #openstack-kolla | 13:55 | |
| *** shyam89 has joined #openstack-kolla | 13:55 | |
| openstackgerrit | Merged openstack/kayobe master: CI: Add a CentOS 8 overcloud job with TLS enabled https://review.opendev.org/736497 | 13:55 |
| *** Luzi has joined #openstack-kolla | 14:00 | |
| *** fbermel has quit IRC | 14:03 | |
| *** abdysn has quit IRC | 14:04 | |
| *** cah_link has quit IRC | 14:07 | |
| Fl1nt | hum... can someone explain me what is the value added by the ansible kolla-toolbox library over using the docker one and give him the command to execute? | 14:07 |
| recyclehero | why I havent found openstack-ansible. I hate kolla ansible not you guys though. you are great. Bye | 14:07 |
| Fl1nt | lol | 14:07 |
| Fl1nt | ok ^^ see ya recyclehero | 14:07 |
| *** TrevorV has joined #openstack-kolla | 14:16 | |
| *** recyclehero has left #openstack-kolla | 14:18 | |
| *** shyam89 has quit IRC | 14:24 | |
| *** shyamb has quit IRC | 14:24 | |
| openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Disable ironic in the overcloud by default https://review.opendev.org/755008 | 14:28 |
| openstackgerrit | Merged openstack/kayobe master: Performance: avoid set_fact in Kolla Ansible host vars https://review.opendev.org/745569 | 14:28 |
| *** ricolin_ has joined #openstack-kolla | 14:31 | |
| r3ap3r | Wow? Spend about a week here and then leaves because he "hate kolla ansible"? I wonder how far he got into the Openstack-Ansible docs before determining he liked it better? | 14:34 |
| r3ap3r | Maybe I missed something though? | 14:34 |
| *** shyamb has joined #openstack-kolla | 14:37 | |
| *** shyam89 has joined #openstack-kolla | 14:37 | |
| *** ricolin has quit IRC | 14:37 | |
| Fl1nt | nevermind r3ap3r it's just a raging troll. | 14:37 |
| r3ap3r | Fl1nt: meh. I pay people like that no attention and generally I can spot them relatively quickly. Maybe it is because of the week I'm having? lol | 14:39 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe stable/ussuri: Performance: avoid set_fact in Kolla Ansible host vars https://review.opendev.org/755548 | 14:41 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: generate certificates automatically https://review.opendev.org/754280 | 14:49 |
| Fl1nt | r3ap3r, or because he used a mix of passive/aggressive questions/complaints ^^ anyway, k-a people are doing a great job and except for few constraints because of Ansible stability, it is rock solid and useful, don't worry. | 14:51 |
| r3ap3r | Fl1nt: not concerned at all! lol Only person I may be a little concerned for is hrw when he goes on his rants. May be worth investing in some blood pressure medicine? lol | 14:52 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: generate certificates automatically https://review.opendev.org/754280 | 14:54 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: generate certificates automatically https://review.opendev.org/754280 | 14:54 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services https://review.opendev.org/676389 | 14:55 |
| yoctozepto | Fl1nt: thanks | 14:56 |
| yoctozepto | mgoddard, mnasiadka, hrw: are we having kall today? or skipping? we failed to establish its agenda... | 14:56 |
| mgoddard | yoctozepto: good question | 14:57 |
| mgoddard | I would probably rather use the time for reviewing etc | 14:57 |
| yoctozepto | mgoddard: today I'm having a day of good questions | 14:57 |
| yoctozepto | ;d | 14:57 |
| yoctozepto | mgoddard: me too | 14:57 |
| mgoddard | see if I can get octavia over the line | 14:57 |
| yoctozepto | duh | 14:57 |
| mnasiadka | yoctozepto: did we fix the docs which we agreed to fix? :) | 14:57 |
| * yoctozepto feels review pain | 14:57 | |
| yoctozepto | mnasiadka: not quite | 14:57 |
| yoctozepto | but the priority dropped considerably :-) | 14:58 |
| *** hjensas|afk has quit IRC | 14:58 | |
| yoctozepto | I'll open up the channel to catch potential attendees | 14:58 |
| yoctozepto | but otherwise announce non-kall today | 14:58 |
| yoctozepto | priteau: kall cancelled, I'm writing to ml now :/ | 15:01 |
| mnasiadka | yoctozepto: I can sweep through the kall etherpad and try to do the changes we agreed to in the next days | 15:01 |
| priteau | I just joined because my calendar told me too ;-) | 15:01 |
| hrw | I just ended an hour long call | 15:05 |
| yoctozepto | priteau: hah :D | 15:06 |
| yoctozepto | hrw: welcome in the club :D | 15:06 |
| hrw | yoctozepto: it was good call ;D | 15:06 |
| yoctozepto | hrw: mine fine as well, just tired because there was another one before that ;d | 15:07 |
| *** shyam89 has quit IRC | 15:07 | |
| *** shyamb has quit IRC | 15:07 | |
| yoctozepto | mnasiadka: you can review/repair my changes to docs in kolla | 15:07 |
| mnasiadka | yoctozepto: sure | 15:07 |
| yoctozepto | mnasiadka: https://review.opendev.org/752512 | 15:07 |
| patchbot | patch 752512 - kolla - Improve docs - 1 patch set | 15:07 |
| mgoddard | anyone want to review octavia? | 15:08 |
| yoctozepto | though octavia sounds like a priority | 15:08 |
| yoctozepto | mgoddard: yeah, I opened it... | 15:08 |
| yoctozepto | :D | 15:08 |
| mnasiadka | mgoddard: which one? I complained in one change already :) | 15:08 |
| mgoddard | mnasiadka: so you did | 15:09 |
| mgoddard | I replied, you didn't :) | 15:09 |
| yoctozepto | headphoneJames: I'll sit to neutron a bit more tomorrow; today I established it might actually be a bug in the wsgi module because the worlds seems to not run neutron with mod_wsgi :D | 15:09 |
| yoctozepto | it could be that it needs one line change as well | 15:09 |
| yoctozepto | but did not find more time to dig deeper | 15:09 |
| yoctozepto | mgoddard: lol | 15:10 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: add precheck for octavia https://review.opendev.org/746002 | 15:10 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/754285 | 15:10 |
| *** Luzi has quit IRC | 15:15 | |
| mnasiadka | mgoddard: replied :) | 15:16 |
| mgoddard | thanks | 15:17 |
| *** ricolin_ has quit IRC | 15:17 | |
| mnasiadka | yoctozepto: Well, question maybe to Neutrinos why the world seems to not run it with mod_wsgi :) | 15:18 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: add releasenote for octavia configuration https://review.opendev.org/746404 | 15:22 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: add octavia openrc file https://review.opendev.org/745997 | 15:26 |
| yoctozepto | mnasiadka: tried that on irc, might redo on ml | 15:26 |
| mgoddard | mnasiadka: I don't mind too much re tenant vs. provider. | 15:31 |
| *** e0ne has quit IRC | 15:32 | |
| mgoddard | maybe yoctozepto has an opinion | 15:32 |
| hrw | provider is simple english | 15:35 |
| hrw | if I have to choose word between provided and tenant then I choose provider. | 15:36 |
| hrw | for tenant I need to look into dictionary ;D | 15:36 |
| yoctozepto | mgoddard: I don't mind it either, but using on-host systemd and dhclient looks bad | 15:38 |
| yoctozepto | mnasiadka: ^ | 15:38 |
| yoctozepto | hrw: it's a different option, not synonymous | 15:38 |
| yoctozepto | hrw: it's a type of neutron network | 15:38 |
| yoctozepto | and here for octavia | 15:38 |
| hrw | ok | 15:38 |
| * hrw off | 15:38 | |
| yoctozepto | mgoddard: qq - one thing I missed | 15:38 |
| yoctozepto | mgoddard: where is dhcp server coming from? | 15:39 |
| mgoddard | yoctozepto: neutron | 15:39 |
| yoctozepto | mgoddard: I mean - it calls dhclient | 15:39 |
| yoctozepto | mgoddard: ah, so it sets up the network with it | 15:39 |
| mgoddard | a port is created for each controller | 15:39 |
| yoctozepto | I missed it | 15:39 |
| yoctozepto | mgoddard: thanks | 15:39 |
| mgoddard | ok, seems that we are swaying towards provider | 15:39 |
| mgoddard | mnasiadka: does it look ok otherwise to you? | 15:40 |
| yoctozepto | I wouldn't let tenant in for the time being to be really honest | 15:40 |
| yoctozepto | I left a bunch of comments as well | 15:40 |
| mgoddard | ok | 15:40 |
| yoctozepto | feeling sorry for reviewing this late | 15:40 |
| yoctozepto | it wasn't that bad :-) | 15:40 |
| yoctozepto | wuchunyang: great job with octavia there | 15:40 |
| yoctozepto | very close | 15:40 |
| mgoddard | the nice thing about the tenant mode is we can test in CI easily | 15:42 |
| mgoddard | shall I pull out tenant mode then? | 15:44 |
| yoctozepto | gimme a sec more, doing more sophisticated review in a local editor | 15:45 |
| yoctozepto | mgoddard: I think we can easily to cater for the provider variant in CI, no? | 15:48 |
| yoctozepto | mgoddard: what do you think is missing? | 15:48 |
| yoctozepto | easily cater* | 15:49 |
| yoctozepto | duh | 15:49 |
| * yoctozepto burnt out after the work day | 15:49 | |
| yoctozepto | mgoddard: you working on splitting tenant out? | 15:53 |
| mgoddard | yoctozepto: it's possible, just a bit more faff to create a VLAN interface etc | 15:54 |
| mgoddard | I'm not splitting yet. Waiting for go/no go | 15:55 |
| yoctozepto | mgoddard: does it have to be a vlan? | 15:55 |
| yoctozepto | i.e. flat is not worky? | 15:55 |
| mgoddard | well could be flat | 15:55 |
| yoctozepto | :-) | 15:55 |
| yoctozepto | we got flat worky | 15:55 |
| mgoddard | well, that would have to be the external network | 15:55 |
| mnasiadka | mgoddard: well, I just feel any non-standard method of creating ports based on OVS will bite us sooner or later - I would prefer it to be a ,,preview'' - I just know that in any case that OVS has problems working - that systemd units stands up - the user sees it as running, but octavia does not work :) | 15:59 |
| mnasiadka | I know that tripleo uses that approach, but they have OVS running on OS level, not in a container. | 15:59 |
| mgoddard | yeah, I guess there could be issues there | 16:00 |
| mnasiadka | mgoddard: with kayobe approach of a bridge and veth pair - I don't see an issue, but with current approach in change - I'm just worried we'll be trying to fix bugs in something, that we can't really fix, but work around and add a couple of checks here and there | 16:00 |
| mnasiadka | maybe we could survive with removing network-online dependency from Docker, but I don't know :) | 16:01 |
| *** bengates has quit IRC | 16:02 | |
| *** bengates has joined #openstack-kolla | 16:03 | |
| *** ozzzo is now known as ozzzo2 | 16:03 | |
| yoctozepto | mnasiadka: https://review.opendev.org/753380 | 16:03 |
| patchbot | patch 753380 - kolla-ansible - Add support for with_frontend and with_backend - 2 patch sets | 16:03 |
| yoctozepto | mgoddard, mnasiadka: let's split out tenant out - mgoddard if you can that would be awesome - then let's amend the other little rough edges/issues it has - I feel we might build a "framework" we can reuse for designate too | 16:04 |
| yoctozepto | looks promising | 16:04 |
| mgoddard | ok, doing it | 16:05 |
| yoctozepto | (I mean framework for the "chicken and egg"-ish problem) | 16:05 |
| yoctozepto | mgoddard: <3 | 16:05 |
| yoctozepto | and another for mnasiadka: https://review.opendev.org/754568 | 16:07 |
| patchbot | patch 754568 - kolla-ansible - Add support for ACME http-01 challenge - 1 patch set | 16:07 |
| jobewan | Are there any plans to create a "kolla_podman" module in lieu of kolla_docker going forward? | 16:07 |
| *** bengates has quit IRC | 16:08 | |
| yoctozepto | jobewan: yeah but they don't have much support, i.e. there is no great gain from using podman | 16:11 |
| yoctozepto | and looking forward it would be quite a pita to get docker->podman upgrade right | 16:11 |
| yoctozepto | team crossing their fingers for docker not dropping out ;d | 16:11 |
| yoctozepto | but yeah, we have in the backs of our heads | 16:12 |
| r3ap3r | Docker vs K8's debates get heated in my experience. Our guys tend to be torn but deploy both Zun and Magnum so our "customers" have the option of using their preferences. | 16:14 |
| yoctozepto | always depends on what one needs | 16:16 |
| r3ap3r | True. ^ | 16:17 |
| * yoctozepto - the author of truisms | 16:17 | |
| r3ap3r | Haha | 16:17 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Implement automatic deploy of octavia https://review.opendev.org/740180 | 16:30 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: support tenant management network https://review.opendev.org/755589 | 16:30 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: support tenant management network https://review.opendev.org/755589 | 16:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: generate certificates automatically https://review.opendev.org/754280 | 16:34 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/754285 | 16:34 |
| *** k_mouza has quit IRC | 16:45 | |
| *** skramaja has quit IRC | 16:46 | |
| *** ozzzo2 has quit IRC | 16:46 | |
| *** ozzzo has joined #openstack-kolla | 16:58 | |
| *** k_mouza has joined #openstack-kolla | 17:00 | |
| yoctozepto | mgoddard: left a final comment for today | 17:04 |
| *** k_mouza has quit IRC | 17:05 | |
| *** wuchunyang has quit IRC | 17:06 | |
| *** k_mouza has joined #openstack-kolla | 17:16 | |
| *** k_mouza has quit IRC | 17:21 | |
| *** k_mouza has joined #openstack-kolla | 17:24 | |
| *** k_mouza has quit IRC | 17:28 | |
| *** k_mouza has joined #openstack-kolla | 17:41 | |
| *** k_mouza has quit IRC | 17:45 | |
| *** k_mouza has joined #openstack-kolla | 18:01 | |
| *** k_mouza has quit IRC | 18:05 | |
| mgoddard | yoctozepto has the final word | 18:09 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: support tenant management network https://review.opendev.org/755589 | 18:17 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: support tenant management network https://review.opendev.org/755589 | 18:19 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/754285 | 18:19 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: support tenant management network https://review.opendev.org/755589 | 18:24 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/754285 | 18:25 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Use Docker healthchecks for core services https://review.opendev.org/676389 | 18:26 |
| *** k_mouza has joined #openstack-kolla | 18:28 | |
| *** k_mouza has quit IRC | 18:33 | |
| *** k_mouza has joined #openstack-kolla | 18:52 | |
| *** k_mouza has quit IRC | 18:57 | |
| *** maharg101 has quit IRC | 19:02 | |
| *** priteau has quit IRC | 19:11 | |
| *** k_mouza has joined #openstack-kolla | 19:26 | |
| *** k_mouza has quit IRC | 19:31 | |
| *** k_mouza has joined #openstack-kolla | 19:46 | |
| *** k_mouza has quit IRC | 19:50 | |
| *** gfidente has quit IRC | 19:57 | |
| *** maharg101 has joined #openstack-kolla | 19:59 | |
| openstackgerrit | Merged openstack/kolla master: Fix Neutron migrations order https://review.opendev.org/755346 | 20:01 |
| *** maharg101 has quit IRC | 20:07 | |
| *** k_mouza has joined #openstack-kolla | 20:09 | |
| *** k_mouza has quit IRC | 20:14 | |
| openstackgerrit | Merged openstack/kayobe master: When EPEL install flag is not set https://review.opendev.org/755350 | 20:34 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: octavia: support tenant management network https://review.opendev.org/755589 | 20:39 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/754285 | 20:39 |
| *** stingrayza has joined #openstack-kolla | 21:05 | |
| *** also_stingrayza has quit IRC | 21:09 | |
| *** mgoddard has quit IRC | 21:30 | |
| *** Fl1nt has quit IRC | 21:33 | |
| *** mgoddard has joined #openstack-kolla | 21:36 | |
| *** k_mouza has joined #openstack-kolla | 22:14 | |
| *** k_mouza has quit IRC | 22:18 | |
| *** jonaspaulo has quit IRC | 22:29 | |
| *** k_mouza has joined #openstack-kolla | 22:51 | |
| *** k_mouza has quit IRC | 22:55 | |
| *** TrevorV has quit IRC | 23:02 | |
| *** k_mouza has joined #openstack-kolla | 23:16 | |
| *** k_mouza has quit IRC | 23:20 | |
| *** k_mouza has joined #openstack-kolla | 23:36 | |
| *** k_mouza has quit IRC | 23:40 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!