Thursday, 2021-02-25

« Wednesday, 2021-02-24 Index Friday, 2021-02-26 »
*** k_mouza has joined #openstack-kolla00:02
*** k_mouza has quit IRC00:07
*** LinPeiWen has joined #openstack-kolla00:33
*** bengates has joined #openstack-kolla00:50
*** bengates has quit IRC00:56
*** xinliang has joined #openstack-kolla01:04
*** Underknowledge has quit IRC01:23
*** Underknowledge has joined #openstack-kolla01:24
*** ricolin_ has joined #openstack-kolla01:55
*** LinPeiWen has quit IRC02:08
*** ricolin_ has quit IRC02:09
*** zzzeek has quit IRC02:23
*** zzzeek has joined #openstack-kolla02:24
*** bengates has joined #openstack-kolla02:32
*** bengates has quit IRC02:38
*** zzzeek has quit IRC02:44
*** zzzeek has joined #openstack-kolla02:46
*** zzzeek has quit IRC02:51
*** zzzeek has joined #openstack-kolla02:53
*** zzzeek has quit IRC03:13
*** zzzeek has joined #openstack-kolla03:15
*** LinPeiWen has joined #openstack-kolla03:21
*** bengates has joined #openstack-kolla03:43
*** bengates has quit IRC03:49
*** xinliang has quit IRC04:32
*** cah_link has joined #openstack-kolla05:14
*** yoctozepto0 has joined #openstack-kolla06:04
*** yoctozepto has quit IRC06:05
*** yoctozepto0 is now known as yoctozepto06:05
*** luksky has joined #openstack-kolla06:13
oklhostmorning06:41
*** skramaja has joined #openstack-kolla06:44
*** dasTor has joined #openstack-kolla07:21
*** wuchunyang has joined #openstack-kolla07:46
hrwelo07:55
*** rpittau|afk is now known as rpittau07:57
*** eliaswimmer has joined #openstack-kolla08:01
*** gfidente|afk is now known as gfidente08:02
*** brinzhang has quit IRC08:05
*** brinzhang has joined #openstack-kolla08:07
*** brinzhang has quit IRC08:07
*** brinzhang has joined #openstack-kolla08:07
*** bengates has joined #openstack-kolla08:08
*** brinzhang has quit IRC08:08
*** brinzhang has joined #openstack-kolla08:09
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer  https://review.opendev.org/c/openstack/kolla-ansible/+/75428508:10
*** bengates has quit IRC08:11
*** bengates has joined #openstack-kolla08:12
*** andrewbonney has joined #openstack-kolla08:13
*** amoralej|off is now known as amoralej08:15
*** maharg101 has joined #openstack-kolla08:28
*** bengates has quit IRC08:28
*** bengates has joined #openstack-kolla08:29
*** cah_link has quit IRC08:31
*** jaicaa has quit IRC08:44
*** jaicaa has joined #openstack-kolla08:47
*** happyhemant has joined #openstack-kolla08:49
*** cah_link has joined #openstack-kolla09:02
openstackgerritMark Goddard proposed openstack/kolla-ansible master: docs: update service configuration  https://review.opendev.org/c/openstack/kolla-ansible/+/77751609:05
*** ewimmer_ has joined #openstack-kolla09:05
*** eliaswimmer has quit IRC09:08
mgoddardmorning09:08
mgoddardhrw: I just spotted this: https://zuul-ci.org/docs/zuul/reference/jobs.html#leaving-warnings09:08
mgoddardhrw: maybe we could use it to leave a warning when there are allowed to fail images?09:09
openstackgerritMark Goddard proposed openstack/kayobe master: DNM: retry on vexxhost  https://review.opendev.org/c/openstack/kayobe/+/77751809:24
openstackgerritMark Goddard proposed openstack/kayobe master: Ubuntu: update Apt cache before package installation  https://review.opendev.org/c/openstack/kayobe/+/77513009:27
hrwmgoddard: interesting09:28
*** e0ne has joined #openstack-kolla09:30
*** k_mouza has joined #openstack-kolla09:31
*** Underknowledge1 has joined #openstack-kolla09:37
*** Underknowledge has quit IRC09:37
*** Underknowledge1 is now known as Underknowledge09:37
*** yoctozepto9 has joined #openstack-kolla09:55
*** gfidente has joined #openstack-kolla09:59
*** hjensas_ has joined #openstack-kolla09:59
openstackgerritMark Goddard proposed openstack/kayobe master: CI: retry overcloud jobs that land on vexxhost clouds  https://review.opendev.org/c/openstack/kayobe/+/77751810:01
*** yoctozepto has quit IRC10:03
*** zzzeek has quit IRC10:03
*** hjensas has quit IRC10:03
*** irclogbot_2 has quit IRC10:03
*** yoctozepto9 is now known as yoctozepto10:03
*** zzzeek has joined #openstack-kolla10:05
*** wuchunyang has quit IRC10:06
*** irclogbot_3 has joined #openstack-kolla10:07
openstackgerritMark Goddard proposed openstack/kayobe master: DNM: try CentOS DIB IPA on vexxhost  https://review.opendev.org/c/openstack/kayobe/+/77755810:14
openstackgerritMark Goddard proposed openstack/kayobe master: CI: retry overcloud jobs that land on vexxhost clouds  https://review.opendev.org/c/openstack/kayobe/+/77751810:21
*** k_mouza has quit IRC10:21
openstackgerritMark Goddard proposed openstack/kayobe master: DNM: try CentOS DIB IPA on vexxhost  https://review.opendev.org/c/openstack/kayobe/+/77755810:21
*** dougsz has joined #openstack-kolla10:33
*** hjensas_ is now known as hjensas10:43
*** dougsz has quit IRC10:52
*** k_mouza has joined #openstack-kolla10:54
*** k_mouza has quit IRC10:58
*** k_mouza has joined #openstack-kolla10:58
*** kevko_ has joined #openstack-kolla11:26
*** kevko has joined #openstack-kolla11:26
*** kevko_ has quit IRC11:27
openstackgerritMark Goddard proposed openstack/kayobe master: DNM: try CentOS DIB IPA on vexxhost  https://review.opendev.org/c/openstack/kayobe/+/77755811:48
openstackgerritMark Goddard proposed openstack/kayobe master: CI: retry overcloud jobs that land on vexxhost clouds  https://review.opendev.org/c/openstack/kayobe/+/77751811:51
openstackgerritMark Goddard proposed openstack/kayobe master: CI: retry overcloud jobs that land on vexxhost clouds  https://review.opendev.org/c/openstack/kayobe/+/77751811:53
*** ewimmer__ has joined #openstack-kolla12:24
openstackgerritMark Goddard proposed openstack/kayobe master: DNM: try CentOS DIB IPA on vexxhost  https://review.opendev.org/c/openstack/kayobe/+/77755812:26
*** ewimmer_ has quit IRC12:26
*** wuchunyang has joined #openstack-kolla12:46
*** ewimmer_ has joined #openstack-kolla12:57
*** ewimmer__ has quit IRC13:00
*** Luzi has joined #openstack-kolla13:01
*** amoralej is now known as amoralej|lunch13:12
*** bengates has quit IRC13:14
*** bengates has joined #openstack-kolla13:20
*** cah_link has quit IRC13:20
*** LinPeiWen has quit IRC13:27
*** skramaja has quit IRC13:42
mgoddardpriteau, mnasiadka: https://review.opendev.org/c/openstack/kayobe/+/777518/13:43
priteauI was waiting for Zuul13:43
priteauAnecdotally, it looks like we're getting pull request limits mostly in limestone13:43
openstackgerritMark Goddard proposed openstack/kayobe-config-dev master: Reduce ironic minimum required memory to 256MB  https://review.opendev.org/c/openstack/kayobe-config-dev/+/77707613:44
mgoddardpriteau: we're running out of CI cloud providers...13:45
mgoddardpriteau: zuul has spoken13:46
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: Add tags for common task  https://review.opendev.org/c/openstack/kolla-ansible/+/77759213:49
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: Add tags for common tasks  https://review.opendev.org/c/openstack/kolla-ansible/+/77759213:49
*** k_mouza has quit IRC13:54
*** k_mouza has joined #openstack-kolla13:55
*** k_mouza has quit IRC14:00
*** k_mouza has joined #openstack-kolla14:00
*** yoctozepto has quit IRC14:01
openstackgerritBartosz Bezak proposed openstack/kolla-ansible master: don't use the same CIDR in octavia_amp_network_cidr and init-run-once.  https://review.opendev.org/c/openstack/kolla-ansible/+/77759514:01
*** yoctozepto has joined #openstack-kolla14:01
openstackgerritBartosz Bezak proposed openstack/kolla-ansible master: don't use the same CIDR in octavia_amp_network_cidr and init-run-once  https://review.opendev.org/c/openstack/kolla-ansible/+/77759514:07
*** amoralej|lunch is now known as amoralej14:14
*** Luzi has quit IRC14:28
*** cah_link has joined #openstack-kolla14:28
openstackgerritMark Goddard proposed openstack/kayobe master: DNM: try CentOS DIB IPA on vexxhost  https://review.opendev.org/c/openstack/kayobe/+/77755814:30
*** zzzeek has quit IRC14:51
*** zzzeek has joined #openstack-kolla14:53
yoctozeptomnasiadka: cinder/ceph active-active got blocked on me14:54
mnasiadkayoctozepto: well, I can take over, if you're willing to share what you've done :)14:55
yoctozeptothere was something on the whiteboard14:55
yoctozeptolemme check14:55
yoctozeptohttps://etherpad.opendev.org/p/KollaWhiteBoard14:55
yoctozeptoL6514:55
yoctozeptoit's even your change that is mentioned there14:56
kevkoyoctozepto: fixed keepalived restart \o/ \o/14:56
yoctozeptoso feel free to improve yerself! :D14:56
kevkojupiii14:56
yoctozeptokevko: nice14:56
yoctozeptowhat was wrong then?14:56
kevkoyoctozepto: net.ipv4.tcp_retries2   <<<14:56
kevkokernel parameter which is set to 15 by default14:57
yoctozeptohmm, interesting14:57
kevkoyoctozepto: https://blog.cloudflare.com/when-tcp-sockets-refuse-to-die/14:57
yoctozeptolol, nice title xD14:57
kevkoyoctozepto: in case of keepalived switch vip (restart keepalived, or networking restart on host)14:57
kevkoyoctozepto: there is connections to DB which refusing to die ..14:57
kevkoyoctozepto: if you restart keystone's apache OR kill process ..everything is fixed   OR let it as it is and it will fixed in around 940 seconds14:59
mnasiadkayoctozepto: ok then, let me mess with this now :)14:59
mnasiadkakevko: restarting haproxy does not help?14:59
kevkoyoctozepto: when i was measuring this timeout it was about 935 - 941 seconds ..so i was googling and googling ..and this definitively helped14:59
kevkomnasiadka: nope, it didn't help ..and logically it can't help15:00
kevkomnasiadka: in keystone there is already broken connection ..you cannot send RST to it ..15:00
mnasiadkakevko: well then, maybe oslo.db/sqlalchemy should have a better way of keepalives/monitoring the connection?15:02
kevkomnasiadka: this is not issue on library layer i think ..there is regular tcp connection ..sqlalchemy/oslo.db think it is OK ...just waiting for reply ..but it never comes15:04
mnasiadkawell, so they shouldn't wait for reply forever and recycle the connection?15:04
*** e0ne has quit IRC15:06
kevkomnasiadka: probably yes, I really don't know why this is happening ..15:06
kevkomnasiadka: I am open to give you ssh key to my env to test it :)15:07
kevkogoing to cigarette ..but w8 few minutes .. i really want to discuss this for a while ...15:07
mnasiadkakevko: I've seen that happenning on multiple environments, would be good to fix, but I don't really have time to investigate - but I think sqlalchemy should work as I mentioned, raise a bug to oslo.db :)15:08
*** imtiazc has joined #openstack-kolla15:15
kevkomnasiadka: well, now i am going to strace apache wsgidaemon and check what timeout is socket open with .. I will be more clever  after15:23
*** stingrayza has quit IRC15:30
*** stingrayza has joined #openstack-kolla15:33
kevkomnasiadka: redhat also suggesting to reduce this kernel param to 315:45
kevkomnasiadka: https://access.redhat.com/solutions/726753    << in HA solutions15:45
kevkomnasiadka https://pracucci.com/linux-tcp-rto-min-max-and-tcp-retries2.html15:46
*** diurnalist has joined #openstack-kolla16:03
openstackgerritwu.chunyang proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer  https://review.opendev.org/c/openstack/kolla-ansible/+/75428516:16
*** wuchunyang has quit IRC16:28
kevkoyoctozepto: hmm, do you mean that my patches for proxysql will be in wallaby as march is behind the door as feature freeze also ..16:35
kevko?16:35
atmarkhello, I'm trying override the haproxy services config for keystone and horizon  but the config doesn't get copied to the controller node16:39
mnasiadkamgoddard, yoctozepto: I'm starting to be fed up with the number of jobs that end up with DISK_FULL16:47
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762316:53
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762316:57
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762317:05
kevkoatmark: how did you try to do it ?17:08
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762317:11
*** cah_link has quit IRC17:16
*** bengates has quit IRC17:17
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762317:21
mnasiadkayoctozepto, mgoddard: Well, found why - we produce 6300 MB of logs on the executor (and the limit is 5000)17:26
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762317:30
*** rpittau is now known as rpittau|afk17:34
*** amoralej is now known as amoralej|off17:38
*** maharg101 has quit IRC17:38
*** diurnalist has quit IRC17:43
*** bengates has joined #openstack-kolla17:47
*** bengates has quit IRC17:52
*** jonaspaulo has joined #openstack-kolla17:56
*** luksky has quit IRC18:01
*** gfidente is now known as gfidente|afk18:01
*** luksky has joined #openstack-kolla18:01
*** wuchunyang has joined #openstack-kolla18:03
*** k_mouza has quit IRC18:05
*** wuchunyang has quit IRC18:07
*** happyhemant has quit IRC18:08
*** amoralej|off is now known as amoralej18:13
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Reduce number of logs and disable ara HTML report  https://review.opendev.org/c/openstack/kolla-ansible/+/77764718:20
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm  https://review.opendev.org/c/openstack/kolla-ansible/+/77141818:21
mnasiadkapriteau: actually your DNM patch makes sense to merge, if we could fail in pre when the limit is too small.18:23
*** bengates has joined #openstack-kolla18:28
*** ewimmer__ has joined #openstack-kolla18:31
*** bengates has quit IRC18:34
yoctozeptomnasiadka: lol18:34
yoctozeptomnasiadka: how did you find that?18:34
yoctozeptoI was curious too18:34
yoctozeptoonly observed18:34
*** ewimmer_ has quit IRC18:34
yoctozeptoit was multinode ubuntu jobs18:34
yoctozeptothat failed like crazy18:34
mnasiadkaWell, I went to ask infra what the hell18:35
mnasiadkaI have a WIP change to remove ara html and stop double copying /var/log/kolla18:36
mnasiadkaThat should give us 250MB less on multinode18:36
mnasiadkaBut we need to find out what happens on Ubuntu :)18:36
*** wuchunyang has joined #openstack-kolla18:41
priteaumnasiadka: It's DNM until I get the code right (had some issues with curl not behaving the same way in Zuul than locally)18:43
priteauBut I am wondering if the rate limit is actually on the mirror itself18:43
priteauIn which case, if it happens reliably on limestone, we could stop using its mirror18:43
mnasiadkapriteau: ah, if we use registry mirror, then yes - but maybe it’s possible to connect to docker api via infra proxy18:44
*** wuchunyang has quit IRC18:45
*** andrewbonney has quit IRC18:49
yoctozeptokevko: it's coming in wallaby I think18:50
yoctozeptowe have time till the end of March18:50
*** kevko has quit IRC18:50
yoctozeptowe are cycle-trailing18:50
yoctozeptopriteau, mnasiadka: one issue is obviously that the rate limit is shared by more nodes so the estimate is not directly reliable18:52
yoctozeptowell, is = might be18:52
yoctozeptomnasiadka: how did you compute 250?18:53
openstackgerritPierre Riteau proposed openstack/kayobe master: [DNM] Test Docker pull rate limits  https://review.opendev.org/c/openstack/kayobe/+/77762318:53
yoctozeptomnasiadka: hmm, so we generate 6300 MB of logs that we actually *transfer*?18:53
mnasiadkayoctozepto: around 120M of ara report and 50M of fluentd logs in two places per node :)18:54
yoctozeptoI think we are dereferencing links18:54
yoctozeptoso perhaps we start copying parts of the system18:54
yoctozeptobecause Ubuntu started having a weird symlink18:55
yoctozeptothat might make sense18:55
mnasiadkaThat might be the case, that’s why I added du in post to check18:55
yoctozeptodu-du-du-du18:56
yoctozeptoah, I see your change now18:56
yoctozeptomnasiadka: 2021-02-25 18:44:30.517421 | primary | du: cannot access '/var/lib/zuul/builds/83c2430160cb46f7984355193d837f8e/work/logs': No such file or directory18:59
yoctozeptooopsie :D18:59
mnasiadkaLol18:59
yoctozeptothat's because it does not exist yet18:59
yoctozeptoas we are creating it for ara19:00
yoctozeptowere*19:00
mnasiadkaArh19:00
yoctozeptoand you dropped that19:00
yoctozeptoand copying happens later19:00
mnasiadkaWill look tomorrow into that :)19:00
yoctozeptoah, no19:01
yoctozeptoI see what happened19:01
yoctozeptowill fix19:02
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: WIP: Reduce number of logs and disable ara HTML report  https://review.opendev.org/c/openstack/kolla-ansible/+/77764719:09
*** alexandreperreau has joined #openstack-kolla19:13
openstackgerritMaksim Malchuk proposed openstack/kolla-ansible master: Correctly configure S3 Token Middleware for Swift  https://review.opendev.org/c/openstack/kolla-ansible/+/77479119:21
openstackgerritMaksim Malchuk proposed openstack/kolla-ansible master: [DNM] test swift deploy 2  https://review.opendev.org/c/openstack/kolla-ansible/+/77765619:25
openstackgerritMaksim Malchuk proposed openstack/kolla-ansible master: [DNM] test swift deploy 1  https://review.opendev.org/c/openstack/kolla-ansible/+/77765619:26
*** zzzeek has quit IRC19:29
*** zzzeek has joined #openstack-kolla19:29
alexandreperreauHi19:30
alexandreperreauI am wondering if someone can help me with the following. We have deployed a multi-node environment with kolla-ansible and it's been in production for awhile (over a year). Recently one of the three controllers has gone down (controller003) and we cannot recover it. I am not sure how to proceed. Can we use a new server and use kolla-ansible to19:30
alexandreperreaudeploy controller003 again as if we are adding a new node? I'm afraid that there are existing dependencies with the broken controller which will affect services like mariadb, rabbit, etc... when adding a new controller. I want to make sure there is no impact especially to the DB. Can I use the same hostname and IPs. Or should I add controller004 to19:30
alexandreperreauinventory to start fresh but then maybe there will still be traces of the old controller. Since we are kind of replacing a controller, I feel that the situation is different than adding a new controller to the platform. Thank you.19:30
*** maharg101 has joined #openstack-kolla19:34
*** maharg101 has quit IRC19:39
kplanthttps://docs.openstack.org/kolla-ansible/latest/user/adding-and-removing-hosts.html20:01
kplanti would think you'd want to follow the remove steps for the failed node, to clean up agents recorded in services like neutron20:01
kplantthen follow steps to add the new node20:02
yoctozeptoalexandreperreau: you should be fine just replacing and running a deploy20:13
yoctozeptoas long as it works already20:13
alexandreperreauThanks kplant and yoctozepto for the quick responses.20:15
alexandreperreauyoctozepto: what do you mean by "as long as it works already"?20:15
yoctozeptoalexandreperreau: I meant if your cluster is operating at the moment20:16
alexandreperreauyes, luckily with HA, the environment is still working20:16
yoctozeptoif you want to use the same hostname and everything else - go my route20:16
yoctozeptoif you want to use a new name20:16
yoctozeptogo with the docs20:16
yoctozeptothere is nothing special as long as the deployment works20:17
kplantyeah, go with yoctozepto's approach20:17
kplantfar simpler20:17
yoctozeptomnasiadka: stopping the logs collection makes CI very green, verrrry greeeen :D20:18
yoctozeptowhat a lol in general20:18
yoctozeptothe DISK_FULL was confusing...20:18
alexandreperreauOk thanks. Should I --limit controller003 (replaced server) or --limit control?20:19
yoctozeptomnasiadka: there are like 0.5 GB logs from each host; still far from the value you mentioned before20:21
yoctozeptoalexandreperreau: if you have not done anything to "undo" the controller you had20:21
yoctozeptoyou can limit to it only20:21
alexandreperreauOk. Thank you for the clarifications. Really appreciated.20:22
*** yankcrime has quit IRC20:35
*** logan- has quit IRC20:35
*** benj_- has joined #openstack-kolla20:36
*** benj_ has quit IRC20:36
*** gary_perkins has quit IRC20:36
*** ewimmer__ has quit IRC20:36
*** dciabrin has quit IRC20:36
*** rockey has quit IRC20:36
*** dabukalam has quit IRC20:36
*** LaurentDumont has quit IRC20:36
*** osmanlicilegi has quit IRC20:36
*** ug_egon has quit IRC20:36
*** oyrogerg has quit IRC20:36
*** Tengu has quit IRC20:36
*** risson_ has joined #openstack-kolla20:37
*** benj_- is now known as benj_20:37
*** eliaswimmer has joined #openstack-kolla20:38
*** logan- has joined #openstack-kolla20:38
*** strigazi has quit IRC20:38
*** heikkine has quit IRC20:38
*** imcsk8 has quit IRC20:38
*** parallax has quit IRC20:38
*** Tengu has joined #openstack-kolla20:38
*** dabukalam_ has joined #openstack-kolla20:39
*** parallax has joined #openstack-kolla20:40
*** zzzeek has quit IRC20:40
*** kevinz has quit IRC20:40
*** ricolin has quit IRC20:40
*** dabukalam_ is now known as dabukalam20:40
*** mnasiadka has quit IRC20:41
*** zzzeek has joined #openstack-kolla20:41
*** gfidente|afk has quit IRC20:42
*** markmcclain has quit IRC20:42
*** mnasiadka has joined #openstack-kolla20:42
*** risson has quit IRC20:42
*** dasp has quit IRC20:43
*** imcsk8 has joined #openstack-kolla20:44
*** oyrogerg has joined #openstack-kolla20:44
*** dasp has joined #openstack-kolla20:45
*** jonaspaulo has quit IRC20:45
*** eliaswimmer has quit IRC20:54
*** k_mouza has joined #openstack-kolla21:25
*** k_mouza has quit IRC21:31
*** jbadiapa has quit IRC21:52
openstackgerritSergei Chekanov proposed openstack/kolla-ansible master: Add support .rules.yml files for prometheus  https://review.opendev.org/c/openstack/kolla-ansible/+/77769522:24
*** luksky has quit IRC22:35
openstackgerritPiotr Parczewski proposed openstack/kayobe master: Add support for custom shm_size for seed container(s)  https://review.opendev.org/c/openstack/kayobe/+/77769822:36
openstackgerritMerged openstack/kayobe master: CI: retry overcloud jobs that land on vexxhost clouds  https://review.opendev.org/c/openstack/kayobe/+/77751822:54
*** yankcrime has joined #openstack-kolla23:33
*** k_mouza has joined #openstack-kolla23:33
*** k_mouza has quit IRC23:34
« Wednesday, 2021-02-24 Index Friday, 2021-02-26 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!