Wednesday, 2020-03-04

*** diurnalist has quit IRC00:01
*** igordc has joined #openstack-kolla00:21
*** k_mouza has joined #openstack-kolla00:28
*** k_mouza has quit IRC00:32
*** k_mouza has joined #openstack-kolla00:34
*** igordc has quit IRC00:40
*** k_mouza has quit IRC01:17
*** k_mouza_ has joined #openstack-kolla01:17
*** zhanglong has joined #openstack-kolla01:20
*** k_mouza_ has quit IRC01:20
*** igordc has joined #openstack-kolla01:20
*** igordc has quit IRC01:33
*** calbers has quit IRC01:40
*** calbers has joined #openstack-kolla01:40
*** calbers has joined #openstack-kolla01:49
*** threestrands has joined #openstack-kolla02:06
*** k_mouza has joined #openstack-kolla02:12
*** k_mouza has quit IRC02:16
*** hongbin has joined #openstack-kolla02:50
*** calbers has quit IRC03:05
*** calbers has joined #openstack-kolla03:08
*** zhanglong has quit IRC03:24
*** Wellie has quit IRC03:39
*** k_mouza has joined #openstack-kolla04:01
*** sm806 has quit IRC04:04
*** sm806 has joined #openstack-kolla04:05
*** k_mouza has quit IRC04:06
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [WIP] Zun: Add zun-cni-daemon to compute node  https://review.opendev.org/70821304:52
*** hongbin has quit IRC04:54
*** skramaja has joined #openstack-kolla05:12
*** ykarel|away is now known as ykarel05:25
*** evrardjp has quit IRC05:35
*** evrardjp has joined #openstack-kolla05:35
*** igordc has joined #openstack-kolla05:38
*** igordc has quit IRC06:11
*** abdysn has joined #openstack-kolla06:34
*** threestrands has quit IRC06:34
*** threestrands has joined #openstack-kolla06:35
*** threestrands has quit IRC06:36
*** threestrands has joined #openstack-kolla06:36
*** threestrands has quit IRC06:37
*** threestrands has joined #openstack-kolla06:38
*** jbadiapa has joined #openstack-kolla06:38
*** threestrands has quit IRC06:39
*** hamzy has quit IRC07:04
*** hamzy has joined #openstack-kolla07:05
*** cah_link has joined #openstack-kolla07:17
*** happyhemant has joined #openstack-kolla07:17
*** sri_ has joined #openstack-kolla07:26
sri_morning07:27
osmanlicilegimorning07:34
*** hjensas has quit IRC07:36
*** JangwonLee_ has quit IRC07:41
*** ykarel is now known as ykarel|lunch07:45
*** kozhukalov has quit IRC07:51
*** tony31 has joined #openstack-kolla07:51
*** diurnalist has joined #openstack-kolla07:53
*** diurnalist has quit IRC07:58
*** tony31 has quit IRC08:04
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Fix elasticsearch configuration in ipv6 environmen  https://review.opendev.org/71117608:06
*** bengates has joined #openstack-kolla08:07
*** tonythomas has joined #openstack-kolla08:10
*** hamzy has quit IRC08:12
*** hamzy has joined #openstack-kolla08:13
*** kozhukalov has joined #openstack-kolla08:13
*** rpittau|afk is now known as rpittau08:16
*** hjensas has joined #openstack-kolla08:21
*** gfidente|AFK is now known as gfidente08:26
*** k_mouza has joined #openstack-kolla08:37
*** kozhukalov has quit IRC08:39
*** k_mouza has quit IRC08:50
*** ykarel|lunch is now known as ykarel08:50
yoctozeptohttps://bugs.launchpad.net/kolla-ansible/+bug/186600808:58
openstackLaunchpad bug 1866008 in kolla-ansible "keystone role failure on deploy" [Critical,Triaged]08:58
yoctozeptoand added to wb08:58
yoctozeptokolla-ansible CI red too08:58
*** ktibi has joined #openstack-kolla08:59
openstackgerritMarcin Juszkiewicz proposed openstack/kolla-ansible master: cirros: upgrade to 0.5.0  https://review.opendev.org/71118209:12
*** scottsol has joined #openstack-kolla09:12
*** dougsz has joined #openstack-kolla09:13
hrwhttps://github.com/cirros-dev/cirros/releases/tag/0.5.0 done09:26
openstackgerritRadosław Piliszek proposed openstack/kolla master: Ensure proper JSON in keystone bootstrap  https://review.opendev.org/71118609:26
*** scottsol has quit IRC09:30
*** hamzy has quit IRC09:34
*** hamzy has joined #openstack-kolla09:34
*** JangwonLee has joined #openstack-kolla09:39
*** scottsol has joined #openstack-kolla09:40
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Use listen port for Placement precheck  https://review.opendev.org/71118709:41
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Use listen port for Placement precheck  https://review.opendev.org/69375009:41
*** hamzy has quit IRC10:03
*** hamzy has joined #openstack-kolla10:04
*** hamzy has quit IRC10:06
*** hamzy_ has joined #openstack-kolla10:06
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CI: Add overcloud host configure job  https://review.opendev.org/71027710:13
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CentOS 8: Support DNF  https://review.opendev.org/71104410:13
openstackgerritMark Goddard proposed openstack/kolla-ansible master: service-rabbitmq: avoid password leak without no_log  https://review.opendev.org/71096910:14
*** k_mouza has joined #openstack-kolla10:18
*** k_mouza_ has joined #openstack-kolla10:19
*** hamzy has joined #openstack-kolla10:20
*** hamzy_ has quit IRC10:21
*** k_mouza has quit IRC10:23
mgoddardmorning all10:26
*** hamzy has quit IRC10:26
mgoddardyoctozepto: great news about kolla gate10:26
yoctozeptomgoddard: about both10:27
yoctozeptomgoddard: one yesterday, one today10:27
yoctozeptomgoddard: life is never boring10:27
*** hamzy has joined #openstack-kolla10:28
mgoddardyoctozepto: http://zuul.openstack.org/builds?project=openstack%2Fkolla&branch=master looks fairly green. Is it just centos/source?10:28
mgoddardas do stables10:29
yoctozeptomgoddard: seems so, must have mischecked10:30
yoctozeptomgoddard: then let's finally drop it from master and we are good10:30
mgoddardyeah, was going to bring it up in today's meeting10:30
mgoddardI'll add to agenda10:30
mgoddardyoctozepto: two blockers10:31
mgoddardhttps://review.opendev.org/#/c/710797/10:32
patchbotpatch 710797 - kolla-ansible - CentOS 8: CI: Add upgrade job - 7 patch sets10:32
mgoddardhttps://review.opendev.org/70953810:32
patchbotpatch 709538 - kolla (stable/train) - CentOS 8: Add deploy jobs in CI - 3 patch sets10:32
mgoddardThe latter fails on centos8/binary10:32
mgoddardactually that's train - ignore the second10:32
*** hamzy has quit IRC10:33
openstackgerritMerged openstack/kolla-ansible master: Add deprecation notes to vmware/xenserver/hyperv guides  https://review.opendev.org/71104010:35
*** scottsol has quit IRC10:35
yoctozeptohrw: https://review.opendev.org/710797 - quick one10:35
patchbotpatch 710797 - kolla-ansible - CentOS 8: CI: Add upgrade job - 7 patch sets10:35
yoctozeptomgoddard: though it will fail at gates due to keystone failure eh10:36
hrwdone yoctozepto10:36
yoctozeptothanks hrw10:37
mgoddardso we have a chicken and egg10:37
yoctozeptomgoddard: any thoughts on keystone anyway?10:37
mgoddardnot yet10:37
yoctozeptowe don't know how to fry the egg10:37
*** Wellie has joined #openstack-kolla10:38
yoctozeptoI can't see us passing this anywhere there10:38
yoctozeptomust be internal only, but then it would break more than us10:38
*** jbadiapa has quit IRC10:39
mgoddardman cat10:40
yoctozeptowoman cat10:46
yoctozeptocatwoman10:46
yoctozeptomgoddard: it's not really chicken&egg problem, first fix k-a and then k, simple10:46
yoctozeptomgoddard: well, unless we have to rebuild something in kolla10:47
*** scottsol has joined #openstack-kolla10:47
mgoddardyoctozepto: https://github.com/openstack/requirements/commit/9bc8626d51e9db66a267d7c6dd8516b1091c0fc310:52
mgoddardoslo.cache was bumped to 2.1.010:52
mgoddardwhich includes the fix for the referenced bug10:53
yoctozeptomgoddard: yeah, but why only us ;p10:53
mgoddardand this was merged recently: https://review.opendev.org/#/c/634457/10:53
patchbotpatch 634457 - oslo.cache - Switch from python-memcached to pymemcache. (MERGED) - 48 patch sets10:53
yoctozeptoyeah, I have that in my bug report10:55
yoctozeptoI wonder10:55
yoctozeptomaybe most devstack jobs do not run memcached10:55
yoctozeptolemme check that10:55
mgoddardwhere is a devstack core when you need one?10:56
yoctozeptomgoddard: no idea, must be in hiding10:56
yoctozeptomgoddard: joking aside, devstack runs memcached by default10:57
yoctozeptomaybe not via oslo.cache?10:57
yoctozeptohttps://d67166a1e0bd5acbef01-ec7c2cda95b1d3573093d93e0e594c95.ssl.cf2.rackcdn.com/710511/2/check/keystone-dsvm-py3-functional/e88e07b/controller/logs/etc/keystone/keystone_conf.txt10:59
yoctozeptodogpile.cache.memcached11:00
yoctozeptoand welcome to hell11:00
yoctozeptowe do11:01
yoctozeptobackend = oslo_cache.memcache_pool11:01
yoctozeptoreport updated11:06
openstackgerritMark Goddard proposed openstack/kolla master: DNM: Test without oslo.cache 2.1.0 in upper-constraints  https://review.opendev.org/71120511:06
mgoddardyoctozepto: this is the only substantial change between 2.0.0 and 2.1.0: https://github.com/openstack/oslo.cache/commit/8a8248d764bbb1db6c0089a58745803c03e38fdb11:08
yoctozeptomgoddard: yeah, it's this one11:09
yoctozeptomgoddard: I'm writing to ml, I see osa, charms and puppet might also be affected11:10
mgoddardyoctozepto: do you know Hervé Beraud's nick? they often comment on kolla patches11:10
mgoddardcool11:10
yoctozeptomgoddard: hberaud11:11
mgoddardok11:11
yoctozeptomgoddard: he's on #openstack-oslo11:11
*** ktibi has quit IRC11:13
yoctozeptomgoddard: I think you replaced reqs with keystone11:19
yoctozeptomgoddard: please check your DNM again11:19
mgoddarddammit11:19
*** zhanglong has joined #openstack-kolla11:20
openstackgerritGorka Eguileor proposed openstack/kolla stable/train: Add cinderlib RPM to cinder-volume  https://review.opendev.org/71120611:21
yoctozeptomgoddard: https://review.opendev.org/71118711:21
patchbotpatch 711187 - kolla-ansible (stable/train) - Use listen port for Placement precheck - 1 patch set11:21
yoctozeptodon't even look at train/stein gates11:22
yoctozeptojust don't11:22
yoctozeptospare yourself some anger11:22
openstackgerritMark Goddard proposed openstack/kolla master: DNM: Test without oslo.cache 2.1.0 in upper-constraints  https://review.opendev.org/71120511:22
yoctozeptomgoddard: same issue in train11:24
yoctozeptomgoddard: checking whether we have not bumped uc by mistake11:24
yoctozeptomgoddard: looking fine11:25
yoctozeptodafsck11:25
yoctozeptosheeee11:28
yoctozeptowe are using master images in stable/train11:28
mgoddardwtf11:29
mgoddardhow?11:30
yoctozeptomgoddard: bad backport approved11:30
mgoddardah, kolla_source_version11:31
mgoddardyoctozepto: looks like we never switched to train...11:32
*** rpittau is now known as rpittau|bbl11:32
yoctozeptomgoddard: yeah, I was about to say that11:32
mgoddardwow11:32
yoctozeptomgoddard: oh my gosh11:32
yoctozeptooh what a fail11:32
mgoddardindeee11:32
mgoddardd11:32
*** JangwonLee_ has joined #openstack-kolla11:35
yoctozeptoand openstack_release to change11:36
*** JangwonLee__ has joined #openstack-kolla11:36
mgoddardthat's the one11:37
yoctozeptomgoddard: best both11:38
yoctozeptomgoddard: you proposing11:38
yoctozepto?11:38
mgoddardwill do11:38
yoctozeptook11:38
mgoddardtrain is the first time we've had to change it - stein it was 'auto'11:38
yoctozeptomgoddard: yeah, confirmed11:38
*** zhanglong has quit IRC11:38
*** JangwonLee has quit IRC11:38
yoctozeptomgoddard: I guess reno-less? :P11:39
mgoddardnah11:39
*** zhanglong has joined #openstack-kolla11:40
*** JangwonLee_ has quit IRC11:40
*** zhanglong has quit IRC11:44
*** bersace has joined #openstack-kolla11:45
*** bersace has joined #openstack-kolla11:46
*** zhanglong has joined #openstack-kolla11:48
*** scottsol has quit IRC11:48
*** JangwonLee_ has joined #openstack-kolla11:50
*** openstackstatus has joined #openstack-kolla11:50
*** ChanServ sets mode: +v openstackstatus11:50
*** scottsol has joined #openstack-kolla11:51
*** reph has joined #openstack-kolla11:53
*** JangwonLee__ has quit IRC11:54
*** scottsol has quit IRC11:56
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CI: Add overcloud host configure job  https://review.opendev.org/71027711:58
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CentOS 8: Support DNF  https://review.opendev.org/71104411:58
*** reph has quit IRC11:59
*** reph has joined #openstack-kolla12:03
*** kplant has joined #openstack-kolla12:11
*** bersace has quit IRC12:15
*** kplant has quit IRC12:16
*** jimcrowleyibm has joined #openstack-kolla12:17
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Set openstack_release and kolla_source_version to train  https://review.opendev.org/71121412:18
*** kplant has joined #openstack-kolla12:21
hrwhttps://marcin.juszkiewicz.com.pl/2020/03/04/cirros-050-released/12:24
yoctozeptomgoddard: sure no reno?12:26
yoctozeptomgoddard: if sure, /me approves12:26
* mgoddard asleep12:26
yoctozeptomgoddard: red colour should wake you up12:27
mgoddardor make you want to go back to bed12:27
yoctozeptomgoddard: oh focus, it's a brekage day for me here as well, as if all devils chose this Wed for their hellish breakparty ;p12:29
hrw+212:29
yoctozeptomgoddard: reno or no reno? last chance; PTL's decision12:29
mgoddardreno, coming up12:30
*** johnthetubaguy has joined #openstack-kolla12:32
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Set openstack_release and kolla_source_version to train  https://review.opendev.org/71121412:33
hrw+212:35
*** zhanglong has quit IRC12:35
yoctozeptomgoddard: and off it goes12:36
yoctozeptolet's finally test train :D12:36
mgoddardthanks12:36
mgoddardyou assume it works :p12:36
yoctozeptomgoddard: you did not check it? :P12:36
yoctozeptomgoddard: I use self-built train images so it worked at some point12:37
*** jopdorp has joined #openstack-kolla12:37
yoctozeptothey* worked12:37
mgoddardyeah, also kolla CI builds them12:37
yoctozeptoyeah, but they were not deployed recently12:38
yoctozepto:D12:38
yoctozeptomgoddard: hberaud has progress with oslo.cache12:38
mgoddardI saw the ML12:38
yoctozeptomgoddard: it will finally come to resolution12:38
yoctozeptomgoddard: more in bug report12:38
openstackgerritMark Goddard proposed openstack/kolla master: Docs: Update release management for openstack_release in k-a  https://review.opendev.org/71121612:41
mgoddardI wonder if we can automate checks for some of the release process12:41
jopdorphi!12:41
*** ykarel is now known as ykarel|afk12:42
jopdorpI'm trying to deploy a kubernetes cluster with magnum12:42
jopdorpbut I'm getting errors in the process12:42
jopdorpit seems to be heat related12:42
jopdorphttps://pastebin.com/ahusu3b912:42
jopdorpthat one is the log from heat12:42
*** zhanglong has joined #openstack-kolla12:42
jopdorpthis from magnum conductor12:42
jopdorphttps://pastebin.com/0RKP3CjM12:42
yoctozeptomgoddard: yeah, and all other wishes ;-)12:43
yoctozeptomgoddard: automate checks... also likely to automate the process itself too12:43
jopdorpi set enable_magnum: "yes" in globals.yml12:44
jopdorpthen i did kolla-ansible -i ./multinode reconfigure --tags magnum, cinder12:44
jopdorpthat seemed to do the trick of deploying magnum12:44
yoctozeptojopdorp: it looks as if it failed to spawn the servers12:45
jopdorpyes12:45
yoctozeptojopdorp: then all that stems from it12:45
jopdorpthe master did spawn12:45
yoctozeptonothing more in logs12:45
jopdorpyeah12:45
yoctozeptocheck nova logs then12:45
jopdorpnot very informative...12:45
jopdorpnova logs12:45
jopdorpokay12:45
jopdorpI'll have a look12:45
*** jimcrowleyibm is now known as jimcrowleyibm[aw12:45
yoctozeptowell, it failed... somewhere :D12:45
jopdorphaha12:46
jopdorpit's weird12:46
jopdorpbecause the master did come up12:46
jopdorpand is accessable12:46
jopdorpI'll investigate in nova12:46
jopdorpanyway Fedora-atomic 29 doesn't deploy at all12:47
jopdorpbecause it is missing entropy12:47
jopdorpit just gets stuck on boot12:47
jopdorpfedora 27 did successfully boot at least12:47
jopdorpatomic12:47
openstackgerritMark Goddard proposed openstack/kayobe stable/train: CentOS 8: Add overcloud CI job  https://review.opendev.org/71106712:51
openstackgerritMark Goddard proposed openstack/kayobe stable/train: CentOS 8: Use same local python version for kolla-ansible  https://review.opendev.org/71121912:51
*** rphillips has quit IRC12:51
*** iniazi_ has joined #openstack-kolla12:53
*** TheJulia has quit IRC12:54
*** TheJulia has joined #openstack-kolla12:54
*** tonythomas has quit IRC12:55
*** mwhahaha has quit IRC12:55
yoctozeptomgoddard:12:55
yoctozepto2020-03-04 12:42:57.864160 | TASK [Record the running state of the environment as seen by the setup module]12:55
yoctozepto2020-03-04 12:43:00.410608 | primary | ERROR! Unexpected Exception, this is probably a bug: invalid syntax (fake_filesystem.py, line 1195)12:55
*** tonythomas has joined #openstack-kolla12:55
yoctozeptodafsck12:55
yoctozeptothis is train fix12:55
yoctozeptodid ansible just break?12:55
*** mwhahaha has joined #openstack-kolla12:56
yoctozepto😂😂😂😂12:56
yoctozeptooh what a day, oh what a joy12:56
*** iniazi has quit IRC12:56
*** ktibi has joined #openstack-kolla13:08
*** hjensas has quit IRC13:11
*** jimcrowleyibm has joined #openstack-kolla13:13
*** zhanglong has quit IRC13:15
*** zhanglong has joined #openstack-kolla13:17
*** reph has quit IRC13:20
*** reph has joined #openstack-kolla13:20
*** jbadiapa has joined #openstack-kolla13:23
*** tonythomas has quit IRC13:23
jopdorpyocozepto: I can't find any errors in nova13:23
jopdorpcould this have something to do with it?13:23
jopdorpWARNING magnum.drivers.heat.template_def stack does not have output_key api_address13:24
*** dannins has quit IRC13:24
*** niedbalski has quit IRC13:24
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CI: Add overcloud host configure job  https://review.opendev.org/71027713:24
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CentOS 8: Support DNF  https://review.opendev.org/71104413:24
jopdorpsame for kube_masters13:24
*** niedbalski has joined #openstack-kolla13:24
*** tonythomas has joined #openstack-kolla13:25
osmanlicilegifolks, you know I'm a little away from kolla for the last few weeks. when I try to build non-ubuntu images on my ubuntu box, I see: INFO:kolla.common.utils.base:Step 1/35 : FROM debian:bionic13:26
osmanlicilegiis this a bug or did I miss something :)13:26
osmanlicilegitell me who backported bionic to debian :P13:27
cosmicsoundgreets osmanlicilegi13:27
*** hjensas has joined #openstack-kolla13:27
mgoddardosmanlicilegi: sounds like we need some check for deb vs ubuntu13:28
yoctozeptowe need many things13:28
osmanlicilegiok, now I feel safe13:30
*** skramaja has quit IRC13:31
osmanlicilegicosmicsound: greets. how's your journey with openstack going?13:32
*** ykarel|afk is now known as ykarel13:32
openstackgerritMark Goddard proposed openstack/kolla master: DNM: Test oslo.cache fix  https://review.opendev.org/71122713:33
*** reph has quit IRC13:42
*** rpittau|bbl is now known as rpittau13:44
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CI: Add overcloud host configure job  https://review.opendev.org/71027713:48
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CentOS 8: Support DNF  https://review.opendev.org/71104413:48
*** jimcrowleyibm has quit IRC13:48
openstackgerritMark Goddard proposed openstack/kolla master: DNM: Test oslo.cache fix  https://review.opendev.org/71122713:50
cosmicsoundosmanlicilegi , very slowly yet keep on it14:00
*** jimcrowleyibm has joined #openstack-kolla14:16
*** ricolin has quit IRC14:25
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: CI: Pin pyfakefs to <4 for Python 2  https://review.opendev.org/71123914:26
*** scottsol has joined #openstack-kolla14:38
*** lile has joined #openstack-kolla14:41
*** sri_ has quit IRC14:44
*** ykarel is now known as ykarel|away14:46
openstackgerritMark Goddard proposed openstack/kolla master: DNM: Test oslo.cache fix  https://review.opendev.org/71122714:48
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CI: Add overcloud host configure job  https://review.opendev.org/71027714:50
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CentOS 8: Support DNF  https://review.opendev.org/71104414:50
mgoddardmgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi14:54
mgoddardmeeting in 6 ^14:54
*** priteau has joined #openstack-kolla14:55
mgoddard#startmeeting kolla15:00
openstackMeeting started Wed Mar  4 15:00:18 2020 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: kolla)"15:00
openstackThe meeting name has been set to 'kolla'15:00
mgoddard#topic rollcall15:00
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:00
mgoddard\o15:00
scottsolo/15:00
yoctozeptoo/15:00
osmanlicilegio/15:00
hrw /o15:00
priteauo/15:00
hrwcrowdy15:01
*** bengates has quit IRC15:02
mgoddard#topic agenda15:03
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:03
mgoddard* Roll-call15:03
mgoddard* Announcements15:03
mgoddard* Review action items from last meeting15:03
mgoddard* CI status15:03
mgoddard* OpenDev PTG in Vancouver - will anyone attend?15:03
*** abdysn has quit IRC15:03
mgoddard* Kolla SIG (aka Kolla Klub?) https://etherpad.openstack.org/p/kolla-sig15:03
mgoddard* Discuss: Ansible may run more handlers than originally planned https://bugs.launchpad.net/kolla-ansible/+bug/186351015:03
openstackLaunchpad bug 1863510 in kolla-ansible "Ansible may run more handlers than originally planned" [Medium,Triaged]15:03
mgoddard* Time to drop CentOS 7 jobs?15:03
mgoddard* Ussuri release planning (kayobe)15:03
mgoddard* Ussuri release planning (kolla & kolla ansible)15:03
mgoddard#topic announcements15:03
*** openstack changes topic to "announcements (Meeting topic: kolla)"15:03
mgoddard#info CI broken in myriad ways15:03
mgoddardWe'll cover that next15:03
*** zhanglong has quit IRC15:03
mgoddardAny others?15:03
hrwCirrOS 0.5.0 released15:04
hrwpatch for k-a in gerrit15:04
mgoddard#info CirrOS 0.5.0 released15:04
mgoddardnice work hrw15:04
mgoddardany new features?15:05
*** bengates has joined #openstack-kolla15:05
yoctozeptocan neutron finally stick to using cirros for their advanced networking testing instead of relying on ubuntu 14.04? :D15:05
hrw5.3 kernel, graphical console working on aarch64, lot of virtio included (rng as well)15:05
hrwyoctozepto: you know that cirros is simple image to test does your cloud works?15:06
hrwyoctozepto: convince 14.04 -> 18.04 move?15:06
yoctozeptohrw: I just asked15:06
yoctozeptoubuntu is bloated15:07
yoctozeptono idea why cirros was not sufficient15:07
mgoddard#topic Review action items from last meeting15:07
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)"15:07
yoctozeptolet's move on15:07
mgoddarddougsz to write bug report about nova SSH nproc issue15:07
mgoddarddougsz: did you get to it?15:07
mgoddardI didn't see it15:07
mgoddard#action dougsz to write bug report about nova SSH nproc issue15:08
mgoddard#topic CI status15:08
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:09
mgoddardok now for some fun15:09
mgoddardwe have a few issues at the moment15:09
mgoddardand to top it off some mirrors seem to be down15:09
yoctozeptoyeah, today is fun all around15:09
mgoddardlet's start with kolla15:10
mgoddardkolla.common.utils.tempest: ERROR: Package 'patrole' requires a different Python: 2.7.5 not in '>=3.6' (C7@master/Ussuri)15:10
yoctozeptowe can drop this from kolla after oslo gets fixed15:11
*** gfidente has quit IRC15:11
yoctozeptobecause we need k-a working15:11
mgoddardexample: https://e3487461d10f0d2cc5e1-c866b68a342a1b98ad74b0accf39cca4.ssl.cf1.rackcdn.com/706042/2/check/kolla-build-centos-source/ae638c7/kolla/build/000_FAILED_tempest.log15:11
yoctozeptosanity check: any idea why it hit master and not train?15:12
yoctozeptowe theoretically build train in c715:12
yoctozeptoeven on master branch15:12
mgoddardhave we done it today?15:12
yoctozeptonope, not touched yet15:13
yoctozeptoin any dimension :-)15:13
mgoddarddo we understand the underlying issue here?15:15
mgoddardtempest dropped py2 support?15:15
mgoddardseems like it happened a while ago though15:16
yoctozeptomhm, patrole dropped it15:17
yoctozeptoand we just take that branch15:17
yoctozeptoit's not from upper-constraints15:17
yoctozeptohence it killed master c715:17
mgoddardok15:17
mgoddardit's centos7, we don't care too much15:17
mgoddardbut will it break stable branches?15:18
mgoddardor will UC save us there?15:18
mgoddardlet's see tomorrows periodic15:18
mgoddardI guess we either drop c7 or drop c7 tempest15:18
yoctozeptodrop c7 altogether15:19
hrwdrop c715:19
yoctozeptoit's a mix-and-match15:19
mgoddardit's coming up next15:19
mgoddardok, onto k-a15:19
yoctozeptostables not affected because they pin to released version15:19
hrwif someone want c7 then train supports it15:19
*** jimcrowleyibm has quit IRC15:20
mgoddard(2020-03-04) Ussuri 100% fail at keystone15:20
mgoddardthis is due to oslo.cache 2.1.015:20
*** jimcrowleyibm has joined #openstack-kolla15:20
mgoddardhberaud proposed a patch, we're testing it now15:20
mgoddardhttps://review.opendev.org/#/c/711227/15:20
patchbotpatch 711227 - kolla - DNM: Test oslo.cache fix - 3 patch sets15:20
mgoddardaffects py3 jobs on master15:21
mgoddardfinally (I hope)15:21
mgoddardARA 0.x broken on py215:21
mgoddardI raised https://bugs.launchpad.net/kolla-ansible/+bug/186607915:21
openstackLaunchpad bug 1866079 in kolla-ansible train "Stable branch CI fails when gathering facts - invalid syntax (fake_filesystem.py, line 1195)" [Critical,In progress] - Assigned to Mark Goddard (mgoddard)15:21
openstackgerritDincer Celik proposed openstack/kolla-ansible stable/stein: Fixes gnocchi-api script name for Ubuntu/Debian  https://review.opendev.org/71124915:21
mgoddardproposed a workaround for us https://review.opendev.org/71123915:21
patchbotpatch 711239 - kolla-ansible (stable/train) - CI: Pin pyfakefs to <4 for Python 2 - 1 patch set15:21
mgoddardand a fix for ARA: https://review.opendev.org/71124015:22
patchbotpatch 711240 - recordsansible/ara (stable/0.x) - Fix ansible execution on Python 2 - 1 patch set15:22
mgoddardoh, we missed one15:22
mgoddardwe're deploying master containers on triain15:23
mgoddardhttps://launchpad.net/bugs/186605415:23
openstackLaunchpad bug 1866054 in kolla-ansible train "Kolla Ansible train deploys master containers by default" [High,In progress] - Assigned to Mark Goddard (mgoddard)15:23
mgoddardthis actually means we're subject to the oslo.cache issue on train too (which is how yoctozepto caught it)15:23
yoctozeptoindeed15:24
yoctozeptoI say we merge your ara and fix to release15:24
yoctozeptoand approve15:24
yoctozeptoas one15:24
*** hjensas has quit IRC15:24
yoctozeptoit's not to backport15:24
yoctozeptoand reno is quite clear15:24
mgoddardwill need to backport ara15:25
mgoddardcan do that first15:25
yoctozeptowell, patches are never removed15:25
yoctozeptoso does not matter too much for order15:25
mgoddardok15:25
mgoddardI think we are done with CI15:26
mgoddardkayobe green \o.15:26
mgoddardkolla cli red15:26
mgoddardfunctional down due to ceph gone (yoctozepto)15:26
mgoddardI saw you were pushing kolla-cli patches yoctozepto, was it for that?15:26
yoctozeptomgoddard: yeah, sorry, forgot to remove it15:27
mgoddardnice and green15:27
mgoddardok, onwards15:27
yoctozeptomgoddard: gmann proposed py2.7 drop in k-c15:27
yoctozeptomgoddard: and it got stuck15:27
*** jimcrowleyibm has quit IRC15:27
mgoddardhmm15:27
yoctozeptomgoddard: hence needed unblock15:28
yoctozeptomgoddard: k-c sees no continued work15:28
mgoddardindeed15:28
mgoddardfeel free to ping interested parties rather than fix yourself15:28
mgoddard#topic OpenDev PTG in Vancouver - will anyone attend?15:28
*** openstack changes topic to "OpenDev PTG in Vancouver - will anyone attend? (Meeting topic: kolla)"15:28
hrwnope15:29
yoctozeptomgoddard: I have no idea who is interested in here :D15:29
yoctozeptonot going myself15:29
hrw#1 budget, #2 coronavirus corpo rules15:29
yoctozeptotoo expensive15:29
mgoddardOnce again I was omitted from the mailing list asking which projects will attend Vancouver. Thankfully priteau noticed15:29
yoctozeptohmph, no love for kolla :/15:29
hrwVancouver is nice city15:30
mgoddardto be fair, kolla has shown no love for ptg recently :)15:30
mgoddardwe haven't really had productive development discussions in person for $years15:30
hrwwas kolla at any PTG after Dublin?15:30
mgoddardit's good for ops feedback etc15:30
mgoddardbut that's summit not PTG15:31
hrwmgoddard: we need european ptg to make it cheaper15:31
mgoddardmaybe, but that just makes it expensive for everyone else :)15:31
yoctozeptonext is Berlin I think?15:31
yoctozeptowell15:31
osmanlicilegiI may attend Vancouver (love my company)15:31
mgoddardosmanlicilegi: great. Just be prepared that there will probably be no official kolla discussions15:32
mgoddardI'm sure it will still be interesting and useful though15:32
mgoddardI think a virtual PTG works well for us15:32
mgoddardand works well for our planet too15:32
osmanlicilegi+1 for vptg15:33
yoctozepto+115:33
hrw+115:33
hrwand we can couch as much as we want ;D15:33
mgoddardseems we have a rough consensus - no official presence from kolla, but there may be some koalas present for informal discussions15:34
mgoddardI'll pass it on15:34
mgoddardGonna mix up the agenda order a little, keep you on your toes15:35
mgoddard#topic Time to drop CentOS 7 jobs?15:35
*** openstack changes topic to "Time to drop CentOS 7 jobs? (Meeting topic: kolla)"15:35
mgoddardTo turn it on its head - are there any reasons why we shouldn't drop centos 7 support in master?15:35
mgoddard1 - kayobe still uses it :D15:35
mgoddardbut we are three patches away from not using it15:36
yoctozeptodrop, drop, drop, drop15:37
hrw+2 for drop15:37
mgoddardoh, there is also bifrost15:37
mgoddard:)15:37
hrw+2 for dropping bifrost too15:37
mgoddardbut I think we can live without it for a time15:37
hrw;D15:37
mgoddardI'm working on it15:37
mgoddardI'm not hearing many reasons to keep it15:38
mgoddardso how do we drop?15:38
mgoddardk-a first?15:39
mgoddardzuul jobs first?15:39
hrwzuul, k-a+k15:39
mgoddarddoes anyone want to do it?15:39
mgoddardI'm sure it will be therapeutic15:39
yoctozeptofirst zuul, project order not relevant15:40
*** scottsol has quit IRC15:41
mgoddardlooking for a volunteer...15:41
yoctozeptoI thought you needed a therapy after today15:42
yoctozeptobut could be me if you insist ;p15:42
mgoddardI'll get the zuul ball rolling15:42
mgoddardthat at least will be easy15:43
mgoddardI have little incentive to remove support given kayobe needs it for now :)15:43
mgoddard#agreed Drop CentOS 7 jobs, starting with kayobe15:44
mgoddard#topic Discuss: Ansible may run more handlers than originally planned https://bugs.launchpad.net/kolla-ansible/+bug/186351015:44
*** openstack changes topic to "Discuss: Ansible may run more handlers than originally planned https://bugs.launchpad.net/kolla-ansible/+bug/1863510 (Meeting topic: kolla)"15:44
openstackLaunchpad bug 1863510 in kolla-ansible "Ansible may run more handlers than originally planned" [Medium,Triaged]15:44
yoctozeptowhat are our options?15:44
mgoddardThanks for spotting this yoctozepto15:44
mgoddardIt's not great tbh15:44
yoctozeptoI don't think we should revert the dry patch15:44
mgoddardI'd really rather not15:45
mgoddardwhen did it land? Stein?15:45
yoctozeptoor Train, not earlier15:45
yoctozeptolet's check then15:45
mgoddardtrain15:45
mgoddardphew15:45
yoctozeptohttps://review.opendev.org/64769915:46
patchbotpatch 647699 - kolla-ansible - Simplify handler conditionals (MERGED) - 13 patch sets15:46
yoctozeptomhm15:46
*** bersace has joined #openstack-kolla15:46
*** TrevorV has joined #openstack-kolla15:46
*** scottsol has joined #openstack-kolla15:46
yoctozeptoadded to bug report15:47
mgoddardditto :)15:47
yoctozepto Radosław Piliszek (yoctozepto) wrote 36 seconds ago:15:47
yoctozepto Mark Goddard (mgoddard) wrote 34 seconds ago:15:47
yoctozeptolate by 2 seconds!15:47
mgoddardit's quite nasty, as it affects all our config file generation and container checking tasks15:47
mgoddardyoctozepto suggested include_tasks in a loop15:48
mgoddardthis would be a significant rewrite of every role's config.yml and check-containers.yml15:48
yoctozeptoas was the one that introduced the bug15:48
mgoddardnot really a rewrite, it mostly removed lines15:49
mgoddarda lot easier to reason about15:49
mgoddard(although apparently still flawed)15:49
mgoddardthere are also performance concerns with include_tasks at scale15:49
mgoddardwhich we haven't fully investigated15:50
mgoddardbut committing to it in a big way seems scary15:50
yoctozeptohmm15:50
*** jimcrowleyibm has joined #openstack-kolla15:52
mgoddardthere is another sort of related issue, I wonder if we could fix both15:52
mgoddardif I run kolla-ansible genconfig, then kolla-ansible deploy15:52
mgoddardif my config changed but container did not, currently the container is not restarted15:52
mgoddardI was thinking about how to address this15:52
yoctozeptotrue15:53
mgoddardwhat if we touched a file in /etc/kolla/<container>15:53
yoctozeptowe should address both at once15:53
yoctozeptootherwise it gets extra work for nothing15:53
mgoddardthen we have tasks/handlers that remove said file, and trigger a restart if it existed15:53
mgoddardjust a thought15:54
mgoddardanother option is kolla_set_configs --check15:55
mgoddardchecks for changes in config files15:56
mgoddardyet another consideration is podman15:56
mgoddardif we were to be forced to switch at some point, would we need to change our model?15:57
hrwsorry guys, I got lost. can patches fixing the current CI issues get RP+2?15:57
mgoddardthat would prevent backporting15:57
yoctozeptogood thinking15:57
mgoddardhrw: will do when we have working patches15:57
yoctozeptohrw: mgoddard needs to merge them15:57
hrwok15:57
yoctozeptomgoddard: just to remember that loops with notify will not save us15:58
hrwI had 2-3 other discussion during meeting ;(15:58
yoctozeptoas they are b0rken15:58
mgoddardyoctozepto: true, but if there is something simple we can do once for each container it would help15:59
yoctozeptothat's the one thing we should not rely on, the rest is a mystery :D15:59
yoctozeptoso to sum up16:00
yoctozeptolet's add to bug report16:00
yoctozeptowhat we should remember when fixing16:00
mgoddardanyway, I don't think it's critical, but it is quite an important problem we should solve16:00
mgoddardok. who is updating bug report?16:00
mgoddardwouldn't want to dup16:00
*** bengates has quit IRC16:01
yoctozeptoI will16:02
mgoddardthanks16:02
yoctozeptolet's wrap it up now16:02
mgoddardand that brings us to the end16:02
mgoddardthanks all16:02
mgoddard#endmeeting16:02
yoctozeptothanks mgoddard16:02
*** openstack changes topic to "Kolla IRC meetings on Wednesdays @ 15:00 UTC - see agenda @ https://goo.gl/OXB0DL | Whiteboard: https://etherpad.openstack.org/p/KollaWhiteBoard | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"16:02
openstackMeeting ended Wed Mar  4 16:02:19 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:02
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-03-04-15.00.html16:02
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-03-04-15.00.txt16:02
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-03-04-15.00.log.html16:02
*** bengates has joined #openstack-kolla16:02
*** Trevor_V has joined #openstack-kolla16:02
generalfuzzI have a question about uwsgi in kolla16:03
* hrw off16:03
generalfuzzAs part of introducing uwsgi into Keystone containers, I need to remove an argument that is passed to Apache: ARGS="-DFOREGROUND16:03
generalfuzzhttps://review.opendev.org/#/c/710879/4/docker/keystone/keystone/extend_start.sh16:03
patchbotpatch 710879 - kolla - Install uwsgi for Keystone - 4 patch sets16:03
generalfuzzHowever when I do that, the kolla ansible tests zuul will fail since they rely on an Apache.16:03
generalfuzzit is possible to merge a patch that has failed zuul tests, since there is a dependency on changes in KA?16:04
yoctozeptohmm, interesting16:05
*** TrevorV has quit IRC16:06
yoctozeptok-a needs to learn it first yeah16:06
*** gfidente has joined #openstack-kolla16:06
generalfuzzbut k-a requires the uwsgi binaries in kolla to work . . .16:07
yoctozeptomgoddard: https://bugs.launchpad.net/kolla-ansible/+bug/1863510/comments/8 please validate :-)16:07
openstackLaunchpad bug 1863510 in kolla-ansible ussuri "Ansible may run more handlers than originally planned" [Medium,Triaged]16:07
*** diurnalist has joined #openstack-kolla16:07
yoctozeptogeneralfuzz: can't they be installed first?16:07
mgoddardgeneralfuzz: sounds like some multi-stage transition will be required16:08
mgoddardgeneralfuzz: e.g. pass through an env var to the container which affects ARGS16:09
yoctozeptogeneralfuzz: btw, was uwsgi confirmed to work with tls?16:11
yoctozeptolast time you said it had some issues?16:11
generalfuzzmgoddard: can you elaborate?16:11
openstackgerritMark Goddard proposed openstack/kolla master: DNM: Test oslo.cache fix  https://review.opendev.org/71122716:11
generalfuzzyoctozepto: I do have a POC working with uwsgi + tls16:12
yoctozeptogeneralfuzz: great! :D16:12
mgoddardgeneralfuzz: at the moment, keystone extend_start unconditionally sets ARGS16:12
mgoddardgeneralfuzz: first, push a patch that makes it conditional based on some (optional) env var16:12
mgoddardif [[ ${USE_HTTPD:-1} -eq 1 ]]; then ARGS=...; fi16:13
mgoddardor similar16:13
mgoddardthen in k-a we can pass USE_HTTPD=0 when we want to switch to uwsgi16:14
mgoddardmake sense?16:14
generalfuzzyes16:14
generalfuzzthank you!16:14
mgoddardhttps://opendev.org/openstack/kolla-ansible/src/branch/master/ansible/roles/keystone/defaults/main.yml#L516:15
mgoddardyou can add an environment key there (also need to update check-containers.yml and handlers/main.yml)16:15
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Set openstack_release and kolla_source_version to train  https://review.opendev.org/71121416:19
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: CI: Pin pyfakefs to <4 for Python 2  https://review.opendev.org/71126716:22
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/rocky: CI: Pin pyfakefs to <4 for Python 2  https://review.opendev.org/71126816:23
mgoddardyoctozepto: thanks for bug summary, looks fine16:25
*** sorin-mihai has quit IRC16:25
*** sorin-mihai has joined #openstack-kolla16:27
*** sorin-mihai_ has joined #openstack-kolla16:29
*** rphillips has joined #openstack-kolla16:31
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CI: Add overcloud host configure job  https://review.opendev.org/71027716:31
openstackgerritMark Goddard proposed openstack/kayobe master: WIP: CentOS 8: Support DNF  https://review.opendev.org/71104416:31
*** sorin-mihai has quit IRC16:32
*** ricolin has joined #openstack-kolla16:37
*** jimcrowleyibm is now known as jimcrowleyibm[aw16:42
*** jimcrowleyibm[aw has quit IRC16:42
*** scottsol has quit IRC16:42
*** scottsol has joined #openstack-kolla16:44
*** jimcrowleyibm has joined #openstack-kolla16:45
*** scottsol has quit IRC16:46
openstackgerritAlfredo Moralejo proposed openstack/kolla master: DNM test new opstools temporary repo  https://review.opendev.org/71127416:47
*** negronjl has quit IRC16:53
*** negronjl has joined #openstack-kolla16:57
*** jimcrowleyibm has quit IRC17:02
openstackgerritMerged openstack/kolla master: Docs: Update release management for openstack_release in k-a  https://review.opendev.org/71121617:03
*** sorin-mihai_ has quit IRC17:03
*** sorin-mihai_ has joined #openstack-kolla17:05
*** bengates has quit IRC17:07
*** openstackgerrit has quit IRC17:20
*** Trevor_V has quit IRC17:21
*** TrevorV has joined #openstack-kolla17:22
*** ktibi has quit IRC17:28
*** igordc has joined #openstack-kolla17:33
*** evrardjp has quit IRC17:35
*** evrardjp has joined #openstack-kolla17:35
*** openstackgerrit has joined #openstack-kolla17:39
openstackgerritAlfredo Moralejo proposed openstack/kolla master: DNM test new opstools temporary repo  https://review.opendev.org/71127417:39
*** rpittau is now known as rpittau|afk17:44
*** ktibi has joined #openstack-kolla17:46
*** dciabrin__ has joined #openstack-kolla17:48
openstackgerritMark Goddard proposed openstack/kolla master: WIP: CentOS 8: Build bifrost image  https://review.opendev.org/71129317:49
*** ktibi has quit IRC17:51
*** dciabrin_ has quit IRC17:52
openstackgerritMark Goddard proposed openstack/kolla stable/train: Use StorageSIG repos for Ceph in CentOS8  https://review.opendev.org/71129517:54
*** hjensas has joined #openstack-kolla17:55
*** priteau has quit IRC17:59
*** dougsz has quit IRC18:00
openstackgerritMark Goddard proposed openstack/kayobe master: CentOS 8: Support DNF  https://review.opendev.org/71104418:03
openstackgerritMark Goddard proposed openstack/kayobe master: CI: Add overcloud host configure job  https://review.opendev.org/71027718:03
*** igordc has quit IRC18:04
openstackgerritMark Goddard proposed openstack/kolla stable/train: Use StorageSIG repos for Ceph in CentOS8  https://review.opendev.org/71129518:04
*** igordc has joined #openstack-kolla18:05
yoctozeptomgoddard: you mean current train c8 images are wrong?18:14
*** tonythomas has quit IRC18:25
*** igordc has quit IRC18:27
*** igordc has joined #openstack-kolla18:33
*** gfidente is now known as gfidente|afk18:46
openstackgerritMerged openstack/kolla-ansible stable/rocky: CI: Pin pyfakefs to <4 for Python 2  https://review.opendev.org/71126818:54
*** k_mouza_ has quit IRC18:56
*** bersace has quit IRC19:05
*** bersace has joined #openstack-kolla19:11
*** jimcrowleyibm has joined #openstack-kolla19:19
*** kplant has quit IRC19:22
*** dmellado has quit IRC19:24
*** dmellado has joined #openstack-kolla19:25
*** mgoddard has quit IRC19:27
*** mgoddard has joined #openstack-kolla19:28
openstackgerritMerged openstack/kolla-ansible stable/stein: CI: Pin pyfakefs to <4 for Python 2  https://review.opendev.org/71126719:28
*** jbadiapa has quit IRC19:29
openstackgerritDincer Celik proposed openstack/kolla master: Upgrade RabbitMQ to 3.8  https://review.opendev.org/69479919:53
*** cosmicsound4 has joined #openstack-kolla20:01
*** cosmicsound has quit IRC20:01
*** cosmicsound4 is now known as cosmicsound20:01
openstackgerritMerged openstack/kolla-ansible stable/train: Set openstack_release and kolla_source_version to train  https://review.opendev.org/71121420:20
*** bersace has quit IRC20:21
jimcrowleyibmyoctozepto hrw Looks like mariadb-backup is include in ubuntu 19.04 repos but not in 18.04s20:33
jimcrowleyibmI think the patch for skipping the package is a good idea. But is skipping it causing use to loose functionality?20:34
yoctozeptowell, mariadb from stock repos is too old20:58
yoctozeptoneutron will probably fail on it20:58
*** reph has joined #openstack-kolla21:21
*** reph has quit IRC21:26
*** reph has joined #openstack-kolla21:26
*** KeithMnemonic has joined #openstack-kolla21:37
*** ktibi has joined #openstack-kolla21:47
*** ktibi has quit IRC21:51
*** Tengu has quit IRC22:04
*** Tengu has joined #openstack-kolla22:06
*** reph has quit IRC22:39
*** cah_link has quit IRC22:49
*** TrevorV has quit IRC22:55
*** threestrands has joined #openstack-kolla23:28
*** sorin-mihai_ has quit IRC23:36
*** sorin-mihai has joined #openstack-kolla23:36
*** jimcrowleyibm is now known as jimcrowleyibm[aw23:46
*** diurnalist has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!