Wednesday, 2019-09-25

*** goldyfruit_ has joined #openstack-kolla00:07
*** k_mouza has quit IRC00:20
*** goldyfruit_ has quit IRC00:33
*** goldyfruit has joined #openstack-kolla00:35
*** goldyfruit has quit IRC01:00
*** goldyfruit has joined #openstack-kolla01:26
*** hongbin has joined #openstack-kolla01:49
*** factor has joined #openstack-kolla02:07
*** hongbin has quit IRC02:36
*** igordc has quit IRC02:39
*** BjoernT_ has quit IRC02:42
*** BjoernT has joined #openstack-kolla02:44
*** BjoernT has quit IRC02:47
*** dave-mccowan has quit IRC02:53
*** goldyfruit has quit IRC04:24
*** dpawlik has joined #openstack-kolla05:13
*** skramaja has joined #openstack-kolla05:21
*** zbr|ruck has quit IRC05:43
*** zbr has joined #openstack-kolla05:44
*** shyam89 has joined #openstack-kolla05:47
crazikpart06:04
*** crazik has left #openstack-kolla06:04
*** radeks has joined #openstack-kolla06:06
*** luksky has joined #openstack-kolla06:21
yoctozeptomorning06:27
*** pcaruana has joined #openstack-kolla06:42
*** gfidente has joined #openstack-kolla06:42
*** rpittau|afk is now known as rpittau06:47
*** shyam89 has quit IRC06:54
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Fix deployment of Grafana  https://review.opendev.org/68453406:59
*** shyam89 has joined #openstack-kolla07:07
*** luksky has quit IRC07:27
*** shyam89 has quit IRC07:27
mnasiadkamorning07:34
*** shyam89 has joined #openstack-kolla07:40
yoctozeptonew interesting CI multinode failure07:42
yoctozepto2019-09-25  7:02:21 2778 [Warning] Access denied for user 'haproxy'@'secondary2' (using password: NO)07:42
yoctozepto2019-09-25  7:02:21 2779 [Warning] Access denied for user 'haproxy'@'primary' (using password: NO)07:42
yoctozepto2019-09-25  7:02:21 0 [Warning] WSREP: 2.0 (secondary2): State transfer to 1.0 (secondary1) failed: -111 (Connection refused)07:42
yoctozepto2019-09-25  7:02:21 0 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():737: Will never receive state. Need to abort.07:42
yoctozeptohttps://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d1/periodic/opendev.org/openstack/kolla-ansible/master/kolla-ansible-centos-source-cinder-lvm/6d115e0/secondary1/logs/kolla/mariadb/mariadb.txt.gz07:42
yoctozeptoWSREP is killing me07:42
*** dims has quit IRC07:46
*** dims has joined #openstack-kolla07:48
*** iniazi_ has joined #openstack-kolla07:53
*** shyam89 has quit IRC07:56
openstackgerritRadosław Piliszek proposed openstack/kolla master: Debian: switch fluentd to td-agent  https://review.opendev.org/68455107:56
openstackgerritRadosław Piliszek proposed openstack/kolla master: Debian: switch fluentd to td-agent  https://review.opendev.org/68455107:56
*** iniazi has quit IRC07:57
mnasiadkayoctozepto: fix WSREP! :)07:57
mnasiadkaso now we are switching Fluentd to td-agent also on Debian?07:57
yoctozeptomnasiadka: yeah and the code got nicer07:58
mnasiadkayeah, now td-agent needs to do builds for arm, and it will be even more nice :)07:59
yoctozeptoindeed, mnasiadka, indeed07:59
*** ivve has joined #openstack-kolla08:01
*** rpittau is now known as rpittau|bbl08:01
*** cah_link has joined #openstack-kolla08:20
stingrayzamorning :)08:20
*** luksky has joined #openstack-kolla08:23
*** k_mouza has joined #openstack-kolla08:26
mgoddardmorning08:31
*** cah_link has quit IRC08:32
*** priteau has joined #openstack-kolla08:34
*** shyam89 has joined #openstack-kolla08:34
*** zbr is now known as zbr|ruck08:40
openstackgerritRadosław Piliszek proposed openstack/kolla master: Debian: switch fluentd to td-agent on x86_64  https://review.opendev.org/68455108:43
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637608:56
*** shyam89 has quit IRC09:00
*** hjensas has joined #openstack-kolla09:01
*** shyam89 has joined #openstack-kolla09:04
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Simplify multi-region configuration documentation  https://review.opendev.org/68463709:04
*** hjensas has quit IRC09:27
*** hjensas has joined #openstack-kolla09:30
shyam89Hi09:32
shyam89I have a queens centos based openstack cloud09:33
*** dpawlik has quit IRC09:33
shyam89deployed through kolla ansible09:33
shyam89Enabled tls on public endpoints09:33
shyam89I see localsettings file of horizon does not mention about ca chain file path09:34
shyam89and ssl verification parameter is commented09:34
shyam89#OPENSTACK_SSL_NO_VERIFY = True09:35
shyam89#OPENSTACK_SSL_CACERT = /path/to/ca-chain.pem09:35
*** dpawlik has joined #openstack-kolla09:36
shyam89In this case, does horizon's requests to other openstack components get ssl verified or not?09:37
yoctozeptoshyam89: then horizon queens defaults are used09:40
yoctozeptocheck the horizon docs09:40
shyam89okay09:42
*** luksky has quit IRC09:54
*** shyam89 has quit IRC09:56
openstackgerritRadosław Piliszek proposed openstack/kolla master: Debian/binary CI: added kolla-build-debian-binary  https://review.opendev.org/68367509:56
*** rpittau|bbl is now known as rpittau09:57
yoctozeptoararar, vmtp permabroken on ubuntu atm09:58
yoctozeptohttps://d7c5d6584507e05a6cd4-ee4c5d809145a8a3246cc4d26d65fbe0.ssl.cf2.rackcdn.com/684551/3/check/kolla-build-ubuntu-source/45b8e29/kolla/build/000_FAILED_vmtp.txt.gz09:58
yoctozeptomgoddard, mnasiadka ^09:58
mgoddarddammit09:58
yoctozeptotabulate breaks install09:58
yoctozeptohttps://pypi.org/project/tabulate/#history09:59
yoctozeptoand yes they had a release today09:59
yoctozeptowriting a bug report09:59
mgoddardI remember hrw looked at a similar issue09:59
*** shyam89 has joined #openstack-kolla10:03
yoctozeptomgoddard: I dunno but infra had this messages in Zuul due to python incompats10:03
yoctozeptowell, it only makes the tiering discussion more urgent10:04
yoctozepto;-)10:04
mgoddardadd to unbuildable for now?10:09
yoctozeptomgoddard: seems best resolution atm10:10
yoctozeptolooking at it I think it mixed python2 and 3 there10:11
mgoddardok I'll propose10:12
yoctozeptonot sure if centos affected as well10:12
yoctozeptoscrolled logs and it builds fine10:16
yoctozeptocentos i mean10:16
openstackgerritMark Goddard proposed openstack/kolla master: Skip vmtp image on Ubuntu/Debian source  https://review.opendev.org/68468710:20
*** k_mouza has quit IRC10:21
*** k_mouza_ has joined #openstack-kolla10:21
yoctozeptohttps://13ca543760d36bac65f9-a7d9059cf68801c3dc92812dc120717e.ssl.cf2.rackcdn.com/684551/3/check/kolla-build-debian-source/ebbbcfa/kolla/build/vmtp.txt.gz10:22
yoctozeptodebian not broken10:22
*** BjoernT has joined #openstack-kolla10:23
*** pcaruana has quit IRC10:23
*** ondrejduchon has joined #openstack-kolla10:24
ondrejduchonmgoddard: Hello, did you check that logging.conf feature? Does it make more sense now? https://review.opendev.org/#/c/679812/ Thank you.10:26
*** BjoernT_ has joined #openstack-kolla10:28
yoctozeptomgoddard: all in all, it does not look tabulate update itself could have been the cause, this is all that has changed in it: https://bitbucket.org/astanin/python-tabulate/commits/cc69141664ee1bb758e68bc88fa2eea5ba4b294110:31
*** BjoernT has quit IRC10:31
*** luksky has joined #openstack-kolla10:33
*** pcaruana has joined #openstack-kolla10:36
yoctozeptomgoddard: nah, the guy seems to drive the repo in a weird way really10:41
yoctozeptothere are more changes between 0.8.3 and 0.8.410:41
yoctozeptoincluding the offending new line being added in there10:41
*** dpawlik has quit IRC10:56
yoctozeptoargh, tabulate installs just fine on python 3.6 on ubuntu bionic10:56
yoctozeptotested on one of my workstations10:57
yoctozepto:/10:57
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637610:58
*** BjoernT_ has quit IRC10:59
mgoddardyoctozepto: perhaps the issue was found and fixed?11:22
mnasiadkathat would be fast:)11:24
yoctozeptomgoddard: issuing one recheck to check11:25
yoctozeptobut that would be odd not to change the package version11:25
yoctozeptoin pypi11:25
mgoddardyou already said it was weird :)11:27
*** kplant has joined #openstack-kolla11:29
*** dpawlik has joined #openstack-kolla11:29
yoctozeptomgoddard: yeah, because at least debian failure was expected, yet it passed :-)11:30
yoctozeptoand then local ubuntu works fine11:30
mgoddardyup11:30
yoctozeptoweird all the way11:30
yoctozeptothe only sure thing is that the offending line was really introduced in that newer package11:31
yoctozeptoall the rest remains a mystery11:31
*** goldyfruit has joined #openstack-kolla11:32
shyam89Hi11:35
shyam89I want to mount a file inside horizon container11:35
shyam89where should place it so that it gets mounted automatically?11:36
shyam89My openstack is queens deployed using kolla ansible11:36
mnasiadkaI don't think we have support for that in queens11:37
yoctozeptoshyam89: assuming you don't mind kolla-ansible being unable to do that for you, you can always make docker mount another volume wherever you want11:43
ivvehey guys, would it be totally weird to implement an extra when condition on handlers neutron-l3-agent, neutron-openvswitch-agent (and possible more agents) called something like allow_restart_critical_service | bool that wouldn't bounch these critical services during an upgrade in a production environment?11:43
yoctozeptoivve: weird-o-meter shows 711:44
kplantrofl11:44
yoctozeptoI guess you just want limit11:44
ivve:)11:44
ivvehaha11:44
ivveim guessing there is no "alternative" way to performing the upgrade script otherwise11:45
shyam89yoctozepto: Horizon not mounting any directory?11:46
ivvesince you can't limit it, especially if controller nodes are hyperconverged as compute11:46
shyam89from host to container11:46
ivveso stamp of approval or denied? :)11:47
yoctozeptoivve: I still don't follow you - you want them to be restarted at some point11:47
mgoddardivve: would it be better to support a rolling restart of those services?11:47
yoctozeptomgoddard: indeed what I had in mind with "limit"11:48
ivvewell, you don't want them restarted if they have instances running on them11:48
mgoddardyoctozepto: limit doesn't always play nicely with upgrade11:48
yoctozeptoshyam89: kolla-ansible queens probably does not allow orchestration of that11:48
yoctozeptosorry11:48
yoctozeptomgoddard: ah :<11:48
ivveopenvswitch-agent is not that bad a few seconds if network loss. but l3 agent restarted can result in over a minute loss of network11:48
mgoddardivve: you can run l3-agent and dhcp-agent in HA mode. OVS agent restart shouldn't take down the data plane as it's in the kernel. Should it?11:49
mgoddardouch11:49
ivveit does11:49
shyam89yoctozepto: I see11:49
shyam89Thank you11:49
yoctozeptoshyam89: yw, newer release have more features though11:49
shyam89yoctozepto: Okay11:50
shyam89That is good11:50
ivvemgoddard: im assuming l3_ha = true11:50
ivveso im not including dhcp, but a simple if statement could check that11:51
ivvei haven't taken everything in consideration yet but im testing in a l3_ha and dvr environment11:51
mgoddardivve: surely the l3 agent failover doesn't take a minute? If not you could roll the restart through the agents11:51
mgoddardah, dvr11:51
ivveyeah i guess i kinda forgot that, but i always assume dvr :P11:52
ivveso only thing really running ha is dhcp11:52
yoctozeptoivve: does it mean restarting ANY of dvr l3 agents cause problems for instances using them for one minute?11:53
ivveif you restart the "local" l3 agent in a host, the instances it is hosting will lose external network for circa a minute, thats how long it takes for the router to come back up11:53
ivveno matter the size of the "cluster"11:54
ivvewell tested with 5 and 60 nodes11:54
yoctozeptohmm, I thought with dvr you still get failover to a random other l3 agent11:55
yoctozeptogood I did not try11:55
ivve:D11:55
*** shyam89 has quit IRC11:55
ivvewhat i did was implement that variable and simply ran (for this test rocky and stein) at the same time11:56
ivveand migrated nodes from "rocky-l3-agents" to "stein-l3-agents"11:57
yoctozeptoivve: https://bugs.launchpad.net/neutron/+bug/160261411:57
openstackLaunchpad bug 1602614 in neutron "DVR + L3 HA loss during failover is higher that it is expected" [High,Fix released] - Assigned to Carl Baldwin (carl-baldwin)11:57
ivveand reconfigured with --limit on hosts11:57
yoctozeptothere seems to be some failover still11:57
yoctozeptobecause it was discussed in newton11:57
yoctozeptocould be something else wrong there11:58
yoctozeptoor a new bug11:58
ivveso regression or still a problem?11:58
ivveor new bug yeah11:58
yoctozeptono idea11:58
ivvei figured it was by design11:58
yoctozeptomaybe discuss with neutrinos11:58
yoctozeptoI doubt11:58
ivveposted the question in the channel12:02
ivveill try to get to the bottom of this :)12:02
*** k_mouza_ has quit IRC12:05
*** k_mouza has joined #openstack-kolla12:05
*** yoctozepto has quit IRC12:10
*** k_mouza has quit IRC12:10
*** yoctozepto has joined #openstack-kolla12:15
*** goldyfruit has quit IRC12:16
*** shyam89 has joined #openstack-kolla12:21
*** dave-mccowan has joined #openstack-kolla12:22
*** dave-mccowan has quit IRC12:26
*** yoctozepto has quit IRC12:26
*** yoctozepto has joined #openstack-kolla12:26
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver  https://review.opendev.org/68473012:29
*** vmixor has joined #openstack-kolla12:40
*** dave-mccowan has joined #openstack-kolla12:43
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver  https://review.opendev.org/68473012:50
openstackgerritMark Goddard proposed openstack/kolla master: Install RabbitMQ in kolla-toolbox image  https://review.opendev.org/68474212:52
*** BjoernT has joined #openstack-kolla12:53
*** shyam89 has quit IRC12:54
*** BjoernT_ has joined #openstack-kolla12:56
*** BjoernT has quit IRC12:59
*** BjoernT_ has quit IRC13:00
*** BjoernT has joined #openstack-kolla13:01
*** dpawlik has quit IRC13:01
openstackgerritMark Goddard proposed openstack/kolla master: Skip vmtp image on Ubuntu/Debian source  https://review.opendev.org/68468713:12
openstackgerritViktor Michalek proposed openstack/kolla master: Placement: Debian pkg has migration script in different path  https://review.opendev.org/68368013:15
*** dpawlik has joined #openstack-kolla13:18
*** dave-mccowan has quit IRC13:26
*** k_mouza has joined #openstack-kolla13:28
*** tonythomas has joined #openstack-kolla13:37
mgoddardondrejduchon: I'll take a look now13:38
*** spiette has quit IRC13:58
*** spiette has joined #openstack-kolla14:00
*** dpawlik has quit IRC14:02
*** dpawlik has joined #openstack-kolla14:14
*** goldyfruit has joined #openstack-kolla14:16
*** chason has joined #openstack-kolla14:16
*** dpawlik has quit IRC14:18
*** rpittau is now known as rpittau|afk14:34
*** luksky has quit IRC14:46
*** goldyfruit has quit IRC14:46
mgoddardmeeting in 10 minutes14:50
mgoddard^ mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne14:50
mnasiadkamgoddard: I won’t be attending, getting more and more sick - but if you need anything - mention my name14:51
*** scottsol has joined #openstack-kolla14:51
mgoddardmnasiadka: got the tungsten flu?14:51
mnasiadkaI think I added something to agenda though... inspired by another breakage in sensu14:52
priteauI am in another meeting, I'll keep an eye on what's being discussed.14:52
mnasiadkamgoddard: something like that14:52
mgoddard:(14:52
mnasiadkamgoddard: we can move discussion about non-openstack images like sensu/prometheus/etc to next meeting14:52
mgoddardok14:53
mnasiadkamgoddard: the idea was to either use their own images (not build ours), or limit the amount of monitoring suites we support14:53
mnasiadkaOr both :)14:53
mnasiadkaSeems like a PTG topic though...14:53
mgoddard+114:54
mgoddardcan always have an early discussion in meetings though14:54
mgoddard#startmeeting kolla15:00
openstackMeeting started Wed Sep 25 15:00:45 2019 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
*** openstack changes topic to " (Meeting topic: kolla)"15:00
openstackThe meeting name has been set to 'kolla'15:00
mgoddard#topic rollcall15:00
*** openstack changes topic to "rollcall (Meeting topic: kolla)"15:00
mgoddardput your hands in the air like you just don't care15:01
mgoddard\o/15:01
yoctozeptoo/15:01
generalfuzzo/15:01
* yoctozepto still caring a bit15:01
egonzalezo/15:01
scottsolo/ (kinda - at a conference)15:01
mgoddard#topic agenda15:03
*** openstack changes topic to "agenda (Meeting topic: kolla)"15:03
mgoddard* Roll-call15:04
mgoddard* Announcements15:04
jovial[m]\o15:04
mgoddard** Kolla feature freeze next week15:04
mgoddard* Review action items from last meeting15:04
mgoddard* CI status15:04
mgoddard* Train release planning15:04
mgoddard* Review priorities15:04
mgoddard* Kayobe Train planning meeting recap15:04
mgoddard* Short-term and long-term proposal for managing non-OpenStack projects Docker images (e.g. sensu case)15:04
mgoddard* Need for review of pre-train TODOs/FIXMEs, e.g. placement separation case https://review.opendev.org/68368015:04
mgoddard#topic announcements15:04
*** openstack changes topic to "announcements (Meeting topic: kolla)"15:04
mgoddard#info Kolla feature freeze next week15:04
mgoddardThis means we will stop accepting patches for features15:05
mgoddardExceptions can be made following core team agreement15:06
mgoddardAny others?15:06
yoctozeptonone here15:06
generalfuzzwill it be open for enabling wsgi in services?15:06
mgoddard#info CentOS 8 released yesterday15:06
mgoddardContainer images expected early next week15:07
*** dpawlik has joined #openstack-kolla15:07
yoctozeptogeneralfuzz: let's discuss exceptions after planned meeting parts15:07
mgoddard+115:07
mgoddard#topic Review action items from last meeting15:07
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)"15:07
*** BjoernT has quit IRC15:07
mgoddardmgoddard to email Jeffrey4l & co about kolla-cli15:07
mgoddardI didn't, but Jeffrey4l saw the ping and caoyuan emailed me15:08
mgoddardIt is not ready for release yet, but should be in ~1 month15:08
mgoddardthey will keep us updated15:08
yoctozeptook15:08
*** BjoernT has joined #openstack-kolla15:08
mgoddard#topic CI status15:08
*** openstack changes topic to "CI status (Meeting topic: kolla)"15:08
mgoddardkolla is RED15:09
mgoddardneed some reviews on https://review.opendev.org/#/c/684687/15:09
mgoddardkayobe is RED15:10
yoctozeptowill be fixed today, I think I know what could have gone wrong there, just need more time, let's not disable it just now unless you have a different urgent patch15:10
yoctozeptothat was re kolla15:10
mgoddardok15:10
yoctozeptonot promising to fix kayobe ;-)15:10
mgoddardwhy not?15:10
mgoddard:p15:10
yoctozeptoI dunno, maybe because I would only break it15:11
yoctozeptowho knows15:11
yoctozeptok-a pretty green, pretty nice15:11
yoctozeptoegonzalez onto the tacker issue in nfv15:11
egonzalezthe current patch in k-ansible fixes it15:11
mgoddardI saw the k-a patch passes CI now15:11
yoctozeptohe actually patched it but awaits glance details15:11
mgoddardis the kolla patch necessary?15:11
egonzalezno, will abandon it15:12
yoctozeptono, I think you should abandon it, egonzalez15:12
mgoddardk15:12
yoctozepto:D15:12
yoctozepto^ example of a little discussion of 3 agreeing people15:12
mgoddardhere's the k-a patch for tacker: https://review.opendev.org/68427515:13
mgoddardmarked RP+115:13
mgoddardand kayobe15:14
mgoddardwe were broken by python-ironicclient 3.0.015:14
yoctozeptomgoddard: it needs glance confirmation15:14
yoctozeptotacker patch I mean15:14
mgoddard3.1.0 is working its way to pypi now, also needs a requirements bump15:14
mgoddardyoctozepto: what do you mean by glance confirmation?15:15
egonzaleztacker uses glance_store and requires some config, not sure about how many of the backends are suported or tested15:16
yoctozepto# NOTE(egonzalez): No idea why this is used instead of glance servers, investigating...15:16
yoctozeptoit looks weird to have this in tacker15:16
yoctozepto[glance_store]15:16
yoctozeptodefault_backend = file15:16
yoctozeptofilesystem_store_datadir = /var/lib/tacker/csar_files/15:16
yoctozepto(weird=wrong)15:17
mgoddardis it a way to bypass the glance API?15:17
mgoddardi.e. it would be configured the same way as in glance15:17
egonzalezi think tacker is storing packages wihtout using glance locally15:17
egonzalezlocally or remotelly depending the backend15:18
yoctozeptomaybe fix for now with a big FIXME instead of just a note?15:18
yoctozeptoif they don't respond timely15:18
egonzalezok15:18
mgoddarddo we have contacts in the tacker team to get this resolved?15:18
egonzalezfile type should be fine for now15:18
*** goldyfruit has joined #openstack-kolla15:18
egonzaleznope, only IRC, people I had contact are not active anymore15:19
yoctozeptolaunchpad and irc already tried15:20
mgoddardok15:20
yoctozeptoml is the next step15:20
mgoddard+115:20
yoctozeptoegonzalez: could you attack them?15:20
egonzalezsure15:20
yoctozeptoonly friendly fire :D15:20
yoctozeptothx15:21
mgoddard#action egonzalez to ask about glance-store config for tacker on openstack-discuss15:21
mgoddardor whatever channel works15:21
*** openstackgerrit has quit IRC15:21
mgoddard#topic Train release planning15:21
*** openstack changes topic to "Train release planning (Meeting topic: kolla)"15:21
mgoddardWe're getting close15:22
mgoddardhopefully we can start on centos 8 / py3 work soon15:23
mgoddardis anyone planning to pick that up?15:23
yoctozeptomgoddard: is there docker there?15:23
yoctozeptoor just podman15:23
*** skramaja has quit IRC15:23
mgoddardyoctozepto: pass15:23
mgoddardwe use upstream docker15:23
mgoddardhopefully it will support centos 815:24
mgoddardhttps://download.docker.com/linux/centos/8 doesn't exist yet15:24
yoctozeptochecked the same15:24
yoctozeptoyup15:24
yoctozeptono el8 packages15:24
yoctozeptogoogle returns only15:25
mgoddardit's still early15:25
yoctozepto"podman replaces docker" stuff15:25
mgoddardanyway, we're talking about images first15:25
yoctozeptoyup, image side15:25
yoctozeptonot host15:25
yoctozeptomight be easier15:25
yoctozeptowill see15:25
yoctozeptoI saw some infographic showing rhel8 containers on rhel7 hosts15:25
yoctozeptoso they presumably support that well15:26
mgoddardI'm working hard on nova cells15:26
*** gfidente has quit IRC15:26
mgoddardoh that's good15:26
yoctozeptohttps://developers.redhat.com/blog/2019/08/23/run-red-hat-enterprise-linux-8-in-a-container-on-rhel-7/15:26
mgoddardgeneralfuzz: now would be a good time to discuss internal TLS15:27
yoctozepto#link https://developers.redhat.com/blog/2019/08/23/run-red-hat-enterprise-linux-8-in-a-container-on-rhel-7/15:27
mgoddard(backends)15:27
generalfuzzso, after this next patch (https://review.opendev.org/#/c/664517)15:27
generalfuzzwe should be ready to start working on enabling wsgi in services15:28
generalfuzzI figure one or two at a time15:28
generalfuzzcan this work begin in train?15:28
mgoddardit can begin in train, whether it will end in train is another matter :)15:29
mgoddardit's a priority feature, so I think we could cut it some slack15:29
generalfuzzdoubtful we will do all services in train timeline, but can def do some15:30
mgoddardbut the main priority must be centos 8 & python 3 after feature freeze, and once that's done we should call time15:30
*** pcaruana has quit IRC15:30
mgoddardit's hard to say yes or no - there are lots of variables15:31
mgoddardwe don't know how long it will take, or how long we have15:31
yoctozeptowe could have started rhel8 with ubi it seems: https://developers.redhat.com/blog/2019/07/29/red-hat-universal-base-image-how-it-works-in-3-minutes-or-less/15:31
generalfuzzwell, if we agree that an approach like https://review.opendev.org/#/c/664516/ is correct, I will start after https://review.opendev.org/#/c/664517 goes in, and it will land where it lands15:33
mgoddardgeneralfuzz: I'd suggest to start working on it and we can see how we go15:33
mgoddardgeneralfuzz: there were quite a few comments on the backend patch15:34
mgoddardI'd quite like to see an approach where we have a role that generates the necessary config that we can reuse15:35
mgoddardanyway, probbably not the time for design discussions15:35
mgoddardI think we're in agreement15:36
generalfuzzyes15:36
mgoddardyoctozepto: do you think ipv6 will land?15:36
generalfuzzand open to suggestions15:36
yoctozeptomgoddard: will try my best15:37
mgoddardyoctozepto: ok15:37
mgoddardbetter to wait & get it right than rush though IMO15:37
yoctozeptoindeed15:37
mgoddardI should have new patches for nova cells up soon. I know it's a big one, but I'd appreciate some reviews15:38
mgoddardI'm focussing on the nova part, and pushing the mariadb/rabbitmq bits out15:39
mgoddardhoping that means we'll get it in sooner rather than later15:39
mgoddardWe have some train feature priorities for kayobe now, but I'll discuss later15:40
mgoddard#topic Review priorities15:40
*** openstack changes topic to "Review priorities (Meeting topic: kolla)"15:40
mgoddardAny patches we need to focus on this week?15:40
mgoddardjovial[m]: kayobe included15:40
* yoctozepto focuses on ipv6 (proponent side)15:41
generalfuzzI would like to get this in: https://review.opendev.org/#/c/66451715:41
yoctozeptowhoa, wait for my reviews :D15:42
*** chason has quit IRC15:42
mgoddardadded RP+115:42
yoctozeptofor some reason it does not conflict with ipv615:43
yoctozepto+0.5 for that15:43
generalfuzznice15:43
jovial[m]kayobe: I will look at https://review.opendev.org/#/c/682701/ now - seems quite important15:43
mgoddardjovial[m]: thanks15:44
mgoddardjovial[m]: bear in mind kayobe CI is busted, so leave off +A for now15:44
mgoddardwe should probably add RP+1 to bug fixes as we get nearer the end of the release15:44
jovial[m]We are just waiting for the new ironic-inspector package?15:44
mgoddardjovial[m]: ironicclient15:45
*** ivve has quit IRC15:45
mgoddard#topic Kayobe Train planning meeting recap15:46
*** openstack changes topic to "Kayobe Train planning meeting recap (Meeting topic: kolla)"15:46
mgoddardYesterday we had a planning meeting for kayobe15:47
mgoddard#link https://etherpad.openstack.org/p/kayobe-train-planning15:47
* yoctozepto eager to learn details15:47
mgoddarddiscussed a number of potential features15:47
mgoddardcentos8 was interesting15:48
yoctozeptos/feature/failure/g15:48
mgoddarddecided we'd support centos 7 and 8 for at least one release, to allow a smooth upgrade of hosts15:48
mgoddardcentos7 will be py2, centos8 py315:48
mgoddardexpecting to need to reinstall hosts to get from 7 to 815:49
mgoddardso we need to think about how we do that smoothly15:49
yoctozeptodrop centos7 in v?15:49
mgoddardmaybe15:49
mgoddardor u15:49
yoctozeptoI think ussuri should have it unless we hit something big15:50
yoctozeptolike python2 more dead than now15:50
mgoddardquestions about running centos 8 containers on centos 8 hosts15:50
mgoddardand implications for live migration with different versions of qemu15:50
mgoddardthen some discussions of python 3, and where we use it15:51
mgoddard"Difficult to estimate - medium size ball of string"15:51
mgoddardWe talked about cleaning up our integration points with kolla-ansible15:52
mgoddarda few ideas proposed, but none without their corner cases15:52
mgoddardfinally, we voted on priorities for the train cycle15:53
mgoddardresults were:15:53
mgoddardcentos815:53
mgoddardpython315:53
mgoddardSupport multiple environments from a single kayobe configuration15:53
mgoddardImproved support for integrating with Ceph-Ansible15:53
mgoddardNext generation discovery & network configuration15:53
mgoddardEasier customization of Kayobe commands15:53
mgoddardNova cells support15:53
mgoddardAs usual, TBC depending on having someone to do the work15:54
mgoddardI added those to the kolla whiteboard15:54
mgoddardAny Qs?15:54
mgoddard#topic Need for review of pre-train TODOs/FIXMEs, e.g. placement separation case https://review.opendev.org/68368015:55
*** openstack changes topic to "Need for review of pre-train TODOs/FIXMEs, e.g. placement separation case https://review.opendev.org/683680 (Meeting topic: kolla)"15:55
*** hjensas has quit IRC15:55
mgoddardI did fix a bunch of things up after we branched stein15:55
mgoddard#link https://review.opendev.org/68368015:55
mgoddardI guess I missed that one, probably because it has no TODO15:56
mgoddarddo we have any others?15:56
mgoddardyoctozepto: I guess this was your topic?15:56
*** itlinux has joined #openstack-kolla15:57
yoctozeptoo/15:57
yoctozeptoyes, mine15:57
yoctozeptosee if we can get more15:58
yoctozeptoteam call to look out for these :-)15:58
mgoddarddo we need to do a periodic grep FIXME|TODO ?15:59
yoctozeptoyeah, let's set CI to fail for old TODO/FIXME still around16:00
*** openstackgerrit has joined #openstack-kolla16:00
openstackgerritViktor Michalek proposed openstack/kolla master: Debian/binary CI: added kolla-build-debian-binary  https://review.opendev.org/68367516:00
yoctozeptothe only sure way to get the job done 8-)16:00
mgoddardhmm, that seems a little heavy handed16:00
yoctozepto(yup, it was a joke but nobody noticed, nobody laughed)16:01
mgoddardah, I have it documented already: https://docs.openstack.org/kolla/latest/contributor/release-management.html#after-rc116:01
yoctozeptoindeed you have16:02
mgoddardwe're over time16:02
mgoddardno open discussion today :(16:02
mgoddardThanks all16:02
openstackgerritScott Solkhon proposed openstack/kolla master: Add Prometheus RabbitMQ exporter  https://review.opendev.org/68479916:02
yoctozeptoand no-pre-ptg talk16:02
yoctozepto:-(16:02
yoctozeptothanks mgoddard16:02
mgoddard#endmeeting16:03
*** openstack changes topic to "Topic for #openstack-kolla is: support: ask.openstack.org | New to Kolla: docs @ https://docs.openstack.org/kolla/latest/ | | Kolla IRC meetngs on Wednesdays @ 15:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b"16:03
openstackMeeting ended Wed Sep 25 16:03:02 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:03
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-25-15.00.html16:03
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-25-15.00.txt16:03
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-25-15.00.log.html16:03
openstackgerritScott Solkhon proposed openstack/kolla-ansible master: Add Prometheus RabbitMQ exporter  https://review.opendev.org/68480016:03
mgoddardyou mean the external images topic? mnasiadka said to push to next week16:03
yoctozeptoah, ok16:03
yoctozeptono need to rush, needs some serious decisions16:03
*** scottsol has quit IRC16:04
generalfuzzmgoddard: thanks16:06
openstackgerritViktor Michalek proposed openstack/kolla master: Debian/binary CI: added kolla-build-debian-binary  https://review.opendev.org/68367516:09
*** goldyfruit_ has joined #openstack-kolla16:12
*** goldyfruit has quit IRC16:15
*** dims has quit IRC16:16
*** dims has joined #openstack-kolla16:23
*** priteau has quit IRC16:23
*** dims has quit IRC16:27
*** ondrejduchon has quit IRC16:27
*** dims has joined #openstack-kolla16:28
*** k_mouza has quit IRC16:35
openstackgerritRadosław Piliszek proposed openstack/kolla master: Use C.UTF-8 locale  https://review.opendev.org/68480816:40
*** dpawlik has quit IRC16:42
yoctozeptomgoddard: ta-da ^16:44
mgoddardyoctozepto: nice catch16:44
mgoddardI think16:44
yoctozeptowe should have done that a long time ago :D16:45
yoctozeptothanks16:45
yoctozeptowell, at least that does break it on ubuntu16:45
mgoddardI'm sure I remember reading some similar issue a couple days ago16:45
yoctozeptoso reversing should unbreak it16:45
mgoddardany unwanted side effects?16:45
yoctozeptoeverything will default to unicode now16:45
yoctozeptowhich some already did (like python on debian for some reason huh)16:46
*** igordc has joined #openstack-kolla16:46
yoctozeptowe test a few things there16:46
yoctozeptoincl. update from POSIX to Unicode16:46
yoctozeptoso we will see16:46
yoctozeptowell, everything is an overstatement obviously16:47
yoctozeptoshould be 'everything that consults locale'16:47
*** eandersson has joined #openstack-kolla16:47
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI  https://review.opendev.org/67637616:48
yoctozeptomgoddard: do you think we should include some releasenote there?16:48
mgoddardyoctozepto: yeah16:48
yoctozeptook, taking note in comment then16:48
yoctozeptolet it build now16:48
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver  https://review.opendev.org/68473016:49
mnasiadkayoctozepto: now you're dwelling in the locale department? Last time I've done that, I forgot what I wanted to do (sucked me in for a week or more) :)16:53
yoctozeptomnasiadka: what? xD16:53
yoctozeptoI did one line in there16:53
mnasiadkaI'm just laughing, that when you touch locale, weird things happen :)16:53
yoctozeptooh yeah16:53
yoctozeptoI know that16:53
*** ivve has joined #openstack-kolla16:53
yoctozeptobeen there, done that16:53
yoctozeptohopefully sticking to C but enabling UTF-8 will be just right16:54
yoctozeptoHOPEFULLY16:54
*** henriqueof has joined #openstack-kolla16:54
*** henriqueof1 has quit IRC16:55
mnasiadkahope dies last, they say16:55
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Copy Nova role as a basis for the Nova cell role  https://review.opendev.org/67565817:02
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Add service-rabbitmq role  https://review.opendev.org/68397417:02
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Factor out nova-cell role  https://review.opendev.org/67565917:02
openstackgerritMark Goddard proposed openstack/kolla-ansible master: WIP: Support shared nova cell controllers  https://review.opendev.org/68480917:02
*** vmixor has quit IRC17:05
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane  https://review.opendev.org/68157317:11
yoctozeptomgoddard: tabulate is now fixed for non-unicode too17:16
yoctozeptowith 0.8.5 landed in pypi17:17
yoctozepto;D17:17
yoctozeptoguy removed the offending line17:17
yoctozeptothat's how you fix it, man17:17
yoctozeptoanyway, just tested vmtp build locally with 0.8.4 forced (and C.UTF-8) and it worked17:18
yoctozeptoso it might be smart to do it anyway for kolla then17:18
yoctozeptohmm, and he fixed the link to point to github one17:25
yoctozeptohe must have published some in-devel version there with 0.8.4, eh17:26
yoctozeptomaybe it was meant to be broken so that kolla could get unicoded ;p17:26
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane  https://review.opendev.org/68157317:42
*** igordc has quit IRC17:52
*** henriqueof1 has joined #openstack-kolla17:53
*** henriqueof has quit IRC17:53
*** hjensas has joined #openstack-kolla18:16
*** luksky has joined #openstack-kolla18:20
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane  https://review.opendev.org/68157318:26
*** k_mouza has joined #openstack-kolla18:37
*** k_mouza has quit IRC18:42
*** gary_perkins has quit IRC18:57
*** gary_perkins has joined #openstack-kolla18:57
*** factor has quit IRC18:59
*** dpawlik has joined #openstack-kolla19:05
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane  https://review.opendev.org/68157319:16
*** hjensas has quit IRC19:16
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver  https://review.opendev.org/68473019:29
*** dpawlik has quit IRC19:29
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver  https://review.opendev.org/68473019:38
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver  https://review.opendev.org/68473019:38
noxoid^ fun fact, redhat backported conntrack support to ce7 3.10 kernels19:42
noxoidfor ovs19:42
*** kplant has quit IRC19:45
mnasiadkanoxoid: still I’m reluctant to set native ovs driver as default :)19:52
noxoidthats fair, ive been using it for a few months though on ce719:53
noxoidhavent seen any issues from it19:53
noxoidi definitely wouldnt set it as the kolla default19:53
noxoidi only set it cause we like to manage our node firewall rules with salt and having other daemons injecting rules makes me sad19:54
*** radeks has quit IRC20:12
*** scottsol has joined #openstack-kolla20:23
*** scottsol has quit IRC20:56
*** igordc has joined #openstack-kolla21:03
*** BjoernT has quit IRC21:17
*** BjoernT has joined #openstack-kolla21:54
*** jonaspaulo has joined #openstack-kolla21:59
*** k_mouza has joined #openstack-kolla22:37
*** k_mouza has quit IRC22:41
*** jonaspaulo has quit IRC22:48
*** luksky has quit IRC23:00
*** igordc has quit IRC23:10
*** goldyfruit_ has quit IRC23:12
*** BjoernT has quit IRC23:21
*** dave-mccowan has joined #openstack-kolla23:28
*** ivve has quit IRC23:28
*** dave-mccowan has quit IRC23:33
openstackgerritMerged openstack/kolla stable/queens: Fix retries when pushing images  https://review.opendev.org/68340623:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!