*** goldyfruit_ has joined #openstack-kolla | 00:07 | |
*** k_mouza has quit IRC | 00:20 | |
*** goldyfruit_ has quit IRC | 00:33 | |
*** goldyfruit has joined #openstack-kolla | 00:35 | |
*** goldyfruit has quit IRC | 01:00 | |
*** goldyfruit has joined #openstack-kolla | 01:26 | |
*** hongbin has joined #openstack-kolla | 01:49 | |
*** factor has joined #openstack-kolla | 02:07 | |
*** hongbin has quit IRC | 02:36 | |
*** igordc has quit IRC | 02:39 | |
*** BjoernT_ has quit IRC | 02:42 | |
*** BjoernT has joined #openstack-kolla | 02:44 | |
*** BjoernT has quit IRC | 02:47 | |
*** dave-mccowan has quit IRC | 02:53 | |
*** goldyfruit has quit IRC | 04:24 | |
*** dpawlik has joined #openstack-kolla | 05:13 | |
*** skramaja has joined #openstack-kolla | 05:21 | |
*** zbr|ruck has quit IRC | 05:43 | |
*** zbr has joined #openstack-kolla | 05:44 | |
*** shyam89 has joined #openstack-kolla | 05:47 | |
crazik | part | 06:04 |
---|---|---|
*** crazik has left #openstack-kolla | 06:04 | |
*** radeks has joined #openstack-kolla | 06:06 | |
*** luksky has joined #openstack-kolla | 06:21 | |
yoctozepto | morning | 06:27 |
*** pcaruana has joined #openstack-kolla | 06:42 | |
*** gfidente has joined #openstack-kolla | 06:42 | |
*** rpittau|afk is now known as rpittau | 06:47 | |
*** shyam89 has quit IRC | 06:54 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Fix deployment of Grafana https://review.opendev.org/684534 | 06:59 |
*** shyam89 has joined #openstack-kolla | 07:07 | |
*** luksky has quit IRC | 07:27 | |
*** shyam89 has quit IRC | 07:27 | |
mnasiadka | morning | 07:34 |
*** shyam89 has joined #openstack-kolla | 07:40 | |
yoctozepto | new interesting CI multinode failure | 07:42 |
yoctozepto | 2019-09-25 7:02:21 2778 [Warning] Access denied for user 'haproxy'@'secondary2' (using password: NO) | 07:42 |
yoctozepto | 2019-09-25 7:02:21 2779 [Warning] Access denied for user 'haproxy'@'primary' (using password: NO) | 07:42 |
yoctozepto | 2019-09-25 7:02:21 0 [Warning] WSREP: 2.0 (secondary2): State transfer to 1.0 (secondary1) failed: -111 (Connection refused) | 07:42 |
yoctozepto | 2019-09-25 7:02:21 0 [ERROR] WSREP: gcs/src/gcs_group.cpp:gcs_group_handle_join_msg():737: Will never receive state. Need to abort. | 07:42 |
yoctozepto | https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d1/periodic/opendev.org/openstack/kolla-ansible/master/kolla-ansible-centos-source-cinder-lvm/6d115e0/secondary1/logs/kolla/mariadb/mariadb.txt.gz | 07:42 |
yoctozepto | WSREP is killing me | 07:42 |
*** dims has quit IRC | 07:46 | |
*** dims has joined #openstack-kolla | 07:48 | |
*** iniazi_ has joined #openstack-kolla | 07:53 | |
*** shyam89 has quit IRC | 07:56 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Debian: switch fluentd to td-agent https://review.opendev.org/684551 | 07:56 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Debian: switch fluentd to td-agent https://review.opendev.org/684551 | 07:56 |
*** iniazi has quit IRC | 07:57 | |
mnasiadka | yoctozepto: fix WSREP! :) | 07:57 |
mnasiadka | so now we are switching Fluentd to td-agent also on Debian? | 07:57 |
yoctozepto | mnasiadka: yeah and the code got nicer | 07:58 |
mnasiadka | yeah, now td-agent needs to do builds for arm, and it will be even more nice :) | 07:59 |
yoctozepto | indeed, mnasiadka, indeed | 07:59 |
*** ivve has joined #openstack-kolla | 08:01 | |
*** rpittau is now known as rpittau|bbl | 08:01 | |
*** cah_link has joined #openstack-kolla | 08:20 | |
stingrayza | morning :) | 08:20 |
*** luksky has joined #openstack-kolla | 08:23 | |
*** k_mouza has joined #openstack-kolla | 08:26 | |
mgoddard | morning | 08:31 |
*** cah_link has quit IRC | 08:32 | |
*** priteau has joined #openstack-kolla | 08:34 | |
*** shyam89 has joined #openstack-kolla | 08:34 | |
*** zbr is now known as zbr|ruck | 08:40 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Debian: switch fluentd to td-agent on x86_64 https://review.opendev.org/684551 | 08:43 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI https://review.opendev.org/676376 | 08:56 |
*** shyam89 has quit IRC | 09:00 | |
*** hjensas has joined #openstack-kolla | 09:01 | |
*** shyam89 has joined #openstack-kolla | 09:04 | |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: WIP: Simplify multi-region configuration documentation https://review.opendev.org/684637 | 09:04 |
*** hjensas has quit IRC | 09:27 | |
*** hjensas has joined #openstack-kolla | 09:30 | |
shyam89 | Hi | 09:32 |
shyam89 | I have a queens centos based openstack cloud | 09:33 |
*** dpawlik has quit IRC | 09:33 | |
shyam89 | deployed through kolla ansible | 09:33 |
shyam89 | Enabled tls on public endpoints | 09:33 |
shyam89 | I see localsettings file of horizon does not mention about ca chain file path | 09:34 |
shyam89 | and ssl verification parameter is commented | 09:34 |
shyam89 | #OPENSTACK_SSL_NO_VERIFY = True | 09:35 |
shyam89 | #OPENSTACK_SSL_CACERT = /path/to/ca-chain.pem | 09:35 |
*** dpawlik has joined #openstack-kolla | 09:36 | |
shyam89 | In this case, does horizon's requests to other openstack components get ssl verified or not? | 09:37 |
yoctozepto | shyam89: then horizon queens defaults are used | 09:40 |
yoctozepto | check the horizon docs | 09:40 |
shyam89 | okay | 09:42 |
*** luksky has quit IRC | 09:54 | |
*** shyam89 has quit IRC | 09:56 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Debian/binary CI: added kolla-build-debian-binary https://review.opendev.org/683675 | 09:56 |
*** rpittau|bbl is now known as rpittau | 09:57 | |
yoctozepto | ararar, vmtp permabroken on ubuntu atm | 09:58 |
yoctozepto | https://d7c5d6584507e05a6cd4-ee4c5d809145a8a3246cc4d26d65fbe0.ssl.cf2.rackcdn.com/684551/3/check/kolla-build-ubuntu-source/45b8e29/kolla/build/000_FAILED_vmtp.txt.gz | 09:58 |
yoctozepto | mgoddard, mnasiadka ^ | 09:58 |
mgoddard | dammit | 09:58 |
yoctozepto | tabulate breaks install | 09:58 |
yoctozepto | https://pypi.org/project/tabulate/#history | 09:59 |
yoctozepto | and yes they had a release today | 09:59 |
yoctozepto | writing a bug report | 09:59 |
mgoddard | I remember hrw looked at a similar issue | 09:59 |
*** shyam89 has joined #openstack-kolla | 10:03 | |
yoctozepto | mgoddard: I dunno but infra had this messages in Zuul due to python incompats | 10:03 |
yoctozepto | well, it only makes the tiering discussion more urgent | 10:04 |
yoctozepto | ;-) | 10:04 |
mgoddard | add to unbuildable for now? | 10:09 |
yoctozepto | mgoddard: seems best resolution atm | 10:10 |
yoctozepto | looking at it I think it mixed python2 and 3 there | 10:11 |
mgoddard | ok I'll propose | 10:12 |
yoctozepto | not sure if centos affected as well | 10:12 |
yoctozepto | scrolled logs and it builds fine | 10:16 |
yoctozepto | centos i mean | 10:16 |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Skip vmtp image on Ubuntu/Debian source https://review.opendev.org/684687 | 10:20 |
*** k_mouza has quit IRC | 10:21 | |
*** k_mouza_ has joined #openstack-kolla | 10:21 | |
yoctozepto | https://13ca543760d36bac65f9-a7d9059cf68801c3dc92812dc120717e.ssl.cf2.rackcdn.com/684551/3/check/kolla-build-debian-source/ebbbcfa/kolla/build/vmtp.txt.gz | 10:22 |
yoctozepto | debian not broken | 10:22 |
*** BjoernT has joined #openstack-kolla | 10:23 | |
*** pcaruana has quit IRC | 10:23 | |
*** ondrejduchon has joined #openstack-kolla | 10:24 | |
ondrejduchon | mgoddard: Hello, did you check that logging.conf feature? Does it make more sense now? https://review.opendev.org/#/c/679812/ Thank you. | 10:26 |
*** BjoernT_ has joined #openstack-kolla | 10:28 | |
yoctozepto | mgoddard: all in all, it does not look tabulate update itself could have been the cause, this is all that has changed in it: https://bitbucket.org/astanin/python-tabulate/commits/cc69141664ee1bb758e68bc88fa2eea5ba4b2941 | 10:31 |
*** BjoernT has quit IRC | 10:31 | |
*** luksky has joined #openstack-kolla | 10:33 | |
*** pcaruana has joined #openstack-kolla | 10:36 | |
yoctozepto | mgoddard: nah, the guy seems to drive the repo in a weird way really | 10:41 |
yoctozepto | there are more changes between 0.8.3 and 0.8.4 | 10:41 |
yoctozepto | including the offending new line being added in there | 10:41 |
*** dpawlik has quit IRC | 10:56 | |
yoctozepto | argh, tabulate installs just fine on python 3.6 on ubuntu bionic | 10:56 |
yoctozepto | tested on one of my workstations | 10:57 |
yoctozepto | :/ | 10:57 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI https://review.opendev.org/676376 | 10:58 |
*** BjoernT_ has quit IRC | 10:59 | |
mgoddard | yoctozepto: perhaps the issue was found and fixed? | 11:22 |
mnasiadka | that would be fast:) | 11:24 |
yoctozepto | mgoddard: issuing one recheck to check | 11:25 |
yoctozepto | but that would be odd not to change the package version | 11:25 |
yoctozepto | in pypi | 11:25 |
mgoddard | you already said it was weird :) | 11:27 |
*** kplant has joined #openstack-kolla | 11:29 | |
*** dpawlik has joined #openstack-kolla | 11:29 | |
yoctozepto | mgoddard: yeah, because at least debian failure was expected, yet it passed :-) | 11:30 |
yoctozepto | and then local ubuntu works fine | 11:30 |
mgoddard | yup | 11:30 |
yoctozepto | weird all the way | 11:30 |
yoctozepto | the only sure thing is that the offending line was really introduced in that newer package | 11:31 |
yoctozepto | all the rest remains a mystery | 11:31 |
*** goldyfruit has joined #openstack-kolla | 11:32 | |
shyam89 | Hi | 11:35 |
shyam89 | I want to mount a file inside horizon container | 11:35 |
shyam89 | where should place it so that it gets mounted automatically? | 11:36 |
shyam89 | My openstack is queens deployed using kolla ansible | 11:36 |
mnasiadka | I don't think we have support for that in queens | 11:37 |
yoctozepto | shyam89: assuming you don't mind kolla-ansible being unable to do that for you, you can always make docker mount another volume wherever you want | 11:43 |
ivve | hey guys, would it be totally weird to implement an extra when condition on handlers neutron-l3-agent, neutron-openvswitch-agent (and possible more agents) called something like allow_restart_critical_service | bool that wouldn't bounch these critical services during an upgrade in a production environment? | 11:43 |
yoctozepto | ivve: weird-o-meter shows 7 | 11:44 |
kplant | rofl | 11:44 |
yoctozepto | I guess you just want limit | 11:44 |
ivve | :) | 11:44 |
ivve | haha | 11:44 |
ivve | im guessing there is no "alternative" way to performing the upgrade script otherwise | 11:45 |
shyam89 | yoctozepto: Horizon not mounting any directory? | 11:46 |
ivve | since you can't limit it, especially if controller nodes are hyperconverged as compute | 11:46 |
shyam89 | from host to container | 11:46 |
ivve | so stamp of approval or denied? :) | 11:47 |
yoctozepto | ivve: I still don't follow you - you want them to be restarted at some point | 11:47 |
mgoddard | ivve: would it be better to support a rolling restart of those services? | 11:47 |
yoctozepto | mgoddard: indeed what I had in mind with "limit" | 11:48 |
ivve | well, you don't want them restarted if they have instances running on them | 11:48 |
mgoddard | yoctozepto: limit doesn't always play nicely with upgrade | 11:48 |
yoctozepto | shyam89: kolla-ansible queens probably does not allow orchestration of that | 11:48 |
yoctozepto | sorry | 11:48 |
yoctozepto | mgoddard: ah :< | 11:48 |
ivve | openvswitch-agent is not that bad a few seconds if network loss. but l3 agent restarted can result in over a minute loss of network | 11:48 |
mgoddard | ivve: you can run l3-agent and dhcp-agent in HA mode. OVS agent restart shouldn't take down the data plane as it's in the kernel. Should it? | 11:49 |
mgoddard | ouch | 11:49 |
ivve | it does | 11:49 |
shyam89 | yoctozepto: I see | 11:49 |
shyam89 | Thank you | 11:49 |
yoctozepto | shyam89: yw, newer release have more features though | 11:49 |
shyam89 | yoctozepto: Okay | 11:50 |
shyam89 | That is good | 11:50 |
ivve | mgoddard: im assuming l3_ha = true | 11:50 |
ivve | so im not including dhcp, but a simple if statement could check that | 11:51 |
ivve | i haven't taken everything in consideration yet but im testing in a l3_ha and dvr environment | 11:51 |
mgoddard | ivve: surely the l3 agent failover doesn't take a minute? If not you could roll the restart through the agents | 11:51 |
mgoddard | ah, dvr | 11:51 |
ivve | yeah i guess i kinda forgot that, but i always assume dvr :P | 11:52 |
ivve | so only thing really running ha is dhcp | 11:52 |
yoctozepto | ivve: does it mean restarting ANY of dvr l3 agents cause problems for instances using them for one minute? | 11:53 |
ivve | if you restart the "local" l3 agent in a host, the instances it is hosting will lose external network for circa a minute, thats how long it takes for the router to come back up | 11:53 |
ivve | no matter the size of the "cluster" | 11:54 |
ivve | well tested with 5 and 60 nodes | 11:54 |
yoctozepto | hmm, I thought with dvr you still get failover to a random other l3 agent | 11:55 |
yoctozepto | good I did not try | 11:55 |
ivve | :D | 11:55 |
*** shyam89 has quit IRC | 11:55 | |
ivve | what i did was implement that variable and simply ran (for this test rocky and stein) at the same time | 11:56 |
ivve | and migrated nodes from "rocky-l3-agents" to "stein-l3-agents" | 11:57 |
yoctozepto | ivve: https://bugs.launchpad.net/neutron/+bug/1602614 | 11:57 |
openstack | Launchpad bug 1602614 in neutron "DVR + L3 HA loss during failover is higher that it is expected" [High,Fix released] - Assigned to Carl Baldwin (carl-baldwin) | 11:57 |
ivve | and reconfigured with --limit on hosts | 11:57 |
yoctozepto | there seems to be some failover still | 11:57 |
yoctozepto | because it was discussed in newton | 11:57 |
yoctozepto | could be something else wrong there | 11:58 |
yoctozepto | or a new bug | 11:58 |
ivve | so regression or still a problem? | 11:58 |
ivve | or new bug yeah | 11:58 |
yoctozepto | no idea | 11:58 |
ivve | i figured it was by design | 11:58 |
yoctozepto | maybe discuss with neutrinos | 11:58 |
yoctozepto | I doubt | 11:58 |
ivve | posted the question in the channel | 12:02 |
ivve | ill try to get to the bottom of this :) | 12:02 |
*** k_mouza_ has quit IRC | 12:05 | |
*** k_mouza has joined #openstack-kolla | 12:05 | |
*** yoctozepto has quit IRC | 12:10 | |
*** k_mouza has quit IRC | 12:10 | |
*** yoctozepto has joined #openstack-kolla | 12:15 | |
*** goldyfruit has quit IRC | 12:16 | |
*** shyam89 has joined #openstack-kolla | 12:21 | |
*** dave-mccowan has joined #openstack-kolla | 12:22 | |
*** dave-mccowan has quit IRC | 12:26 | |
*** yoctozepto has quit IRC | 12:26 | |
*** yoctozepto has joined #openstack-kolla | 12:26 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver https://review.opendev.org/684730 | 12:29 |
*** vmixor has joined #openstack-kolla | 12:40 | |
*** dave-mccowan has joined #openstack-kolla | 12:43 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver https://review.opendev.org/684730 | 12:50 |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Install RabbitMQ in kolla-toolbox image https://review.opendev.org/684742 | 12:52 |
*** BjoernT has joined #openstack-kolla | 12:53 | |
*** shyam89 has quit IRC | 12:54 | |
*** BjoernT_ has joined #openstack-kolla | 12:56 | |
*** BjoernT has quit IRC | 12:59 | |
*** BjoernT_ has quit IRC | 13:00 | |
*** BjoernT has joined #openstack-kolla | 13:01 | |
*** dpawlik has quit IRC | 13:01 | |
openstackgerrit | Mark Goddard proposed openstack/kolla master: Skip vmtp image on Ubuntu/Debian source https://review.opendev.org/684687 | 13:12 |
openstackgerrit | Viktor Michalek proposed openstack/kolla master: Placement: Debian pkg has migration script in different path https://review.opendev.org/683680 | 13:15 |
*** dpawlik has joined #openstack-kolla | 13:18 | |
*** dave-mccowan has quit IRC | 13:26 | |
*** k_mouza has joined #openstack-kolla | 13:28 | |
*** tonythomas has joined #openstack-kolla | 13:37 | |
mgoddard | ondrejduchon: I'll take a look now | 13:38 |
*** spiette has quit IRC | 13:58 | |
*** spiette has joined #openstack-kolla | 14:00 | |
*** dpawlik has quit IRC | 14:02 | |
*** dpawlik has joined #openstack-kolla | 14:14 | |
*** goldyfruit has joined #openstack-kolla | 14:16 | |
*** chason has joined #openstack-kolla | 14:16 | |
*** dpawlik has quit IRC | 14:18 | |
*** rpittau is now known as rpittau|afk | 14:34 | |
*** luksky has quit IRC | 14:46 | |
*** goldyfruit has quit IRC | 14:46 | |
mgoddard | meeting in 10 minutes | 14:50 |
mgoddard | ^ mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne | 14:50 |
mnasiadka | mgoddard: I won’t be attending, getting more and more sick - but if you need anything - mention my name | 14:51 |
*** scottsol has joined #openstack-kolla | 14:51 | |
mgoddard | mnasiadka: got the tungsten flu? | 14:51 |
mnasiadka | I think I added something to agenda though... inspired by another breakage in sensu | 14:52 |
priteau | I am in another meeting, I'll keep an eye on what's being discussed. | 14:52 |
mnasiadka | mgoddard: something like that | 14:52 |
mgoddard | :( | 14:52 |
mnasiadka | mgoddard: we can move discussion about non-openstack images like sensu/prometheus/etc to next meeting | 14:52 |
mgoddard | ok | 14:53 |
mnasiadka | mgoddard: the idea was to either use their own images (not build ours), or limit the amount of monitoring suites we support | 14:53 |
mnasiadka | Or both :) | 14:53 |
mnasiadka | Seems like a PTG topic though... | 14:53 |
mgoddard | +1 | 14:54 |
mgoddard | can always have an early discussion in meetings though | 14:54 |
mgoddard | #startmeeting kolla | 15:00 |
openstack | Meeting started Wed Sep 25 15:00:45 2019 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
*** openstack changes topic to " (Meeting topic: kolla)" | 15:00 | |
openstack | The meeting name has been set to 'kolla' | 15:00 |
mgoddard | #topic rollcall | 15:00 |
*** openstack changes topic to "rollcall (Meeting topic: kolla)" | 15:00 | |
mgoddard | put your hands in the air like you just don't care | 15:01 |
mgoddard | \o/ | 15:01 |
yoctozepto | o/ | 15:01 |
generalfuzz | o/ | 15:01 |
* yoctozepto still caring a bit | 15:01 | |
egonzalez | o/ | 15:01 |
scottsol | o/ (kinda - at a conference) | 15:01 |
mgoddard | #topic agenda | 15:03 |
*** openstack changes topic to "agenda (Meeting topic: kolla)" | 15:03 | |
mgoddard | * Roll-call | 15:04 |
mgoddard | * Announcements | 15:04 |
jovial[m] | \o | 15:04 |
mgoddard | ** Kolla feature freeze next week | 15:04 |
mgoddard | * Review action items from last meeting | 15:04 |
mgoddard | * CI status | 15:04 |
mgoddard | * Train release planning | 15:04 |
mgoddard | * Review priorities | 15:04 |
mgoddard | * Kayobe Train planning meeting recap | 15:04 |
mgoddard | * Short-term and long-term proposal for managing non-OpenStack projects Docker images (e.g. sensu case) | 15:04 |
mgoddard | * Need for review of pre-train TODOs/FIXMEs, e.g. placement separation case https://review.opendev.org/683680 | 15:04 |
mgoddard | #topic announcements | 15:04 |
*** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:04 | |
mgoddard | #info Kolla feature freeze next week | 15:04 |
mgoddard | This means we will stop accepting patches for features | 15:05 |
mgoddard | Exceptions can be made following core team agreement | 15:06 |
mgoddard | Any others? | 15:06 |
yoctozepto | none here | 15:06 |
generalfuzz | will it be open for enabling wsgi in services? | 15:06 |
mgoddard | #info CentOS 8 released yesterday | 15:06 |
mgoddard | Container images expected early next week | 15:07 |
*** dpawlik has joined #openstack-kolla | 15:07 | |
yoctozepto | generalfuzz: let's discuss exceptions after planned meeting parts | 15:07 |
mgoddard | +1 | 15:07 |
mgoddard | #topic Review action items from last meeting | 15:07 |
*** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)" | 15:07 | |
*** BjoernT has quit IRC | 15:07 | |
mgoddard | mgoddard to email Jeffrey4l & co about kolla-cli | 15:07 |
mgoddard | I didn't, but Jeffrey4l saw the ping and caoyuan emailed me | 15:08 |
mgoddard | It is not ready for release yet, but should be in ~1 month | 15:08 |
mgoddard | they will keep us updated | 15:08 |
yoctozepto | ok | 15:08 |
*** BjoernT has joined #openstack-kolla | 15:08 | |
mgoddard | #topic CI status | 15:08 |
*** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:08 | |
mgoddard | kolla is RED | 15:09 |
mgoddard | need some reviews on https://review.opendev.org/#/c/684687/ | 15:09 |
mgoddard | kayobe is RED | 15:10 |
yoctozepto | will be fixed today, I think I know what could have gone wrong there, just need more time, let's not disable it just now unless you have a different urgent patch | 15:10 |
yoctozepto | that was re kolla | 15:10 |
mgoddard | ok | 15:10 |
yoctozepto | not promising to fix kayobe ;-) | 15:10 |
mgoddard | why not? | 15:10 |
mgoddard | :p | 15:10 |
yoctozepto | I dunno, maybe because I would only break it | 15:11 |
yoctozepto | who knows | 15:11 |
yoctozepto | k-a pretty green, pretty nice | 15:11 |
yoctozepto | egonzalez onto the tacker issue in nfv | 15:11 |
egonzalez | the current patch in k-ansible fixes it | 15:11 |
mgoddard | I saw the k-a patch passes CI now | 15:11 |
yoctozepto | he actually patched it but awaits glance details | 15:11 |
mgoddard | is the kolla patch necessary? | 15:11 |
egonzalez | no, will abandon it | 15:12 |
yoctozepto | no, I think you should abandon it, egonzalez | 15:12 |
mgoddard | k | 15:12 |
yoctozepto | :D | 15:12 |
yoctozepto | ^ example of a little discussion of 3 agreeing people | 15:12 |
mgoddard | here's the k-a patch for tacker: https://review.opendev.org/684275 | 15:13 |
mgoddard | marked RP+1 | 15:13 |
mgoddard | and kayobe | 15:14 |
mgoddard | we were broken by python-ironicclient 3.0.0 | 15:14 |
yoctozepto | mgoddard: it needs glance confirmation | 15:14 |
yoctozepto | tacker patch I mean | 15:14 |
mgoddard | 3.1.0 is working its way to pypi now, also needs a requirements bump | 15:14 |
mgoddard | yoctozepto: what do you mean by glance confirmation? | 15:15 |
egonzalez | tacker uses glance_store and requires some config, not sure about how many of the backends are suported or tested | 15:16 |
yoctozepto | # NOTE(egonzalez): No idea why this is used instead of glance servers, investigating... | 15:16 |
yoctozepto | it looks weird to have this in tacker | 15:16 |
yoctozepto | [glance_store] | 15:16 |
yoctozepto | default_backend = file | 15:16 |
yoctozepto | filesystem_store_datadir = /var/lib/tacker/csar_files/ | 15:16 |
yoctozepto | (weird=wrong) | 15:17 |
mgoddard | is it a way to bypass the glance API? | 15:17 |
mgoddard | i.e. it would be configured the same way as in glance | 15:17 |
egonzalez | i think tacker is storing packages wihtout using glance locally | 15:17 |
egonzalez | locally or remotelly depending the backend | 15:18 |
yoctozepto | maybe fix for now with a big FIXME instead of just a note? | 15:18 |
yoctozepto | if they don't respond timely | 15:18 |
egonzalez | ok | 15:18 |
mgoddard | do we have contacts in the tacker team to get this resolved? | 15:18 |
egonzalez | file type should be fine for now | 15:18 |
*** goldyfruit has joined #openstack-kolla | 15:18 | |
egonzalez | nope, only IRC, people I had contact are not active anymore | 15:19 |
yoctozepto | launchpad and irc already tried | 15:20 |
mgoddard | ok | 15:20 |
yoctozepto | ml is the next step | 15:20 |
mgoddard | +1 | 15:20 |
yoctozepto | egonzalez: could you attack them? | 15:20 |
egonzalez | sure | 15:20 |
yoctozepto | only friendly fire :D | 15:20 |
yoctozepto | thx | 15:21 |
mgoddard | #action egonzalez to ask about glance-store config for tacker on openstack-discuss | 15:21 |
mgoddard | or whatever channel works | 15:21 |
*** openstackgerrit has quit IRC | 15:21 | |
mgoddard | #topic Train release planning | 15:21 |
*** openstack changes topic to "Train release planning (Meeting topic: kolla)" | 15:21 | |
mgoddard | We're getting close | 15:22 |
mgoddard | hopefully we can start on centos 8 / py3 work soon | 15:23 |
mgoddard | is anyone planning to pick that up? | 15:23 |
yoctozepto | mgoddard: is there docker there? | 15:23 |
yoctozepto | or just podman | 15:23 |
*** skramaja has quit IRC | 15:23 | |
mgoddard | yoctozepto: pass | 15:23 |
mgoddard | we use upstream docker | 15:23 |
mgoddard | hopefully it will support centos 8 | 15:24 |
mgoddard | https://download.docker.com/linux/centos/8 doesn't exist yet | 15:24 |
yoctozepto | checked the same | 15:24 |
yoctozepto | yup | 15:24 |
yoctozepto | no el8 packages | 15:24 |
yoctozepto | google returns only | 15:25 |
mgoddard | it's still early | 15:25 |
yoctozepto | "podman replaces docker" stuff | 15:25 |
mgoddard | anyway, we're talking about images first | 15:25 |
yoctozepto | yup, image side | 15:25 |
yoctozepto | not host | 15:25 |
yoctozepto | might be easier | 15:25 |
yoctozepto | will see | 15:25 |
yoctozepto | I saw some infographic showing rhel8 containers on rhel7 hosts | 15:25 |
yoctozepto | so they presumably support that well | 15:26 |
mgoddard | I'm working hard on nova cells | 15:26 |
*** gfidente has quit IRC | 15:26 | |
mgoddard | oh that's good | 15:26 |
yoctozepto | https://developers.redhat.com/blog/2019/08/23/run-red-hat-enterprise-linux-8-in-a-container-on-rhel-7/ | 15:26 |
mgoddard | generalfuzz: now would be a good time to discuss internal TLS | 15:27 |
yoctozepto | #link https://developers.redhat.com/blog/2019/08/23/run-red-hat-enterprise-linux-8-in-a-container-on-rhel-7/ | 15:27 |
mgoddard | (backends) | 15:27 |
generalfuzz | so, after this next patch (https://review.opendev.org/#/c/664517) | 15:27 |
generalfuzz | we should be ready to start working on enabling wsgi in services | 15:28 |
generalfuzz | I figure one or two at a time | 15:28 |
generalfuzz | can this work begin in train? | 15:28 |
mgoddard | it can begin in train, whether it will end in train is another matter :) | 15:29 |
mgoddard | it's a priority feature, so I think we could cut it some slack | 15:29 |
generalfuzz | doubtful we will do all services in train timeline, but can def do some | 15:30 |
mgoddard | but the main priority must be centos 8 & python 3 after feature freeze, and once that's done we should call time | 15:30 |
*** pcaruana has quit IRC | 15:30 | |
mgoddard | it's hard to say yes or no - there are lots of variables | 15:31 |
mgoddard | we don't know how long it will take, or how long we have | 15:31 |
yoctozepto | we could have started rhel8 with ubi it seems: https://developers.redhat.com/blog/2019/07/29/red-hat-universal-base-image-how-it-works-in-3-minutes-or-less/ | 15:31 |
generalfuzz | well, if we agree that an approach like https://review.opendev.org/#/c/664516/ is correct, I will start after https://review.opendev.org/#/c/664517 goes in, and it will land where it lands | 15:33 |
mgoddard | generalfuzz: I'd suggest to start working on it and we can see how we go | 15:33 |
mgoddard | generalfuzz: there were quite a few comments on the backend patch | 15:34 |
mgoddard | I'd quite like to see an approach where we have a role that generates the necessary config that we can reuse | 15:35 |
mgoddard | anyway, probbably not the time for design discussions | 15:35 |
mgoddard | I think we're in agreement | 15:36 |
generalfuzz | yes | 15:36 |
mgoddard | yoctozepto: do you think ipv6 will land? | 15:36 |
generalfuzz | and open to suggestions | 15:36 |
yoctozepto | mgoddard: will try my best | 15:37 |
mgoddard | yoctozepto: ok | 15:37 |
mgoddard | better to wait & get it right than rush though IMO | 15:37 |
yoctozepto | indeed | 15:37 |
mgoddard | I should have new patches for nova cells up soon. I know it's a big one, but I'd appreciate some reviews | 15:38 |
mgoddard | I'm focussing on the nova part, and pushing the mariadb/rabbitmq bits out | 15:39 |
mgoddard | hoping that means we'll get it in sooner rather than later | 15:39 |
mgoddard | We have some train feature priorities for kayobe now, but I'll discuss later | 15:40 |
mgoddard | #topic Review priorities | 15:40 |
*** openstack changes topic to "Review priorities (Meeting topic: kolla)" | 15:40 | |
mgoddard | Any patches we need to focus on this week? | 15:40 |
mgoddard | jovial[m]: kayobe included | 15:40 |
* yoctozepto focuses on ipv6 (proponent side) | 15:41 | |
generalfuzz | I would like to get this in: https://review.opendev.org/#/c/664517 | 15:41 |
yoctozepto | whoa, wait for my reviews :D | 15:42 |
*** chason has quit IRC | 15:42 | |
mgoddard | added RP+1 | 15:42 |
yoctozepto | for some reason it does not conflict with ipv6 | 15:43 |
yoctozepto | +0.5 for that | 15:43 |
generalfuzz | nice | 15:43 |
jovial[m] | kayobe: I will look at https://review.opendev.org/#/c/682701/ now - seems quite important | 15:43 |
mgoddard | jovial[m]: thanks | 15:44 |
mgoddard | jovial[m]: bear in mind kayobe CI is busted, so leave off +A for now | 15:44 |
mgoddard | we should probably add RP+1 to bug fixes as we get nearer the end of the release | 15:44 |
jovial[m] | We are just waiting for the new ironic-inspector package? | 15:44 |
mgoddard | jovial[m]: ironicclient | 15:45 |
*** ivve has quit IRC | 15:45 | |
mgoddard | #topic Kayobe Train planning meeting recap | 15:46 |
*** openstack changes topic to "Kayobe Train planning meeting recap (Meeting topic: kolla)" | 15:46 | |
mgoddard | Yesterday we had a planning meeting for kayobe | 15:47 |
mgoddard | #link https://etherpad.openstack.org/p/kayobe-train-planning | 15:47 |
* yoctozepto eager to learn details | 15:47 | |
mgoddard | discussed a number of potential features | 15:47 |
mgoddard | centos8 was interesting | 15:48 |
yoctozepto | s/feature/failure/g | 15:48 |
mgoddard | decided we'd support centos 7 and 8 for at least one release, to allow a smooth upgrade of hosts | 15:48 |
mgoddard | centos7 will be py2, centos8 py3 | 15:48 |
mgoddard | expecting to need to reinstall hosts to get from 7 to 8 | 15:49 |
mgoddard | so we need to think about how we do that smoothly | 15:49 |
yoctozepto | drop centos7 in v? | 15:49 |
mgoddard | maybe | 15:49 |
mgoddard | or u | 15:49 |
yoctozepto | I think ussuri should have it unless we hit something big | 15:50 |
yoctozepto | like python2 more dead than now | 15:50 |
mgoddard | questions about running centos 8 containers on centos 8 hosts | 15:50 |
mgoddard | and implications for live migration with different versions of qemu | 15:50 |
mgoddard | then some discussions of python 3, and where we use it | 15:51 |
mgoddard | "Difficult to estimate - medium size ball of string" | 15:51 |
mgoddard | We talked about cleaning up our integration points with kolla-ansible | 15:52 |
mgoddard | a few ideas proposed, but none without their corner cases | 15:52 |
mgoddard | finally, we voted on priorities for the train cycle | 15:53 |
mgoddard | results were: | 15:53 |
mgoddard | centos8 | 15:53 |
mgoddard | python3 | 15:53 |
mgoddard | Support multiple environments from a single kayobe configuration | 15:53 |
mgoddard | Improved support for integrating with Ceph-Ansible | 15:53 |
mgoddard | Next generation discovery & network configuration | 15:53 |
mgoddard | Easier customization of Kayobe commands | 15:53 |
mgoddard | Nova cells support | 15:53 |
mgoddard | As usual, TBC depending on having someone to do the work | 15:54 |
mgoddard | I added those to the kolla whiteboard | 15:54 |
mgoddard | Any Qs? | 15:54 |
mgoddard | #topic Need for review of pre-train TODOs/FIXMEs, e.g. placement separation case https://review.opendev.org/683680 | 15:55 |
*** openstack changes topic to "Need for review of pre-train TODOs/FIXMEs, e.g. placement separation case https://review.opendev.org/683680 (Meeting topic: kolla)" | 15:55 | |
*** hjensas has quit IRC | 15:55 | |
mgoddard | I did fix a bunch of things up after we branched stein | 15:55 |
mgoddard | #link https://review.opendev.org/683680 | 15:55 |
mgoddard | I guess I missed that one, probably because it has no TODO | 15:56 |
mgoddard | do we have any others? | 15:56 |
mgoddard | yoctozepto: I guess this was your topic? | 15:56 |
*** itlinux has joined #openstack-kolla | 15:57 | |
yoctozepto | o/ | 15:57 |
yoctozepto | yes, mine | 15:57 |
yoctozepto | see if we can get more | 15:58 |
yoctozepto | team call to look out for these :-) | 15:58 |
mgoddard | do we need to do a periodic grep FIXME|TODO ? | 15:59 |
yoctozepto | yeah, let's set CI to fail for old TODO/FIXME still around | 16:00 |
*** openstackgerrit has joined #openstack-kolla | 16:00 | |
openstackgerrit | Viktor Michalek proposed openstack/kolla master: Debian/binary CI: added kolla-build-debian-binary https://review.opendev.org/683675 | 16:00 |
yoctozepto | the only sure way to get the job done 8-) | 16:00 |
mgoddard | hmm, that seems a little heavy handed | 16:00 |
yoctozepto | (yup, it was a joke but nobody noticed, nobody laughed) | 16:01 |
mgoddard | ah, I have it documented already: https://docs.openstack.org/kolla/latest/contributor/release-management.html#after-rc1 | 16:01 |
yoctozepto | indeed you have | 16:02 |
mgoddard | we're over time | 16:02 |
mgoddard | no open discussion today :( | 16:02 |
mgoddard | Thanks all | 16:02 |
openstackgerrit | Scott Solkhon proposed openstack/kolla master: Add Prometheus RabbitMQ exporter https://review.opendev.org/684799 | 16:02 |
yoctozepto | and no-pre-ptg talk | 16:02 |
yoctozepto | :-( | 16:02 |
yoctozepto | thanks mgoddard | 16:02 |
mgoddard | #endmeeting | 16:03 |
*** openstack changes topic to "Topic for #openstack-kolla is: support: ask.openstack.org | New to Kolla: docs @ https://docs.openstack.org/kolla/latest/ | | Kolla IRC meetngs on Wednesdays @ 15:00 UTC - see agenda @ https://goo.gl/OXB0DL - IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:03 | |
openstack | Meeting ended Wed Sep 25 16:03:02 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:03 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-25-15.00.html | 16:03 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-25-15.00.txt | 16:03 |
openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2019/kolla.2019-09-25-15.00.log.html | 16:03 |
openstackgerrit | Scott Solkhon proposed openstack/kolla-ansible master: Add Prometheus RabbitMQ exporter https://review.opendev.org/684800 | 16:03 |
mgoddard | you mean the external images topic? mnasiadka said to push to next week | 16:03 |
yoctozepto | ah, ok | 16:03 |
yoctozepto | no need to rush, needs some serious decisions | 16:03 |
*** scottsol has quit IRC | 16:04 | |
generalfuzz | mgoddard: thanks | 16:06 |
openstackgerrit | Viktor Michalek proposed openstack/kolla master: Debian/binary CI: added kolla-build-debian-binary https://review.opendev.org/683675 | 16:09 |
*** goldyfruit_ has joined #openstack-kolla | 16:12 | |
*** goldyfruit has quit IRC | 16:15 | |
*** dims has quit IRC | 16:16 | |
*** dims has joined #openstack-kolla | 16:23 | |
*** priteau has quit IRC | 16:23 | |
*** dims has quit IRC | 16:27 | |
*** ondrejduchon has quit IRC | 16:27 | |
*** dims has joined #openstack-kolla | 16:28 | |
*** k_mouza has quit IRC | 16:35 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Use C.UTF-8 locale https://review.opendev.org/684808 | 16:40 |
*** dpawlik has quit IRC | 16:42 | |
yoctozepto | mgoddard: ta-da ^ | 16:44 |
mgoddard | yoctozepto: nice catch | 16:44 |
mgoddard | I think | 16:44 |
yoctozepto | we should have done that a long time ago :D | 16:45 |
yoctozepto | thanks | 16:45 |
yoctozepto | well, at least that does break it on ubuntu | 16:45 |
mgoddard | I'm sure I remember reading some similar issue a couple days ago | 16:45 |
yoctozepto | so reversing should unbreak it | 16:45 |
mgoddard | any unwanted side effects? | 16:45 |
yoctozepto | everything will default to unicode now | 16:45 |
yoctozepto | which some already did (like python on debian for some reason huh) | 16:46 |
*** igordc has joined #openstack-kolla | 16:46 | |
yoctozepto | we test a few things there | 16:46 |
yoctozepto | incl. update from POSIX to Unicode | 16:46 |
yoctozepto | so we will see | 16:46 |
yoctozepto | well, everything is an overstatement obviously | 16:47 |
yoctozepto | should be 'everything that consults locale' | 16:47 |
*** eandersson has joined #openstack-kolla | 16:47 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Ceph-Ansible CI https://review.opendev.org/676376 | 16:48 |
yoctozepto | mgoddard: do you think we should include some releasenote there? | 16:48 |
mgoddard | yoctozepto: yeah | 16:48 |
yoctozepto | ok, taking note in comment then | 16:48 |
yoctozepto | let it build now | 16:48 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver https://review.opendev.org/684730 | 16:49 |
mnasiadka | yoctozepto: now you're dwelling in the locale department? Last time I've done that, I forgot what I wanted to do (sucked me in for a week or more) :) | 16:53 |
yoctozepto | mnasiadka: what? xD | 16:53 |
yoctozepto | I did one line in there | 16:53 |
mnasiadka | I'm just laughing, that when you touch locale, weird things happen :) | 16:53 |
yoctozepto | oh yeah | 16:53 |
yoctozepto | I know that | 16:53 |
*** ivve has joined #openstack-kolla | 16:53 | |
yoctozepto | been there, done that | 16:53 |
yoctozepto | hopefully sticking to C but enabling UTF-8 will be just right | 16:54 |
yoctozepto | HOPEFULLY | 16:54 |
*** henriqueof has joined #openstack-kolla | 16:54 | |
*** henriqueof1 has quit IRC | 16:55 | |
mnasiadka | hope dies last, they say | 16:55 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Copy Nova role as a basis for the Nova cell role https://review.opendev.org/675658 | 17:02 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add service-rabbitmq role https://review.opendev.org/683974 | 17:02 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Factor out nova-cell role https://review.opendev.org/675659 | 17:02 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: WIP: Support shared nova cell controllers https://review.opendev.org/684809 | 17:02 |
*** vmixor has quit IRC | 17:05 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane https://review.opendev.org/681573 | 17:11 |
yoctozepto | mgoddard: tabulate is now fixed for non-unicode too | 17:16 |
yoctozepto | with 0.8.5 landed in pypi | 17:17 |
yoctozepto | ;D | 17:17 |
yoctozepto | guy removed the offending line | 17:17 |
yoctozepto | that's how you fix it, man | 17:17 |
yoctozepto | anyway, just tested vmtp build locally with 0.8.4 forced (and C.UTF-8) and it worked | 17:18 |
yoctozepto | so it might be smart to do it anyway for kolla then | 17:18 |
yoctozepto | hmm, and he fixed the link to point to github one | 17:25 |
yoctozepto | he must have published some in-devel version there with 0.8.4, eh | 17:26 |
yoctozepto | maybe it was meant to be broken so that kolla could get unicoded ;p | 17:26 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane https://review.opendev.org/681573 | 17:42 |
*** igordc has quit IRC | 17:52 | |
*** henriqueof1 has joined #openstack-kolla | 17:53 | |
*** henriqueof has quit IRC | 17:53 | |
*** hjensas has joined #openstack-kolla | 18:16 | |
*** luksky has joined #openstack-kolla | 18:20 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane https://review.opendev.org/681573 | 18:26 |
*** k_mouza has joined #openstack-kolla | 18:37 | |
*** k_mouza has quit IRC | 18:42 | |
*** gary_perkins has quit IRC | 18:57 | |
*** gary_perkins has joined #openstack-kolla | 18:57 | |
*** factor has quit IRC | 18:59 | |
*** dpawlik has joined #openstack-kolla | 19:05 | |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Implement IPv6 support in the control plane https://review.opendev.org/681573 | 19:16 |
*** hjensas has quit IRC | 19:16 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver https://review.opendev.org/684730 | 19:29 |
*** dpawlik has quit IRC | 19:29 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver https://review.opendev.org/684730 | 19:38 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Allow to set neutron_firewall_driver https://review.opendev.org/684730 | 19:38 |
noxoid | ^ fun fact, redhat backported conntrack support to ce7 3.10 kernels | 19:42 |
noxoid | for ovs | 19:42 |
*** kplant has quit IRC | 19:45 | |
mnasiadka | noxoid: still I’m reluctant to set native ovs driver as default :) | 19:52 |
noxoid | thats fair, ive been using it for a few months though on ce7 | 19:53 |
noxoid | havent seen any issues from it | 19:53 |
noxoid | i definitely wouldnt set it as the kolla default | 19:53 |
noxoid | i only set it cause we like to manage our node firewall rules with salt and having other daemons injecting rules makes me sad | 19:54 |
*** radeks has quit IRC | 20:12 | |
*** scottsol has joined #openstack-kolla | 20:23 | |
*** scottsol has quit IRC | 20:56 | |
*** igordc has joined #openstack-kolla | 21:03 | |
*** BjoernT has quit IRC | 21:17 | |
*** BjoernT has joined #openstack-kolla | 21:54 | |
*** jonaspaulo has joined #openstack-kolla | 21:59 | |
*** k_mouza has joined #openstack-kolla | 22:37 | |
*** k_mouza has quit IRC | 22:41 | |
*** jonaspaulo has quit IRC | 22:48 | |
*** luksky has quit IRC | 23:00 | |
*** igordc has quit IRC | 23:10 | |
*** goldyfruit_ has quit IRC | 23:12 | |
*** BjoernT has quit IRC | 23:21 | |
*** dave-mccowan has joined #openstack-kolla | 23:28 | |
*** ivve has quit IRC | 23:28 | |
*** dave-mccowan has quit IRC | 23:33 | |
openstackgerrit | Merged openstack/kolla stable/queens: Fix retries when pushing images https://review.opendev.org/683406 | 23:48 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!