15:00:48 #startmeeting kolla 15:00:49 Meeting started Wed Oct 16 15:00:48 2019 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:50 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:52 The meeting name has been set to 'kolla' 15:00:53 #topic rollcall 15:00:55 \o 15:01:05 0/ 15:01:19 o/ 15:01:51 o/ 15:01:55 o/ 15:02:15 o/ 15:02:42 #topic agenda 15:02:46 o/ 15:02:50 * Roll-call 15:02:52 * Announcements 15:02:54 ** OpenStack Train final release this week 15:02:56 ** OpenStack Queens becomes EM next week on Friday 15:02:58 * Review action items from last meeting 15:03:00 * CI status 15:03:02 * Train release planning 15:03:04 * Review priorities 15:03:06 * Which images should we mark as maintained in the support matrix? 15:03:08 * Reducing load of build & publishing jobs on CI & Dockerhub (continued) https://etherpad.openstack.org/p/kolla-train-image-evaluation 15:03:10 #topic announcements 15:03:12 #info OpenStack Train final release this week 15:03:14 I think it should happen today 15:03:28 #info OpenStack Train final release this week 15:03:34 #info OpenStack Queens becomes EM next week on Friday 15:03:51 mgoddard: it happened 15:04:00 final release? 15:04:09 #link http://lists.openstack.org/pipermail/openstack-announce/2019-October/002024.html 15:04:14 oh yeah 15:04:19 thanks 15:04:31 np 15:05:06 I think we agreed to stop publishing images in extended maintenance (EM), so we should disable queens soon 15:05:15 next Friday 15:05:24 or later if they postpone 15:05:35 but yeah, we should do this as soon 15:05:51 #topic Review action items from last meeting 15:05:54 yoctozepto to investigate tacker + glance store 15:05:56 mgoddard to EOL pike 15:05:58 mgoddard to backport publishing job timeout increase 15:06:00 mnasiadka to look at not building or publishing deprecated images 15:06:02 we had a few 15:06:05 yoctozepto: I saw your email 15:06:18 so we need shared storage now? 15:07:30 mgoddard: indeed :< 15:07:54 it does not look like tacker is too organized now 15:08:03 Mark Goddard proposed openstack/kolla stable/stein: CI: increase both limits to 3h https://review.opendev.org/688954 15:08:04 they are not too responsive either 15:08:35 should we do the same as glance and enable only on one host? 15:08:53 and mark that limitation in docs? 15:08:59 (or a red note) 15:09:03 rel note :) 15:09:15 if only we had some location independent way of storing images :/ 15:09:28 sounds fair 15:09:57 yoctozepto: are you going to pick it up? 15:11:25 mgoddard: one instance of tacker? 15:11:31 yes 15:11:41 Mark Goddard proposed openstack/kolla stable/rocky: CI: increase both limits to 3h https://review.opendev.org/688956 15:11:53 ok, let me see to that 15:11:56 thanks 15:12:02 I don't know what tacker does really 15:12:07 will need to read up ;-) 15:12:27 #action yoctozepto to fix tacker & NFV scenario job 15:12:36 something NFV something 15:13:31 that's what I know :-) 15:13:31 I proposed a patch for EOL pike, still waiting for +A 15:13:46 just now I did the other action item :) 15:14:05 mnasiadka: I don't think you did yours did you? 15:14:35 #action mnasiadka to look at not building or publishing deprecated images 15:14:49 mgoddard: nope, I started locally, it wasn't a two-liner - and got a lot of work on the OVN PoC this week, but hopefully I can look into that tomorrow 15:15:00 k 15:15:06 #topic CI status 15:15:20 NFV still borked 15:15:38 we have some issues on ubuntu train in kolla? 15:15:47 yes 15:15:53 #link https://bugs.launchpad.net/kolla/+bug/1848295 15:15:53 Launchpad bug 1848295 in kolla train "trove and rally unbuildable for ubuntu binary" [Critical,Triaged] 15:15:54 ubuntu ceph scenario is broken atm 15:15:59 ah, that too 15:16:00 and this 15:16:04 yeah, both 15:16:12 is there a bug for the ceph issue? 15:16:32 looks like no 15:16:33 nope 15:16:35 I don't think so 15:16:37 let me raise it 15:16:44 but proposal is there 15:16:51 :D 15:17:17 thanks 15:17:52 otherwise I think we're ok on the kolla side? 15:18:06 jovial[m] is looking at baremetal boot issues in kayobe CI 15:18:55 mnasiadka: fails bslurp now https://1ef7fd5154b0aa7f60f5-cfb50cc5198678afd23d9756bd7ac094.ssl.cf5.rackcdn.com/688392/6/check/kolla-ansible-ubuntu-source-ceph/f4a235d/primary/logs/ansible/deploy 15:19:08 mgoddard: kolla yup 15:19:32 yeah, not much luck so far. I've got a wild hunch that it may be because we are using `cpu-model` and this is broken with haswell cpus. 15:19:37 yoctozepto: it was the same with previous version of the patch - I'll look into it tomorrow (or you can) 15:19:38 `host-model` 15:20:37 mnasiadka: let it be on you then 15:20:49 jovial[m]: certain it's not a networking issue? 15:20:56 reproduced it locally? 15:21:28 pretty sure - unless the kernel is getting corrupted on transit 15:21:58 would be good if you could pass a checksum argument 15:22:24 we could try switching from pxe to ipxe? 15:22:31 udp -> tcp 15:23:00 anyways, better move on 15:23:11 #topic Train release planning 15:23:16 The train approaches 15:23:39 and the opportunity for puns is running out 15:23:48 yeah, using IPXE seems like a good idea. 15:24:36 we have IPv6 and cells with FFEs, both have a +2 but last to merge will need a rebase 15:25:05 generalfuzz asked about FFE for TLS CA patches 15:25:27 #link https://review.opendev.org/#/c/686024/ 15:26:16 I've had my head down on cells, and am away tomorrow and friday 15:26:20 I sure did 15:26:55 train releases just today 15:27:00 in two weeks we could get it in 15:27:06 or three months 15:27:07 ffe +1 from me ;-) 15:27:12 mnasiadka: then not 15:27:14 but yeah 15:27:20 we have 3 months formally ;p 15:27:26 yes 15:27:28 well we did agree on this friday as a deadline for features 15:27:55 hmm, indeed 15:28:05 regarding order 15:28:14 I am in favor of ipv6 first 15:28:22 then I can rebase cells on that 15:28:27 since mgoddard is gone 15:28:37 CI and dougsz verify that 15:28:49 and mnasiadka + hrw make it w+1 15:28:55 if we have at least two cores sign up to review TLS and its dependency I'd accept an FFE 15:29:33 I can sign up to review it - it would be a waste to merge the installation method in base container, but not the deployment part. 15:30:42 me too 15:30:51 ok, thanks 15:31:21 +1 thanks 15:31:23 mnasiadka: opv6 -> https://review.opendev.org/681573 15:31:28 ipv6* 15:31:41 or we can accept hrw's +2 15:32:25 #topic Review priorities 15:32:54 We have a few feature patches with RP+1 15:33:15 I just added it to the TLS CA patch too 15:33:35 yoctozepto: just to be sure - are we merging ipv6 first and rebasing cells afterwards, or the other way around? :) 15:34:25 I'd say it would be easier to merge cells first, as ipv6 is more search + replace friendly 15:34:31 but whatever works 15:35:34 mnasiadka: ipv6 first for less CI work 15:35:51 mgoddard: you do realize it's more or less the same stuff either way? ;p 15:35:59 you either get it incoming or existing 15:36:01 hello guys 15:36:02 {"msg": "The conditional check 'easy_install_available' failed. The error was: template error while templating string: no test named 'version'. String: {{ not (ansible_distribution == 'Ubuntu' and\n ansible_distribution_major_version is version(18, 'ge'))\n and\n not (ansible_distribution == 'Debian' and\n 15:36:02 ansible_distribution_major_version is version(10, 'ge')) }}\n\nThe error appears to have been in '/usr/share/kolla-ansible/ansible/roles/baremetal/tasks/install.yml': line 104, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Install pip\n ^ here\n"} 15:36:07 oh my 15:36:27 yoctozepto: sort of 15:36:53 ipv6 can lean on grep to verify correctness, cells cannot 15:37:26 anyways 15:37:36 neolee: we are currently in a meeting 15:37:56 mgoddard: what diffs are for 15:38:11 I guess you just have a different approach at things ;p 15:38:11 ok 15:38:12 Will Szumski proposed openstack/kayobe-config-dev master: Switch to using IPXE https://review.opendev.org/688962 15:38:15 anyways, let's continue 15:38:20 sorry 15:38:27 for priorities, there will be some release patches coming. I'll RP+1 those 15:38:42 Then let's focus on bugs 15:39:22 Please check launchpad bug lists, so we can merge any fixes we have proposed 15:39:31 Also gerrit review dashboards 15:40:16 #link https://bugs.launchpad.net/kolla/train/+bugs 15:40:19 #link https://bugs.launchpad.net/kolla-ansible/train/+bugs 15:40:31 and beware any bugs not yet tagged as train 15:41:11 Anything else we need to prioritise? 15:41:29 *chuckles* nfv *chuckles* 15:41:32 but it's on me 15:41:55 #topic Which images should we mark as maintained in the support matrix? 15:42:18 #link https://docs.openstack.org/kolla/latest/support_matrix.html 15:43:05 that is the question 15:43:19 #link https://etherpad.openstack.org/p/kolla-train-image-evaluation 15:43:26 I made a new section 15:43:31 Images to mark as maintained 15:43:33 L58 15:43:47 throw up your suggestions, comment on others 15:44:17 The definition of 'maintained' in this case is 'kolla core team is maintaining versions' 15:44:30 but not tested in CI 15:45:03 I am building the container using local horizon source code. but somehow my customization changes are not coming into the horizon container 15:45:27 pbing19: meeting ongoing 15:45:46 mgoddard: images we care about I guess 15:45:50 we can repeat those T 15:46:09 I guess anything with T & C, we bump C to M 15:46:36 yoctozepto:ok 15:48:06 mgoddard: yup 15:48:08 anyone seeing the list in that etherpad? 15:49:29 yup 15:50:59 are there others we should bump to M? 15:51:42 does 'maintaining versions' have meaning for binary images? 15:52:09 and for source images can we apply M to anything managed by version-check.py? i.e. most official openstack 15:52:36 mgoddard: not sure, I think M needs some knowledge from us 15:52:37 so yeah 15:52:40 at least source 15:52:44 binary excluded 15:52:54 and those we have experience with 15:53:03 (or at least pretend) 15:53:14 what do you mean by knowledge? 15:54:11 mgoddard: have you ever deployed dragonflow 15:54:24 yoctozepto: no, and it is deprecated :) 15:54:32 but surely you could find more :-) 15:54:52 maybe we need a better definition of M 15:55:14 we're not saying it's tested and working, that's T 15:55:25 we're just saying we bump versions from time to time right? 15:56:10 yes 15:56:16 mgoddard: hmm, I think that's too lazy 15:56:35 compare with elasticsearch, it will only be updated if someone proposes it 15:56:36 I thought it would be something like we don't test but we generally know how to troubleshoot 15:56:55 let's see what the list looks like when your idea is applied 15:56:59 aodh though will be updated each release, and point releases within it 15:57:01 and then excluded exotic things 15:57:03 shall we 15:57:26 ok 15:59:57 morning 16:02:19 new list up 16:03:15 I think considering our job as maintainers here might help 16:03:25 we don't promise to fix all bugs 16:03:48 but part of our job for the images involves keeping the versions of deployed software up to date 16:04:17 M could be interpreted as those things we regularly keep updated 16:04:32 C is more community updated 16:04:53 hmm, ok 16:05:19 if you have a better deinition I'm open to it 16:06:21 anyway, we're out of time 16:06:28 thanks all 16:06:34 #endmeeting