15:07:41 #startmeeting kolla 15:07:41 Meeting started Wed Jun 30 15:07:41 2021 UTC and is due to finish in 60 minutes. The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:07:41 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:07:41 The meeting name has been set to 'kolla' 15:07:47 #chair mgoddard 15:07:47 Current chairs: mgoddard yoctozepto 15:07:53 #topic Roll-call 15:07:55 o/ 15:07:57 erm 15:08:03 oop 15:08:04 s 15:08:13 no problem 15:08:20 I'm not sure where today went :) 15:08:26 let's harvest participants 15:08:32 crazy day here as well :-) 15:08:43 mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt 15:08:52 koalas assemble 15:08:52 meeting now, folks :-) 15:08:55 <3 15:08:59 seriously? 15:09:02 :) 15:09:06 but you have to whisper "assemble" 15:09:41 #topic agenda 15:09:43 * Roll-call 15:09:45 * Agenda 15:09:47 * Announcements 15:09:49 * Review action items from the last meeting 15:09:51 * CI status 15:09:53 * Release tasks 15:09:55 * Xena cycle planning 15:09:57 * Open discussion 15:09:59 #topic announcements 15:10:41 #info Kolla Wallaby releases now available 15:11:34 YEAH 15:11:42 #link http://lists.openstack.org/pipermail/openstack-discuss/2021-June/023289.html 15:11:49 #topic Review action items from the last meeting 15:12:03 there were none 15:12:07 #topic CI status 15:12:57 need to merge patches I think? 15:13:07 hmm 15:13:25 on the stable branches 15:14:19 for kolla 15:14:20 #link https://review.opendev.org/q/topic:%22ci-emergency-fix-for-zuul-4-6%22 15:15:19 added my +2 15:16:08 upgrades to Ussuri fail sporadically on Ubuntu due to Neutron migrations failing 15:16:12 monasca scenario fails all the time 15:16:38 yes, sir 15:16:52 something to do with the ES index 15:16:58 unsure of the cause 15:17:15 I think dougsz is aware 15:17:32 ok 15:17:52 #topic Release tasks 15:17:55 ubuntu ussuri upgrades are a mystery though 15:18:55 I think we still have some R-17 release patches open 15:19:19 Yes, and Heat is a mystery 15:19:41 https://review.opendev.org/c/openstack/kolla-ansible/+/796422 15:19:43 refresh our memory please 15:20:00 and only on debuntu 15:20:37 https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d4c/796415/3/check/kolla-ansible-debian-source/d4c358b/primary/logs/kolla/heat/heat-api-error.txt 15:20:46 Can we at least merge this one? https://review.opendev.org/c/openstack/kolla-ansible/+/796422 15:20:51 https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d4c/796415/3/check/kolla-ansible-debian-source/d4c358b/primary/logs/kolla/heat/apache-error.txt 15:21:34 mgoddard: I guess we can 15:24:29 why enable chrony again? 15:24:37 has anyone tried speaking to the heat team? 15:25:20 yoctozepto: only for upgrades, to test the cleanup command 15:25:53 mgoddard: well, once somebody will debug it properly, and find some meaningful error message - it would make sense, but I haven't made it to this point. 15:28:38 mgoddard: are not we testing that in wallaby? 15:28:50 yoctozepto: yes, testing the wallaby code 15:28:53 ah, you want to avoid regressions 15:28:59 in master we test master code 15:29:04 indeed 15:29:12 there is one issue 15:29:27 enabled will kill the host one 15:29:33 "we disabled CI testing in master because we test ussuri" :) 15:29:51 ? 15:30:00 I'm being silly 15:30:23 I see 15:30:33 if that is an issue in master then it is an issue in wallaby also 15:30:59 yeah, we don't restore it for users 15:31:11 ok 15:31:21 as long as it works in ci 15:32:09 hold on, don't we disable the host one first, then start it after cleaning the container? 15:32:14 to avoid having them both running 15:33:23 on the heat issue, should we disable testing of heat in CI for now? 15:34:13 that's also an option 15:34:26 I'll do a recheck to check if the issue still persists, if it does will disable heat testing in CI 15:34:27 it's only doing openstack orchestration service list 15:35:37 does it break on the api call? 15:35:41 or earlier? 15:36:04 looks like api call https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d4c/796415/3/check/kolla-ansible-debian-source/d4c358b/primary/logs/kolla/heat/heat-api-error.txt 15:36:19 yeah, it's breaking on that 15:37:34 Any other release tasks? 15:37:38 can we call something else? 15:37:43 on that heat api 15:38:27 it's crashing, so what do you want to call? 15:38:47 I'm just curious if it's failing listing these 15:39:09 like the whole api wsgi service is not being run, it's crashing on mod_wsgi running the binary 15:39:26 yeah, but mgoddard said only after that api call 15:39:31 and it looks like it 15:40:08 - - - [22/Jun/2021:06:36:44 +0000] "GET / HTTP/1.1" 500 531 1321112 "-" "curl-healthcheck" 15:40:08 seems to affect ubuntu only 15:40:09 hmm 15:40:17 it seems to work on version discovery from curl 15:40:22 or does it? 15:40:24 500 15:40:33 well, docker healthcheck does something 15:40:35 * yoctozepto blind 15:40:45 let's see the same for centos 15:41:13 and gets 500 15:42:54 I think we should move on 15:43:42 I propose we disable the heat service list on ubuntu, and add it to the workarounds list to be fixed before release 15:44:17 will do 15:45:36 - - - [22/Jun/2021:06:35:03 +0000] "GET / HTTP/1.1" 300 117 2108380 "-" "curl-healthcheck" 15:45:39 working gets 300 15:45:43 so it's utterly broken 15:45:47 hmm 15:45:55 must be mod_wsgi version 15:45:57 re other release tasks, we should probably do some stable releases soon 15:46:51 #action mgoddard check open backports & propose stable releases 15:48:24 #topic Xena cycle planning 15:49:20 Do we need to take some time to look at open patches, and try to agree some things to focus on 15:49:23 ? 15:49:47 Would like to get the Lets Encrypt integration in Xena :) 15:49:53 +1 15:49:56 https://review.opendev.org/c/openstack/kolla-ansible/+/741340 15:50:02 part 1 is there 15:50:18 we could borrow an idea from ironic 15:50:54 how so? 15:50:56 core reviewers can sign up to review particular features 15:51:01 ah 15:51:45 which means that the developer of the feature has an idea if their patch is likely to get merged, and who to ask/bug for help/reviews 15:52:00 it might also help focus attention of cores 15:52:20 and maybe get more cores involved in reviewing 15:52:35 since the review firehose can be a bit overwhelming 15:52:45 true that 15:53:07 and it is easy to focus on easy wins 15:53:24 just to get them out of the queue :) 15:53:27 any thoughts? 15:53:43 (sorry for mentioning but I need to before I forget, there is a thread on ml which could use mgoddard replying - some kayobe slides are gone) 15:53:56 mgoddard: I agree we could adopt this strategy 15:53:57 k, thanks 15:54:09 yw 15:54:22 (the one from Amit) 15:54:54 I see it 15:56:11 right now we don't have a great view of features being developed 15:57:38 well, something to think about 15:57:44 bright ideas on a postcard please 15:57:47 Well, it would be nice to have a list of features that need reviewing on some board/etherpad 15:58:02 we previously had priorities on https://etherpad.opendev.org/p/KollaWhiteBoard 15:58:11 but we stopped doing priorities :) 15:58:24 I guess they had two flaws 15:58:36 1. no guarantee that anyone picks up priorities 15:59:05 2. people often contribute unanticipated features that do not get added to the list 15:59:42 yeah, need to adapt that approach 15:59:45 to reality 15:59:54 so maybe we need more of a separation between 'community priorities as a guide for project direction' 15:59:54 ok, we have to wrap up I guess 16:00:00 vs work in progress 16:00:01 yup 16:00:34 sounds like a good challenge for the next PTL :) 16:00:44 on that note, thanks all 16:00:52 #endmeeting