09:01:00 <jakeyip> #startmeeting magnum
09:01:00 <opendevmeet> Meeting started Wed Feb 21 09:01:00 2024 UTC and is due to finish in 60 minutes.  The chair is jakeyip. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:01:00 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:01:00 <opendevmeet> The meeting name has been set to 'magnum'
09:01:05 <jakeyip> #link https://etherpad.opendev.org/p/magnum-weekly-meeting
09:01:08 <jakeyip> Please put your topics into to Agenda
09:01:11 <jakeyip> #topic Roll Call
09:01:13 <jakeyip> o/
09:01:18 <dalees> o/
09:01:31 <jakeyip> mnasiadka: courtesy ping :)
09:04:41 <jakeyip> hm let's start without him. I think he'll join later
09:06:07 <jakeyip> #topic Announcements
09:06:19 <jakeyip> https://releases.openstack.org/caracal/schedule.html
09:06:46 <jakeyip> Caracal-3 milestone and Feature freeze
09:07:08 <jakeyip> is next week. So this week we will need to prioritise anything that we want to get in this cycle
09:07:28 <jakeyip> especially deprecations, cos those need to sit for at least 1 cycle
09:07:36 <dalees> feature freeze next week; hmm I need to get back to that control plane resize spec!
09:07:47 <mnasiadka> sorry, got distracted
09:08:27 <dalees> do we have deprecation patchsets proposed this cycle that need attention?
09:08:33 <jakeyip> please help check under the couch cushions for any reviews that might be missed
09:08:46 <jakeyip> hmm, I can't think of any now, I will ping
09:08:57 <jakeyip> (if I find any)
09:08:59 <mnasiadka> deprecations? we deprecated nearly all drivers
09:09:16 <mnasiadka> and I think we have patches from dropping everything apart heat
09:09:28 <mnasiadka> we could have a look at any labels if we still need them
09:09:45 <dalees> yes, thanks for linking those removals in agenda; i need to look at them and review.
09:10:01 <jakeyip> I think there's atomic still too, but I can do that
09:10:17 <mnasiadka> ah right
09:10:23 <jakeyip> yes thanks for the great work with ironic, coreos dropping mnasiadka
09:10:45 <mnasiadka> well, everyone likes to remove some files every now and then ;-)
09:11:20 <dalees> you're a net positive in my books, mnasiadka. Even if not in LoC
09:11:32 <mnasiadka> haha, thanks :)
09:11:59 <jakeyip> removing lines should count for more effort than adding lines. any monkey can add lines.
09:12:31 <jakeyip> (they don't have to work...) but removing... people get mad if things break :)
09:13:05 <jakeyip> ok let's get on to next item
09:13:13 <jakeyip> #topic Review Action Items
09:13:26 <jakeyip> Delete trusts https://review.opendev.org/c/openstack/magnum/+/905032
09:14:03 <jakeyip> dalees: thanks for this. I had a question - do you know when trusts are not deleted?
09:14:37 <dalees> yes, I saw your question but need to get the code back into my head to reply. I'll do this tomorrow (only had 1/2 day today).
09:14:51 <jakeyip> you mentioned in your commit "Not every cluster will clean up trusts, even though they are always created."
09:15:19 <dalees> yeah, there are two code paths to delete completion; one had the delete trusts in it, and another conductor code path didn't.
09:15:41 <jakeyip> I see, may be helpful to have that in commit message
09:15:57 <dalees> i think it was a periodic update that affected the outcome.
09:16:13 <dalees> Yes, good point. Once I review and reply I will update the commit message too
09:16:17 <jakeyip> I do see trusts building up in my DB although I haven't dug too deeply (like you have), so I would appreciate understanding why
09:16:34 <jakeyip> but... if it is too difficult or takes too much time, you don't have to answer
09:16:59 <jakeyip> it's good to go in.
09:17:07 <jakeyip> is there anything else re this? since you added that topic?
09:17:09 <dalees> all good, I'll update it tomorrow!
09:17:17 <mnasiadka> Do we need a periodic job to compare trusts with existing clusters to clean up old trusts that accumulated on environments?
09:17:35 <dalees> no nothing else on it, just needed review attention ;)
09:18:05 <dalees> mnasiadka: that's interesting; the trusts are created in the `magnum` domain....
09:18:26 <jakeyip> hmm I would say it is a good idea yet I am scared. :P
09:18:58 <dalees> maybe a CLI once-off, not a periodic.
09:19:23 <jakeyip> I think would be good as a `magnum-manage` kind of tool
09:19:44 <dalees> (and let the operator decide if the output was trustworthy or worth actioning)
09:20:17 <jakeyip> that's how nova does it too for purging old records
09:20:32 <jakeyip> there isn't such a thing in magnum yet so it'll be something new.
09:21:08 <dalees> i think it's worth recording that idea, we havea a lot of old trusts too. They aren't too harmful, but would be nice to clean them up after this bug is sorted out.
09:21:31 <dalees> I'd like to visit that sometime, but not right now.
09:23:16 <jakeyip> yeap cool. let's go on to the next review?
09:23:34 <dalees> ok
09:23:41 <jakeyip> Drop fedora ironic driver https://review.opendev.org/c/openstack/magnum/+/907319
09:23:46 <jakeyip> Drop coreos driver https://review.opendev.org/c/openstack/magnum/+/907300
09:24:44 <jakeyip> I've reviewed those, appreciate another pair of eyes :)
09:24:48 <jakeyip> nothing else from me
09:25:19 <dalees> sounds like a job for me; i'll add myself to them.
09:26:05 <jakeyip> thanks :)
09:26:16 <jakeyip> Update OCCM registery https://review.opendev.org/c/openstack/magnum/+/909344
09:27:56 <jakeyip> a bit of history - as you may know, we don't prefer to update tags, etc. this is due to the CT design - if a CT doesn't have a certain label and Magnum changes the default, that will potentially break the CT
09:28:40 <jakeyip> DON'T BREAK PRODUCTION is rule number 1 in my books :)
09:29:24 <jakeyip> but having said that, now that we have good tests for new versions of Kubernetes, we will need a change to pull in newer versions of OCCM, as the registry has moved
09:30:35 <dalees> k, seems sensible. If it only breaks older than 1.24 then that's fine for Magnum C
09:30:50 <jakeyip> guess what this comes down to is a decision to change this 'policy', and supplement this change with reno, etc, so operators upgrading will have a warning
09:32:04 <jakeyip> I would like it to be a bigger (more generic) announcement than for these labels, cos flannel faces the same issue too.
09:32:57 <jakeyip> so a big generic announcement that - "Magnum Core Team is not keeping labels static anymore", then we don't have to create reno for each label we change the default of
09:33:52 <jakeyip> of cos it doesn't mean we go all crazy updating all labels, we only do for those that are seriously holding us back in the beginning. like this one
09:35:22 <jakeyip> comments?
09:35:53 <dalees> ok - i don't have a big opinion on this. We override nearly every label and mirror all the images so we also set CONTAINER_INFRA_PREFIX :)
09:36:39 <jakeyip> yeah it doesn't affect Nectar too, but I am just concerned for others
09:37:30 <jakeyip> I think mnasiadka would prob be ok with this since he has +2. I will add reno to that review
09:37:36 <jakeyip> next
09:38:02 <mnasiadka> yeah, that looks ok
09:38:24 <mnasiadka> jakeyip: did you want +w https://review.opendev.org/c/openstack/magnum/+/905032 - or is there a reason we're holding from this?
09:39:05 <dalees> ^ for that, I'll answer Jake's Q tomorrow and update the commit message.
09:39:20 <dalees> so hold +w for a day
09:39:27 <jakeyip> mnasiadka: holding off until dalees have a chance to update commit message. if dalees can't remember then he can +w :)
09:40:39 <jakeyip> #agreed Change 'policy' to allow update defaults for labels if it blocks progress.
09:41:10 <jakeyip> #action jakeyip to update reno for https://review.opendev.org/c/openstack/magnum/+/909344 with decision
09:41:26 <jakeyip> ok, continuing on
09:41:29 <jakeyip> Switch Calico deployment to Helm chart https://review.opendev.org/q/topic:%22calico-helm%22
09:41:41 <jakeyip> dalees: how is calico working for you? cos it isn't working in CI
09:42:29 <dalees> well, I'm using much newer manifests than in Magnum upstream.
09:42:30 <jakeyip> I assume StackHPC has those patches to make it work.
09:43:19 <jakeyip> manifests? can contribute?
09:44:28 <dalees> yeah, I can. The calico_tag is used to switch which manifest is included. There's a limit though to how many Calico versions you can have supported, because they're big and overload Heat template size
09:46:25 <jakeyip> ok. after that is done can you look at the contributed ones, one of them is to remove calico_v3_3
09:47:09 <dalees> I'll check, but I've got a manifest for Calico 3.13, 3.23 and 3.26 that we're using.
09:47:12 <jakeyip> hm, does that need deprecation cycle. it's very old. deferring to both of you, since you are more experienced with calico. (I run flannel)
09:47:39 <dalees> Yeah, 3.3 is not so helpful, but maybe it need to stay and we just add one newer to Magnum.
09:49:12 <jakeyip> ok. I can send in a deprecation reno for that, get it in before next week.
09:49:27 <jakeyip> #action jakeyip deprecate calico 3.3
09:50:00 <dalees> calico 3.12 was tested on k8s 1.14 through 1.17. So 3.3 must be quite old
09:50:30 <jakeyip> yeah I'm not sure if anyone is using it
09:51:22 <jakeyip> let's go with deprecation reno first, if we change our mind we can update the reno
09:52:20 <mnasiadka> 3.3 only matches 3.3
09:52:24 <mnasiadka> and nothing else ;-)
09:53:32 <jakeyip> mnasiadka: ?
09:54:02 <mnasiadka> If we're talking about the patch dropping calico_v3_3 ,,matcher''
09:55:24 <mnasiadka> and surely calico v3.3 is not tested against any new Kubernetes release
09:55:34 <mnasiadka> basically what we have in Caracal now won't work for anybody
09:55:34 <jakeyip> ah yeah
09:56:10 <jakeyip> mnasiadka: I had a bunch of comments on Jakub patches
09:56:21 <mnasiadka> are those comments in Gerrit? ;-)
09:56:25 <jakeyip> yeah
09:56:46 <mnasiadka> ok, we'll have a look - but probably in a day or two
09:57:27 <jakeyip> the very first patch in the chain is removing calico_v3.3. . if he doesn't get to it by feature freeze we will merge the reno and leave it for next cycle. is that ok?
09:58:01 <mnasiadka> if we don't remove calico_v3.3 - we can't remove the whole manifest and convert to do Helm chart
09:58:39 <jakeyip> for subsequent changes that uses Calico helm chart, probably next cycle looking at LoC. would need dalees input too
09:59:29 <mnasiadka> let me update the patch now
10:00:44 <dalees> hm, yeah i'll look more. I'm not too keen on the helm chart for our use, as the static manifest for calico 3.26 is enough to see us through to the End of Heat.  but I don't want to prevent others using/merging it.
10:00:59 <jakeyip> there's a change in the chain that might need discussion to - remove tiller. https://review.opendev.org/c/openstack/magnum/+/908414/1
10:01:44 <opendevreview> Michal Nasiadka proposed openstack/magnum master: Removing legacy calico v3.3  https://review.opendev.org/c/openstack/magnum/+/908407
10:02:32 <mnasiadka> And I'll add a reno in some minutes
10:02:41 <mnasiadka> but yes, the removal of calico-service.sh was not intentional
10:03:14 <jakeyip> dalees: also wonder if helm chart will break your patches. please help to check. I will be able to know more when you send it up.
10:03:45 <mnasiadka> jakeyip: Helm2 and Tiller are deprecated since Nov 2020 (https://helm.sh/blog/helm-v2-deprecation-timeline/)
10:04:07 <jakeyip> I guess the ideal situation is that we can carry both helm chart and static manifests so neither one of you have to carry those patches any more :)
10:05:01 <dalees> jakeyip: yep, we might be able to have some label magic that allows that. let's see
10:05:34 <mnasiadka> I think our goal was to do that once, instead of updating the manifest when even a minor bump to Calico would be required
10:05:56 <mnasiadka> And we're close to the Heat content limit, so it might be tight...
10:06:28 <jakeyip> we need to dump the deadweight :)
10:06:37 <jakeyip> *looks for things to throw overboard*
10:07:05 <mnasiadka> haha
10:07:20 <jakeyip> mnasiadka: thanks for updating that patch. :)
10:07:53 <jakeyip> mnasiadka: yeah I don't think anyone will be using Tiller code anymore. dalees ?
10:08:08 <dalees> haha, no. tiller can go
10:08:13 <jakeyip> also, 1 cycle or straight removal?
10:09:18 <jakeyip> oh man we are past time. anyone needs to go? 1 more point left
10:09:37 <dalees> i'm sleepy, but i can stay a bit ;)
10:09:45 <jakeyip> thanks, appreciate it
10:10:00 <jakeyip> ok let's talk about last thing
10:10:15 <jakeyip> beta feature - should it go in this cycle?
10:10:42 <jakeyip> it was meant to use to mark StackHPC CAPI driver as beta, so it won't be loaded. And have to be explicitly enabled
10:11:07 <mnasiadka> even if we don't use it - I guess it's a shame to not merge it
10:12:11 <dalees> yeah, if it's ready it seems reasonable to merge
10:12:44 <dalees> one less thing to line up merging of, if we need it.
10:13:37 <jakeyip> hm ok. thanks for the input. I was pretty keen to merge it because it's useful. I might try to clean it up a bit to see if I can make it to feature freeze
10:14:23 <jakeyip> OH WAIT I forgot most important patch
10:14:31 <jakeyip> Add feature to specify driver explicitly https://review.opendev.org/c/openstack/magnum/+/907297
10:14:52 <jakeyip> this one NEEDS +2 this cycle
10:15:05 <jakeyip> dalees: wake up :P
10:15:07 <dalees> yes, that is more important ;)
10:15:34 <dalees> my list of things to review is getting quite long
10:16:10 <jakeyip> bump this to the top :P
10:16:20 <jakeyip> *pushes the stack of paper off the table*
10:16:57 * dalees converts List to FIFO
10:16:59 <jakeyip> mnasiadka: need to upgrade to +2 too if you can :)
10:17:30 <mnasiadka> remove that space in reno and I can :D
10:18:17 <jakeyip> ah! :)
10:18:37 <jakeyip> good call.
10:19:33 <jakeyip> ok. nothing else from me. there's a bunch of misc reviews that mnasiadka can hopefully get to, trival things
10:19:44 <jakeyip> anything else ?
10:20:03 <dalees> I've read that patch and like it; so review and then +2 won't be long coming.
10:22:49 <mnasiadka> Yeah, I'll have a look in the review queue and also work on some improvements in CI
10:22:51 <opendevreview> Jake Yip proposed openstack/magnum master: Add feature to specify driver explicitly  https://review.opendev.org/c/openstack/magnum/+/907297
10:23:14 <jakeyip> ok let's end this meeting
10:23:23 <jakeyip> #endmeeting