| jakeyip | hi all, meeting in about 10 mins. anyone around? ping mnasiadka / dalees | 08:52 |
|---|---|---|
| dalees | hi jakeyip | 08:52 |
| mnasiadka | I'm off - public holiday today in Poland | 08:52 |
| dalees | happy labour day | 08:53 |
| jakeyip | ah what about dalees ? | 08:54 |
| dalees | I'm about, preparing to submit a few patches for magnum-ui, but they're not ready to discuss just yet. | 08:54 |
| jakeyip | alright let's have a quick on then. | 08:56 |
| opendevreview | Dale Smith proposed openstack/magnum master: Change network driver test to use non-default driver. https://review.opendev.org/c/openstack/magnum/+/905632 | 09:00 |
| jakeyip | #startmeeting magnum | 09:01 |
| opendevmeet | Meeting started Wed May 1 09:01:44 2024 UTC and is due to finish in 60 minutes. The chair is jakeyip. Information about MeetBot at http://wiki.debian.org/MeetBot. | 09:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 09:01 |
| opendevmeet | The meeting name has been set to 'magnum' | 09:01 |
| jakeyip | #link https://etherpad.opendev.org/p/magnum-weekly-meeting | 09:01 |
| jakeyip | #topic Roll Call | 09:01 |
| jakeyip | o/ | 09:01 |
| jakeyip | ping dalees | 09:02 |
| dalees | o/ | 09:02 |
| jakeyip | #topic Review | 09:02 |
| jakeyip | Update control-plane nodes taint | 09:02 |
| jakeyip | Update control-plane nodes taint - https://review.opendev.org/c/openstack/magnum/+/917407 | 09:02 |
| jakeyip | dalees: that's ok? | 09:03 |
| jakeyip | my motivation is to get the heat driver working with v1.28, passing sonobuoy, updating docs, etc, for D cycle | 09:03 |
| dalees | it looks okay to me if we can't have master taint's anymore. A shame we didn't overlap with control-plane taint for a while, but this should only affect freshly built clusters | 09:04 |
| jakeyip | yeah we all missed that | 09:04 |
| dalees | and if it breaks old k8s <1.20, then that's okay :D | 09:04 |
| dalees | on a similar topic - I'm updating wording from `master` to `control plane` in magnum-ui. Any issues with that in general? | 09:05 |
| dalees | it'll be a bigger effort to update the internal variables and api; I'm not tackling that yet. But we can do some user facing text | 09:05 |
| jakeyip | no we should go with that everywhere, since it's the word upstream uses now | 09:05 |
| dalees | cool | 09:05 |
| * dalees notes the reno in that patchset. Yep operators will need that. | 09:06 | |
| jakeyip | feilong did minion to node a while back - https://review.opendev.org/c/openstack/magnum/+/608799 | 09:07 |
| jakeyip | one concern is new cluster for existing template will have a behaviour change too | 09:09 |
| jakeyip | I think I'll add that sentence to be more clear to operators | 09:09 |
| jakeyip | I think that's prob all we should do | 09:10 |
| dalees | yeah, it will. That'll be a change that some might not expect - hard to make it fully backwards compatible though. easier to roll forwards for those folk | 09:10 |
| jakeyip | yeap | 09:11 |
| jakeyip | any other concern with this review? I will update reno | 09:11 |
| dalees | no, LGTM. just my comment about that duplicate `toleration` | 09:12 |
| jakeyip | yeap I think I've deleted that just haven't sent it up | 09:13 |
| jakeyip | next. Change network driver test to use non-default driver https://review.opendev.org/c/openstack/magnum/+/905632 | 09:13 |
| jakeyip | still needed? I've rebased | 09:13 |
| dalees | i recall it improves test coverage | 09:14 |
| dalees | which we need | 09:14 |
| dalees | will wait and see zuul coverage results and compare. | 09:14 |
| jakeyip | ok | 09:15 |
| jakeyip | next: Update autoscaler clusterrole permissions to support 1.22 https://review.opendev.org/c/openstack/magnum/+/892846 | 09:15 |
| jakeyip | another one of yours :) | 09:15 |
| dalees | not too much to say, it allows using a recent autoscaler for Heat driver clusters | 09:16 |
| dalees | do you enable cluster autoscaler? | 09:16 |
| jakeyip | no we didn't, do y ou? | 09:18 |
| dalees | yep, some of our customers do. | 09:18 |
| dalees | we carry that patch locally, otherwise autoscaler doesn't run :) | 09:18 |
| dalees | on that topic, there's an interesting problem with CAPI driver and cluster autoscaler. Node counts won't update in Magnum currently - only in CAPI. | 09:20 |
| jakeyip | ok I'll rebase, if it passes test I'll merge. | 09:20 |
| opendevreview | Dale Smith proposed openstack/magnum master: Update autoscaler clusterrole permissions to support 1.22 https://review.opendev.org/c/openstack/magnum/+/892846 | 09:20 |
| jakeyip | :D ha you beat me to it | 09:20 |
| dalees | :) | 09:21 |
| jakeyip | any ideas for the CAPI node count mismatch? | 09:24 |
| jakeyip | maybe driver can update | 09:24 |
| dalees | yeah - but it requires lots of changes to the magnum provider in cluster-autoscaler (kubernetes project). Right now it reaches into Heat Stacks, because Magnum API cannot yet return a list of node group members. | 09:25 |
| dalees | driver could poll and update, but that feels the wrong way around. Might be simpler tohugh. cluster autoscaler should probably just talk to Magnum API to do the job. | 09:26 |
| dalees | I raised a bug here https://github.com/stackhpc/capi-helm-charts/issues/317 - so it's tracked *somewhere* :) | 09:27 |
| jakeyip | yeah ok let's see how it goes, maybe someone from there will pick it up :D | 09:30 |
| jakeyip | I'm not familiar with that code so can't help much | 09:32 |
| jakeyip | dalees: on the topic of capi-helm-charts, when do you think we'll be ready for openstack/magnum-capi-helm-charts ? | 09:34 |
| dalees | What are the blockers? CI pipelines? | 09:35 |
| jakeyip | we will fork so we don't have to bring in all their CI | 09:37 |
| jakeyip | how are you handling the chart now for catalyst? | 09:37 |
| dalees | we forked it locally, and publish it to our OCI registry for Magnum to use. We have several modifications like ignoring the keypair, and Calico BGP (which I do need to submit upstream) | 09:39 |
| dalees | however, we will continue to sync with upstream, and push changes that would be useful to others | 09:39 |
| jakeyip | once Magnum forks it to openstack/magnum-capi-helm-charts, your upstream should then be this repo? | 09:41 |
| dalees | and the management loadbalancer - which would be really useful to others who want to allow private clusters... but that requires CAPI and CAPO builds. | 09:41 |
| dalees | yeah, we would switch to that - as i understand stackhpc would sync with it too. | 09:41 |
| jakeyip | yeah matt will take care of openstack <-> stackhpc, we are aware some things might clash cos they use it for Azimuth | 09:43 |
| jakeyip | will sort that out when we get to it | 09:43 |
| opendevreview | Jake Yip proposed openstack/magnum master: Update control-plane nodes taint https://review.opendev.org/c/openstack/magnum/+/917407 | 09:46 |
| dalees | I've got a question about your usage of magnum-ui | 09:46 |
| jakeyip | sure | 09:47 |
| dalees | have you updated magnum-ui to Antelope(?), and have you tried ricolin's "Get Cluster Config" button? | 09:47 |
| dalees | I rebased onto 2024.1 today, and that button made the browser download 3 certificate files and a kubeconfig - but the kubeconfig doesn't reference the cert files. I'm a bit puzzled by this. Did it ever work? | 09:49 |
| jakeyip | I'll have check and get back to you, I believe our dashboard is at Bobcat but we tear out a bunch of panes. | 09:49 |
| jrosser_ | ^ we came across the same thing, not knowing what to do with the downloaded files | 09:50 |
| jakeyip | I don't have an existing cluster to check now | 09:50 |
| jakeyip | if you have a link to the patch that'll be helpful | 09:51 |
| dalees | my other question is - do many others use keystone auth? It's useful for us to provide a button for Kubeconfig with Keystone Auth, and a button for Admin Kubeconfig. | 09:51 |
| jakeyip | it doesn't work out of the box for us and I haven't patched it yet to make it work | 09:52 |
| dalees | jrosser_: thanks, useful to know I'm not the only one. I think I'll propose a change to embed the certs inline in the kubeconfig. That will make it the same as the CLI `openstack coe cluster config`. | 09:52 |
| jakeyip | basically because our role names are different from keystone. `Member' instead of 'member', etc. | 09:52 |
| dalees | ah righto. One day we'll catch up with these role names... | 09:53 |
| dalees | I think we still have `_member_` ;) | 09:53 |
| dalees | but `k8s_admin`, `k8s_viewer`, `k8s_developer` are used mostly in keystoneauth. | 09:54 |
| jakeyip | yeah the good ole _member_, we have some clouds with that too :D | 09:55 |
| jrosser_ | adding an implied role making _member_ and member equivalent is a handy way to migrate out of that | 09:55 |
| jakeyip | nice :) | 09:56 |
| jakeyip | caveat is implied roles don't work well with app cred, there's an open bug | 09:56 |
| jrosser_ | there were recent fixes to keystone to make that also work for existing app creds i think | 09:56 |
| jrosser_ | ahha snap :) | 09:56 |
| jakeyip | _member_ -> member is easier than Member -> member. | 09:57 |
| jakeyip | keystone says names are not case sensitive (so you can't have two names with different cases), but some places are case sensitive so a wrong case won't work | 09:57 |
| jakeyip | :q | 09:58 |
| jakeyip | dalees: your keystone-auth issue is with CAPI driver? | 09:58 |
| jakeyip | jrosser_: :D heee I remember cos I was just looking at the keystone reviews | 10:00 |
| dalees | jakeyip: it applies to magnum-ui, so it's not driver specific. I'll propose two buttons in the UI: "Download KeystoneAuth Kubeconfig" and "Download Admin Kubeconfig". If I can make the KeystoneAuth one only appear for those clusters with it enabled, that'll be ideal. | 10:00 |
| jrosser_ | https://review.opendev.org/c/openstack/keystone/+/910337 | 10:01 |
| jakeyip | dalees: sorry I mean, keystoneauth is working for you now? for clusters spun up by CAPI or Heat? | 10:02 |
| dalees | jakeyip: yes, we use it for both Heat and CAPI(helm) | 10:03 |
| dalees | though there's a snag in v1.29 which travisholton is working on. | 10:04 |
| jakeyip | jrosser_: I like this one more https://review.opendev.org/c/openstack/keystone/+/893737 | 10:04 |
| jrosser_ | ah yes that is a patch from my team | 10:04 |
| jrosser_ | but adding tests is just soooo hard /o\ | 10:04 |
| jakeyip | oh nice! I'll comment on this :P | 10:05 |
| jakeyip | dalees: I think that sounds good. give it a go | 10:08 |
| dalees | alright, incoming magnum-ui patchsets soon. | 10:09 |
| jakeyip | so if memory serves, the files that you downloaded are actually from the certificates endpoint. | 10:13 |
| jakeyip | python-magnumclient grabs them and formats them for kubeconfig | 10:14 |
| dalees | the CA is, the key and CSR(not downloaded) are generated, and posted to the certificates endpoint, yeah. | 10:14 |
| jakeyip | you may know this already... | 10:14 |
| dalees | I was looking at this code today ;) | 10:14 |
| dalees | anything else for meeting? | 10:16 |
| jakeyip | ok I'll leave you to it then. | 10:16 |
| jakeyip | nothing | 10:16 |
| jakeyip | let's call it then, we are over time | 10:16 |
| jakeyip | #endmeeting | 10:17 |
| opendevmeet | Meeting ended Wed May 1 10:17:11 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 10:17 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/magnum/2024/magnum.2024-05-01-09.01.html | 10:17 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/magnum/2024/magnum.2024-05-01-09.01.txt | 10:17 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/magnum/2024/magnum.2024-05-01-09.01.log.html | 10:17 |
| jakeyip | dalees: sorry about the time, and thanks for coming. :) | 10:17 |
| jakeyip | jrosser_: thanks for coming too | 10:17 |
| dalees | no worries; daylight savings means it's 10pm not 11pm | 10:17 |
| jakeyip | ha yeah that's good for me too | 10:18 |
| jakeyip | oh btw let me know if you going openstack koera or kubecon hong kong, :) | 10:18 |
| dalees | jakeyip I already made a few changes, most in the Create Cluster. Here's a little preview: https://i.imgur.com/n7YBwhM.png | 10:18 |
| jakeyip | timezone friendly conferences :D | 10:18 |
| dalees | jakeyip: no budget this year for overseas, just virtual tickets to kubecon usa and kiwi pycon. | 10:19 |
| jakeyip | ok | 10:19 |
| dalees | well, in-person kiwi pycon. | 10:21 |
| jakeyip | nice, never been to pycon :( | 10:23 |
| dalees | neither! I hope there's pie | 10:23 |
| jakeyip | hahaha I hope I'll make it over there one day, never been to NZ either. | 10:24 |
| jakeyip | btw if you can let me know what's the issue with keystoneauth and v1.29 that'll be great | 10:25 |
| jakeyip | not urgent, please do it when you can find time. I'll let you go cos it's bedtime for you! :) | 10:26 |
| dalees | jakeyip: https://kubernetes.slack.com/archives/CFKJB65G9/p1713217132482869?thread_ts=1713217132.482869&cid=CFKJB65G9 | 10:32 |
| jakeyip | thanks | 10:34 |
| jakeyip | ah this issue! that was why I had to push https://review.opendev.org/c/openstack/magnum-capi-helm/+/915274 to disable keystone-auth | 10:37 |
| jakeyip | glad it's getting worked on | 10:37 |
| jakeyip | I wonder how vexxhost solve this | 10:38 |
| travisholton | jakeyip: this issue only comes up >= 1.29 from what I've seen | 18:07 |
| travisholton | there was another issue caused by the k8s-keystone-auth helm chart using registry.k8s.io/provider-os/k8s-keystone-auth:v1.26.0 which doesn't exist | 18:10 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
