| opendevreview | Dale Smith proposed openstack/magnum-capi-helm master: Use API LB floating IPs according to template and label https://review.opendev.org/c/openstack/magnum-capi-helm/+/946857 | 01:45 |
|---|---|---|
| opendevreview | Stephen Finucane proposed openstack/magnum master: api: Use application_url, not host_url https://review.opendev.org/c/openstack/magnum/+/952762 | 13:21 |
| opendevreview | Stephen Finucane proposed openstack/magnum master: api: Use application_url, not host_url https://review.opendev.org/c/openstack/magnum/+/952762 | 13:31 |
| -opendevstatus- NOTICE: Zuul jobs reporting POST_FAILURE were due to an incident with one of our cloud providers; this provider has been temporarily disabled and changes can be rechecked | 22:36 | |
| dalees | andrewbogott_: yeah, the Magnum driver (on conductor) grabs that chart each time a cluster is created or updated. We publish ours onto an OCI registry and set magnum.conf to use that instead, removing part of the internet requirement. | 23:26 |
| andrewbogott_ | dalees: that sounds like just what I want! Can you tell me what the magnum.conf setting looks like? | 23:26 |
| andrewbogott_ | Or... I'd love to just have the file sitting locally on the control node, if I pre-install it will it just find it and skip the download? | 23:27 |
| dalees | andrewbogott_: Magnum conductor will run a `helm upgrade`, so if that can take a file you might be able to. An OCI registry is more flexible for us as we regularly publish updated charts and refer to the chart version in Cluster Templates. The config to use an OCI hosted chart looks like this: https://paste.openstack.org/show/byE6ThBKPDjFNRtWesYn/ | 23:30 |
| dalees | the default_helm_chart_version is overridden by Cluster Template label "capi_helm_chart_version". | 23:31 |
| dalees | and helm_chart_repo is intentionally empty. | 23:31 |
| * dalees will review the capi-helm driver docs, and make sure this info is in there. | 23:32 | |
| andrewbogott_ | cool, thank you! I don't really know what's involved in making an OCI registry but I can read up. Will also have to think about where to host that. | 23:32 |
| andrewbogott_ | Our control nodes are managed by puppet so it's easy to have puppet drop in an updated file if it's just a static file on the node. | 23:33 |
| dalees | if you need to be separated from the internet, you'll need one to pull any container images. Or if you have an allowlist of registries you could publish to, you can look up how helm can push to an OCI registry and do that. | 23:33 |
| andrewbogott_ | So, hang on, I think I still don't understand something... | 23:33 |
| andrewbogott_ | the 'helm upgrade' is happening on the magnum control node? Or happening on the k8s worker cluster? | 23:33 |
| andrewbogott_ | ...for context, I'm not running magnum itself on a k8s cluster, just on a puppet-managed bare metal server. | 23:34 |
| andrewbogott_ | It looked to me like magnum was trying to download the file /there/ which doesn't make much sense, because... no k8s, no helm | 23:34 |
| dalees | 'helm upgrade' runs from Magnum conductor (within the capi helm driver code). It targets the K8s CAPI management cluster. That management cluster runs CAPI/CAPO and creates the workload clusters. | 23:34 |
| andrewbogott_ | Then I have a VM w/k3s to host capi | 23:35 |
| andrewbogott_ | so the download is happening in the management cluster (in my case, on the k3s VM) not on the host running magnum-conductor | 23:35 |
| andrewbogott_ | ^ correct? | 23:35 |
| andrewbogott_ | ...because in my case the k3s cluster /does/ have public internet access but the host running magnum-conductor does not | 23:36 |
| andrewbogott_ | I hope I'm making sense :) | 23:36 |
| dalees | helm (run from magnum-conductor) will pull the chart before sending rendered manifests to the k3s cluster. | 23:36 |
| andrewbogott_ | got it, ok, then I'm not completely misunderstanding what's happening. | 23:37 |
| dalees | :) | 23:38 |
| andrewbogott_ | Is it really just that one helm chart that gets pulled from the outside, or is it doing that kind of thing all the time? | 23:38 |
| dalees | I think it's just that one chart; the other charts (which deploy to workload clusters from HelmRelease CRDs) are pulled from within the management cluster. | 23:40 |
| andrewbogott_ | cool | 23:40 |
| andrewbogott_ | I'm still trying to parse 'Magnum conductor will run a `helm upgrade`, so if that can take a file you might be able to.' but probably I just need to read the source. | 23:41 |
| andrewbogott_ | If I wind up needing to alter the driver for this would you accept a patch? | 23:41 |
| dalees | I suggest you give yourself the ability to have multiple charts available at once, even if you make it work from disk. Old clusters might use chart version X, and your newer ones version Y (as specified in cluster template). Every resize event needs to run 'helm upgrade', so they need to remain accessible until the clusters are upgraded or deleted. | 23:42 |
| * andrewbogott_ nods | 23:43 | |
| andrewbogott_ | It feels like... it should be possible to direct the mgmt cluster to download its own chart? But I guess that would only fix my weird use case and not really be better for anyone else. | 23:44 |
| dalees | andrewbogott_: seems like a reasonable use case. I think I have made it work from local disk before in development, so with the right combination of config it should be possible already, but improvements are always welcome!. | 23:44 |
| andrewbogott_ | cool. I'll dig around, please ping me if you wake up in the night and suddenly remember how you did it :) | 23:45 |
| dalees | my understanding of helm is it runs on the client system and renders the manifests. To do this it has to pull the chart on the client system not on the target k8s cluster. So you wouldn't proxy that chart pull via the k8s cluster, nor have the k8s cluster pull the chart. It must be done where the `helm` binary is executed (the client - in this case | 23:49 |
| dalees | a child of magnum-conductor). | 23:49 |
| andrewbogott_ | ok, that makes sense I think! I imagine I will understand helm a lot better at the end of this road | 23:59 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!