| *** sreejithp has quit IRC | 00:14 | |
| airship-irc-bot1 | <sidney.shiba> Hey all, while troubleshooting the "`manifest/site/test-site`" I found that the kustomize "directed graph" refers to the same catalogue folder (_*test-site/target/catalogue*_) in two different places and wondering if this is ok? Wondering if this is the cause of some errors (see below) when adding my `ReplacementTransformer`/`VariableCatalogue` in the mix. Following these dependencies is giving me headaches: 1- | 01:04 |
|---|---|---|
| airship-irc-bot1 | `test-site/ephemeral/controlplane/kustomization.yaml/resources` -> _*test-site/target/catalogue*_, and -> `test-site/ephemeral/controlplane/nodes` 2- `test-site/ephemeral/controlplane/nodes/kustomization.yaml/generators` -> `test-site/ephemeral/controlplane/hostgenerator` 3- `test-site/ephemeral/controlplane/hostgenerator/kustomization.yaml/resources` -> `test-site/ephemeral/catalogues` 4- `test-site/ephemeral/catalogues/resources` -> | 01:04 |
| airship-irc-bot1 | _*test-site/target/catalogue*_ I am getting when adding the transformer, which does not touch at all `test-site/ephemeral/controlplane/nodes` so not sure what it means. Any clues or help? `$ KUSTOMIZE_PLUGIN_HOME=$(pwd) kustomize build --enable-alpha-plugins .` `Error: accumulating resources: accumulation err='accumulating resources from 'nodes': '/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/nodes' must | 01:04 |
| airship-irc-bot1 | resolve to a file': recursed accumulation of path '/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/nodes': loading generator plugins: expected file with Go object code at: /home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/airshipit.org/v1alpha1/variablecatalogue/VariableCatalogue.so` | 01:04 |
| airship-irc-bot1 | <sidney.shiba> Note that it works without my `ReplacementTransformer`/`VariableCatalogue` but cannot figure out what is the issue and kind of blocked here. | 01:07 |
| *** SRao has joined #airshipit | 01:38 | |
| *** SRao has quit IRC | 03:04 | |
| *** ianychoi_ has joined #airshipit | 03:40 | |
| *** ianychoi has quit IRC | 03:43 | |
| *** uzumaki has quit IRC | 07:04 | |
| *** iamweswilson_ has joined #airshipit | 08:28 | |
| *** uzumaki has joined #airshipit | 08:29 | |
| *** lamt_ has joined #airshipit | 08:29 | |
| *** ildikov_ has joined #airshipit | 08:29 | |
| *** parallax_ has joined #airshipit | 08:29 | |
| *** parallax has quit IRC | 08:43 | |
| *** lamt has quit IRC | 08:43 | |
| *** iamweswilson has quit IRC | 08:43 | |
| *** ildikov has quit IRC | 08:43 | |
| *** lamt_ is now known as lamt | 08:43 | |
| *** ildikov_ is now known as ildikov | 08:43 | |
| *** iamweswilson_ is now known as iamweswilson | 08:43 | |
| *** parallax_ is now known as parallax | 08:43 | |
| *** roman_g has joined #airshipit | 09:58 | |
| airship-irc-bot1 | <alexander.hughes> Hello @sidney.shiba great questions, sharing a few thoughts and lessons I've learned trying to make sense of the Kustomize structure in Treasuremap's manifests 1. The inheritance structure is set up to minimize how much code is duplicated, but it sometimes feels like trying to find a needle in a haystack when you're determining where the documents inherit from and what they do. To resolve this I'd recommend not using | 12:32 |
| airship-irc-bot1 | kustomize directly, but instead using airshipctl. Commands `airshipctl phase list` and `airshipctl phase render <phaseName>` and `airshipctl phase tree <phaseName>` are all helpful in determining the phases, making sure they render and tree especially to determine where all the resources are for that phase. Note - if you're using a recent commit of Treasuremap or airshipctl you'll want to set up your SOPS keys to decrypt the manifests, | 12:32 |
| airship-irc-bot1 | link: https://opendev.org/airship/airshipctl/src/branch/master/manifests/site/test-site/target/generator/README.md 2. For Treasuremap's test-site, the networking catalogue uses the airshipctl catalogue as a basis since we know it works in the gates https://opendev.org/airship/airshipctl/src/branch/master/manifests/function/airshipctl-base-catalogues/networking.yaml then because the core manifests need some additional entries to make it useful | 12:32 |
| airship-irc-bot1 | for operators we've patched them in at the type level here https://opendev.org/airship/treasuremap/src/branch/v2/manifests/type/airship-core/shared/catalogues/kustomization.yaml and for our specific site requirements we've patched again at the site level for targets https://opendev.org/airship/treasuremap/src/branch/v2/manifests/site/test-site/target/catalogues/kustomization.yaml - this is a lot of patching and leads to my "needle in a | 12:32 |
| airship-irc-bot1 | haystack" analogy earlier, this structure would be very helpful if you have a large number of sites that are mostly the same except for a few site specific or type specific changes that's where the patches come in. if you prefer you can break the entire inheritance structure and define the catalogue directly in one place of your choosing. As for the ephemeral catalogue, it uses essentially the same catalogue, it just patches a few values as | 12:32 |
| airship-irc-bot1 | the IPs for ephemeral and target are different https://opendev.org/airship/treasuremap/src/branch/v2/manifests/site/test-site/ephemeral/catalogues/kustomization.yaml 3. The VariableCatalogue.so error when I experienced it earlier this week was difficult to figure out. What I ended up finding is that when we identify a resource, and then transform it, the wrapper that is the transformer gets carried forward into future steps of Kustomize - but | 12:32 |
| airship-irc-bot1 | because the transformer wrapper isn't really useful it throws the error. To address this I'd recommend essentially deleting the wrapper via Kustomize when you're done with the transform step. We've done this in a few places, here's a couple examples a. Using the secrets template https://opendev.org/airship/treasuremap/src/branch/v2/manifests/type/airship-core/target/generator/secret-template.yaml b. Deleting the transformer when done | 12:32 |
| airship-irc-bot1 | https://opendev.org/airship/treasuremap/src/branch/v2/manifests/type/airship-core/target/generator/cleanup/secret-cleanup.yaml | 12:32 |
| *** uzumaki has quit IRC | 14:12 | |
| airship-irc-bot1 | <ih616h> Good morning team. Could I get some reviews on this SIP PS? https://review.opendev.org/c/airship/sip/+/778584 - Remove the ClusterName from SIPCluster CR | 14:19 |
| *** ianychoi__ has joined #airshipit | 14:46 | |
| *** ianychoi_ has quit IRC | 14:49 | |
| *** uzumaki has joined #airshipit | 14:53 | |
| airship-irc-bot1 | <sidney.shiba> Hi @alexander.hughes, thanks for this detailed explanation and now need to digest it. Yesterday, I was trying to create a mental graph of the kustomize rendering so to figure out what was going on with my modifications but didn't have enough memory capacity in my head to visualize it in its entirety. So, start searching for a tool that someone might have created to help me here. After a few Googling I found this tool | 15:03 |
| airship-irc-bot1 | (https://github.com/jpreese/kustomize-graph) but didn't try it, yet. | 15:03 |
| airship-irc-bot1 | <alexander.hughes> I haven't tried Kustomize graph, but I'll give it a look. Here's a sample output though of phase tree, really helps tracking everything down | 15:12 |
| airship-irc-bot1 | <alexander.hughes> ```ubuntu@ubuntu-VirtualBox:~$ airshipctl --airshipconf /home/ubuntu/airshipcfg phase tree bootstrap-iso gpg: keybox '/tmp/pubring.kbx' created gpg: /tmp/trustdb.gpg: trustdb created gpg: key 3D16CEE4A27381B4: public key "SOPS Functional Tests Key 1 (https://github.com/mozilla/sops/) <secops@mozilla.com>" imported gpg: key 3D16CEE4A27381B4: secret key imported gpg: key D8720D957C3D3074: public key "SOPS Functional Tests Key | 15:12 |
| airship-irc-bot1 | 2 (https://github.com/mozilla/sops/) <secops@mozilla.com>" imported gpg: key D8720D957C3D3074: secret key imported gpg: key 3D16CEE4A27381B4: "SOPS Functional Tests Key 1 (https://github.com/mozilla/sops/) <secops@mozilla.com>" not changed gpg: key D8720D957C3D3074: "SOPS Functional Tests Key 2 (https://github.com/mozilla/sops/) <secops@mozilla.com>" not changed gpg: key 19F9B5DAEA91FF86: public key "SOPS Functional Tests Key 3 | 15:12 |
| airship-irc-bot1 | (https://github.com/mozilla/sops/) <secops@mozilla.com>" imported gpg: Total number processed: 5 gpg: imported: 3 gpg: unchanged: 2 gpg: secret keys read: 2 gpg: secret keys imported: 2 bootstrap [manifests/site/test-site/ephemeral/bootstrap] ├── Transformers │ └── replacements [/tmp/default/treasuremap/manifests/type/airship-core/ephemeral/bootstrap/replacements] │ └── Resource | 15:12 |
| airship-irc-bot1 | replacements [/tmp/default/airshipctl/manifests/function/ephemeral/replacements] ├── Resources │ ├── bootstrap [/tmp/default/treasuremap/manifests/type/airship-core/ephemeral/bootstrap] │ │ └── Resources │ │ └── ephemeral [/tmp/default/airshipctl/manifests/function/ephemeral] │ └── catalogues [/tmp/default/treasuremap/manifests/site/test-site/ephemeral/catalogues] │ └── R | 15:12 |
| airship-irc-bot1 | [/tmp/default/treasuremap/manifests/site/test-site/target/catalogues] │ └── Resources │ ├── catalogues [/tmp/default/treasuremap/manifests/type/airship-core/shared/catalogues] │ │ └── Resources │ │ └── airshipctl-base-catalogues [/tmp/default/airshipctl/manifests/function/airshipctl-base-catalogues] │ └── r | 15:12 |
| airship-irc-bot1 | [/tmp/default/treasuremap/manifests/site/test-site/target/generator/results] │ └── Transformers │ └── decrypt-secrets [/tmp/default/treasuremap/manifests/site/test-site/target/generator/results/decrypt-secrets] └── Generators └── hostgenerator [/tmp/default/treasuremap/manifests/site/test-site/ephemeral/bootstrap/hostgenerator] ├── Resources | 15:12 |
| airship-irc-bot1 | [/tmp/default/airshipctl/manifests/function/hostgenerator-m3] │ ├── hardwareprofile-example [/tmp/default/airshipctl/manifests/function/hardwareprofile-example] │ └── catalogues [/tmp/default/treasuremap/manifests/site/test-site/ephemeral/catalogues] │ └── Resources │ └── catalogues [/tmp/default/treasuremap/manifests/site/test-site/target/catalogues] | 15:12 |
| airship-irc-bot1 | │ ├── catalogues [/tmp/default/treasuremap/manifests/type/airship-core/shared/catalogues] │ │ └── Resources │ │ └── airshipctl-base-catalogues [/tmp/default/airshipctl/manifests/function/airshipctl-base-catalogues] │ └── results [/tmp/default/treasuremap/manifests/site/test-site/target/generator/resu | 15:12 |
| airship-irc-bot1 | └── Transformers │ └── decrypt-secrets [/tmp/default/treasuremap/manifests/site/test-site/target/generator/results/decrypt-secrets] └── Transformers ├── replacements [/tmp/default/airshipctl/manifests/function/hostgenerator-m3/replacements] ├── replacements [/tmp/default/airshipctl/manifests/function/hardwareprofile-example/replacements] | 15:12 |
| airship-irc-bot1 | [/tmp/default/airshipctl/manifests/function/hostgenerator-m3/cleanup] ├── cleanup [/tmp/default/treasuremap/manifests/site/test-site/ephemeral/catalogues/cleanup] │ └── Resources │ └── cleanup [/tmp/default/treasuremap/manifests/site/test-site/target/generator/results/cleanup] │ └── Resources │ └── override | 15:12 |
| airship-irc-bot1 | [/tmp/default/treasuremap/manifests/site/test-site/target/generator/results/cleanup/override] │ └── Resources │ └── cleanup [/tmp/default/treasuremap/manifests/type/airship-core/target/generator/cleanup] └── cleanup [/tmp/default/airshipctl/manifests/function/hardwareprofile-example/cleanup]``` | 15:12 |
| airship-irc-bot1 | <sirishagopigiri> Hi Team, I would like to request some more reviews on these PSs: https://review.opendev.org/c/airship/hostconfig-operator/+/775332 == adding exec role to HCO https://review.opendev.org/c/airship/hostconfig-operator/+/773389 == adding apt/yum package update/install feature to HCO https://review.opendev.org/c/airship/airshipctl/+/777930 == Update capm3 to 0.4.0 version to fix BMH move bug Thank you in advance! | 15:24 |
| airship-irc-bot1 | <sidney.shiba> thanks for the example. Let me try for troubleshooting my issue. | 17:52 |
| *** roman_g has quit IRC | 17:59 | |
| *** roman_g has joined #airshipit | 17:59 | |
| *** roman_g has quit IRC | 17:59 | |
| *** roman_g has joined #airshipit | 18:00 | |
| *** roman_g has quit IRC | 18:00 | |
| *** roman_g has joined #airshipit | 18:01 | |
| *** roman_g has quit IRC | 18:01 | |
| *** roman_g has joined #airshipit | 18:01 | |
| *** roman_g has quit IRC | 18:02 | |
| *** roman_g has joined #airshipit | 18:02 | |
| *** roman_g has quit IRC | 18:03 | |
| airship-irc-bot1 | <sidney.shiba> It shows me the tree, which is really great. Now I need to understand how `kustomize` is parsing the tree and figure out where my issue is. Btw, I used `kustomize` CLI for troubleshooting prior to executing the `controplane-ephemeral` phase. Thank you very much for assisting me here. Greatly appreciated. | 18:17 |
| airship-irc-bot1 | <alexander.hughes> my pleasure! the VariableCatalogue.so issue I have a strong suspicion is related to the transformer you've added and you'll want to add a corresponding patch deleting the transformer | 18:25 |
| airship-irc-bot1 | <sidney.shiba> I have a cleanup but seems it was not invoked. Follow an example provided by Alexey. | 18:31 |
| airship-irc-bot1 | <rishabh.k.jain> Hi , I need some core reviews here. :slightly_smiling_face: thank you. In this ps, I have updated the cluster template for docker provider (capd). https://review.opendev.org/c/airship/airshipctl/+/772425 | 20:21 |
| airship-irc-bot1 | <sidney.shiba> I notice a difference in how the tree has been rendered with my changes and without. With my changes it renders with `Transformers` first, while without (working one) shows `Resources` first. Does it mean how `kustomize` is parsing the tree? With my changes below (erroring): ```sx3394@airship:~/projects/treasuremap$ airshipctl phase tree controlplane-ephemeral controlplane [manifests/site/test-site/ephemeral/controlplane] ├┠| 21:25 |
| airship-irc-bot1 | Transformers │ └── replacements [/home/sx3394/projects/treasuremap/manifests/type/airship-core/ephemeral/controlplane/replacements] │ └── Resources │ ├── replacements [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/api-server/replacements] │ │ └── Resources │ │ └── cleanup [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/catalo | 21:25 |
| airship-irc-bot1 | [/home/sx3394/projects/airshipctl/manifests/function/k8scontrol/replacements] └── Resources ├── controlplane [/home/sx3394/projects/treasuremap/manifests/type/airship-core/ephemeral/controlplane] │ └── Resources │ ├── api-server [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/api-server] │ │ └── Resources │ │ └── catalogue | 21:25 |
| airship-irc-bot1 | [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/catalogue] │ └── k8scontrol [/home/sx3394/projects/airshipctl/manifests/function/k8scontrol] ├── catalogues [/home/sx3394/projects/treasuremap/manifests/site/test-site/target/catalogues] │ └── Resources │ └── catalogues [/home/sx3394/projects/treasuremap/manifests/type/airship-core/shared/catalogues] │ â””â | 21:25 |
| airship-irc-bot1 | └── airshipctl-base-catalogues [/home/sx3394/projects/airshipctl/manifests/function/airshipctl-base-catalogues] └── nodes [/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/nodes] └── Generators └── hostgenerator [/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/hostgenerator] ├── Resources │ | 21:25 |
| airship-irc-bot1 | [/home/sx3394/projects/airshipctl/manifests/function/hostgenerator-m3] │ └── catalogues [/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/catalogues] │ └── Resources │ └── catalogues [/home/sx3394/projects/treasuremap/manifests/site/test-site/target/catalogues] │ └── Resources │ | 21:25 |
| airship-irc-bot1 | [/home/sx3394/projects/treasuremap/manifests/type/airship-core/shared/catalogues] │ └── Resources │ └── airshipctl-base-catalogues [/home/sx3394/projects/airshipctl/manifests/function/airshipctl-base-catalogues] └── Transformers └── replacements | 21:25 |
| airship-irc-bot1 | [/home/sx3394/projects/airshipctl/manifests/function/hostgenerator-m3/replacements]``` | 21:25 |
| airship-irc-bot1 | <sidney.shiba> Without my changes (working): ```sx3394@airship:~/projects$ airshipctl phase tree controlplane-ephemeral controlplane [manifests/site/test-site/ephemeral/controlplane] ├── Resources │ ├── controlplane [/home/sx3394/projects/treasuremap/manifests/type/airship-core/ephemeral/controlplane] │ │ └── Resources │ │ ├── api-server [/home/sx3394/projects/treasuremap/manifests/function/dex- | 21:26 |
| airship-irc-bot1 | └── Resources │ │ │ └── catalogue [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/catalogue] │ │ └── k8scontrol [/home/sx3394/projects/airshipctl/manifests/function/k8scontrol] │ ├── catalogues [/home/sx3394/projects/treasuremap/manifests/site/test-site/target/catalogues] │ │ └── Resources │ │ └── catalogues [/home/sx3394/projects/treasur | 21:26 |
| airship-irc-bot1 | │ └── Resources │ │ └── airshipctl-base-catalogues [/home/sx3394/projects/airshipctl/manifests/function/airshipctl-base-catalogues] │ └── nodes [/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/nodes] │ └── Generators │ └── hostgenerator [/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/controlplane/hostge | 21:26 |
| airship-irc-bot1 | │ ├── hostgenerator-m3 [/home/sx3394/projects/airshipctl/manifests/function/hostgenerator-m3] │ │ └── catalogues [/home/sx3394/projects/treasuremap/manifests/site/test-site/ephemeral/catalogues] │ │ └── Resources │ │ └── catalogues [/home/sx3394/projects/treasuremap/manifests/site/test-site/target/catalogues] │ │ | 21:26 |
| airship-irc-bot1 | │ └── catalogues [/home/sx3394/projects/treasuremap/manifests/type/airship-core/shared/catalogues] │ │ └── Resources │ │ └── airshipctl-base-catalogues [/home/sx3394/projects/airshipctl/manifests/function/airshipctl-base-catalogues] │ └── Transformers │ └── replacements | 21:26 |
| airship-irc-bot1 | [/home/sx3394/projects/airshipctl/manifests/function/hostgenerator-m3/replacements] └── Transformers └── replacements [/home/sx3394/projects/treasuremap/manifests/type/airship-core/ephemeral/controlplane/replacements] └── Resources ├── replacements [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/api-server/replacements] │ └── Resources │ ┠| 21:26 |
| airship-irc-bot1 | [/home/sx3394/projects/treasuremap/manifests/function/dex-aio/catalogue/cleanup] └── replacements [/home/sx3394/projects/airshipctl/manifests/function/k8scontrol/replacements]``` | 21:26 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!