| *** vishalmanchanda has joined #openstack-containers | 00:00 | |
| *** threestrands has joined #openstack-containers | 00:12 | |
| *** xinliang13 has joined #openstack-containers | 01:48 | |
| *** sapd1 has joined #openstack-containers | 02:14 | |
| *** xinliang13 has quit IRC | 02:18 | |
| *** xinliang has joined #openstack-containers | 02:19 | |
| *** xinliang has quit IRC | 02:37 | |
| *** xinliang has joined #openstack-containers | 02:44 | |
| *** xinliang has quit IRC | 02:50 | |
| *** xinliang has joined #openstack-containers | 02:50 | |
| *** ramishra has joined #openstack-containers | 03:27 | |
| *** dave-mccowan has joined #openstack-containers | 03:42 | |
| *** ricolin has quit IRC | 03:43 | |
| *** ykarel|away is now known as ykarel | 04:24 | |
| *** ricolin has joined #openstack-containers | 04:27 | |
| *** dave-mccowan has quit IRC | 04:30 | |
| *** sapd1 has quit IRC | 05:18 | |
| *** sapd1 has joined #openstack-containers | 05:31 | |
| *** sapd1 has quit IRC | 05:58 | |
| *** sapd1 has joined #openstack-containers | 06:16 | |
| *** threestrands has quit IRC | 06:42 | |
| *** ricolin has quit IRC | 06:52 | |
| *** rcernin has quit IRC | 07:06 | |
| *** ykarel is now known as ykarel|lunch | 07:39 | |
| *** ricolin_ has joined #openstack-containers | 08:33 | |
| *** ykarel|lunch is now known as ykarel | 08:59 | |
| *** ricolin_ has quit IRC | 09:00 | |
| *** ricolin_ has joined #openstack-containers | 09:02 | |
| *** yolanda has quit IRC | 09:11 | |
| *** yolanda has joined #openstack-containers | 09:11 | |
| *** ricolin_ has quit IRC | 09:23 | |
| yankcrime | brtknr: ah, looks like i'm on 9.0.0 | 09:40 |
|---|---|---|
| brtknr | yankcrime: 9.1.0 will also bring support for fcos but better if you install 9.2.0... we are almost about to release 9.3.0 if you wanna wait | 09:53 |
| yankcrime | brtknr: looks like i need to build custom images since uca (and hence my kolla container images) are still on 9.0.0 | 09:58 |
| yankcrime | been a while since i've done this! | 09:58 |
| brtknr | yankcrime: whats uca? | 10:00 |
| yankcrime | ubuntu cloud archive | 10:00 |
| brtknr | yankcrime: aah | 10:08 |
| brtknr | i think kolla has the latest train release on its dockerhub for magnum | 10:09 |
| *** vishalmanchanda has quit IRC | 10:09 | |
| yankcrime | the ubuntu binary images for the train release are still on 9.0.0 because that's what's in uca | 10:10 |
| yankcrime | i've just pulled them and checked | 10:10 |
| *** pcaruana has joined #openstack-containers | 10:10 | |
| brtknr | yankcrime: ouch | 10:13 |
| brtknr | 9.0.0 was autoreleased before all the changes we wanted merged got merged | 10:14 |
| brtknr | hence it is quite buggy | 10:14 |
| yankcrime | first time i've tried it since upgrading | 10:14 |
| yankcrime | rolled out octavia over the weekend and wanted to start testing magnum again | 10:14 |
| brtknr | yankcrime:there's octavia on sausage now? awesome! | 10:17 |
| yankcrime | yup! | 10:17 |
| brtknr | btw when is support for cinder volumes coming? | 10:17 |
| yankcrime | whenever we can find enough customers to pay for the additional power! | 10:18 |
| yankcrime | "customers" | 10:18 |
| brtknr | you already have the disks? | 10:19 |
| yankcrime | not yet but i think we can sort the hardware | 10:19 |
| yankcrime | it's the power that's the problem | 10:19 |
| brtknr | wonder if you could use converged local storage on the hypervisors for the cinder volumes? | 10:19 |
| yankcrime | i've thought about that and it's a bad idea / pain in the butt for a number of reasons | 10:20 |
| brtknr | yes something about it doesnt sound right | 10:21 |
| brtknr | btw "Error: Unable to retrieve load balancers." on horizon | 10:22 |
| brtknr | yankcrime: | 10:22 |
| yankcrime | oh yeah you need a role adding, hang on | 10:23 |
| yankcrime | brtknr: try now | 10:24 |
| yankcrime | prob have to login again | 10:25 |
| yankcrime | brtknr: any luck? | 10:39 |
| brtknr | yankcrime: no longer getting errors | 10:48 |
| brtknr | will wait for you to deploy new release of magnum before testing this with a k8s cluster | 10:49 |
| *** ricolin_ has joined #openstack-containers | 11:13 | |
| *** ykarel is now known as ykarel|afk | 11:26 | |
| yankcrime | brtknr: done, magnum is now on 9.2.0 | 11:34 |
| brtknr | yankcrime: neat! | 11:35 |
| yankcrime | whether it works or not... ¯\_(ツ)_/¯ | 11:35 |
| brtknr | yankcrime: you might like to try github.com/stackhpc/magnum-terraform | 11:45 |
| yankcrime | brtknr: nice! | 11:45 |
| yankcrime | have you (or anyone) tried flatcar linux as a drop-in replacement for coreos btw? | 11:46 |
| brtknr | yankcrime: no, sadly not | 12:03 |
| brtknr | yankcrime: is Fedora CoreOS 31 20200118 from the stable branch? | 12:07 |
| brtknr | there is a newer image btw | 12:07 |
| yankcrime | brtknr: i don't remember uploading that | 12:13 |
| brtknr | i just uploaded fedora-coreos-31.20200210.3.0-openstack.x86_64, feel free to make it public if you like but please dont rename :) | 12:14 |
| *** ykarel|afk is now known as ykarel | 12:17 | |
| brtknr | yankcrime: | 12:20 |
| brtknr | also uploaded Fedora-AtomicHost-29-20191126.0.x86_64 | 12:20 |
| brtknr | the last FA image which is now EOL | 12:20 |
| yankcrime | brtknr: nice, ta | 12:50 |
| yankcrime | just did a quick test, cluster status 'create complete' but i get a cluster not found error when doing openstack coe cluster config | 12:51 |
| yankcrime | think there's a missing cert somehow | 12:51 |
| *** ricolin_ has quit IRC | 13:00 | |
| *** ricolin_ has joined #openstack-containers | 13:10 | |
| brtknr | yankcrime: hmm also have you run magnum-db-manage upgrade? | 13:31 |
| yankcrime | i would assume k-a ran that as part of the upgrade from stein to train | 13:31 |
| yankcrime | i don't know if it's been re-run since i went from 9.0.0 to 9.2.0 | 13:31 |
| brtknr | yankcrime: i am getting this "Failed to pre-delete resources for cluster e31cbf49-5105-42c4-823c-d6282fc3b96e, error: Unrecognized schema in response body. (HTTP 403) (Request-ID: req-38b57f8e-6e35-46e6-b1b0-346b31d5072a)" | 13:31 |
| *** ricolin_ has quit IRC | 13:33 | |
| brtknr | yankcrime: i have noticed in the past that k-a fails to run magnum-db-manage | 13:33 |
| *** ykarel is now known as ykarel|afk | 13:33 | |
| yankcrime | brtknr: ok, give me a few mins to take a look and run it manually if necessary | 13:39 |
| brtknr | magnum-db-manage upgrade is idempotent, running it again is fine but you cant go back unless you have a backup | 13:40 |
| brtknr | sorry if i am telling you something you already know | 13:40 |
| brtknr | yankcrime: | 13:40 |
| yankcrime | brtknr: the db was already at the right revision apparently | 13:44 |
| brtknr | yankcrime: hmm not sure why the cluster is then refusing to delete | 13:45 |
| yankcrime | hmm, i deleted it ok as an admin | 13:48 |
| *** sapd1 has quit IRC | 14:03 | |
| brtknr | yankcrime: {'default-master': 'Resource CREATE failed: OctaviaClientException: resources.etcd_lb.resources.loadbalancer: Policy does not allow this request to be performed. (HTTP 403) (Request-ID: req-8349a9c2-fc7b-4d77-89e5-2c346f4cb315)', 'default-worker': 'Resource CREATE failed: OctaviaClientException: resources.etcd_lb.resources.loadbalancer: Policy does not allow this request to be | 14:07 |
| brtknr | performed. (HTTP 403) (Request-ID: req-8349a9c2-fc7b-4d77-89e5-2c346f4cb315)'} | 14:07 |
| brtknr | i think the heat user needs the octavia roles too | 14:07 |
| *** ykarel|afk is now known as ykarel | 14:07 | |
| yankcrime | surely heat should inherit the role of the user creating the stack | 14:09 |
| yankcrime | i think this might be related | 14:09 |
| yankcrime | 2020-03-09 14:06:21.865 30 ERROR magnum.drivers.heat.k8s_fedora_template_def [req-13345c2f-5881-4f2e-9844-9c400d688fed - - - - -] Failed to load default keystone auth policy: FileNotFoundError: [Errno 2] No such file or directory: '/etc/magnum/keystone_auth_default_policy.json' | 14:09 |
| yankcrime | also weird how i'm able to create a cluster ok... | 14:13 |
| *** sapd1 has joined #openstack-containers | 14:17 | |
| brtknr | yankcrime: that is not a real error | 14:22 |
| brtknr | it should be a warning really | 14:22 |
| yankcrime | ok | 14:22 |
| brtknr | you can override the keystone auth policy for k8s by placing it in the file | 14:23 |
| yankcrime | brtknr: could the tag you have set 'heat_container_agent_tag': 'ussuri-dev' cause a problem? | 14:44 |
| brtknr | yankcrime: ussuri-dev is good, train-stable-2 is even better | 14:45 |
| brtknr | yankcrime: i still am not able to create a cluser | 14:48 |
| *** lpetrut has joined #openstack-containers | 14:49 | |
| yankcrime | brtknr: could it be because you don't have a fixed_network specified in the template you're using? | 14:53 |
| brtknr | yankcrime: not sure what that has to do with this error: | 14:55 |
| brtknr | {'default-master': 'Resource CREATE failed: OctaviaClientException: resources.etcd_lb.resources.loadbalancer: Policy does not allow this request to be performed. (HTTP 403) (Request-ID: req-8349a9c2-fc7b-4d77-89e5-2c346f4cb315)', 'default-worker': 'Resource CREATE failed: OctaviaClientException: resources.etcd_lb.resources.loadbalancer: Policy does not allow this request to be performed. (HTTP | 14:55 |
| brtknr | 403) (Request-ID: req-8349a9c2-fc7b-4d77-89e5-2c346f4cb315)'} | 14:55 |
| yankcrime | i wonder if it's because it's trying to create a loadbalancer with an interface in the wrong network | 14:55 |
| brtknr | what network should i use? | 14:55 |
| *** ykarel is now known as ykarel|away | 15:05 | |
| brtknr | yankcrime: is heat user an admin? | 15:08 |
| brtknr | yankcrime: i believe a user needs "heat_stack_owner" role | 15:11 |
| yankcrime | it's whatever k-a sets up for heat | 15:11 |
| yankcrime | as for which network, whatever network you have access to in the project you're using | 15:12 |
| yankcrime | you using that demo tenancy? | 15:12 |
| brtknr | yeah | 15:13 |
| *** sapd1 has quit IRC | 15:20 | |
| yankcrime | so in the template i created (i've been testing in the same project) i used 242916d4-4d37-4e3f-bddb-3166b7d6f1ef for the fixed-network and a4e680f9-98b0-461d-bd4f-3015e8b9461a for the subnet | 15:31 |
| *** sapd1 has joined #openstack-containers | 15:37 | |
| brtknr | yankcrime: i cant delete the cluster i created again, something is weird about the roles assigned to my user | 16:02 |
| brtknr | what roles have i got assigned? | 16:02 |
| yankcrime | hmmm let me look | 16:03 |
| yankcrime | weird how you can create but not delete | 16:03 |
| yankcrime | brtknr: try again, penny just dropped when i remembered you're using that demo project | 16:05 |
| *** lpetrut has quit IRC | 16:05 | |
| brtknr | `ERROR: You are not authorized to use stacks:delete.` | 16:06 |
| brtknr | yankcrime: | 16:06 |
| yankcrime | lolwut | 16:06 |
| yankcrime | is that after logging out and back in again? | 16:07 |
| yankcrime | you need a new token scoped with the new roles | 16:07 |
| yankcrime | or is this via the cli? | 16:07 |
| brtknr | i believe i need heat_stack_owner permissing | 16:07 |
| brtknr | im using the cli | 16:07 |
| yankcrime | brtknr: ok try again | 16:08 |
| brtknr | delete in progress | 16:09 |
| yankcrime | \o/ | 16:09 |
| brtknr | yankcrime: Authorization failed: SSL exception connecting to https://compute.sausage.cloud:5000/v3/auth/tokens: | 16:54 |
| brtknr | Source [heat] Unavailable. | 16:54 |
| yankcrime | hmm i wonder if heat has died | 16:56 |
| yankcrime | transient error? looks ok here | 16:57 |
| brtknr | yankcrime: have you managed to create a cluster successfully yet? | 17:02 |
| brtknr | me, no | 17:03 |
| yankcrime | brtknr: yeah earlier it completed successfully but i couldn't get the config info | 17:12 |
| yankcrime | it still thinks it's creating your stack | 17:12 |
| brtknr | im trying again with atomic | 17:28 |
| brtknr | yankcrime: | 17:29 |
| yankcrime | aye ok, i suspect something didn't work in the vm that was provisioned | 17:39 |
| *** sapd1 has quit IRC | 17:52 | |
| brtknr | yankcrime: btw you need to set cluster_user_trust=true | 18:20 |
| yankcrime | where does that go? | 18:21 |
| yankcrime | magum's config? | 18:21 |
| yankcrime | *magnum | 18:21 |
| brtknr | inside /etc/magnum/magnum.conf under [trust] | 18:21 |
| yankcrime | ahh look at that, it's false by default with k-a | 18:23 |
| yankcrime | pls hold | 18:23 |
| brtknr | looks like the podman cluster created okay but i see the calico pods failing for wahtever reason | 18:23 |
| brtknr | yes cluster_user_trust is off by default in k-a due to a CVE | 18:24 |
| brtknr | as the trust_id gives a user complete access to the cluster | 18:25 |
| yankcrime | brtknr: ok, done | 18:32 |
| *** pcaruana has quit IRC | 19:23 | |
| *** dave-mccowan has joined #openstack-containers | 20:12 | |
| *** trident has quit IRC | 20:57 | |
| *** trident has joined #openstack-containers | 20:58 | |
| *** trident has quit IRC | 21:04 | |
| *** trident has joined #openstack-containers | 21:05 | |
| *** rcernin has joined #openstack-containers | 21:36 | |
| *** zigo has quit IRC | 22:13 | |
| *** zigo has joined #openstack-containers | 22:18 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!