| *** Florian has joined #openstack-containers | 00:08 | |
| *** itlinux_ has joined #openstack-containers | 00:11 | |
| *** itlinux has quit IRC | 00:13 | |
| *** itlinux has joined #openstack-containers | 00:38 | |
| *** itlinux_ has quit IRC | 00:40 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: Add server group for cluster worker nodes https://review.openstack.org/613825 | 00:54 |
|---|---|---|
| *** sapd1 has joined #openstack-containers | 00:58 | |
| *** itlinux has quit IRC | 01:03 | |
| *** itlinux has joined #openstack-containers | 01:13 | |
| *** dave-mccowan has joined #openstack-containers | 01:23 | |
| openstackgerrit | Merged openstack/magnum master: Add hidden flag to cluster template https://review.openstack.org/634948 | 01:28 |
| *** itlinux has quit IRC | 01:38 | |
| *** sdake has joined #openstack-containers | 01:59 | |
| *** sdake has quit IRC | 02:18 | |
| *** sdake has joined #openstack-containers | 02:20 | |
| *** mgoddard has quit IRC | 02:32 | |
| *** sdake has quit IRC | 02:44 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: [fedora_atomic] Support auto healing for k8s https://review.openstack.org/631378 | 03:01 |
| *** sapd1 has quit IRC | 03:07 | |
| *** hongbin has joined #openstack-containers | 03:10 | |
| *** ykarel|away has joined #openstack-containers | 03:10 | |
| *** janki has joined #openstack-containers | 03:21 | |
| *** sapd1 has joined #openstack-containers | 03:23 | |
| openstackgerrit | Huang.Xiangdong proposed openstack/magnum master: suport multi discovery service https://review.openstack.org/637450 | 03:28 |
| *** ricolin has joined #openstack-containers | 03:38 | |
| *** udesale has joined #openstack-containers | 03:44 | |
| *** ykarel|away is now known as ykarel | 04:00 | |
| *** ramishra has joined #openstack-containers | 04:07 | |
| *** dave-mccowan has quit IRC | 04:07 | |
| *** _fragatina has quit IRC | 04:26 | |
| *** _fragatina has joined #openstack-containers | 04:26 | |
| *** lpetrut has joined #openstack-containers | 04:43 | |
| *** sapd1 has quit IRC | 04:48 | |
| *** ykarel has quit IRC | 04:58 | |
| *** sapd1 has joined #openstack-containers | 05:00 | |
| *** lpetrut has quit IRC | 05:10 | |
| *** ykarel has joined #openstack-containers | 05:19 | |
| *** hongbin has quit IRC | 05:42 | |
| *** sdake has joined #openstack-containers | 06:38 | |
| *** belmoreira has joined #openstack-containers | 06:49 | |
| *** sdake has quit IRC | 07:30 | |
| *** pcaruana has joined #openstack-containers | 07:43 | |
| *** ykarel is now known as ykarel|lunch | 07:58 | |
| *** mgoerens has joined #openstack-containers | 08:21 | |
| mgoerens | Hi team! I'm using DevStack Queens, and I'm hitting some issue when enabling SSL. On cluster creation, Nova complains that the provided user_data is too long, see this issue: https://bugs.launchpad.net/nova/+bug/1669979. Without SSL, we don't hit this limit. | 08:23 |
| openstack | Launchpad bug 1669979 in OpenStack Compute (nova) "failed to boot instance when user data is larger than 65535 bytes" [Wishlist,Expired] | 08:23 |
| mgoerens | On NON-SSL DevStack: user_data sent from HEAT to NOVA has 59676 bytes. | 08:23 |
| mgoerens | On SSL DevStack: user_data sent from HEAT to NOVA has 66488 bytes (limit is 65535 bytes) | 08:23 |
| mgoerens | Decoding the base64 user_data shows only one difference: In the second case, the SSL certificates are added. | 08:24 |
| mgoerens | I was wondering if this is an issue you already encountered, and if so, how you worked around it. Thanks for reading! | 08:24 |
| *** mgoddard has joined #openstack-containers | 08:33 | |
| *** ArchiFleKs has quit IRC | 08:41 | |
| *** ykarel|lunch is now known as ykarel | 08:59 | |
| *** ArchiFleKs has joined #openstack-containers | 09:00 | |
| *** pcaruana|afk| has joined #openstack-containers | 09:01 | |
| *** pcaruana has quit IRC | 09:02 | |
| *** ramishra has quit IRC | 09:05 | |
| *** ramishra has joined #openstack-containers | 09:09 | |
| *** sdake has joined #openstack-containers | 09:14 | |
| *** ramishra_ has joined #openstack-containers | 09:21 | |
| *** ramishra has quit IRC | 09:23 | |
| *** ttsiouts has joined #openstack-containers | 09:23 | |
| *** sapd1 has quit IRC | 09:33 | |
| *** trident has joined #openstack-containers | 09:38 | |
| *** Florian has quit IRC | 09:49 | |
| *** sapd1 has joined #openstack-containers | 09:49 | |
| *** Horrorcat has joined #openstack-containers | 09:55 | |
| Horrorcat | imdigitaljim, sorry for the delay. I’m back on my Fedora-27-does-not-work-but-CoreOS-does with my mixture of Rocky (magnum, heat) and Pike (everything else) services. Here’s the output of kubectl get all --all-namespaces=true of a fresh Fedora 27 Atomic-based cluster: http://paste.debian.net/hidden/d1976547/ | 10:01 |
| *** pcaruana|afk| has quit IRC | 10:01 | |
| strigazi | Horrorcat: this looks ok ^^ can you kubectl get all --all-namespaces | 10:03 |
| strigazi | mgoerens in queens you shouldn't see this anymore | 10:05 |
| *** ramishra_ has quit IRC | 10:06 | |
| *** pcaruana has joined #openstack-containers | 10:07 | |
| *** ramishra has joined #openstack-containers | 10:09 | |
| *** dioguerra has joined #openstack-containers | 10:25 | |
| Horrorcat | strigazi, that’s what I did, isn’t it? | 10:27 |
| Horrorcat | strigazi, http://paste.debian.net/hidden/3066f8e0/ | 10:27 |
| dioguerra | Feilong Wang: Responded to your query at https://review.openstack.org/#/c/632392/13 | 10:27 |
| strigazi | Horrorcat: I see three lines, the shell, the headers and the kubernetes svc | 10:29 |
| mgoerens | strigazi, thanks for your answer. I deploy DevStack using "TARGET_BRANCH=stable/queens", and all components seems to be in Queens: magnum-api --version 6.3.1.dev1; heat-api --version 10.0.3; nova-api --version 17.0.10 | 10:30 |
| strigazi | mgoerens: and the stack craetion fails? | 10:31 |
| strigazi | with the user_data thing? | 10:33 |
| mgoerens | yes, the Heat stack creation fails. Nova complains that the user_data is too long (exceeds the limit of 65535 bytes) | 10:33 |
| strigazi | mgoerens: You pass the openstack_ca in magnum conf? Is that file too big? | 10:34 |
| mgoerens | Also, the heat stack status goes to CREATE_FAILED, but Magnum doesn't update the cluster status accordingly. In the Magnum logs, the MagnumPeriodicTasks.sync_cluster_status fails with a DBDataError "Data too long for column 'status_reason'" | 10:35 |
| mgoerens | strigazi, let me check that | 10:35 |
| mgoerens | $ sudo du -sk /opt/stack/data/ca-bundle.pem | 10:38 |
| mgoerens | 4/opt/stack/data/ca-bundle.pem | 10:38 |
| mgoerens | strigazi, it's not big no | 10:38 |
| strigazi | mgoerens: which commit of magnum are you using? HEAD of stable/queens? | 10:39 |
| mgoerens | strigazi, last commit in git log is "commit f86de684d2dcca018d056c44be4e4b164b8a4b32" | 10:43 |
| mgoerens | branch is stable/queens | 10:43 |
| mgoerens | $ magnum-api --version | 10:43 |
| mgoerens | 6.3.1.dev1 | 10:43 |
| strigazi | so HEAD, yes | 10:44 |
| openstackgerrit | Spyros Trigazis proposed openstack/magnum stable/queens: DNM test queens https://review.openstack.org/626154 | 10:44 |
| strigazi | have modified anything? mgoerens let's see what the ci thinks ^^ | 10:45 |
| mgoerens | (have to go will be back a bit later, strigazi thanks for your help) | 10:48 |
| *** sdake has quit IRC | 11:16 | |
| *** sdake has joined #openstack-containers | 11:21 | |
| *** ttsiouts has quit IRC | 11:28 | |
| *** ttsiouts has joined #openstack-containers | 11:29 | |
| *** ttsiouts has quit IRC | 11:33 | |
| *** udesale has quit IRC | 11:33 | |
| *** udesale has joined #openstack-containers | 11:34 | |
| Horrorcat | strigazi, exactly, that’s the output I get | 11:39 |
| strigazi | Horrorcat: you don't get pods, deployment etc? | 11:40 |
| strigazi | Horrorcat: kubectl get cs | 11:40 |
| Horrorcat | strigazi, yes, exactly. | 11:41 |
| Horrorcat | strigazi, kubectl get cs: http://paste.debian.net/hidden/7440dd1a/ | 11:41 |
| strigazi | Horrorcat: ssh fedora@<master ip> "sudo journcalctl -u heat-container-agent --no-pager" | 11:43 |
| strigazi | Horrorcat: ssh fedora@<master ip> "sudo docker ps -a" | 11:43 |
| strigazi | Horrorcat: ssh fedora@<master ip> "sudo atomic containers list" | 11:43 |
| Horrorcat | (that’ll take a moment) | 11:43 |
| strigazi | Horrorcat: the firsh one contains endpoints and maybe credentials | 11:44 |
| strigazi | Horrorcat: check the content before sharing | 11:44 |
| Horrorcat | will do, thanks for the warning | 11:44 |
| strigazi | after we should check, journcalctl -u kube-{apiseserver, scheduler, controller-manager} --no-pager | 11:46 |
| strigazi | and the most obvious :), kubectl get nodes | 11:46 |
| Horrorcat | strigazi, re journalctl -u heat-container-agent: http://paste.debian.net/hidden/3d579ede/ I think this might already show the cause to you; I omitted about 8000 repetitions of the last line. | 11:48 |
| Horrorcat | sudo docker ps -a is empty | 11:48 |
| Horrorcat | sudo atomic containers list: http://paste.debian.net/hidden/0ed70d19/ | 11:49 |
| *** janki has quit IRC | 11:49 | |
| Horrorcat | (on the master) kubectl get nodes: http://paste.debian.net/hidden/6e95550c/ | 11:49 |
| Horrorcat | journalctl -u kube-apiserver: http://paste.debian.net/hidden/b30622eb/ | 11:52 |
| Horrorcat | journalctl -u kube-scheduler: http://paste.debian.net/hidden/fd3e0f18/ | 11:52 |
| Horrorcat | aaand I got ratelimited by the pastebin :D | 11:52 |
| *** ttsiouts has joined #openstack-containers | 11:53 | |
| strigazi | paste.openstack.org | 11:53 |
| *** janki has joined #openstack-containers | 11:54 | |
| Horrorcat | journalctl -u kube-controller-manager: http://paste.openstack.org/raw/745254/ | 11:55 |
| Horrorcat | thanks | 11:55 |
| Horrorcat | that should be it | 11:55 |
| strigazi | one last thing, /var/log/cloud-init-output.log (contains creds) | 11:56 |
| strigazi | also check for containers in the node (docker ps) | 11:56 |
| strigazi | also check for containers in the node (docker ps -a) | 11:56 |
| Horrorcat | docker ps -a on the node is empty, too | 11:59 |
| Horrorcat | (still reviewing the cloud-init-output.log) | 11:59 |
| Horrorcat | strigazi, http://paste.openstack.org/raw/COxfSgMCnQeaR7ZFZyJZ/ I hope I got everything and didn’t elide too much :) | 12:09 |
| Horrorcat | (cloud-init-output.log) | 12:09 |
| *** sdake has quit IRC | 12:12 | |
| strigazi | I think I found it | 12:13 |
| strigazi | Horrorcat: can you check in /etc/sysconfig/heat-params if the trust_id has a value? | 12:14 |
| *** sdake has joined #openstack-containers | 12:17 | |
| Horrorcat | no, it is set to "" | 12:18 |
| strigazi | Horrorcat: in [trust] set cluster_user_trust = true https://docs.openstack.org/magnum/latest/configuration/sample-config.html | 12:18 |
| strigazi | and create a new cluster | 12:19 |
| strigazi | can you also show me your cluster's labels | 12:19 |
| Horrorcat | I will try that, thank you very much | 12:19 |
| Horrorcat | I wonder though, why would it work with CoreOS? | 12:20 |
| Horrorcat | | labels | {'flannel_backend': 'host-gw'} | | 12:20 |
| strigazi | it is a new regression :( because we unified the config scripts and when the trust_id is empty it exits | 12:20 |
| *** kaiokmo has joined #openstack-containers | 12:25 | |
| *** janki has quit IRC | 13:06 | |
| mgoerens | strigazi, I'm back. Shall I check something else on my DevStack Queens deployment ? | 13:06 |
| Horrorcat | strigazi, from reading the comment cluster_user_trust is not a recommended thing to do. So do I understand it correctly that it should not be required in my case and it is a regression? If so, will the regression be fixed in a minor release? | 13:11 |
| *** sdake has quit IRC | 13:28 | |
| *** _fragatina has quit IRC | 13:34 | |
| *** _fragatina has joined #openstack-containers | 13:34 | |
| *** jmlowe has quit IRC | 13:43 | |
| *** itlinux has joined #openstack-containers | 13:51 | |
| *** _fragatina has quit IRC | 13:53 | |
| *** _fragatina has joined #openstack-containers | 13:53 | |
| *** sdake has joined #openstack-containers | 13:55 | |
| openstackgerrit | Ricardo Rocha proposed openstack/magnum master: [k8s] Add trustee as a secret in kube-system https://review.openstack.org/636725 | 13:58 |
| *** itlinux has quit IRC | 14:08 | |
| *** itlinux has joined #openstack-containers | 14:11 | |
| *** ykarel is now known as ykarel|away | 14:14 | |
| *** ykarel|away is now known as ykarel | 14:15 | |
| *** ttsiouts has quit IRC | 14:19 | |
| *** ttsiouts has joined #openstack-containers | 14:20 | |
| *** ttsiouts has quit IRC | 14:23 | |
| *** ttsiouts has joined #openstack-containers | 14:23 | |
| strigazi | cluster_user_trust well, it is not recommended is a strong statement made by the creator of the patch. Without this you can not use the cloud provider or anything that needs authentication with openstack. | 14:28 |
| strigazi | Horrorcat: ^^ | 14:28 |
| *** mgoerens has quit IRC | 14:29 | |
| strigazi | this trust id, give access to the cluster to the openstack project, to create volumes, lbaas and so on | 14:29 |
| *** mgoerens has joined #openstack-containers | 14:31 | |
| *** ricolin_ has joined #openstack-containers | 14:40 | |
| *** jmlowe has joined #openstack-containers | 14:41 | |
| *** ricolin has quit IRC | 14:42 | |
| *** ykarel is now known as ykarel|away | 14:42 | |
| *** ykarel|away has quit IRC | 14:52 | |
| Horrorcat | strigazi, I’m confused though, why does it work with CoreOS then? | 14:52 |
| *** sdake has quit IRC | 14:58 | |
| *** dave-mccowan has joined #openstack-containers | 14:58 | |
| *** janki has joined #openstack-containers | 15:00 | |
| *** dave-mccowan has quit IRC | 15:03 | |
| strigazi | Horrorcat: in fedora we use this: https://github.com/openstack/magnum/blob/stable/rocky/magnum/drivers/common/templates/kubernetes/fragments/kube-apiserver-to-kubelet-role.sh#L84 | 15:06 |
| strigazi | Horrorcat: in coreos we don't | 15:06 |
| strigazi | makese sense? | 15:06 |
| strigazi | makes sense? | 15:06 |
| Horrorcat | ah! yes | 15:10 |
| Horrorcat | thanks! | 15:11 |
| *** munimeha1 has joined #openstack-containers | 15:12 | |
| *** henriqueof has joined #openstack-containers | 15:14 | |
| *** ricolin_ has quit IRC | 15:16 | |
| *** ricolin_ has joined #openstack-containers | 15:17 | |
| *** jmlowe has quit IRC | 15:20 | |
| *** jmlowe has joined #openstack-containers | 15:21 | |
| *** itlinux has quit IRC | 15:37 | |
| *** udesale has quit IRC | 15:41 | |
| *** sdake has joined #openstack-containers | 15:44 | |
| *** ykarel|away has joined #openstack-containers | 15:50 | |
| *** itlinux has joined #openstack-containers | 15:50 | |
| *** itlinux has quit IRC | 15:51 | |
| *** ykarel|away is now known as ykarel | 15:51 | |
| *** itlinux has joined #openstack-containers | 16:03 | |
| *** itlinux has quit IRC | 16:06 | |
| *** itlinux has joined #openstack-containers | 16:07 | |
| *** itlinux has quit IRC | 16:08 | |
| *** janki has quit IRC | 16:08 | |
| *** pcaruana has quit IRC | 16:10 | |
| *** ramishra has quit IRC | 16:13 | |
| *** ykarel is now known as ykarel|away | 16:13 | |
| *** itlinux has joined #openstack-containers | 16:17 | |
| *** ramishra has joined #openstack-containers | 16:17 | |
| *** mgoerens has quit IRC | 16:26 | |
| *** flwang1 has joined #openstack-containers | 16:26 | |
| flwang1 | strigazi: around for a catch up? | 16:31 |
| *** mgoerens has joined #openstack-containers | 16:33 | |
| *** ramishra has quit IRC | 16:42 | |
| *** ttsiouts has quit IRC | 16:47 | |
| *** ttsiouts has joined #openstack-containers | 16:48 | |
| *** ppetit has joined #openstack-containers | 16:50 | |
| *** ttsiouts has quit IRC | 16:52 | |
| *** ppetit has quit IRC | 16:53 | |
| *** ricolin_ has quit IRC | 17:00 | |
| *** sdake has quit IRC | 17:00 | |
| *** jmlowe has quit IRC | 17:02 | |
| openstackgerrit | Diogo Guerra proposed openstack/magnum master: Install prometheus monitoring with helm https://review.openstack.org/629577 | 17:02 |
| *** Florian has joined #openstack-containers | 17:04 | |
| *** sdake has joined #openstack-containers | 17:05 | |
| *** ppetit_ has joined #openstack-containers | 17:06 | |
| *** ppetit_ has quit IRC | 17:14 | |
| *** ricolin has joined #openstack-containers | 17:16 | |
| *** jmlowe has joined #openstack-containers | 17:21 | |
| *** mgoerens has quit IRC | 17:23 | |
| openstackgerrit | Merged openstack/magnum stable/rocky: Allow cluster template being renamed https://review.openstack.org/635035 | 17:31 |
| *** jmlowe has quit IRC | 17:31 | |
| *** ttsiouts has joined #openstack-containers | 17:42 | |
| *** jmlowe has joined #openstack-containers | 17:45 | |
| openstackgerrit | Merged openstack/magnum master: [k8s] helm install metrics service https://review.openstack.org/632392 | 17:46 |
| *** _fragatina has quit IRC | 17:48 | |
| *** ricolin has quit IRC | 17:52 | |
| *** flwang1 has quit IRC | 17:53 | |
| *** sapd1 has quit IRC | 17:59 | |
| *** ykarel|away has quit IRC | 18:16 | |
| *** ykarel|away has joined #openstack-containers | 18:17 | |
| *** ykarel|away has quit IRC | 18:22 | |
| *** sdake has quit IRC | 18:29 | |
| *** sdake_ has joined #openstack-containers | 18:31 | |
| *** _fragatina has joined #openstack-containers | 19:00 | |
| *** hongbin has joined #openstack-containers | 19:36 | |
| *** hongbin has quit IRC | 19:43 | |
| *** sdake_ has quit IRC | 19:54 | |
| *** mrodriguez has joined #openstack-containers | 20:08 | |
| *** ttsiouts has quit IRC | 20:09 | |
| *** ttsiouts has joined #openstack-containers | 20:10 | |
| *** ttsiouts has quit IRC | 20:14 | |
| *** ttsiouts has joined #openstack-containers | 20:28 | |
| *** jmlowe has quit IRC | 20:56 | |
| *** ttsiouts has quit IRC | 21:07 | |
| *** ttsiouts has joined #openstack-containers | 21:08 | |
| *** ttsiouts has quit IRC | 21:13 | |
| *** rcernin has joined #openstack-containers | 21:40 | |
| *** ttsiouts has joined #openstack-containers | 21:43 | |
| *** henriqueof_ has joined #openstack-containers | 21:51 | |
| *** ttsiouts has quit IRC | 21:53 | |
| *** henriqueof has quit IRC | 21:53 | |
| *** ttsiouts has joined #openstack-containers | 21:54 | |
| *** ttsiouts has quit IRC | 21:59 | |
| *** dave-mccowan has joined #openstack-containers | 22:16 | |
| *** dave-mccowan has quit IRC | 22:21 | |
| *** _fragatina has quit IRC | 22:25 | |
| *** sdake has joined #openstack-containers | 22:25 | |
| *** _fragatina has joined #openstack-containers | 22:26 | |
| *** sdake has quit IRC | 22:27 | |
| *** jmlowe has joined #openstack-containers | 22:32 | |
| *** sdake has joined #openstack-containers | 22:33 | |
| *** sdake has quit IRC | 22:46 | |
| *** sdake has joined #openstack-containers | 22:47 | |
| *** ArchiFleKs has quit IRC | 23:04 | |
| *** munimeha1 has quit IRC | 23:12 | |
| *** sdake has quit IRC | 23:24 | |
| *** sdake has joined #openstack-containers | 23:24 | |
| *** ArchiFleKs has joined #openstack-containers | 23:25 | |
| *** sdake has quit IRC | 23:32 | |
| *** ttsiouts has joined #openstack-containers | 23:51 | |
| *** ttsiouts has quit IRC | 23:57 | |
| *** spsurya has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!