Thursday, 2020-03-12

*** openstackstatus has joined #openstack-containers00:43
*** ChanServ sets mode: +v openstackstatus00:43
*** hongbin has joined #openstack-containers01:12
*** vishalmanchanda has joined #openstack-containers01:51
*** sapd1_x has joined #openstack-containers02:45
*** xinliang has joined #openstack-containers02:47
*** xinliang has quit IRC02:57
*** hongbin has quit IRC03:03
*** KeithMnemonic1 has joined #openstack-containers03:28
*** KeithMnemonic has quit IRC03:31
*** ricolin_ has joined #openstack-containers03:31
*** ricolin_ has quit IRC03:33
*** ricolin_ has joined #openstack-containers03:34
*** ykarel|away is now known as ykarel03:48
*** dave-mccowan has quit IRC03:58
*** KeithMnemonic1 has quit IRC04:06
*** udesale has joined #openstack-containers04:17
*** strigazi has quit IRC05:49
*** dtomasgu has quit IRC05:50
*** strigazi has joined #openstack-containers05:51
*** dtomasgu has joined #openstack-containers05:52
*** dtomasgu_ has joined #openstack-containers07:00
*** strigazi has quit IRC07:03
*** dtomasgu has quit IRC07:03
*** sapd1_x has quit IRC08:23
*** ramishra has quit IRC08:37
*** ramishra has joined #openstack-containers08:39
*** pcaruana has joined #openstack-containers08:54
*** ykarel is now known as ykarel|lunch09:19
openstackgerritSpyros Trigazis proposed openstack/magnum master: calido: Add node/status in ClusterRole  https://review.opendev.org/71262909:37
cosmicsoundbrtknr , managed to add the train-stable-2 tag and check the new logs, they do not say more then previously09:42
cosmicsoundwill time out after 60 mis09:42
*** sapd1 has joined #openstack-containers09:50
*** tobias-urdin has joined #openstack-containers10:03
openstackgerritSpyros Trigazis proposed openstack/magnum master: calico: Add node/status in ClusterRole  https://review.opendev.org/71262910:05
tobias-urdinhm trying to troubleshoot or fix an issue, deployed a kubenetes 1.15.7 cluster with magnum10:07
tobias-urdinbut the minions doesn't populate the InternalIP address, only the master10:07
tobias-urdinso the octavia-ingress-controller can't get the ips for the members10:08
tobias-urdinhttp://paste.openstack.org/show/790584/10:08
tobias-urdintried overriding kubelet --node-ip without success, it's running flannel10:08
tobias-urdinanybody has a good idea? tried to find a workaround yesterday, only thing i have left is trying to override in kubelet-config.yaml but can't find any good options to set when browsing the struct in the golang code10:09
openstackgerritThomas Hartland proposed openstack/magnum master: Add node groups documentation  https://review.opendev.org/71234010:13
*** ykarel|lunch is now known as ykarel10:19
*** pcaruana has quit IRC10:42
tobias-urdinsolved the above, the minions kubelet has --cloud-provider=external -- removing that and restart kubelet populates Addresses with the InternalIP10:54
*** pcaruana has joined #openstack-containers10:55
*** vishalmanchanda has quit IRC11:00
*** sapd1 has quit IRC11:00
cosmicsound2020-03-12 11:01:42,082 - stages.py[ERROR]: Unable to render networking. Network config is likely broken: No available network renderers found. Searched through list: ['eni', 'sysconfig', 'netplan']11:07
cosmicsoundanyone seen this on cloud-init.log11:07
cosmicsoundis only one error shown11:07
tobias-urdinare you running inside something that doesn't have neither of ifconfig, sysconfig (you are running NetworkManager?) or netplan (ubuntu)?11:08
tobias-urdinto answer myself, correct thing to do for my issue is setting the label cloud_provider_enabled to false11:12
cosmicsoundtobias-urdin , i use default fedora atomichost latest stable 29.11.0611:59
*** udesale_ has joined #openstack-containers12:14
*** udesale has quit IRC12:17
*** rcernin has quit IRC12:25
*** ianychoi has quit IRC13:13
*** dave-mccowan has joined #openstack-containers14:02
brtknrtobias-urdin: sounds like you need to set cluster_user_trust=true inside magnum.conf14:05
yankcrimebrtknr: think i've fixed that timeout problem - it's a systemd default timeout value for (re)starting and stopping services14:10
yankcrimebumping that value for the unit def for the master and the minions gives it time to complete14:10
tobias-urdinbrtknr: ok, the trust is then used to collect instance ips etc? to populate all that (probably also external ips if floating ip is enabled?)14:12
*** dave-mccowan has quit IRC14:25
*** dave-mccowan has joined #openstack-containers14:31
*** markguz_ has joined #openstack-containers14:31
brtknrcloud_provider_enabled defaults to the value of cluster_user_trust unless set to false, it sounds like your openstack cloud controller manager is not running which is used to give node internal ip14:34
brtknryankcrime: would you like to propose a fix?14:35
yankcrimebrtknr: yeah i'll try and find some time later today14:35
brtknri think it was bumped up to 10 mins recently14:36
brtknrwhat did you bump it up to14:36
yankcrimeit can't be that14:36
brtknryankcrime: did you change the value of TimeoutStartSec?14:36
yankcrimeunless i'm mistaken it looks like it's the default of 90s14:36
brtknror something else?14:36
yankcrimeTimeoutSec14:36
yankcrimewhich affects TimeoutStartSec and TimeoutStopSec14:37
yankcrimejust for units in configure-kubernetes-master.sh and configure-kubernetes-minion.sh14:37
yankcrimethere's a systemd global setting of DefaultTimeoutStartSec14:37
yankcrimethe default for that is 90s14:37
yankcrimewhich is what all service units inherit if it's not overridden14:38
brtknryankcrime: what I meant was kube-apiserver is already configured with TimeoutStartSec of 10 minutes: https://github.com/openstack/magnum/blob/fa45002e21ef6de3b4a9da35d590a4c5b3d0d7a4/magnum/drivers/common/templates/kubernetes/fragments/configure-kubernetes-master.sh#L10614:39
yankcrimeoh hmm, maybe this was fixed recently14:39
yankcrimeyeah i've just spotted that in master14:39
yankcrimeit's not in train which is where i've been troubleshooting14:40
yankcrimeok cool, i'll carry my own patch for now knowing that it's been fixed upstream already14:41
*** ricolin_ has quit IRC14:41
*** ricolin_ has joined #openstack-containers14:41
brtknryankcrime: just not backported to train14:42
yankcrimebrtknr: looks that way14:42
*** ricolin_ has quit IRC14:48
openstackgerritBharat Kunwar proposed openstack/magnum stable/train: Support TimeoutStartSec for k8s systemd services  https://review.opendev.org/71270815:18
brtknryankcrime: ^15:18
openstackgerritSpyros Trigazis proposed openstack/magnum master: k8s-fedora: Set max-size to 10m for containers  https://review.opendev.org/71247515:18
yankcrimebrtknr: awesome!15:18
openstackgerritSpyros Trigazis proposed openstack/magnum master: fcos: Disable zincati auto-updates  https://review.opendev.org/71247615:19
brtknryankcrime: we'll make sure its included in 9.3.015:20
*** ykarel is now known as ykarel|away15:34
*** sapd1 has joined #openstack-containers15:42
*** xinliang has joined #openstack-containers16:11
*** xinliang has quit IRC16:20
*** sapd1 has quit IRC16:39
*** udesale_ has quit IRC16:49
*** trident has quit IRC19:51
*** trident has joined #openstack-containers19:59
*** pcaruana has quit IRC20:20
*** markguz_ has quit IRC20:56
openstackgerritMerged openstack/magnum stable/train: Support TimeoutStartSec for k8s systemd services  https://review.opendev.org/71270821:49
*** rcernin has joined #openstack-containers22:04
*** rcernin has quit IRC22:04
*** rcernin has joined #openstack-containers22:05
*** dave-mccowan has quit IRC22:50
openstackgerritFeilong Wang proposed openstack/magnum master: Update default calico_ipv4pool  https://review.opendev.org/71281523:06
*** dave-mccowan has joined #openstack-containers23:09
*** threestrands has joined #openstack-containers23:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!