Tuesday, 2019-01-29

« Monday, 2019-01-28 Index Wednesday, 2019-01-30 »
*** _fragatina has joined #openstack-containers00:01
*** itlinux has joined #openstack-containers00:10
*** sdake has quit IRC00:23
*** sdake has joined #openstack-containers00:24
*** mrodriguez has quit IRC00:27
*** hongbin has joined #openstack-containers00:27
*** flwang has quit IRC00:45
*** hongbin has quit IRC01:18
*** hongbin has joined #openstack-containers01:20
*** _fragatina has quit IRC01:27
*** sdake has quit IRC01:28
*** _fragatina has joined #openstack-containers01:28
*** _fragatina has quit IRC01:32
*** ricolin_ has joined #openstack-containers02:01
openstackgerritFeilong Wang proposed openstack/python-magnumclient master: Keystone auth support  https://review.openstack.org/62309202:07
openstackgerritFeilong Wang proposed openstack/python-magnumclient master: Keystone auth support  https://review.openstack.org/62309202:21
*** sdake has joined #openstack-containers02:31
*** sdake has quit IRC02:37
*** sdake has joined #openstack-containers02:38
openstackgerritFeilong Wang proposed openstack/python-magnumclient master: Keystone auth support  https://review.openstack.org/62309202:53
*** ykarel|away has joined #openstack-containers02:55
*** sdake has quit IRC02:58
*** sdake has joined #openstack-containers02:59
*** _fragatina has joined #openstack-containers03:12
*** _fragatina has quit IRC03:13
*** _fragatina has joined #openstack-containers03:14
*** sdake has quit IRC03:26
*** sdake has joined #openstack-containers03:30
*** sdake has quit IRC03:42
*** ykarel|away is now known as ykarel03:42
openstackgerritFeilong Wang proposed openstack/magnum master: Support multi k8s image versions  https://review.openstack.org/63365003:49
*** ykarel is now known as ykarel|afk03:50
*** udesale has joined #openstack-containers04:02
*** ykarel|afk is now known as ykarel04:06
*** _fragatina has quit IRC04:08
*** ricolin_ has quit IRC04:11
*** ricolin has joined #openstack-containers04:11
*** itlinux has quit IRC04:41
*** sdake has joined #openstack-containers04:50
*** spsurya has joined #openstack-containers05:05
*** udesale has quit IRC05:29
*** sdake has quit IRC05:41
*** sdake has joined #openstack-containers05:50
*** udesale has joined #openstack-containers05:51
*** udesale has quit IRC05:59
*** udesale has joined #openstack-containers06:00
*** ykarel has quit IRC06:09
*** hongbin has quit IRC06:21
*** ykarel has joined #openstack-containers06:21
*** ramishra has joined #openstack-containers06:33
*** strigazi has quit IRC06:42
*** strigazi has joined #openstack-containers06:43
*** belmoreira has quit IRC06:53
*** udesale has quit IRC07:10
*** udesale has joined #openstack-containers07:18
*** udesale has quit IRC07:22
*** udesale has joined #openstack-containers07:26
*** sapd1 has joined #openstack-containers07:48
openstackgerritFeilong Wang proposed openstack/magnum master: Support multi k8s image versions  https://review.openstack.org/63365007:54
*** sdake has quit IRC08:11
*** ttsiouts has joined #openstack-containers08:27
*** ramishra_ has joined #openstack-containers08:34
*** ramishra has quit IRC08:35
*** janki has joined #openstack-containers08:36
*** ttsiouts has quit IRC08:39
*** ttsiouts has joined #openstack-containers08:40
*** ttsiouts has quit IRC08:44
*** pcaruana has joined #openstack-containers08:51
*** ykarel is now known as ykarel|lunch08:52
*** ttsiouts has joined #openstack-containers09:02
*** ricolin has quit IRC09:07
*** ign0tus has joined #openstack-containers09:15
*** ykarel|lunch is now known as ykarel09:20
*** ttsiouts has quit IRC10:02
*** ttsiouts has joined #openstack-containers10:02
*** ttsiouts has quit IRC10:07
*** ramishra_ has quit IRC10:23
*** sapd1 has quit IRC10:23
*** ramishra has joined #openstack-containers10:30
*** ttsiouts has joined #openstack-containers10:35
openstackgerritMerged openstack/magnum stable/queens: support http/https proxy for discovery url  https://review.openstack.org/63306410:53
*** belmoreira has joined #openstack-containers10:55
openstackgerritDiogo Guerra proposed openstack/magnum master: [WIP] [k8s] helm install metrics service  https://review.openstack.org/63239210:57
*** janki has quit IRC11:02
*** mkuf_ has quit IRC11:07
*** mkuf has joined #openstack-containers11:07
*** udesale has quit IRC11:09
*** sapd1 has joined #openstack-containers11:17
*** ign0tus has quit IRC11:17
*** ign0tus has joined #openstack-containers11:19
*** sapd1 has quit IRC11:36
*** ttsiouts has quit IRC11:40
*** ttsiouts has joined #openstack-containers11:40
*** ttsiouts has quit IRC11:44
*** ttsiouts has joined #openstack-containers12:11
*** ttsiouts has quit IRC12:16
*** ttsiouts has joined #openstack-containers12:18
*** mkuf has quit IRC12:20
*** mkuf has joined #openstack-containers12:26
*** pcaruana has quit IRC12:40
*** pcaruana has joined #openstack-containers12:50
*** kaiokmo has joined #openstack-containers12:50
*** mkuf_ has joined #openstack-containers12:59
*** mkuf has quit IRC13:02
*** ttsiouts has quit IRC13:32
*** ttsiouts has joined #openstack-containers13:33
*** ttsiouts has quit IRC13:37
*** zul has quit IRC13:38
*** mkuf has joined #openstack-containers13:43
*** mkuf_ has quit IRC13:45
*** ttsiouts has joined #openstack-containers13:50
*** ign0tus has quit IRC13:54
*** ign0tus has joined #openstack-containers13:57
*** ykarel is now known as ykarel|away14:01
*** ign0tus has quit IRC14:10
*** zul has joined #openstack-containers14:10
*** udesale has joined #openstack-containers14:24
*** ykarel|away has quit IRC14:25
*** ign0tus has joined #openstack-containers14:25
*** ign0tus has quit IRC14:37
*** dave-mccowan has joined #openstack-containers14:40
*** dave-mccowan has quit IRC14:45
*** pcaruana has quit IRC14:45
openstackgerritDiogo Guerra proposed openstack/magnum master: [WIP] [k8s] helm install metrics service  https://review.openstack.org/63239214:46
*** sdake has joined #openstack-containers14:49
*** sdake has quit IRC14:51
*** sdake has joined #openstack-containers14:53
*** pcaruana has joined #openstack-containers14:53
openstackgerritKeith Berger proposed openstack/magnum stable/pike: support http/https proxy for discovery url  https://review.openstack.org/63375514:58
*** hongbin has joined #openstack-containers15:13
*** salmankhan has joined #openstack-containers15:14
*** salmankhan has quit IRC15:18
*** udesale has quit IRC15:18
*** ttsiouts has quit IRC15:23
*** ttsiouts has joined #openstack-containers15:23
*** ttsiouts has quit IRC15:28
*** sdake has quit IRC15:32
*** sdake has joined #openstack-containers15:36
*** livelace has joined #openstack-containers15:37
*** ykarel|away has joined #openstack-containers15:38
*** ykarel|away is now known as ykarel15:39
*** livelace has quit IRC15:39
*** ttsiouts has joined #openstack-containers15:42
*** openstackgerrit has quit IRC15:51
*** Nel1x has joined #openstack-containers16:36
*** sdake has quit IRC16:46
*** pcaruana has quit IRC17:01
*** ttsiouts has quit IRC17:06
*** ttsiouts has joined #openstack-containers17:06
*** ttsiouts has quit IRC17:10
*** hongbin has quit IRC17:14
*** ramishra has quit IRC17:25
colby_flwang: for what its worth, its just the minions joining the cluster that is the problem. Im seeing the error on the minion only. I can run get nodes on master and it just returns the master nodes. This worked fine with cluster_user_trust=False. But then trustID is never set in the openstack config and I cant use cinder volumes in kubernetes. I switched that config to attempt to use cinder volumes for pvc's and now the minions dont join with17:37
colby_the error I gave before. Do you want the magnum conductor logs or logs from the the actual nodes?17:37
*** ykarel is now known as ykarel|away17:41
*** sdake has joined #openstack-containers17:43
colby_it appears to get the availibility zone from the cloud providor calls: Adding node label from cloud provider: failure-domain.beta.kubernetes.io/zone=West Datacenter17:59
colby_then fails when joining the cluster: invalid: metadata.labels: Invalid value: "West Datacenter"17:59
*** sdake has quit IRC18:10
*** sdake has joined #openstack-containers18:38
*** _fragatina has joined #openstack-containers19:01
*** ykarel|away has quit IRC19:06
*** ttsiouts has joined #openstack-containers20:26
*** openstackgerrit has joined #openstack-containers20:26
openstackgerritKeith Berger proposed openstack/magnum stable/pike: support http/https proxy for discovery url  https://review.openstack.org/63375520:26
*** sdake has quit IRC20:45
strigazimeeting anyone?21:01
cbrummI'm here, not sure who else it21:02
strigazi#startmeeting containers21:02
openstackMeeting started Tue Jan 29 21:02:26 2019 UTC and is due to finish in 60 minutes.  The chair is strigazi. Information about MeetBot at http://wiki.debian.org/MeetBot.21:02
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:02
*** openstack changes topic to " (Meeting topic: containers)"21:02
openstackThe meeting name has been set to 'containers'21:02
strigazi#topic Roll Call21:02
*** openstack changes topic to "Roll Call (Meeting topic: containers)"21:02
cbrummo/21:02
strigazio/21:02
jakeyipo/21:02
*** imdigitaljim has joined #openstack-containers21:03
imdigitaljimo/21:04
strigazi#topic Stories/Tasks21:04
*** openstack changes topic to "Stories/Tasks (Meeting topic: containers)"21:04
strigaziFrom my side:21:05
strigazik8s v1.11.6 images rebuilt https://review.openstack.org/#/c/633478/21:05
strigazi(v1.11.7 was out just yesterday)21:06
strigazitiller deployment can be taken in, please review https://review.openstack.org/#/c/612336/21:06
imdigitaljimfrom us:21:07
strigazitwo patches for upgrades: add openssh clients to the heat-agent https://review.openstack.org/#/c/633504/ AND add the agent to all nodes: https://review.openstack.org/#/c/561858/21:07
imdigitaljimwe just got kubernetes org approval to submit21:07
imdigitaljimso we might have some openstack-ccm updates to provide in the near future21:08
imdigitaljimcluster autoscaler work and inplace upgrades for the centos driver21:08
imdigitaljimonce this is in place and stable ill be making effort to upstreaming the driver21:08
imdigitaljimwe run v1.13.2 atm as well21:09
imdigitaljimwe also want to finalize any core magnum changes we'd probably need to do21:09
strigaziimdigitaljim: please build also the ci for the special centos image.21:10
imdigitaljimcluster-autoscaler work is k8s specific (mostly), trivial driver changes if any i forget21:10
colin-o/21:10
imdigitaljimyes i definitely will21:10
imdigitaljimmake an mvp centos CI21:10
*** munimeha1 has joined #openstack-containers21:11
imdigitaljimalthough strigazi: i might need some help getting that setup im unfamiliar21:11
imdigitaljimto connect with existing stuff to be easily consumed21:12
strigazifor the cluster-autoscaler I'm testing our own https://github.com/cernops/autoscaler/pull/321:12
*** flwang has joined #openstack-containers21:12
imdigitaljimill forward to our cas man21:12
strigaziI'm giving priority to that one.21:12
flwangsorry, i'm late21:12
colin-nice strigazi looks like a lot of progress21:13
*** schaney has joined #openstack-containers21:13
flwangstrigazi: what are we discussing?21:15
flwangi'd like to discuss the k8s image versions we support, related to patch https://review.openstack.org/63365021:16
strigaziflwang: looks ok, maybe we can use the vars file21:16
strigaziother than that it is ok21:17
strigaziflwang: what do you think?21:17
flwangstrigazi: use vars is also ok for me and it's more clean i think, i will propose another patchset21:18
strigaziflwang: thanks, building all of them is fine21:19
strigaziI have another thing I want to discuss21:19
strigaziWe have an issue in k8s_fedora21:19
strigazimaybe in centos that imdigitaljim is solved21:19
strigazik8s uostrean beeds three CAs21:20
strigaziCA= certificate authority not cluster autoscaler21:20
strigazione  for etcd21:20
strigazione for front-proxy21:20
strigaziand one for the API21:20
strigaziIn magnum we create only one.21:20
strigaziIn the same manner that we create the service account keys, we need two more CAs.21:21
flwangwhat's the affection now?21:22
flwangaffection/impact i mean21:22
strigaziFor example, a new feature like the metrics-server doesn't work correctly21:22
flwangis there any security hole now?21:22
strigazisee here:21:22
imdigitaljimoh?21:22
imdigitaljimhelm chart metrics server?21:23
strigazihttps://review.openstack.org/#/c/632392/21:23
strigazihttps://review.openstack.org/#/c/632392/4/magnum/drivers/common/templates/kubernetes/fragments/configure-kubernetes-master.sh@7521:23
strigaziThis option kind of works21:23
strigazibut the proper thing to do is to have tree CAs21:23
strigazisee docs here:21:23
strigazihttps://github.com/kubernetes/website/blob/master/content/en/docs/setup/certificates.md21:24
flwangso you mean though we can use the current, only one, CA, but we'd better to create 3, right?21:25
strigaziinstalling the metrics-server with helm or not, it doesn'r matter21:25
strigaziflwang: yes21:25
strigaziquoting docs: You can create a single root CA, controlled by an administrator. This root CA can then create multiple intermediate CAs, and delegate all further creation to Kubernetes itself.21:25
imdigitaljimyeah21:26
imdigitaljim1 CA is okay21:26
strigazihaving these three CAs would make the transition to kubeadm ~trivial21:26
imdigitaljim3 CA is better21:26
flwangstrigazi: i see21:26
strigazikubeadm will try to create these three CAs.21:26
flwangstrigazi: so i think it's important but not urgent issue21:26
strigaziif you provide them it will use them21:27
flwangcool21:27
imdigitaljimalso note: we dont use kubeadm at this time21:27
imdigitaljimin case you were wondering21:27
imdigitaljimtheres a few issues in using kubeadm that we need to be better before/if we transition to them21:27
imdigitaljimi think they are publicly listed in their design docs21:28
strigazikubeadm or not, to use API aggregators we need to deploy front-proxy certs properly21:28
openstackgerritFeilong Wang proposed openstack/magnum stable/pike: support http/https proxy for discovery url  https://review.openstack.org/63375521:29
strigazidocs for front proxy: https://kubernetes.io/docs/tasks/access-kubernetes-api/configure-aggregation-layer/21:31
imdigitaljimWarning: Do not reuse a CA that is used in a different context unless you understand the risks and the mechanisms to protect the CA’s usage.21:32
imdigitaljim:D21:32
imdigitaljimi see your concern21:32
openstackgerritKeith Berger proposed openstack/magnum stable/pike: support http/https proxy for discovery url  https://review.openstack.org/63375521:32
strigaziI can mention a horrible side-effect of using one CA. using the same CA to sign certs for kubelets and etcd, means compromise of kubelet certs gives access to etcd.21:33
imdigitaljimsure21:34
strigaziprovided that there is a route from from the kubelet node to etcd21:34
imdigitaljimpreventing lateral movement is always good21:34
strigazisame for the etcd that calico uses21:34
imdigitaljimill probably start working on solving some of these in the upcoming weeks21:35
imdigitaljimgood point out strigazi21:35
colin-yeah. been working on Octavia on the side and have been going through this due to all the CAs it uses to secure communications between components21:35
colin-and wanting them to be signed uniquely by service made deployment more complex but ultimately better for the security, imo21:36
strigaziit is a ~chicken-egg problem. Who sings whose certs :)21:37
imdigitaljimwe were looking at other solutions like vault as a CA21:38
imdigitaljimand such21:38
strigaziThe problem to cert management in kubernetes is more CAs21:38
imdigitaljimhttps://github.com/hashicorp/vault21:38
strigazis/problem/solution/21:38
imdigitaljimnot sure their CA capabilities21:38
strigaziAnything else anyone? (the day is ending for me)21:41
imdigitaljimnot here21:42
imdigitaljimo/21:42
strigaziflwang colin- jakeyip anything else?21:42
strigazijakeyip: for tempest python3, all good?21:43
jakeyipNothing to report :)21:43
strigazicool21:44
colin-nope21:44
strigazilet's end the meeting then21:45
colin-have a good night!21:45
strigazisee you next week21:45
strigazicolin-: cheers21:45
cbrummbye all21:45
strigazi#endmeeting21:45
*** openstack changes topic to "OpenStack Containers Team"21:45
openstackMeeting ended Tue Jan 29 21:45:49 2019 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:45
openstackMinutes:        http://eavesdrop.openstack.org/meetings/containers/2019/containers.2019-01-29-21.02.html21:45
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/containers/2019/containers.2019-01-29-21.02.txt21:45
openstackLog:            http://eavesdrop.openstack.org/meetings/containers/2019/containers.2019-01-29-21.02.log.html21:45
openstackgerritMerged openstack/magnum stable/rocky: Support Keystone AuthN and AuthZ for k8s  https://review.openstack.org/63357122:13
*** sdake has joined #openstack-containers22:15
*** munimeha1 has quit IRC22:43
openstackgerritFeilong Wang proposed openstack/magnum master: Support multi k8s image versions  https://review.openstack.org/63365022:53
-openstackstatus- NOTICE: http://zuul.openstack.org is not working. https://zuul.openstack.org does work. Please use that while we investigate.23:12
colby_flwang: any suggestions for me? Is renaming our availability zone going to be our solution?23:31
*** sdake has quit IRC23:35
*** sdake has joined #openstack-containers23:37
*** rcernin has quit IRC23:53
flwangcolby_: did you upgrade your cinder recently?23:55
*** sdake has quit IRC23:55
*** sdake has joined #openstack-containers23:55
openstackgerritFeilong Wang proposed openstack/magnum master: Support multi k8s image versions  https://review.openstack.org/63365023:59
« Monday, 2019-01-28 Index Wednesday, 2019-01-30 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!