| jakeyip | thanks flwang, we are thinking of looking into it. lloks like the csi plugin runs with kubernetes 1.13+. Would you have any idea what CERN is running? | 00:33 |
|---|---|---|
| jakeyip | what are you running by the way? we are running 1.11.8 | 00:33 |
| flwang | we're running v1.11, v1.12 and v1.13 | 00:49 |
| flwang | and we're going to support v1.14 and above | 00:49 |
| *** rookiehuang has joined #openstack-containers | 01:09 | |
| *** ricolin has joined #openstack-containers | 02:51 | |
| *** ricolin_ has joined #openstack-containers | 03:35 | |
| *** ricolin has quit IRC | 03:38 | |
| *** ramishra has joined #openstack-containers | 04:04 | |
| *** udesale has joined #openstack-containers | 04:07 | |
| *** ykarel has joined #openstack-containers | 04:23 | |
| *** ramishra has quit IRC | 04:45 | |
| *** ramishra has joined #openstack-containers | 04:45 | |
| *** danil has quit IRC | 04:49 | |
| *** janki has joined #openstack-containers | 05:02 | |
| *** ramishra has quit IRC | 06:06 | |
| *** ramishra has joined #openstack-containers | 06:06 | |
| *** ricolin__ has joined #openstack-containers | 06:41 | |
| *** ricolin__ is now known as ricolin | 06:41 | |
| *** ricolin_ has quit IRC | 06:45 | |
| *** rcernin has quit IRC | 07:04 | |
| *** pcaruana has joined #openstack-containers | 07:08 | |
| *** ykarel is now known as ykarel|lunch | 07:59 | |
| *** janki has quit IRC | 08:43 | |
| *** ykarel|lunch is now known as ykarel | 09:08 | |
| *** janki has joined #openstack-containers | 09:45 | |
| *** dasp has quit IRC | 09:49 | |
| *** dasp has joined #openstack-containers | 09:49 | |
| *** spsurya has joined #openstack-containers | 09:50 | |
| brtknr | jakeyip: we are using manila with magnum | 10:25 |
| brtknr | jakeyip: however, it wasn't smooth sailing to get it working... | 10:25 |
| brtknr | jakeyip: there were some permission issues... | 10:26 |
| brtknr | jakeyip: especially when mounting manila PV to non-root containers | 10:26 |
| brtknr | we (StackHPC) | 10:26 |
| *** janki has quit IRC | 10:27 | |
| tbarron | jakeyip: brtknr: flwang: i've run manila-csi with 1.15.0, just POC, no magnum | 10:47 |
| tbarron | brtknr: can you elaborate a bit on the permissions issues? | 10:47 |
| brtknr | tbarron: the permission issue was basically that the persistent volumes created by manila-csi were by default being created with root, 755 permission... which when mounted by a non root container was unwritable | 10:50 |
| tbarron | brtknr: ack | 10:50 |
| brtknr | we had to change this behaviour in the manila api so that the volumes are created with 777 | 10:51 |
| brtknr | bit of a hack but got things working... | 10:51 |
| tbarron | brtknr: yeah we added this option to manila ceph driver https://github.com/openstack/manila/blob/master/manila/share/drivers/cephfs/driver.py#L103 | 10:51 |
| tbarron | brtknr: and I think 0775 is sufficient? | 10:52 |
| brtknr | tbarron: yeah perhaps... i never tried 0775, only 0777... | 10:53 |
| tbarron | brtknr: at least when tested with openshift | 10:53 |
| brtknr | we ended up using cephfs_volume_mode i think... | 10:53 |
| brtknr | thanks for implemeting it :) | 10:53 |
| tbarron | brtknr: well I think manila is pretty compelling as a csi plugin for k8s, | 10:54 |
| tbarron | rwx storage | 10:54 |
| tbarron | hard multitenancy (keystone) | 10:54 |
| tbarron | and service over net rather than via hypervisor so it works fine for VMs and baremetal | 10:55 |
| tbarron | nodes | 10:55 |
| brtknr | yep, i second that | 11:10 |
| brtknr | it works quite reliably | 11:10 |
| brtknr | i havent used cinder so dont know how it comapres against cinder volumes | 11:10 |
| brtknr | tbarron: are you from cern? | 11:12 |
| *** ramishra has quit IRC | 11:24 | |
| *** ramishra has joined #openstack-containers | 11:26 | |
| *** mkuf_ is now known as mkuf | 11:35 | |
| tbarron | brtknr: no, I work for Red Hat | 11:52 |
| tbarron | brtknr: cinder-csi and manila-csi both give you keystone based hard multitenancy | 11:52 |
| tbarron | brtknr: that is, you can deploy lots of departmental k8s clusters using the same big scale out storage from OpenStack but | 11:53 |
| tbarron | brtknr: provide hard separation between the storage used for one k8s cluster and another while | 11:54 |
| tbarron | brtknr: maintaining elastic and dynamic provisioning, i.e. without having to dedicate separate storage pools per tenant | 11:54 |
| tbarron | brtknr: main diffs for cinder vs manila is that manila gives genernal purpose RWX mode storage and | 11:55 |
| tbarron | brtknr: manila serves storage over network rather than hypervisor so it's easy to provide it | 11:56 |
| tbarron | brtknr: doesn't know or care abot the difference between serving storage to k8s with minion nodes running on VMs or on bare metal | 11:56 |
| tbarron | brtknr: flwang: jakeyip: if any of you are deploying manila csi multi-protocol (native cephfs and nfs) this makes deployment of the CSI plugins a lot easier: | 12:01 |
| tbarron | https://github.com/kubernetes/cloud-provider-openstack/pull/711 | 12:01 |
| brtknr | tbarron: thanks for sharing, looks useful | 12:06 |
| *** udesale has quit IRC | 12:06 | |
| *** udesale has joined #openstack-containers | 12:07 | |
| *** ramishra has quit IRC | 12:46 | |
| *** goldyfruit has joined #openstack-containers | 13:06 | |
| *** goldyfruit has quit IRC | 13:11 | |
| *** ykarel is now known as ykarel|afk | 13:22 | |
| *** pcaruana has quit IRC | 13:47 | |
| *** ykarel|afk is now known as ykarel|away | 13:52 | |
| *** pcaruana has joined #openstack-containers | 14:00 | |
| *** ramishra has joined #openstack-containers | 14:07 | |
| *** danil has joined #openstack-containers | 14:07 | |
| *** rookiehuang has quit IRC | 15:02 | |
| *** goldyfruit has joined #openstack-containers | 15:26 | |
| *** lpetrut has joined #openstack-containers | 16:09 | |
| *** henriqueof has quit IRC | 16:12 | |
| *** ramishra has quit IRC | 16:44 | |
| *** ricolin has quit IRC | 16:57 | |
| *** udesale has quit IRC | 17:07 | |
| *** goldyfruit has quit IRC | 17:18 | |
| *** goldyfruit_ has joined #openstack-containers | 17:18 | |
| *** ykarel|away has quit IRC | 17:56 | |
| *** spsurya has quit IRC | 18:49 | |
| *** lpetrut has quit IRC | 19:05 | |
| *** henriqueof has joined #openstack-containers | 20:20 | |
| henriqueof | Have anyone found this error on Stein?: failed to initialize keystone client: open /etc/kubernetes/ca-bundle.crt: no such file or directory | 20:20 |
| henriqueof | The k8s-keystone-auth keeps on CrashLoopBackOff state. | 20:20 |
| *** rcernin has joined #openstack-containers | 22:06 | |
| *** henriqueof has quit IRC | 23:00 | |
| jakeyip | henriqueof: any other failures in '/var/log/cloud-init-output.log' ? | 23:47 |
| jakeyip | Thanks brtknr, tbarron, flwang for your input. Sorry I wasn't around to reply, I'm on Australia time. | 23:51 |
| jakeyip | I'm looking at using the manila csi and cephfs csi plugins. Unfortunately I think it needs Natalius Ceph so I'll have to wait for the Ceph team to upgrade their clusters. https://github.com/ceph/ceph-csi | 23:53 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!