Wednesday, 2018-02-28

*** yasemin has quit IRC00:20
*** Guest5320 has quit IRC00:20
*** Guest5320 has joined #openstack-containers00:23
*** yasemin has joined #openstack-containers00:25
*** yamamoto has joined #openstack-containers00:28
*** yamamoto has quit IRC00:33
*** fragatina has quit IRC00:41
*** fragatina has joined #openstack-containers00:42
*** kiennt26 has joined #openstack-containers00:48
*** AlexeyAbashkin has joined #openstack-containers01:04
*** AlexeyAbashkin has quit IRC01:08
*** ianychoi has joined #openstack-containers01:13
*** hongbin has quit IRC01:26
*** yamamoto has joined #openstack-containers01:39
*** yamamoto has quit IRC01:44
*** harlowja has quit IRC01:50
*** kiennt26 has quit IRC01:56
*** fragatina has quit IRC02:03
*** fragatina has joined #openstack-containers02:03
*** oikiki has joined #openstack-containers02:04
*** AlexeyAbashkin has joined #openstack-containers02:04
*** vijaykc4 has joined #openstack-containers02:04
*** fragatina has quit IRC02:04
*** itlinux has joined #openstack-containers02:07
*** kiennt26 has joined #openstack-containers02:07
*** AlexeyAbashkin has quit IRC02:08
*** daidv has quit IRC02:09
*** daidv has joined #openstack-containers02:09
*** gsimondon has joined #openstack-containers02:13
*** vijaykc4 has quit IRC02:14
*** gsimondon has quit IRC02:18
*** vijaykc4 has joined #openstack-containers02:21
*** kiennt26 has quit IRC02:31
*** kiennt26 has joined #openstack-containers02:31
*** ykarel has joined #openstack-containers02:39
*** oikiki has quit IRC02:40
*** yamamoto has joined #openstack-containers02:41
*** yamamoto has quit IRC02:46
*** oikiki has joined #openstack-containers02:49
*** fragatina has joined #openstack-containers02:51
*** fragatina has quit IRC02:51
*** gsimondon has joined #openstack-containers03:07
*** gsimondon has quit IRC03:11
*** fragatina has joined #openstack-containers03:33
*** ykarel has quit IRC03:37
*** fragatina has quit IRC03:38
*** oikiki has quit IRC03:42
*** yamamoto has joined #openstack-containers03:43
*** yamamoto has quit IRC03:48
*** oikiki has joined #openstack-containers03:52
openstackgerritFeilong Wang proposed openstack/magnum master: Fix network issue with calico  https://review.openstack.org/54813903:57
*** gsimondon has joined #openstack-containers04:01
*** flwang has quit IRC04:03
*** gsimondon has quit IRC04:06
*** udesale has joined #openstack-containers04:18
*** ykarel has joined #openstack-containers04:33
*** oikiki has quit IRC04:42
*** yamamoto has joined #openstack-containers04:45
*** yamamoto has quit IRC04:50
*** janki has joined #openstack-containers04:53
*** gsimondon has joined #openstack-containers04:55
*** gsimondon has quit IRC05:00
*** mdnadeem has joined #openstack-containers05:02
*** AlexeyAbashkin has joined #openstack-containers05:05
*** dave-mccowan has quit IRC05:08
*** AlexeyAbashkin has quit IRC05:09
*** dpawar has joined #openstack-containers05:29
*** fragatina has joined #openstack-containers05:40
*** harlowja has joined #openstack-containers05:40
*** fragatina has quit IRC05:40
*** fragatina has joined #openstack-containers05:41
*** yamamoto has joined #openstack-containers05:46
*** sidx64 has joined #openstack-containers05:47
*** gsimondon has joined #openstack-containers05:49
*** sidx64 has joined #openstack-containers05:50
*** yamamoto has quit IRC05:52
*** pcichy has joined #openstack-containers05:53
*** gsimondon has quit IRC05:54
*** sidx64 has quit IRC05:56
*** sidx64 has joined #openstack-containers05:57
*** sidx64 has quit IRC05:58
*** udesale_ has joined #openstack-containers06:03
*** ykarel_ has joined #openstack-containers06:04
*** udesale has quit IRC06:05
*** udesale__ has joined #openstack-containers06:06
*** ykarel has quit IRC06:07
*** udesale_ has quit IRC06:09
*** harlowja has quit IRC06:13
*** sidx64 has joined #openstack-containers06:16
*** fragatina has quit IRC06:16
*** fragatina has joined #openstack-containers06:17
*** sidx64 has quit IRC06:17
*** ianychoi_ has joined #openstack-containers06:19
*** sidx64 has joined #openstack-containers06:19
*** sidx64 has quit IRC06:21
*** ianychoi has quit IRC06:22
*** sidx64 has joined #openstack-containers06:23
openstackgerritOpenStack Proposal Bot proposed openstack/magnum-ui master: Imported Translations from Zanata  https://review.openstack.org/54845506:41
*** ricolin has joined #openstack-containers06:42
*** gsimondon has joined #openstack-containers06:43
*** gsimondon has quit IRC06:48
*** yamamoto has joined #openstack-containers06:48
*** vijaykc4 has quit IRC06:49
*** vijaykc4 has joined #openstack-containers06:52
*** yamamoto has quit IRC06:54
*** sidx64 has quit IRC07:03
*** ykarel__ has joined #openstack-containers07:04
*** sidx64 has joined #openstack-containers07:04
*** sidx64 has quit IRC07:06
*** ykarel_ has quit IRC07:06
*** sidx64 has joined #openstack-containers07:07
*** sidx64 has quit IRC07:09
*** mjura has joined #openstack-containers07:17
*** ramishra has joined #openstack-containers07:18
*** ykarel__ is now known as ykarel07:20
*** trinaths has joined #openstack-containers07:26
*** jianhua has joined #openstack-containers07:34
*** AlexeyAbashkin has joined #openstack-containers07:43
*** yamamoto has joined #openstack-containers07:50
*** pcaruana has joined #openstack-containers07:51
*** lpetrut has joined #openstack-containers07:52
*** lpetrut has quit IRC07:54
*** yamamoto has quit IRC07:55
*** vijaykc4 has quit IRC07:56
*** rcernin has quit IRC07:57
*** udesale_ has joined #openstack-containers08:02
*** mdnadeem_ has joined #openstack-containers08:02
*** ykarel_ has joined #openstack-containers08:02
*** b_bezak has joined #openstack-containers08:04
*** gsimondon has joined #openstack-containers08:04
*** udesale__ has quit IRC08:05
*** mdnadeem has quit IRC08:05
*** ykarel has quit IRC08:06
*** yamamoto has joined #openstack-containers08:06
*** yamamoto has quit IRC08:06
*** armaan has quit IRC08:07
*** armaan has joined #openstack-containers08:07
*** udesale__ has joined #openstack-containers08:07
*** ykarel__ has joined #openstack-containers08:07
*** gsimondo1 has joined #openstack-containers08:09
*** gsimondon has quit IRC08:09
*** sidx64 has joined #openstack-containers08:10
*** udesale_ has quit IRC08:10
*** ykarel_ has quit IRC08:10
*** mdnadeem_ has quit IRC08:10
*** ykarel__ is now known as ykarel08:11
*** ramishra has quit IRC08:12
*** _slunkad has quit IRC08:13
*** ykarel is now known as ykarel|lunch08:14
*** ricolin has quit IRC08:19
*** vijaykc4 has joined #openstack-containers08:19
*** vijaykc4 has quit IRC08:19
*** vijaykc4 has joined #openstack-containers08:20
*** vijaykc4 has quit IRC08:20
*** sidx64 has quit IRC08:23
*** vijaykc4 has joined #openstack-containers08:23
*** mdnadeem_ has joined #openstack-containers08:24
*** sidx64 has joined #openstack-containers08:24
*** sidx64 has quit IRC08:25
*** sidx64 has joined #openstack-containers08:27
*** trinaths has quit IRC08:27
*** ykarel|lunch is now known as ykarel08:31
*** vijaykc4 has quit IRC08:31
*** vijaykc4 has joined #openstack-containers08:36
*** trinaths has joined #openstack-containers08:36
*** ramishra has joined #openstack-containers08:39
*** flwang has joined #openstack-containers08:41
flwangstrigazi: around?08:41
*** ricolin has joined #openstack-containers08:41
*** chhagarw has joined #openstack-containers08:49
*** olivierbourdon38 has joined #openstack-containers08:56
*** ricolin has quit IRC08:57
*** lpetrut has joined #openstack-containers08:57
*** ricolin has joined #openstack-containers08:59
*** mgoddard_ has joined #openstack-containers09:03
strigaziflwang: hey09:07
flwangstrigazi: when will the magnum sessions start?09:08
*** yamamoto has joined #openstack-containers09:09
strigaziin a few minutes, we just wait for a few more people to show up. we are two so far :)09:10
flwanghah, ok09:12
*** kiennt26 has quit IRC09:12
flwanghow can I join? is that ok if I just put my opinions on the etherpad?09:13
*** yolanda has quit IRC09:13
strigaziflwang: yes, or I can create a hangout or vidyo09:16
flwangthat's ok, here is midnight, so let me keep quiet09:17
strigaziflwang: makes sense :)09:17
flwangi have replied your question on https://review.openstack.org/#/c/54813909:18
flwangthere are some regression issue introduced by the etcd patch and the dashboard patch09:18
flwangadd add the two mounts, all pods work now09:19
flwangbut I'm not sure if there is a bug between calico and the latest k8s dashboard or proxy, now I can't use kubectl proxy to access dashboard09:19
strigaziit is strange, since f27 has docker inside and therefore /var/lib/docker exists09:20
flwangstrigazi: sorry, i'm still using 2609:20
flwangshould I just upgrade to 27 and don't care about this?09:20
flwangand does the /var/run exists in f27?09:21
strigaziyes09:21
strigazieg for /var/run/secrets09:21
flwangstrigazi: great, i will test it again with f2709:22
flwangbut then same question, do we need to care about this? for f2609:23
flwangor we just need to document it09:23
*** vijaykc4 has quit IRC09:23
strigaziwe can document this09:24
jianhuai think ,after every magnum verion released, we should provide related fedora version09:26
*** vijaykc4 has joined #openstack-containers09:27
strigazijianhua: yes, so far it was always the latest09:28
strigaziand with this release it still is09:28
strigazilet's create a matrix then09:28
flwangstrigazi: ok, seems it's back to my original suggestion09:28
*** avi_ has joined #openstack-containers09:28
*** robputt has joined #openstack-containers09:28
robputtgood morning :-)09:29
*** hongbin has joined #openstack-containers09:29
flwangstrigazi: so shall we create a support matrix since now and maintain it in the future?09:29
strigaziflwang: yes, maintaining matrices will end up being undermaintained09:29
strigazirobputt: good morning09:30
flwangstrigazi: cool, i will test it with f27 and reupload a new patch set to add the matrix09:30
robputtI am having some trouble with Magnum and was hoping that somebody could help me :-), I have it setup and configured on top of a known working Nova, Heat, Neutron etc... Pike install09:31
strigazifyi it is in the release notes09:31
strigazihttps://docs.openstack.org/releasenotes/magnum/queens.html09:31
*** _slunkad has joined #openstack-containers09:31
*** kiennt26 has joined #openstack-containers09:31
robputtit all seems to be behaving just fine but when the instance comes up with Fedora Atomic latest image it fails the cloud init stuff to configure the master on stage 1309:31
strigazibut yes, it seems for older releases there is confusion09:32
strigazirobputt: for magnum pike you use f26 right?09:32
_slunkadstrigazi:  ERROR heat.engine.resource ResourceFailure: OverLimit: resources[0].resources.kube-master: Request is too large.<br /><br />09:32
robputterrr let me SSH to this host and see what version it is09:32
*** ykarel_ has joined #openstack-containers09:33
*** mdnadeem__ has joined #openstack-containers09:33
*** udesale_ has joined #openstack-containers09:33
robputtseems not, it seems ca[fedora@my-cluster-s3zzkura2y4d-master-0 ~]$ cat /etc/fedora-release09:33
robputtFedora release 25 (Twenty Five)09:33
robputtcan you link me to a f26 atomic image that is known to work please?09:34
*** sidx64 has quit IRC09:34
strigazihttp://mirror.datacenter.by/pub/fedoraproject.org/alt/atomic/stable/09:34
*** mdnadeem_ has quit IRC09:34
jianhuastrigazi,I asked a question last time,how to build a docker image for specified k8s version that can run on atomic host09:35
strigazihttps://download.fedoraproject.org/pub/alt/atomic/stable09:35
jianhuado you remember?09:35
robputtthanks @strigazi09:35
jianhuais there any further progress?09:35
*** ykarel has quit IRC09:35
flwangstrigazi: should we focus on PTG now?09:35
robputtI will try this out and confirm if I am successful, many thanks09:35
flwangjianhua: robputt: now it's the PTG time technically09:35
*** ykarel__ has joined #openstack-containers09:35
flwangmaybe we can discuss issues later than today09:36
jianhuahow to join you09:36
flwangjianhua: https://etherpad.openstack.org/p/magnum-ptg-rocky09:36
*** udesale__ has quit IRC09:36
strigazijianhua: right, I'll create a document on how to build them09:36
*** udesale has joined #openstack-containers09:37
strigaziThe only problem is that the repo is hosted on github09:37
jianhua i can open github09:38
*** udesale_ has quit IRC09:38
*** ykarel_ has quit IRC09:38
jianhuai buy a vps,^-^09:38
strigazihttps://github.com/projectatomic/atomic-system-containers09:39
strigazijianhua: you can build these containers kubernetes-*09:39
*** mdnadeem__ has quit IRC09:39
strigazifyi v1.9.3 is pushed to dockerhub you can take it from there09:39
strigaziback to the etherpad09:40
*** yolanda has joined #openstack-containers09:43
*** lpetrut has quit IRC09:49
*** lpetrut has joined #openstack-containers09:50
*** sidx64 has joined #openstack-containers09:50
*** mdnadeem__ has joined #openstack-containers09:50
*** sidx64 has quit IRC09:52
*** sidx64 has joined #openstack-containers10:00
*** sidx64 has quit IRC10:00
*** sidx64 has joined #openstack-containers10:03
*** kiennt26 has quit IRC10:04
*** fragatina has quit IRC10:16
*** fragatina has joined #openstack-containers10:16
*** salmankhan has joined #openstack-containers10:19
*** daidv has quit IRC10:21
*** salmankhan has quit IRC10:23
*** salmankhan has joined #openstack-containers10:23
*** pc_m has quit IRC10:26
*** ykarel__ is now known as ykarel10:29
*** yamamoto has quit IRC10:29
*** ykarel is now known as ykarel|afk10:39
*** lpetrut has quit IRC10:41
*** yamamoto has joined #openstack-containers10:41
*** lpetrut has joined #openstack-containers10:41
*** lpetrut has quit IRC10:42
*** lpetrut has joined #openstack-containers10:42
openstackgerritVabada proposed openstack/python-magnumclient master: OSC command for magnum quota's  https://review.openstack.org/50610010:43
*** udesale has quit IRC10:44
*** ejat has quit IRC10:44
*** ejat has joined #openstack-containers10:44
*** ejat has quit IRC10:44
*** ejat has joined #openstack-containers10:44
*** salmankhan has quit IRC10:46
*** lpetrut has quit IRC10:46
*** lpetrut has joined #openstack-containers10:47
*** sidx64 has quit IRC10:49
openstackgerritVabada proposed openstack/python-magnumclient master: OSC command for magnum quota's  https://review.openstack.org/50610010:49
*** salmankhan has joined #openstack-containers10:53
openstackgerritVabada proposed openstack/python-magnumclient master: OSC command for magnum quota's  https://review.openstack.org/50610010:57
*** trinaths has quit IRC11:03
*** sidx64 has joined #openstack-containers11:04
*** mgoddard_ has quit IRC11:10
openstackgerritVabada proposed openstack/python-magnumclient master: OSC command for magnum quota's  https://review.openstack.org/50610011:13
openstackgerritVabada proposed openstack/python-magnumclient master: OSC command for magnum quota's  https://review.openstack.org/50610011:14
robputtHi can you let me know when PTG time is over so I can poke you guys some more about my issue?11:14
*** mgoddard_ has joined #openstack-containers11:16
openstackgerritMerged openstack/magnum stable/queens: Change swarm ClusterTemplate coe to swarm-mode  https://review.openstack.org/54769311:18
*** b_bezak has quit IRC11:18
*** b_bezak has joined #openstack-containers11:19
*** b_bezak_ has joined #openstack-containers11:20
*** b_bezak has quit IRC11:23
*** ykarel|afk is now known as ykarel11:32
*** sidx64 has quit IRC11:32
*** salmankhan has quit IRC11:34
strigazirochapor1o: can you join us in the etherpad for cluster healing?11:37
*** sidx64 has joined #openstack-containers11:40
*** salmankhan has joined #openstack-containers11:45
*** jianhua has quit IRC11:47
*** jmlowe_ has quit IRC11:48
*** b_bezak_ has quit IRC11:56
*** mgoddard_ has quit IRC11:56
*** yolanda has quit IRC12:02
*** b_bezak has joined #openstack-containers12:05
*** hongbin has quit IRC12:13
*** sidx64 has quit IRC12:14
*** yamamoto has quit IRC12:14
*** lpetrut has quit IRC12:14
openstackgerritVabada proposed openstack/python-magnumclient master: OSC command for magnum quota's  https://review.openstack.org/50610012:15
*** pcichy has quit IRC12:21
*** vijaykc4 has quit IRC12:27
*** vijaykc4 has joined #openstack-containers12:27
*** _slunkad has quit IRC12:27
*** vijaykc4 has quit IRC12:28
*** pcichy has joined #openstack-containers12:29
*** ramishra has quit IRC12:32
*** vijaykc4 has joined #openstack-containers12:32
*** ricolin has quit IRC12:32
*** sidx64 has joined #openstack-containers12:34
*** sidx64 has quit IRC12:36
*** chhagarw has quit IRC12:36
*** gabrielvinha has quit IRC12:38
*** udesale has joined #openstack-containers12:43
*** dpawar has quit IRC12:50
*** vijaykc4 has quit IRC12:51
*** vijaykc4 has joined #openstack-containers12:52
*** pc_m has joined #openstack-containers12:52
*** vijaykc4 has quit IRC12:52
*** yolanda has joined #openstack-containers12:52
*** vijaykc4 has joined #openstack-containers12:53
*** vijaykc4 has quit IRC12:53
*** vijaykc4 has joined #openstack-containers12:53
*** gabrielvinha has joined #openstack-containers13:00
*** ricolin has joined #openstack-containers13:00
*** jmlowe has joined #openstack-containers13:03
*** jmlowe has quit IRC13:07
*** jmlowe has joined #openstack-containers13:08
*** b_bezak has quit IRC13:15
*** robputt has quit IRC13:16
*** lpetrut has joined #openstack-containers13:17
*** b_bezak has joined #openstack-containers13:18
*** dave-mccowan has joined #openstack-containers13:18
*** ktibi has joined #openstack-containers13:23
*** jmlowe has quit IRC13:31
*** ricolin has quit IRC13:31
*** ricolin has joined #openstack-containers13:31
*** mgoddard_ has joined #openstack-containers13:33
*** jmlowe has joined #openstack-containers13:33
*** yamamoto has joined #openstack-containers13:34
*** mdnadeem__ has quit IRC13:40
*** hongbin has joined #openstack-containers13:51
*** ramishra has joined #openstack-containers13:54
*** ramishra has quit IRC13:59
*** vijaykc4 has quit IRC13:59
*** vijaykc4 has joined #openstack-containers13:59
*** vijaykc4 has quit IRC14:00
*** vijaykc4 has joined #openstack-containers14:00
*** _slunkad has joined #openstack-containers14:00
*** vijaykc4 has quit IRC14:00
*** vijaykc4 has joined #openstack-containers14:01
*** vijaykc4 has quit IRC14:02
*** b_bezak has quit IRC14:12
*** armaan has quit IRC14:13
*** ramishra has joined #openstack-containers14:15
*** vijaykc4 has joined #openstack-containers14:18
*** dsariel has joined #openstack-containers14:20
*** ykarel is now known as ykarel|away14:26
*** ykarel|away has quit IRC14:34
*** b_bezak has joined #openstack-containers14:37
*** robputt has joined #openstack-containers14:48
robputtstrigazi you there?14:51
robputtI tried out the Fedora 26 image as suggested earlier14:51
robputtbut it still fails on part 11 of the cloud init stuff with the cert dict not having a key 'pem'14:51
*** armaan has joined #openstack-containers14:54
*** dpawar has joined #openstack-containers14:55
robputtok actually maybe its related to my flavour having a tiny disk local disk... let me try a bigger flavour14:56
*** dpawar has quit IRC14:56
*** ykarel|away has joined #openstack-containers14:56
robputtok nope, same issue15:10
robputtKeyError: 'pem' when running part 11, any ideas why this would be the case?15:10
*** dpawar has joined #openstack-containers15:13
robputtI remade the curl request by hand and it returns a dict containing a key pem, not sure why it fails when its ran via cloud init though :-\15:16
*** vijaykc4 has quit IRC15:17
*** vijaykc4 has joined #openstack-containers15:18
*** chhagarw has joined #openstack-containers15:23
vabadarobputt, could you share the entire /var/log/cloud-init-output.log please?15:24
*** yolanda has quit IRC15:26
*** armaan has quit IRC15:27
*** yee37936 has joined #openstack-containers15:37
*** yee379 has quit IRC15:40
*** yamamoto has quit IRC15:43
*** yolanda has joined #openstack-containers15:46
*** dsariel has quit IRC15:56
*** zhubingbing has joined #openstack-containers15:57
*** mjura has quit IRC16:03
robputtsure16:05
robputtill gist it vabada16:05
*** yamamoto has joined #openstack-containers16:05
robputtp.s. ignore the secrets, this is a test env on my local network so nobody is getting in ;-)16:07
robputthttps://gist.github.com/robputt796/20f7fb64a1ca0ac3373e27f87b373452 < vabada16:07
robputtthese lines seem significant https://gist.github.com/robputt796/20f7fb64a1ca0ac3373e27f87b373452#file-gistfile1-txt-L87-L8916:08
*** oikiki has joined #openstack-containers16:10
*** pcaruana has quit IRC16:10
*** dsariel has joined #openstack-containers16:12
*** gsimondo1 has quit IRC16:12
*** vabada2 has joined #openstack-containers16:15
*** AlexeyAbashkin has quit IRC16:16
*** ykarel|away is now known as ykarel16:17
*** yamamoto has quit IRC16:24
*** vabada has quit IRC16:24
*** PagliaccisCloud has quit IRC16:24
*** OGtrilliams has joined #openstack-containers16:24
*** OGtrilliams is now known as PagliaccisCloud16:25
vabada2robputt, that line seems the cause for Failed running /var/lib/cloud/instance/scripts/part-01116:27
*** vabada2 has quit IRC16:27
*** vabada has joined #openstack-containers16:27
robputtsure16:28
vabadarobputt, can you cat /var/lib/cloud/instance/scripts/part-01116:28
robputtthing is I can manually do that curl just fine and get the output16:28
robputtso i think its either a bad keystone token passed to cloud init16:28
robputtor something16:28
robputtthing is it uses curl --silent so its not so verbose16:28
*** udesale has quit IRC16:32
ykarelrobputt, your cluster vm's can reach openstack.beaconfield.robertputt.co.uk?16:32
robputtyeah its on local public network16:33
*** armaan has joined #openstack-containers16:33
robputti can curl it directly from the VM16:34
ykarelack16:34
ykarelthen it must be some token issue16:35
ykarelrobputt, try running the script manually: bash -x /var/lib/cloud/instance/scripts/part-01116:37
robputtthing is if I run it manually it doesn't have all the env variables setup16:37
robputt:-\16:37
robputtbecause it doesn't have the stuff injected by cloud init16:38
robputtwhat would the procedure be for replicating this manually?16:38
ykarelrobputt, sudo su -16:38
ykarelrobputt, source /etc/sysconfig/heat-params16:38
ykarelthen,  bash -x /var/lib/cloud/instance/scripts/part-01116:38
ykarelbefore running update the script16:39
ykarelcert_dir=/etc/kubernetes/certs --> cert_dir=/etc/kubernetes/certs216:39
robputtok sure ill try this and report back16:39
robputt:D16:39
*** armaan has quit IRC16:39
* ykarel will be back after some time16:41
*** dsariel has quit IRC16:43
*** hongbin has quit IRC16:45
*** _slunkad has quit IRC16:45
robputtso ykarel we get to here...16:46
robputtcurl -X GET -H 'X-Auth-Token: gAAAAABaltwU7ER_QlNpVUnpFoF08Gkdar3dG4HPcfDeV1s6HjeB_7DVD1T4QYt69E8b3Q5ZbU2FkGloDQ_rE0_jjfWVwAwjjP5Bc3Czu3CSnsJpZqat-xWjO4eK23M9kMdolMk3nPLysoE1Pq5Q8KNcS_SqUOcXCw' -H 'OpenStack-API-Version: container-infra latest' http://openstack.beaconfield.robertputt.co.uk:9511/v1/certificates/dc61ee20-b066-44bc-a0e4-7997c36f627016:46
robputtand then the response is... {"errors": [{"status": 500, "code": "server", "links": [], "title": "Remote error: BadRequest Invalid input for field 'identity/password/user/password': None is not of type 'string'\n\nFailed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:\n    {'type': 'string'}\n\nOn instance['identity']['password']['user']['password']:\n    None (16:47
robputtHTTP 400) (Request-ID: req-d72a1c5c-4d08-4dfa-96a8-251a90853455)\n[u'", "detail": "Remote error: BadRequest Invalid input for field 'identity/password/user/password': None is not of type 'string'\n\nFailed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:\n    {'type': 'string'}\n\nOn instance['identity']['password']['user']['password']:\n    None (HTTP 400) (Req16:47
robputtuest-ID: req-d72a1c5c-4d08-4dfa-96a8-251a90853455)\n[u'", "request_id": ""}]}16:47
robputtsorry for multi message :facepalm:16:47
*** zhubingbing has quit IRC16:48
*** armaan has joined #openstack-containers16:48
robputtthat user gets generated immediately before via this....16:49
robputthttp://openstack.beaconfield.robertputt.co.uk:5000/v3//auth/tokens16:49
ykarelrobputt, which version of magnum?16:57
robputtlatest from Git master branch16:57
robputtits built from source as per the Pike install instructions16:58
ykarelhow you installed it?16:58
ykarelok16:58
ykarelyour magnum.conf?16:58
robputtliterally followed this16:58
robputthttps://docs.openstack.org/magnum/pike/install/install-guide-from-source.html16:58
robputtlet me grab it for you16:58
robputtsorryy its long17:01
robputthttps://gist.github.com/robputt796/35a70108186ca20095c8cb152d8620d017:02
*** mgoddard_ has quit IRC17:02
ykarelrobputt, you missed some configs17:02
robputtohhhh damn17:03
robputtwhat am I missing?17:03
ykarelin [keystone_authtoken]17:03
robputterrr which keys?17:03
*** yolanda has quit IRC17:04
robputtmagnum has some special stuff in there more than other services?17:04
ykareladmin_user, admin_password and admin_tenant_name17:04
*** salmankhan has quit IRC17:04
robputtohhh ok17:04
*** fragatina has quit IRC17:04
robputtlet me update and try again17:04
ykarelafter adding, restart magnum services, and run that script after updating cert_dir17:05
robputtis this the magnum domain admin or my openstack main admin account?17:05
ykarelmagnum user17:05
robputtok17:06
robputtthanks17:06
*** olivierbourdon38 has quit IRC17:06
ykarelso ossvcsuser17:06
*** salmankhan has joined #openstack-containers17:07
ykarelrobputt, ^^ is magnum user?17:07
*** lpetrut has quit IRC17:07
robputtossvcuser is the service account user17:07
robputtit has admin role in service project17:07
robputtthen there is magnum_domain_admin17:07
robputtwhich has admin role in magnum domain17:07
robputtas per the docs17:07
ykarelrobputt, use admin_user=ossvcuser, admin_password=ycmgbwiwgjttefjc3q8 and admin_tenant_name=service17:08
robputtok17:08
*** gsimondon has joined #openstack-containers17:08
robputtill do this and retry the curl17:08
robputtjust waiting for magnum services to restart17:08
ykarelok17:08
robputtok17:09
robputtthat totally works17:09
robputtbut why does Magnum require these extra keys in keystone_authtoken?17:09
robputtit is the only OpenStack service I seen so far requiring these keys17:09
robputtwell17:10
ykarelThis is the way it works, we have some plan to get rid of these, probably a bug is there17:10
robputtthanks for your help17:10
robputtIll try a new cluster build and I guess this time it should succeed :D17:10
robputtbtw17:10
robputtdo any public clouds deploy Magnum? or is it mostly private clouds with this?17:10
ykarelmay be if u use [keystone_auth] then these should not be required, if you want to try that17:10
ykarelrobputt, strigazi could answer this17:10
robputtkeystone_auth is deprecating keystone_authtoken soon?17:10
ykarelrobputt, yes17:11
robputtok well thanks for the help17:13
robputtmega useful17:13
robputt:-)17:13
*** ktibi has quit IRC17:24
*** oikiki has quit IRC17:26
armaanfolks, any suggestion on how can i implement basic authentication for K8s dashboard via Magnum. something like this https://github.com/kubernetes/dashboard/wiki/Access-control#basic17:28
*** oikiki has joined #openstack-containers17:30
*** ktibi has joined #openstack-containers17:30
armaanstrigazi: Hello!17:32
*** dpawar_ has joined #openstack-containers17:33
*** dpawar has quit IRC17:35
*** AlexeyAbashkin has joined #openstack-containers17:36
*** dpawar_ has quit IRC17:37
*** serlex has joined #openstack-containers17:39
*** fragatina has joined #openstack-containers17:39
robputtle sigh17:40
robputtso the thing totally worked ykarel17:40
*** AlexeyAbashkin has quit IRC17:40
robputtbut now after I get the cert config etc... from Magnum API and I try to use KubeCTL17:40
robputtError from server (Forbidden): deployments.extensions is forbidden: User "Magnum User" cannot create deployments.extensions in the namespace "default"17:41
robputt:-(17:41
*** vijaykc4 has quit IRC17:42
*** vijaykc4 has joined #openstack-containers17:42
*** chhavi__ has joined #openstack-containers17:43
*** gus_ has quit IRC17:45
*** pcichy has quit IRC17:45
ykarelrobputt, when are you seeing it, at what command17:46
robputtany command17:46
robputt:D17:46
robputtkubectl get pods17:46
robputtError from server (Forbidden): pods is forbidden: User "Magnum User" cannot list pods in the namespace "default"17:46
robputtError from server (Forbidden): pods is forbidden: User "Magnum User" cannot list pods in the namespace "default"17:46
robputt(general_dev) Roberts-MacBook-Pro:kubeconf robe8437$ kubectl run nginx --image nginx17:46
robputtError from server (Forbidden): deployments.extensions is forbidden: User "Magnum User" cannot create deployments.extensions in the namespace "default"17:46
robputtfor example17:46
robputt:-)17:46
*** chhagarw has quit IRC17:46
*** gus has joined #openstack-containers17:46
*** serlex has left #openstack-containers17:47
ykarelrobputt, kubectl version17:47
robputt(general_dev) Roberts-MacBook-Pro:kubeconf robe8437$ kubectl version17:48
robputtClient Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-02-07T12:22:21Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}17:48
robputtServer Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"archive", BuildDate:"2018-02-13T11:42:06Z", GoVersion:"go1.10rc2", Compiler:"gc", Platform:"linux/amd64"}17:48
ykarelcat /etc/redhat-release17:48
robputton the server?17:48
ykarelin cluster vm17:48
ykarelmaster or minion17:48
robputt[fedora@test-mbjzb3e3e7bg-master-0 ~]$ cat /etc/redhat-release17:49
robputtFedora release 26 (Twenty Six)17:49
ykarelrobputt, am not sure about this17:50
ykarelcan you try with Fedora atomic 2717:51
robputtyou think maybe it failed to build because I have resource constrained VMs17:51
ykarelis what we have in master17:51
robputtmy testing cloud is very small so I only gave it 1024mb instances (RAM)17:51
robputti can try with Fedora Atomic 27 sure17:51
*** ramishra has quit IRC17:51
ykarelrobputt, resources should not be a problem with this error17:51
robputtok will rebuild with Fedora Atomic 27 and let you know17:52
robputt:-)17:52
robputtthanks for your help again17:52
ykarelrobputt, https://download.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-27-20180212.2/CloudImages/x86_64/images/Fedora-Atomic-27-20180212.2.x86_64.qcow217:52
armaanykarel: Hello, will there be basic auth or RBAC available in Queens k8s dashboard?17:52
ykarelarmaan, i think it's there17:52
ykarelRBAC17:52
armaanykarel: awesome! but not available in Pike?17:52
ykarelarmaan, in queens it's there: https://review.openstack.org/#/c/547096/17:53
ykarelin pike, no17:53
armaanykarel: Yay! \O/17:53
armaanthanks for confirming :)17:53
ykarelrobputt, have you checked logs for kube services17:54
ykarelin case to find if there is some issue17:55
robputtno but i killed the cluster now17:55
robputtso... i will check if it fails on Fedora 2717:55
robputt:D17:55
*** vijaykc4 has quit IRC17:55
*** ricolin has quit IRC17:55
ykarelrobputt, Ok17:55
*** ykarel is now known as ykarel|away17:56
*** janki has quit IRC18:03
*** ykarel|away is now known as ykarel18:03
*** ricolin has joined #openstack-containers18:04
*** yee37936 has quit IRC18:12
*** yee379 has joined #openstack-containers18:12
*** salmankhan has quit IRC18:20
*** fragatina has quit IRC18:23
*** fragatina has joined #openstack-containers18:24
robputtykarel, exactly the same issue in Fedora 2718:31
robputtlooking at logs now18:31
openstackgerritKirsten G. proposed openstack/magnum master: Add minimum system requirements to docs  https://review.openstack.org/54868218:31
*** ricolin has quit IRC18:32
robputthmmm 2018-02-28 18:10:51,841 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_scripts_user.py'>) failed18:32
ykareloikiki, did you face issue with less resources?18:34
ykarelrobputt, yes you should finx something in /var/log/cloud-init-output.log18:35
ykarelsearch for part18:35
robputthang on i see18:36
robputtERROR: There is not enough free space in volume group atomicos to create data volume of size MIN_DATA_SIZE=2G.18:36
robputti need bigger disk again?18:36
robputti will move to bigger flavour size :D18:36
robputtagain18:36
robputt:D18:36
robputthow much disk does this need I am building 20GB instances18:36
ykarelyes18:37
ykarelreally18:37
ykareli faced this with 10 GB18:37
robputtok18:37
ykarel20 GB should be enough18:37
robputtright18:37
robputtmy flavour... https://imgur.com/a/OLEVJ18:38
ykarellooks ok18:38
ykarelhave you specified this flavor only while creating cluster-template18:39
robputti also see https://gist.github.com/robputt796/1a35902251f00729b00fd2f4bbe237d718:39
ykarelthis is fine18:39
*** b_bezak has quit IRC18:40
robputtohh ok18:40
*** b_bezak has joined #openstack-containers18:40
robputtthe flavour is specified in the template... here is the template...18:42
robputthttps://gist.github.com/robputt796/a9989c3c0ea418570e13a20bf2de2f9a18:42
*** b_bezak has quit IRC18:45
oikikiykarel: yes18:45
oikikiykarel: i had to increase things bc the cluster create was really slow18:46
oikikiykarel: I saw strigazi wanted to add a section on the ptg etherpad page. so i updated it with the specs he told us to use on the intern chat18:47
*** chhavi__ has quit IRC18:48
ykareloikiki, Ok.18:49
ykareli use 8GB, 4 CPU and some swap, as sometime i don't get big environment18:50
ykarelrobputt, it looks ok18:51
robputtykarel, 8GB and 4 CPU O_o, wow thats half of my compute node hehe (I do Openstack on a super budget setup)18:51
robputti will keep digging the logs18:51
robputt:-)18:51
ykarelrobputt, check, may be: df -h18:52
robputtsure I see the docker vol mounted and the root vol18:53
ykarelERROR: There is not enough free space in volume group atomicos to create data volume of size MIN_DATA_SIZE=2G this is definely the issue18:53
robputtbut the docker vol is very small...18:53
robputti think i will increase disk size18:53
robputterrr18:53
robputtis there anyway todo boot from volume for these ?18:53
robputtin the template or something like give it 200GB cinder vol?18:53
ykarelyou have cinder installed,18:54
ykarelthen you can use docker_volume_size parameter18:54
*** armaan has quit IRC18:54
oikikiykarel: if you'd like any changes to that docfix, just let me know :)18:54
*** armaan has joined #openstack-containers18:55
ykareloikiki, ok18:55
robputtin the template? ykarel ?18:55
ykarelrobputt, yes18:55
robputtok18:55
robputtlet me update and try again18:55
ykarelrobputt, share output for df -h18:55
ykareli want to look at it18:55
robputtyes18:55
robputtthe docker isnt mounted18:55
robputt[fedora@my-cluster-mlkjhr5uxv4b-master-0 ~]$ df -h18:55
robputtFilesystem                 Size  Used Avail Use% Mounted on18:55
robputtdevtmpfs                   466M     0  466M   0% /dev18:55
robputttmpfs                      493M     0  493M   0% /dev/shm18:55
robputttmpfs                      493M  812K  492M   1% /run18:55
robputttmpfs                      493M     0  493M   0% /sys/fs/cgroup18:55
robputt/dev/mapper/atomicos-root   19G  2.6G   17G  14% /sysroot18:55
robputt/dev/vda1                  976M   77M  833M   9% /boot18:56
robputttmpfs                       99M     0   99M   0% /run/user/100018:56
robputtonly the root18:56
robputt:-(18:56
ykareli think this is ok, but not sure18:56
*** mgoddard_ has joined #openstack-containers18:56
ykarelno env currently to confirm18:56
robputti will delete this cluster, and then try the docker vol size option in the template and see if we can use cinder18:57
robputt;-)18:57
*** gsimondon has quit IRC18:57
ykarelyou deleted?18:58
ykarelrobputt, ^^18:58
robputtyeah so,18:58
robputtwith docker_volume_size how do we specify this18:58
robputtand do we need to specify a volume driver?18:58
ykarelok18:58
ykarelno volume driver not required18:59
ykareljust docker-volume-size=<size in GB>18:59
robputtok18:59
robputtand this will create a cinder vol of that size at build?18:59
ykarel--docker-volume-size18:59
ykarelrobputt, yes19:00
robputtok will try now and report back19:00
ykarelack19:00
robputtok i see the VM get created, but no cinder volume yet :-\19:04
*** flwang has quit IRC19:06
ykarelrobputt, wait for some time19:07
ykarelalso --docker-volume-size can be specified during cluster create as well19:08
robputtok i will wait :D19:09
robputtbut if i get bored ill ssh in and tail the log19:09
robputt:D19:09
ykarelsure19:14
ykarel:)19:15
*** robputt has quit IRC19:16
*** harlowja has joined #openstack-containers19:22
*** dsariel has joined #openstack-containers19:23
*** robputt has joined #openstack-containers19:31
robputtok19:34
*** robputt has left #openstack-containers19:34
*** robputt has joined #openstack-containers19:34
robputtit failed again, with ERROR: There is not enough free space in volume group atomicos to create data volume of size MIN_DATA_SIZE=2G.19:34
robputtbut... no cinder was created despite adding that thing to the template19:34
robputt:-(19:34
*** itlinux has quit IRC19:47
*** mgoddard_ has quit IRC19:52
robputteurghhh even with 40GB disk - ERROR: There is not enough free space in volume group atomicos to create data volume of size MIN_DATA_SIZE=2G.19:58
robputtcloud init seems to be expanding the root to the full size19:58
robputt/dev/mapper/atomicos-root   39G  2.9G   37G   8% /sysroot19:58
robputt:-(19:58
*** AlexeyAbashkin has joined #openstack-containers20:03
*** AlexeyAbashkin has quit IRC20:07
oikikirobputt - i thought the minimum for devstack alone was 60gb?20:08
robputtthis isnt really devstack deploy20:08
robputtso what it is is a 8GB RAM VM on my NAS with the control plane :D20:09
robputtand then a 16GB compute node20:10
robputtthat is all20:10
robputt:P20:10
oikikiahhh20:10
robputtanyway, the point is, the kubernetes master VM is a flavour with 40GB disk - but... "ERROR: There is not enough free space in volume group atomicos to create data volume of size MIN_DATA_SIZE=2G." so there is space but it seems the root gets expanded to the full size before this cloud init script configures the lvm20:10
*** ricolin has joined #openstack-containers20:11
robputtbut yes if anyone wants to donate better hardware for my playground cloud :-)20:13
robputtbecause 60GB RAM devstack node sounds wonderful20:14
robputt:D20:14
ykarelrobputt, cat /etc/sysconfig/docker-storage-setup20:14
*** oikiki has quit IRC20:16
*** oikiki has joined #openstack-containers20:17
robputthey ykarel https://gist.github.com/robputt796/c4aaedc26eb072fb20abb37abc2fd53c20:17
ykarelrobputt, while creating cluster-template what storage driver you specified20:20
robputti didn't specify any20:20
robputti literally used the same shizzles as on the Pike install guide20:20
robputt:-)20:20
ykarelokk20:21
robputtlet me show ya my template20:22
robputthttps://gist.github.com/robputt796/f650b8b8562e6382481c47a2d0af5b7820:22
robputtit seems to default to devicemapper20:22
robputtshould I specify overlay or auks?20:22
robputt*aufs?20:22
ykarelbut the file you shared has overlay220:23
robputtyes20:23
robputtbut it was built from this template...20:23
ykareland as default is devicemapper it should contain devicemapper20:23
robputtok so20:24
robputtthis is interesting20:24
robputtwith Fedora 26 Atomic image20:24
robputtit makes the device mapper work20:24
robputton Fedora 27 it doesnt20:24
robputthow odd20:25
robputt:-\20:25
ykarelhmm, strange20:25
robputtlike if i swap out the image on the template for fedora26 the device mapper gives me this in df -h20:26
ykareljournalctl --no-pager -u docker-storage-setup20:26
robputt/dev/mapper/atomicos-root              5.0G  2.0G  3.1G  39% /sysroot20:26
robputt/dev/mapper/atomicos-docker--root--lv   15G   48M   15G   1% /var/lib/docker20:26
ykarel^^ looks Ok20:26
robputtyou wanna see this on the fedora 26 box or the fedora 27 box?20:26
ykarelboth20:26
robputtykarel - https://gist.github.com/robputt796/3d91490e6905b5933a5f9ee8c180180f20:28
*** ykarel_ has joined #openstack-containers20:29
robputtdamn you must hate users like me :joy:20:29
robputtykarel - https://gist.github.com/robputt796/3d91490e6905b5933a5f9ee8c180180f20:30
*** ykarel has quit IRC20:31
ykarel_robputt, and what is there in /etc/sysconfig/docker-storage-setup in fedora 26 one20:32
robputt[fedora@my-cluster26-eqr3h7ahfnvl-master-0 ~]$ cat /etc/sysconfig/docker-storage-setup20:34
robputtGROWROOT=True20:34
robputtSTORAGE_DRIVER=devicemapper20:34
robputtROOT_SIZE=5GB20:34
robputtDATA_SIZE=95%FREE20:34
robputtits mega weird why the file is not getting injected as such on the f27 box20:34
ykarel_This is correct and should be same in f2720:35
*** fragatina has quit IRC20:35
*** fragatina has joined #openstack-containers20:36
*** fragatina has quit IRC20:36
*** fragatina has joined #openstack-containers20:36
robputtyeah is the same20:36
robputtbut then the df -h  output doesnt have the second block device20:37
robputtonly the root one20:37
robputtand its expanded to the full size of the disk20:37
*** ykarel_ is now known as ykarel20:39
*** flwang has joined #openstack-containers20:43
*** fragatina has quit IRC20:47
*** gsimondon has joined #openstack-containers20:54
ykarelrobputt, i will try a setup locally to check tomorrow20:59
robputtit is ok, i will try and investigate more thanks for pointing out these key files to check21:02
robputtit has been useful21:02
robputti will let you know how I get on21:02
robputt:D21:02
ykarelkk21:02
*** gsimondo1 has joined #openstack-containers21:11
*** gsimondon has quit IRC21:13
*** dsariel has quit IRC21:16
*** armaan has quit IRC21:24
*** armaan has joined #openstack-containers21:24
*** ykarel is now known as ykarel|away21:28
*** dsariel has joined #openstack-containers21:30
*** mgoddard_ has joined #openstack-containers21:37
*** gsimondo1 has quit IRC21:40
*** armaan has quit IRC21:41
*** armaan has joined #openstack-containers21:41
oikikiflwang / flwang1: Can you have another look/review please? https://review.openstack.org/#/c/534364/21:42
*** ktibi has quit IRC21:44
flwang1oikiki: sure, i will take a look, thanks for reminder21:45
oikikiflwang1: thank you!21:45
flwang1np21:45
*** fragatina has joined #openstack-containers21:47
*** salmankhan has joined #openstack-containers21:51
*** fragatina has quit IRC21:52
*** fragatina has joined #openstack-containers21:53
*** salmankhan has quit IRC21:56
*** yolanda has joined #openstack-containers22:02
*** ykarel|away has quit IRC22:09
*** mgoddard_ has quit IRC22:15
*** rcernin has joined #openstack-containers22:25
*** fragatina has quit IRC22:52
*** fragatina has joined #openstack-containers22:52
*** gsimondon has joined #openstack-containers23:09
*** gsimondon has quit IRC23:13
*** oikiki has quit IRC23:13
*** oikiki has joined #openstack-containers23:14
*** mnaser has quit IRC23:19
*** mnaser has joined #openstack-containers23:23
*** serverascode has quit IRC23:23
*** mnaser has quit IRC23:24
*** mnaser has joined #openstack-containers23:24
*** kong has quit IRC23:25
*** mnaser has quit IRC23:25
*** mnaser has joined #openstack-containers23:26
*** serverascode has joined #openstack-containers23:26
*** kong has joined #openstack-containers23:26
*** kbyrne has quit IRC23:28
*** kbyrne has joined #openstack-containers23:33
*** robputt has quit IRC23:39
*** ejat has quit IRC23:39
*** fragatina has quit IRC23:39
*** fragatina has joined #openstack-containers23:39
*** fragatina has quit IRC23:41
*** fragatina has joined #openstack-containers23:41
*** ejat has joined #openstack-containers23:41
*** ejat has quit IRC23:41
*** ejat has joined #openstack-containers23:41
*** fragatina has quit IRC23:50
*** fragatina has joined #openstack-containers23:50
*** fragatina has quit IRC23:55
*** fragatina has joined #openstack-containers23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!