Friday, 2015-04-24

apmelton_awaycool, I'll give it a look00:00
sdake_i already had a copr repo ready to go00:00
sdake_for kube 0.1500:00
sdake_the docs are woefully out of date00:00
sdake_but if you understand docker well and docker build systems it shoudl make sense00:00
apmelton_awayalrighty00:01
*** sdake has quit IRC00:03
apmelton_awaysdake_: do you have your new image uncompressed anywhere?00:04
apmelton_awayjust wondering if it's only 700M00:04
sdake_why not decopressing?00:04
apmelton_awaysdake_: I did00:04
sdake_ya thats about right00:04
apmelton_awayaight00:04
*** apmelton_away is now known as apmelton00:05
sdake_did it take awhile to decompress?00:05
apmeltona bit, but no where near the 20 minutes it usually took to download that image before ;P00:06
madhurisdake_, Hi00:06
madhurigood morning folks00:06
sdake_hey madhuri00:06
apmelton'morning!00:06
sdake_got a new image for you to test if you like00:06
madhuriYes00:06
madhuriI saw your mail00:06
madhuriI will test it00:06
*** achanda has quit IRC00:07
*** yuanying-alt has joined #openstack-containers00:12
apmeltonsdake_: flannel and cloud-final issues: http://paste.openstack.org/show/205449/00:13
sdake_journalctl on flannel service plz00:15
*** vilobhmm11 has quit IRC00:15
*** vilobhmm1 has joined #openstack-containers00:15
apmeltonsdake_: nadda00:15
sdake_rpm -qa | grep flannel00:15
apmeltonnadda00:16
*** yuanying-alt has quit IRC00:16
*** Marga_ has quit IRC00:23
*** fangfeng_ has joined #openstack-containers00:26
*** fangfeng_ has left #openstack-containers00:27
*** fangfenghua_ has joined #openstack-containers00:29
*** jay-lau-513 has quit IRC00:30
sdake_apmelton found the problem00:36
apmeltonyea?00:36
sdake_give my computer 20 minutes to think00:36
apmeltonalright00:36
apmeltonmay or may not having a working devstack :/00:36
*** Marga_ has joined #openstack-containers00:45
*** suro-patz1 has quit IRC00:48
sdake_apmelton build failed will be anothe 20 minutes00:52
apmelton:(00:52
*** Marga_ has joined #openstack-containers00:53
*** Marga_ has quit IRC00:54
sdake_apmelton compressing now01:10
apmeltonk01:10
*** Tango has quit IRC01:11
sdake_the updates repo was issing from teh build01:11
sdake_as a result nothing in updates was being pulled in01:11
sdake_including flannel01:11
apmeltongotcha01:11
sdake_i hav espent literally 5+ years of my life building image of one type or another01:14
sdake_kernel vm container etc01:14
*** vahidh has quit IRC01:15
sdake_still compressing01:15
*** suro-patz has joined #openstack-containers01:17
apmeltonwaiting on devstack to build, waiting on the gate, waiting on nova unit tests: http://i.imgur.com/HVJRZwQ.jpg01:18
apmeltonthough I do have to say, last time I ran nova unittests they'd got pretty quick01:18
sdake_nothing is as bad as kernel dev was01:19
sdake_wait 30 minutes for one printf to make it into the run01:19
sdake_you reboot01:19
apmeltonjeez01:19
sdake_"fuck I forgot what I added the print to."01:20
sdake_groan01:20
apmeltonhaha01:20
sdake_ok dl away01:20
sdake_same filename01:20
apmeltonalrighty01:21
sdake_madhuri ^^01:21
madhuriyup01:21
apmeltonman downloading from fedorapeople at night isn't so bad01:21
apmelton20.8MB/s01:21
*** sdake has joined #openstack-containers01:22
*** suro-patz1 has joined #openstack-containers01:22
madhurisdake_, Are you uploading a new image?01:22
apmeltonsdake_: 15 seconds to unzip01:22
*** suro-patz has quit IRC01:23
sdakeits been uploaded just now01:23
sdake15 minutes to copress yay ;)01:23
madhuriOk I downloaded the fedora-atomic-3 image01:24
*** sdake_ has quit IRC01:26
sdakethe new one i hope ;)01:26
madhuriLet us know once done01:27
sdakei'm done uploading01:27
*** liusheng has joined #openstack-containers01:29
apmeltonsdake: odd: var-lib-nfs-rpc_pipefs.mount                                                              loaded failed failed01:29
apmeltonit does look like kubernetes started01:30
apmeltonlet me try a pod create01:30
sdakethats the 420 mount module01:30
sdakeapmelton any success?01:33
apmeltonsdake: nope01:34
apmeltonstill flannel issues on the node01:34
sdakerpm -qa | grep flannel01:34
apmeltonit's there01:34
apmeltonlet me gist ya the journal01:34
sdakeatlesst you have the new image ;)01:34
apmeltonhttp://paste.openstack.org/show/205465/01:35
apmeltonit does have the proper ip in the /etc/sysconfig/flannel as well01:35
sdakelog into that node01:35
sdakeand restart flannel01:35
sdakeand see what happens01:35
apmeltonsudo01:36
apmeltonoops01:36
*** erkules_ has joined #openstack-containers01:36
apmeltonnow this is odd: unable to execute /bin/ping: Invalid argument01:36
sdakedo other binaries work?01:36
apmeltoncurl www.google.com worked01:37
sdakemy mortal form has grown weak - I need sustinence01:37
apmeltonheh01:37
apmeltonyea... I need to start turning in for the night01:37
sdakebnjil[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[01:38
sdake[[oo;[01:38
apmeltonwat01:38
*** erkules has quit IRC01:38
sdakefood jumped on keyboard apologie01:38
sdakes01:38
apmeltonlol01:39
apmeltonsdake: anything quick you want me to try before hopping off?01:39
sdakerestart flanneld01:39
*** vilobhmm1 has quit IRC01:40
*** vilobhmm1 has joined #openstack-containers01:40
*** vilobhmm1 has quit IRC01:40
apmeltonsdake: same issue01:42
apmeltonseems it's having issues communicating01:42
sdaketelnet 400101:42
apmeltonsdake: no telnet01:44
sdakecurl01:44
apmeltonyea... that works on the master, but it's blocked elsewhere01:44
sdakewhat do you mean blocked elsewhere01:45
sdakeyou mean from the master you can connect to the master port 4001?01:45
sdakebut from a differnet hosty ouc annot connect to port 4001 of the master?01:45
apmeltonyes, but it seems node to master is timing out01:45
sdakerun iptables --flush01:45
sdaketry again01:45
apmeltonsdake: on the host or the master?01:46
sdakeon the master and nodes01:46
sdakeboth01:46
sdakeminion and master01:46
apmeltonsdake: nope01:46
sdakeis firewall-cmd installed?01:47
apmeltonsdake: no01:48
sdakemadhuri i hope yu have better news :)01:48
sdakenight apmelton01:48
sdakei think your system isbusted ;)01:48
apmeltonone thing I did notice was ETCD_LISTEN_CLIENT_URLS is set to http://localhost:400101:49
madhurisdake vm is not yet up01:49
apmeltondoes that mean it's only listening on 127.0.0.101:49
apmelton?01:49
*** sdake_ has joined #openstack-containers01:50
apmeltonI think that's what it is01:50
apmeltonif I change it to http://0.0.0.0:4001, it works01:50
sdake_you think what is what it is01:51
sdake_change what to 0.0.0.0:4000101:51
apmeltonon the master /etc/etcd/etcd.conf, ETCD_LISTEN_CLIENT_URLS is set to http://localhost:4001, if I set ETCD_LISTEN_CLIENT_URLS to http://0.0.0.0:400101:51
sdake_ya that would make sene01:52
apmeltonI can curl the master's etcd addr/port and it'll respond01:52
sdake_try starting foalnnel now01:52
sdake_we need to change that in the masater setup shell01:52
apmeltonsdake: now it starts01:52
sdake_nice01:53
sdake_should be an easy patch then01:53
apmelton\o/01:53
*** sdake has quit IRC01:53
apmeltonalright, I'm out01:54
apmeltonhave a good day and night everyone01:54
*** jjlehr has joined #openstack-containers01:54
*** Kennan2 is now known as Kennan01:55
*** apmelton is now known as apmelton_away01:57
*** yuanying-alt has joined #openstack-containers02:00
*** harlowja is now known as harlowja_away02:02
*** yuanying-alt has quit IRC02:05
*** kebray has joined #openstack-containers02:06
*** suro-patz1 has quit IRC02:12
*** suro-patz has joined #openstack-containers02:13
*** jjlehr has quit IRC02:14
openstackgerritSteven Dake proposed openstack/magnum: Add etcd 2.0 config file support  https://review.openstack.org/17705602:18
sdake_madhuri can you test that please with the 0.15 image02:18
sdake_and vote on the review02:18
openstackgerritSteven Dake proposed openstack/magnum: Change from kubernetes 0.11 to 0.15  https://review.openstack.org/17705802:22
openstackgerritKai Qiang Wu(Kennan) proposed openstack/magnum: Fix the versionedobject version issue  https://review.openstack.org/17705902:26
*** achanda has joined #openstack-containers02:37
*** unicell has quit IRC02:39
madhurisdake_, sure. Thanks02:47
*** suro-patz has quit IRC02:50
*** EricGonczer_ has joined #openstack-containers02:52
*** EricGonczer_ has quit IRC02:57
KennanHi :sdake_02:57
sdake_sup kennan03:00
Kennanif you check your new reviews patches03:00
*** EricGonczer_ has joined #openstack-containers03:00
Kennanit failed03:00
Kennanit was because of the versionedobject issue03:00
*** sdake has joined #openstack-containers03:00
Kennansince we not branch, and I have one patch for workaround that03:01
Kennanhttps://review.openstack.org/17705903:01
Kennan:sdake_ this is the failed logs03:03
Kennanhttp://logs.openstack.org/58/177058/1/check/check-functional-dsvm-magnum/d94f521/logs/devstacklog.txt.gz03:03
Kennan:sdake_ I will be out now for some issue. Good night :)03:04
*** sdake_ has quit IRC03:04
*** EricGonczer_ has quit IRC03:04
*** sdake_ has joined #openstack-containers03:04
*** sdake has quit IRC03:08
sdake_yuanying i put a decompressed copy on the site03:11
sdake_night all03:11
yuanyingsdake_ good night!03:11
*** rongze has joined #openstack-containers03:12
*** liusheng has left #openstack-containers03:24
*** suro-patz has joined #openstack-containers03:35
*** jjlehr has joined #openstack-containers03:36
*** achanda has quit IRC03:44
openstackgerritJanek proposed openstack/magnum: Add coe attribute to BayModel  https://review.openstack.org/17604003:45
sdake_madhuri did the patch work03:48
*** yuanying-alt has joined #openstack-containers03:49
sdake_yuanying-alt did that image work?03:49
*** yuanying-alt has quit IRC03:54
*** vilobhmm1 has joined #openstack-containers03:57
sdake_maduri any luck with that patch + iamge?04:05
madhurisdake_, I was out for lunch04:10
madhuriBut it seems I can' ping my vm04:10
madhuriI am checking04:10
*** fangfenghua_ has quit IRC04:27
*** fangfenghua_ has joined #openstack-containers04:28
*** subscope_ has joined #openstack-containers04:30
*** achanda has joined #openstack-containers04:35
*** suro-patz has quit IRC04:36
*** fangfenghua_ has quit IRC04:36
*** Marga_ has joined #openstack-containers04:41
*** unicell has joined #openstack-containers04:42
*** chandankumar has joined #openstack-containers04:45
*** gose has joined #openstack-containers04:49
*** Marga_ has quit IRC05:10
*** Marga_ has joined #openstack-containers05:11
openstackgerritMerged openstack/magnum: Use container's bay for docker endpoint  https://review.openstack.org/17647005:21
*** jjlehr has quit IRC05:26
*** Tango has joined #openstack-containers05:31
*** oro has joined #openstack-containers05:33
openstackgerritMerged openstack/magnum: Correct spelling mistake in dev-quickstart.  https://review.openstack.org/17673005:34
*** fangfenghua_ has joined #openstack-containers05:35
*** Marga_ has quit IRC05:36
*** Marga_ has joined #openstack-containers05:36
*** Marga_ has joined #openstack-containers05:37
*** yuanying-alt has joined #openstack-containers05:38
*** fangfenghua_ has quit IRC05:41
*** Marga_ has quit IRC05:42
*** yuanying-alt has quit IRC05:43
*** vilobhmm1 has quit IRC05:44
*** kebray has quit IRC05:51
*** david-ly_ has joined #openstack-containers06:01
*** david-lyle has quit IRC06:03
*** jay-lau-513 has joined #openstack-containers06:03
*** oro has quit IRC06:05
*** suro-patz has joined #openstack-containers06:05
*** subscope_ has quit IRC06:11
*** suro-patz has quit IRC06:12
*** unicell has quit IRC06:21
*** unicell has joined #openstack-containers06:23
madhuriHi, have anyone tested the new fedora-21-atomic-3 image?06:30
*** suro-patz has joined #openstack-containers06:32
*** gose has quit IRC06:33
*** ashishb has joined #openstack-containers06:34
madhurisdake_, ping?06:35
*** fangfenghua_ has joined #openstack-containers06:40
*** fangfenghua_ has quit IRC06:46
*** suro-patz has quit IRC06:51
*** fangfenghua_ has joined #openstack-containers07:00
*** coolsvap|afk is now known as coolsvap07:08
*** coolsvap is now known as coolsvap|afk07:09
*** nshaikh has joined #openstack-containers07:11
*** sdake_ has quit IRC07:20
*** Tango has quit IRC07:23
*** achanda has quit IRC07:24
*** oro has joined #openstack-containers07:27
*** achanda has joined #openstack-containers07:36
*** rongze has quit IRC07:39
*** oro_ has joined #openstack-containers07:42
*** achanda has quit IRC07:57
madhurisdake: ping me whenever you are free08:04
digamadhuri: I am creating a new setup08:05
digaplease ping the new fedora image link08:05
madhuriSure08:06
madhurihttps://fedorapeople.org/groups/magnum/08:06
madhurifedora-21-atomic-3.qcow208:06
madhuridiga, I will be away for few hours08:07
madhuriSo please leave comments for me here about the image status08:07
madhuridiga, Thanks08:07
digamadhuri: sure08:08
openstackgerritLan Qi Song proposed openstack/magnum: Bay show return api address and node addresses  https://review.openstack.org/17711808:30
openstackgerritLan Qi Song proposed openstack/magnum: Remove duplicated test case  https://review.openstack.org/17712508:52
openstackgerritMerged openstack/magnum: Add timeout parameter to bay create  https://review.openstack.org/17406108:58
*** jay-lau-513 has quit IRC08:59
*** yuanying-alt has joined #openstack-containers09:15
*** yuanying-alt has quit IRC09:20
*** sdake has joined #openstack-containers09:37
*** sdake_ has joined #openstack-containers09:39
*** sdake has quit IRC09:43
*** Marga_ has joined #openstack-containers09:48
*** Marga_ has quit IRC09:50
*** Marga_ has joined #openstack-containers09:50
sdake_madhuri around?09:53
*** fangfenghua_ has quit IRC09:54
*** Marga_ has quit IRC10:06
yuanyingshe is away10:07
*** yuanying has quit IRC10:39
*** fangfeng_ has joined #openstack-containers10:50
*** sdake_ has quit IRC10:53
*** fangfeng_ has quit IRC10:56
*** fangfeng_ has joined #openstack-containers10:57
*** fangfeng_ is now known as fangfenghua_10:58
*** yuanying-alt has joined #openstack-containers11:04
*** yuanying-alt has quit IRC11:09
*** fangfenghua_ has quit IRC11:14
*** fangfenghua_ has joined #openstack-containers11:16
*** fangfenghua_ has quit IRC11:28
*** fangfenghua_ has joined #openstack-containers11:34
*** fangfenghua_ has quit IRC11:35
*** rongze has joined #openstack-containers11:38
*** Marga_ has joined #openstack-containers11:41
*** rongze has quit IRC11:42
*** funzo has joined #openstack-containers11:51
*** Marga_ has quit IRC11:57
*** fangfenghua_ has joined #openstack-containers12:10
*** dboik_ has quit IRC12:10
*** erkules_ is now known as erkules12:11
*** erkules has joined #openstack-containers12:11
*** oro has quit IRC12:22
*** oro_ has quit IRC12:23
*** dboik has joined #openstack-containers12:27
*** dboik has quit IRC12:27
*** dboik has joined #openstack-containers12:27
*** rpothier has joined #openstack-containers12:33
*** dboik_ has joined #openstack-containers12:35
*** yuanying-alt has joined #openstack-containers12:35
*** dboik_ has quit IRC12:35
*** dboik_ has joined #openstack-containers12:36
*** dboik has quit IRC12:38
*** yuanying-alt has quit IRC12:39
*** dboik_ has quit IRC12:55
*** dboik has joined #openstack-containers12:56
*** dboik has quit IRC12:56
*** dboik has joined #openstack-containers12:57
*** funzo has quit IRC13:01
*** oro has joined #openstack-containers13:07
*** rlrevell has joined #openstack-containers13:13
*** oro has quit IRC13:14
*** yuanying-alt has joined #openstack-containers13:21
*** yuanying-alt has quit IRC13:22
*** rlrevell has quit IRC13:31
*** rlrevell has joined #openstack-containers13:32
*** chandankumar has quit IRC13:34
*** jjlehr has joined #openstack-containers13:35
*** rongze has joined #openstack-containers13:46
*** apmelton_away is now known as apmelton13:50
openstackgerritJanek proposed openstack/magnum: Add coe attribute to BayModel  https://review.openstack.org/17604013:54
*** jay-lau-513 has joined #openstack-containers14:12
*** EricGonczer_ has joined #openstack-containers14:17
*** oro has joined #openstack-containers14:29
*** rongze has quit IRC14:41
*** Tango has joined #openstack-containers14:48
*** Marga_ has joined #openstack-containers14:50
*** kebray has joined #openstack-containers14:53
*** julim has joined #openstack-containers14:54
*** ashishb has quit IRC14:57
*** achanda has joined #openstack-containers15:03
*** achanda has quit IRC15:08
*** rcleere is now known as rcleere_away15:10
*** rcleere_away is now known as rcleere15:11
*** david-ly_ is now known as david-lyle15:22
openstackgerritJanek proposed openstack/python-magnumclient: Add coe attribute to BayModel creation  https://review.openstack.org/17609615:25
*** nshaikh has quit IRC15:26
*** jjlehr has quit IRC15:26
fangfenghua_https://blueprints.launchpad.net/kolla/+spec/enable-docker-compose-container15:31
fangfenghua_sorry15:31
fangfenghua_wrong place15:31
*** chandankumar has joined #openstack-containers15:34
*** fangfenghua_ has quit IRC15:35
*** EricGonczer_ has quit IRC15:42
*** Marga_ has quit IRC15:45
*** adrian_otto has joined #openstack-containers15:46
*** sdake has joined #openstack-containers15:51
openstackgerritJennifer Carlucci proposed openstack/python-magnumclient: Add heat timeout to bay-create for magnum client  https://review.openstack.org/17647615:51
*** Marga_ has joined #openstack-containers15:55
*** vilobhmm1 has joined #openstack-containers15:59
*** vilobhmm11 has joined #openstack-containers16:01
*** vilobhmm1 has quit IRC16:04
*** ashishb has joined #openstack-containers16:11
*** vilobhmm11 has quit IRC16:12
*** unicell has quit IRC16:20
*** chandankumar has quit IRC16:20
*** daneyon_ has joined #openstack-containers16:22
*** daneyon has quit IRC16:25
*** oro has quit IRC16:26
*** chandankumar has joined #openstack-containers16:33
*** adrian_otto has quit IRC16:36
*** unicell has joined #openstack-containers16:42
*** vilobhmm1 has joined #openstack-containers16:47
*** vilobhmm1 has quit IRC16:47
*** vilobhmm1 has joined #openstack-containers16:47
*** vilobhmm1 has quit IRC16:47
*** vilobhmm1 has joined #openstack-containers16:48
*** achanda has joined #openstack-containers16:52
*** vilobhmm1 has quit IRC16:53
*** sdake has quit IRC16:53
*** vilobhmm1 has joined #openstack-containers16:53
*** sdake has joined #openstack-containers17:12
*** suro-patz has joined #openstack-containers17:13
*** EricGonczer_ has joined #openstack-containers17:19
*** EricGonc_ has joined #openstack-containers17:21
*** oro has joined #openstack-containers17:21
*** EricGonczer_ has quit IRC17:24
*** ashishb has quit IRC17:25
*** prad_ has joined #openstack-containers17:44
*** ashishb has joined #openstack-containers17:44
*** prad_ is now known as prad17:44
*** harlowja_away is now known as harlowja17:51
*** Marga_ has quit IRC17:56
*** diga_ has joined #openstack-containers17:58
*** ashishb has quit IRC17:59
*** adrian_otto has joined #openstack-containers18:03
*** gose has joined #openstack-containers18:15
*** ashishb has joined #openstack-containers18:23
*** Marga_ has joined #openstack-containers18:26
*** kebray has quit IRC18:35
*** diga_ has quit IRC18:43
*** ashishb has quit IRC18:55
*** daneyon has joined #openstack-containers18:58
*** daneyon_ has quit IRC18:59
*** chandankumar has quit IRC19:02
*** diga_ has joined #openstack-containers19:03
sdakeanyone tried out the new image yet?19:06
apmeltonsdake: I'm in the process of rebuilding my magnum env19:07
apmeltonmy old environment finally got too trashed19:07
sdakeapmelton there are a few outstanding git reviews - pelase run with those19:07
sdakethe ones that fix the client cnfig for etcd19:07
apmeltonsdake: I saw that19:08
sdakei need to get my environment runningagain19:08
sdakeswit hing between kolla nad magnum dev on same box is pita19:08
apmeltonI'm hitting an issue where devstack tries to run our migrations19:08
apmeltonI think fangfenghua has a fix19:08
apmeltonoh no I think it's Kennan19:09
*** nshaikh has joined #openstack-containers19:18
*** kebray has joined #openstack-containers19:31
*** petertr7 has joined #openstack-containers19:34
*** prad has quit IRC19:39
apmeltonsdake: :( etcd.service                                                                         loaded failed     failed19:53
sdakeapmelton can you look at the config file pls19:53
sdakepaste it19:53
sdakemy sed fu is not strong :)19:53
apmeltonsdake: one sec, I see the problem19:54
apmeltonsdake: see comment: https://review.openstack.org/#/c/177056/1/magnum/templates/heat-kubernetes/fragments/configure-kubernetes-master.sh19:55
apmeltonhurray: | k8s3 | 2          | CREATE_COMPLETE |19:59
openstackgerritSteven Dake proposed openstack/magnum: Add etcd 2.0 config file support  https://review.openstack.org/17705620:01
openstackgerritSteven Dake proposed openstack/magnum: Change from kubernetes 0.11 to 0.15  https://review.openstack.org/17705820:01
sdakeapmelton can you verify the updated patch pls20:01
apmeltonsdake: the create worked20:06
apmeltonlet me try a pod create20:06
apmeltoncloud-final.service is still failing20:08
apmeltonand the pod never leaves pending20:08
sdakeflannel setting up tho?20:09
sdakehave time to debug cloud-final.service?20:09
apmeltoncc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)20:10
apmeltonflannel does seem to be running20:10
apmeltonfactory.go:244] Error scheduling default redis-master: no minions available to schedu20:12
apmeltonthis is odd: http://paste.openstack.org/show/205794/20:19
apmeltonthat's from 10.0.0.4's kubelet service20:19
apmeltonsdake: on the minions, in /etc/kubernetes/apiserver, KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:4001"20:22
apmeltonshould that also be pointing to the master's address?20:23
sdakeapmelton should be yes20:35
sdakeapmelton sorry got distracted, i'm back :)20:35
sdakeoh the minions don't need apiserver modified20:35
sdakeapiserver is only run on the master20:35
apmeltonk20:35
apmeltonso yea... I'm not sure what's going on here20:36
sdakewhere was that paste from?20:36
sdakethe master?20:36
apmeltonnope20:36
apmeltonthe minion20:36
sdakewhat is 10.0.0.4?20:36
apmeltonspecifically the one at 10.0.0.420:36
sdakefrom which service, kubelet?20:37
apmeltonyes20:37
sdakethere is  kubelet.conf20:37
sdakeor some file like that20:37
sdakecan yu apste it20:37
sdake in /etc/kubernetes20:37
apmeltonI think I already changed it20:37
sdakewas it busted?20:37
apmeltonsdake: http://paste.openstack.org/show/205806/20:37
apmeltonI think when it booted KUBELET_API_SERVER="--api_servers=10.0.0.3:8080" as KUBELET_API_SERVER="--api_servers=localhost:8080"20:38
apmeltoneither localhost or 127.0.0.1, can't remember which20:38
sdakewell tht will caus ebad thigns to happen i am sure20:38
sdakecan you go fresh and show me the file20:39
sdakeso I can modify the template20:39
apmeltonalright20:39
apmeltonI don't think that change fixed anything though20:39
sdakeif conf files are not correct during startup the entire state gets wedged20:39
sdakeso they need to be right the first time through20:39
sdakeespecially once you start a pod20:39
apmeltongotcha20:39
apmeltonsdake: looks like fresh minion is set up properly already: http://paste.openstack.org/show/205807/20:43
*** oro has quit IRC20:45
sdakeshould have http:// in front of it20:45
apmeltonsdake: I'm actually wondering if this some how failed: https://github.com/openstack/magnum/blob/master/magnum/templates/heat-kubernetes/fragments/configure-kubernetes-minion.sh#L37-L4020:48
apmeltonand the minion actually can't find itself20:48
sdakev1beta120:49
sdakebad sign20:49
apmeltonsdake: yea, I can see that file failed in the cloud init logs20:51
sdaketry running that curl command manually20:51
openstackgerritSteven Dake proposed openstack/magnum: Change from kubernetes 0.11 to 0.15  https://review.openstack.org/17705820:53
openstackgerritSteven Dake proposed openstack/magnum: Add http:// prefix to kubelet api server  https://review.openstack.org/17744920:53
sdakewe should be using v1beta3 i think20:53
sdakei think beta1 has ben deprecated20:53
apmeltonsdake: I don't get any response20:55
apmeltonrather, it's an empty response20:55
sdakeecho $?20:55
sdakeor maybe its echo $020:55
sdaketry the echo $?20:55
apmelton[minion@k8-xelgh6i2siru-1-cnchbd4rxgtk-kube-node-kqdqkderb27b ~]$ echo $?20:55
apmelton720:56
sdakeitshould be the return code20:56
sdakehttp://curl.haxx.se/mail/archive-2003-10/0083.html20:56
*** rlrevell has quit IRC20:56
apmeltonsorry, had the wrong ip20:56
sdaketry v1beta320:56
apmeltonreturn code is 22 with the proper ip20:56
sdakedon't try v1beta3 yet20:56
sdakeCURLE_HTTP_RETURNED_ERROR (22)20:57
sdakeThis is returned if CURLOPT_FAILONERROR is set TRUE and the HTTP server returns an error code that is >= 400.20:57
sdakeCURLE_WRITE_ERROR (23)20:57
sdaketry v1beta320:57
apmeltonstill 2220:58
sdaketry v1beta3 now and see if that returns 020:58
sdakelets go to #google-containers20:58
sdakecan you check the logs on the master21:03
sdakein the api server21:03
sdakesee what it doesnt' like21:03
*** sdake_ has joined #openstack-containers21:04
apmeltonsdake: http://paste.openstack.org/show/205808/21:06
*** nshaikh has left #openstack-containers21:07
*** sdake has quit IRC21:08
apmeltonsdake_: "message": "the object provided is unrecognized (must be of type Node): couldn't get version/kind: invalid character '\\'' looking for beginning of object key string",21:09
sdake_there ya go21:10
sdake_type is wrong :)21:10
sdake_change kind to "Node"21:10
sdake_and v1betav321:11
sdake_or whatever it is21:11
apmeltonwait21:11
apmelton' isn't valid json21:11
apmeltonright?21:11
apmeltonit's saying:  invalid character '\\''21:12
sdake_the problem is "Minion"should be changed to "node"21:12
sdake_it may also be invalid json - not sure21:12
apmeltonsdake_: tried that21:12
apmelton╯°□°)╯︵ ┻━┻21:13
apmeltonstack@magnum-metal:~/kubernetes/examples/redis$ kubectl get minions -s http://172.24.4.26:808021:13
apmeltonNAME                LABELS              STATUS21:13
apmelton10.0.0.4            <none>              Ready21:13
apmeltonnot valid json21:13
apmeltonhow did this work before?21:13
sdake_who knows :)21:13
sdake_do you have a change to fix it then?21:14
apmeltonyea, I can do something real quick21:14
*** dboik_ has joined #openstack-containers21:15
sdake_paste a dif and i'll integrate it in my change set21:16
apmeltonsdake_: here's a bug: https://bugs.launchpad.net/magnum/+bug/144831321:16
openstackLaunchpad bug 1448313 in Magnum "Kubernetes minion discovery call fails due to invalid json" [Undecided,New]21:16
sdake_cool can yu paste a diff :)21:17
apmeltonsdake_: http://paste.openstack.org/show/205815/21:18
*** dboik has quit IRC21:18
*** sdake has joined #openstack-containers21:19
*** dboik_ has quit IRC21:19
apmeltonsdake: dunno if you saw:  http://paste.openstack.org/show/205815/21:19
apmeltonalso, does cox internet just really suck out there?21:19
apmeltonyou drop and re-join quite a bit21:19
openstackgerritDigambar proposed openstack/magnum: Implementation of Cluster distro for baymodel.  https://review.openstack.org/17234821:21
*** oro has joined #openstack-containers21:21
*** diga_ has quit IRC21:23
*** EricGonc_ has quit IRC21:23
sdakeapmelton try that change set21:23
*** sdake_ has quit IRC21:23
openstackgerritSteven Dake proposed openstack/magnum: Change from kubernetes 0.11 to 0.15  https://review.openstack.org/17705821:23
openstackgerritSteven Dake proposed openstack/magnum: Invalid JSON in dynamic registration of minion  https://review.openstack.org/17746121:23
*** Marga_ has quit IRC21:23
sdaketo connect to cisco's vpn, I have to connect o a different wireless network in my hosue21:23
sdaketo connect to my dev boxes, I have to change my wireless21:23
sdakehence the dropping quite a bit21:24
apmeltonjeez21:24
sdakecisco doesn't support split routing21:24
sdakesecurity thing21:24
* sdake groans21:24
apmeltonyou need yourself a bouncer21:24
* apmelton <3's quassel21:24
sdakecox internet is really solid21:25
*** diga_ has joined #openstack-containers21:25
sdakeat rht I could stay connected to vpn and irc for weeks at a time21:25
sdakebutrht allowed split routing21:25
apmeltongotcha21:25
sdakeso with all those changes, does this damn thing work yet :)21:26
apmeltonsdake: still resetting my environment21:26
sdakeif possible I'd like you to confirm my patch set once we get all theese changes fixed21:26
sdakeI hope madhuri appreciates this :)21:27
* sdake hates changing versions of kubernetes21:28
sdake3 days of jerking around with images and tempalte changes21:28
apmeltonsdake: http://paste.openstack.org/show/205816/21:29
apmeltonit does look like it's running21:29
diga_sdake, apmelton : Just finished the implementation of cluster_distro, please review my patch as you get some time - https://review.openstack.org/#/c/172348/321:30
apmeltonnot sure if it's alright for events to be rejectyed21:30
apmeltonsdake: let me try a pod create21:30
sdakeare you using k8s 15 client?21:30
sdakeapmelton ^21:30
sdakeif not, you should use kubectl version 1521:31
diga_sdake: 1 question21:31
sdakewound me diga_21:31
diga_will 8G RAM machine work for magnum ?21:32
sdakeI think so21:32
diga_to setup all these things21:32
sdakeI have 128gb so ymmv :)21:32
diga_:O21:32
sdakehongbin runs on 8gb i think21:33
diga_okay21:33
diga_thanks21:33
apmeltonsdake: I guess we don't sync pod status up to magnum yet?21:33
apmeltonkubectl shows pod as running, but magnum pod-show has it as pending still21:34
sdakeapmelton probably not21:34
diga_Its late here, need to go to bed21:34
diga_have a nice day!21:34
sdakeapmelton that may be because your version of kubectl is too old21:34
sdakewhen you do a pod show, it hsould access k8s directly for that info21:34
apmeltonsdake: do we even still use kubectl?21:34
sdakeyes it it is still used -that is the whole point of going to kube 0.1521:34
sdakeso we can get rid of it :)21:34
apmeltonhttp://paste.openstack.org/show/205817/21:35
*** diga_ has quit IRC21:35
sdakemadhuri is blocked on this atm21:35
sdakeapmelton so it looks like its working?21:36
apmeltonthink so21:36
sdakejust with a bug in there21:36
apmeltonI think there may also be a bug with bay-create at the moment21:36
sdakemore detail?21:36
apmeltonsdake: still need to investigate21:36
sdakeapmelton in your latest run were you using my change set?21:37
apmeltonsdake: yes21:37
sdakenice +2's then? :)21:37
sdakelet me ptu you as coauthor first21:37
sdakesec21:37
apmeltonsdake: can only +1 ;P21:37
sdakeoh right21:37
sdakewe need to fix that :)21:37
apmeltonit'll certainly be easier to argue I should get more time on magnum if I'm core ;)21:38
sdakestackalytics is down, how are your review stats21:38
apmeltonsdake: low, but I'm working on it21:39
sdakecool - need new cores to be in the middle of the review pack21:39
sdakeor better ;)21:39
apmeltonsdake: stackalytics is up for me21:40
apmeltonsay's I've only got 2021:40
sdakeCo-Authored-By: Andrew Melton <apmelton@rackspace.com>21:40
sdakeis that the correct line?21:40
apmeltonCo-Authored-By: Andrew Melton <andrew.melton@rackspace.com>21:41
openstackgerritSteven Dake proposed openstack/magnum: Add etcd 2.0 config file support  https://review.openstack.org/17705621:42
openstackgerritSteven Dake proposed openstack/magnum: Invalid JSON in dynamic registration of minion  https://review.openstack.org/17746121:42
openstackgerritSteven Dake proposed openstack/magnum: Change from kubernetes 0.11 to 0.15  https://review.openstack.org/17705821:42
openstackgerritSteven Dake proposed openstack/magnum: Add http:// prefix to kubelet api server  https://review.openstack.org/17744921:42
sdakecan you file bug for that sync state issue21:42
sdakeand whichever bay creat eprolem you saw21:42
sdakeplease note in yur +1 that you tested th entire changeset with 0.1521:42
sdakeapmelton thanks a bunch for your debug assistance :)21:43
apmeltonsdake: definitely!21:43
sdakehongbin did that work last time we made the -2 image21:43
*** vilobhmm11 has joined #openstack-containers21:51
apmeltonsdake: https://bugs.launchpad.net/magnum/+bug/144833221:51
openstackLaunchpad bug 1448332 in Magnum "Pod status not synced from kubernetes to magnum" [Undecided,New]21:51
*** vilobhmm12 has joined #openstack-containers21:53
*** vilobhmm1 has quit IRC21:53
sdakeapmelton thanks21:53
*** dboik has joined #openstack-containers21:54
*** vilobhmm11 has quit IRC21:56
*** rpothier has quit IRC22:09
*** sdake_ has joined #openstack-containers22:11
*** Kennan2 has joined #openstack-containers22:12
*** Kennan has quit IRC22:12
*** Marga_ has joined #openstack-containers22:13
*** sdake has quit IRC22:15
*** oro has quit IRC22:18
*** sdake has joined #openstack-containers22:23
*** sdake_ has quit IRC22:27
*** adrian_otto1 has joined #openstack-containers22:33
*** adrian_otto has quit IRC22:36
*** harlowja_ has joined #openstack-containers22:41
*** harlowja has quit IRC22:41
*** sdake_ has joined #openstack-containers22:53
*** achanda has quit IRC22:53
*** sdake__ has joined #openstack-containers22:55
*** sdake has quit IRC22:56
*** apmelton is now known as apmelton_away22:58
*** sdake_ has quit IRC22:59
*** petertr7 has quit IRC23:31
*** kebray has quit IRC23:44
*** sdake__ has quit IRC23:52
*** achanda has joined #openstack-containers23:53
*** achanda has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!