Tuesday, 2016-08-16

*** diogogmt has joined #openstack-kuryr00:11
*** banix has joined #openstack-kuryr00:24
*** vikasc has joined #openstack-kuryr00:33
*** tonanhngo has joined #openstack-kuryr00:34
*** tonanhngo has quit IRC00:39
*** banix has quit IRC00:44
*** limao has joined #openstack-kuryr00:53
*** limao has quit IRC00:54
*** limao_ has joined #openstack-kuryr00:54
*** limao_ has quit IRC01:01
*** limao has joined #openstack-kuryr01:01
*** tonanhngo has joined #openstack-kuryr01:22
*** tonanhngo has quit IRC01:26
openstackgerritLiping Mao proposed openstack/kuryr-libnetwork: Fix hard-code listen ip "0.0.0.0"  https://review.openstack.org/35456301:35
*** banix has joined #openstack-kuryr02:04
*** banix has quit IRC02:26
*** limao has quit IRC02:42
*** hongbin has joined #openstack-kuryr02:44
*** yuanying has quit IRC02:49
*** limao has joined #openstack-kuryr02:51
*** yuanying has joined #openstack-kuryr02:52
*** limao has quit IRC03:07
*** shashank_hegde has joined #openstack-kuryr03:30
*** diogogmt has quit IRC03:34
*** shashank_hegde has quit IRC03:36
*** limao has joined #openstack-kuryr03:36
*** hongbin has quit IRC03:45
*** yuanying has quit IRC03:48
*** yuanying has joined #openstack-kuryr03:51
openstackgerritLiping Mao proposed openstack/kuryr: Set up MTU docker interface  https://review.openstack.org/35571204:10
openstackgerritLiping Mao proposed openstack/kuryr-libnetwork: Set up MTU docker interface  https://review.openstack.org/35571404:16
*** irenab has quit IRC04:42
*** irenab has joined #openstack-kuryr05:11
*** limao has quit IRC05:29
*** limao has joined #openstack-kuryr05:31
*** shashank_hegde has joined #openstack-kuryr05:42
openstackgerritLiping Mao proposed openstack/kuryr-libnetwork: Fix hard-code listen ip "0.0.0.0"  https://review.openstack.org/35456305:45
*** shashank_hegde has quit IRC05:49
*** vikasc has quit IRC05:54
*** oanson has joined #openstack-kuryr05:58
*** vikasc has joined #openstack-kuryr06:06
openstackgerritLiping Mao proposed openstack/kuryr-libnetwork: Fix hard-code listen ip "0.0.0.0"  https://review.openstack.org/35456306:19
*** diga_ has joined #openstack-kuryr06:22
*** shashank_hegde has joined #openstack-kuryr06:26
*** shashank_hegde has quit IRC07:18
openstackgerritjanonymous proposed openstack/kuryr-libnetwork: Modify fullstack test cases for kuryr-libnetwork  https://review.openstack.org/35578608:44
*** yamamoto has quit IRC08:44
*** janki has joined #openstack-kuryr08:47
*** devvesa has joined #openstack-kuryr09:18
*** limao has quit IRC09:19
vikascdevvesa, ping09:20
*** pablochacin has joined #openstack-kuryr09:22
*** limao has joined #openstack-kuryr09:24
vikascapuimedo,  ping09:25
vikascapuimedo, devvesa i want help in preparing /etc/cni/net.d/network.conf on kubelete node09:26
vikasc{09:26
vikasc    "name": "kuryr-k8s-network",09:26
vikasc    "type": "kuryr",09:26
vikasc    "etcd_authority": "fed-master:2379",09:26
vikasc    "driver": {09:26
vikasc        "type": "kuryr-network"09:26
vikasc    },09:26
vikasc}09:26
openstackgerritjanonymous proposed openstack/kuryr-libnetwork: trivial: nosetest .xml removal as testr is used now  https://review.openstack.org/35581209:30
*** yamamoto has joined #openstack-kuryr09:31
janonymousvikasc: can i help ? i have little idea of kubernetes though09:35
vikascjanonymous, i would appreciate09:36
vikasc:)09:36
janonymousso, the above one is problem statement ..?09:37
vikascjanonymous, i wanted to get my network.conf validated09:37
janonymousOhh, i will read and get back. :)09:39
irenabvikasc: did you check the midonet kuryr fork to see the CNI config?09:41
vikascirenab, you mean config.py?09:41
*** jchhatbar has joined #openstack-kuryr09:41
*** yamamoto has quit IRC09:41
vikasckuryr/cni/driver.py this file i am putting at /opt/cni/bin09:42
vikascconfigured and restarted the kubelet with --network-plugin-dir and --network-plugin options09:43
irenabvikasc: there shuld be the deployment guide, let me find it09:44
irenabpablochacin: Can you please advise on it?09:44
*** jchhatbar has quit IRC09:44
*** janki has quit IRC09:44
pablochacinsure, my pleasure09:44
vikascpablochacin, currently this is my network file in /etc/cni/net.d/09:45
vikasc{09:46
vikasc    "name": "kuryr-k8s-network",09:46
vikasc    "type": "kuryr",09:46
vikasc    "etcd_authority": "fed-master:2379",09:46
vikasc    "driver": {09:46
vikasc        "type": "kuryr-network"09:46
vikasc    }09:46
vikasc}09:46
*** yamamoto has joined #openstack-kuryr09:46
*** yamamoto has quit IRC09:46
*** yamamoto has joined #openstack-kuryr09:48
pablochacinvikasc I'm just entering a meeting, I'll be back to you later.09:49
vikascpablochacin, sure09:49
*** limao has quit IRC10:08
devvesavikas: We have the following on the Kubelet machine10:15
*** janki has joined #openstack-kuryr10:16
devvesain the file /usr/libexec/kubernetes/kubelet-plugins/net/exec/kuryr.conf the following contents10:16
devvesa{ "name": "kuryr", "type": "kuryr" }10:16
devvesaand the binary installed by midonet/kuryr should be on /opt/cni/bin/kuryr10:17
vikascdevvesa, hw this binary will be installed on /opt/cni/bin/kuryr10:22
devvesaand kubelet should run with `--network-plugin=cni`10:22
vikascdevvesa, that i did10:22
vikascdevvesa, how can i get "kuryr" binary10:23
vikascdevvesa, Aug 16 15:50:32 fed-node kubelet[8843]: E0816 15:50:32.889178    8843 cni.go:167] Error adding network: could not find "." plugin10:23
devvesavikasc: It is installed by when you do `python setup.py install` on kuryr directory10:24
devvesavikasc: https://github.com/midonet/kuryr/blob/k8s/setup.cfg#L34-L3610:24
vikascohh10:24
devvesa(please note we are using the k8s branch)10:24
*** yamamoto has quit IRC10:25
vikascdevvesa, thanks, understood10:26
devvesavikasc: I think the the CNI spec says that the conf files should be on /etc/cni/net.d, but kubernetes ignores this 'detail' and uses the /usr/libexec/...foo... directory10:27
vikascdevvesa, i was thinking on same point10:27
vikascdevvesa, thanks for confirming10:27
vikasc:)10:27
vikascdevvesa, it should work fine now10:28
vikascdevvesa, let me try it out10:28
vikascdevvesa,10:31
vikascbyte-compiling /usr/lib/python2.7/site-packages/kuryr/tests/unit/test_aio_methods.py to test_aio_methods.pyc10:31
vikasc  File "/usr/lib/python2.7/site-packages/kuryr/tests/unit/test_aio_methods.py", line 17410:31
vikasc    chunk = yield from response.read_chunk()10:31
vikasc                     ^10:31
vikascSyntaxError: invalid syntax10:31
vikascdevvesa, i am using python2.710:31
devvesapython 3.410:31
vikascok10:31
devvesablame apuimedo, I am the most conservative guy on coding ever :)10:32
vikaschehe10:32
vikascdevvesa,  Error syncing pod 68f1c5ef-639e-11e6-94a6-08002778068c, skipping: failed to "SetupNetwork" for "nginx_default" with SetupNetworkError: "Failed to setup network for pod \"nginx_default(68f1c5ef-639e-11e6-94a6-08002778068c)\" using network plugins \"cni\": netplugin failed but error parsing its diagnostic message \"CNI aborted due to exception: HTTPConnectionPool(host='localhost', port=8080)10:47
vikascdevvesa, nevermind10:48
devvesawell, at least cni plugin is loaded :)10:48
vikascdevvesa, yes. :)10:48
vikascdevvesa, this is due to configuration error10:48
vikascdevvesa, i think i  can fix it10:48
devvesacool. ping me otherwise10:48
vikascdevvesa, sure10:48
*** dingboopt has joined #openstack-kuryr11:02
*** yamamoto has joined #openstack-kuryr11:03
vikascdevvesa, i provided configuration such as api_server details through env variables11:08
vikascdevvesa, now when trying to create a pod,11:09
vikascE0816 16:36:22.659445   32025 cni.go:118] Error while adding to cni network: netplugin failed but error parsing its diagnostic message "CNI aborted due to exception: [Errno 2] No such file or directory: '/usr/lib/python3.5/site-packages/usr/libexec/kuryr/unbound'\n":11:09
vikascdevvesa, i thought of setting pdb by running kubelet in foreground but breakpoint is not hitting11:09
vikascdevvesa, i mean it hits the pdb line in code but does not give control11:10
devvesaI think it is because of the bindir11:10
devvesaconfiguration variable11:10
devvesahttps://github.com/midonet/kuryr/blob/k8s/kuryr/common/config.py#L3211:11
vikascdevvesa, thanks for pointer11:11
devvesait uses this $pybasedir11:11
vikascdevvesa, make sense11:11
devvesabad news are there is no way to pass configuration explicitly to cni driver (since it is not a service and does not spawns, just a binary waiting for being called)11:11
vikascso just env variables11:12
devvesaI propose you to modify the config.py and run the `python setup.py install` again11:12
vikasc?11:12
vikasci set env variables and rerun kubelet in foreground11:12
devvesabut this particular attribute does not have env var, right?11:12
vikascohh11:12
vikasci dint notice that :)11:13
vikascdevvesa, will do as you suggested11:13
devvesaI think you start suffering what I said in yesterday's meeting that "No ready for production and lot of changes needed" :)11:13
vikascdevvesa, :)11:14
*** vikasc is now known as vikasc|brb11:16
*** vikasc|brb is now known as vikasc11:18
vikascdevvesa, is there a way to hit pdb breakpoint in driver code?11:18
*** devvesa has quit IRC12:07
*** janki is now known as janki|afk12:14
*** devvesa has joined #openstack-kuryr12:20
vikascdevvesa, somehow binding:vif_type value is reaching "unbound" to driver12:22
vikascdevvesa, even in raven logs, it is always showing "unbound"12:23
vikascdevvesa, but if i do "neutron port-show <port-id>", there it shows "ovs" correctly12:23
*** diga__ has joined #openstack-kuryr12:24
*** diga_ has quit IRC12:25
*** diga has quit IRC12:27
vikascdevvesa, here are the raven logs, http://pastebin.com/wXnvxEsF12:28
devvesasorry. I was AFK, let me see12:28
vikascdevvesa, binding:vif_type is always "unbound"12:28
vikascdevvesa, no worries12:28
devvesauhm... the binding:vif_type is not the interface_driver attribute on Neutron?12:31
vikascdevvesa, it is12:32
devvesaAnd it is well configured?12:32
vikascdevvesa, one of the port attributes12:32
vikascdevvesa, neutron port-show <port-id>12:33
vikascshows "ovs" as value of this attribute12:33
vikascwhich is expected12:33
vikascbut in raven logs it shows "unbound"12:33
*** yamamoto has quit IRC12:33
vikascdevvesa, and i guess same is passed to driver through annotations12:34
vikascdevvesa, then in binding.py based on value of this attribute binding executable is picked12:34
vikascdevvesa, there it is getting "unbound", which is the fallback script12:35
devvesayes. that's my guess12:35
devvesabut I wonder why sin the annotatoin there is the unbound12:36
vikascdevvesa, did you take a look at logs12:37
vikascdevvesa, that might give some clue12:37
devvesaYes. Too much info12:37
devvesastill digging in12:38
vikascdevvesa, Can it be some issue while copying port attributes and framing json in raven code?12:38
vikascdevvesa, sure12:38
vikascdevvesa,  i will also look.12:38
*** yamamoto has joined #openstack-kuryr12:39
*** yamamoto has quit IRC12:43
*** sdake has joined #openstack-kuryr12:46
sdakehey folks12:46
sdakejust a quick question - when will kuryr introduce keystone v3 support12:46
*** diga has joined #openstack-kuryr12:46
sdakewe can't merge it into kolla atm because it lacks keystone v3 support (and kolla is keystone v3+ only)12:47
sdakei'd like to see it merge this cycle12:47
sdakethe work is 99% done12:47
sdakejust neds config optiosn for keystone v3 ;)12:47
sdakebut the work on the kuryr end - don't know what status of that is12:47
devvesaare you using ml2 with ovs, vikas?12:49
devvesasdake: gsagie may know the status.12:49
devvesaapuimedo knows for sure, but I think today he is on PTO12:49
sdakedevvesa his geo is?12:49
sdake(so I can determine when to ping him :)12:49
devvesaIsrael12:50
sdakeiirc he is in EMEA12:50
sdakecool thanks12:50
sdakei'll try later tonight before heading to bed12:50
sdakeor ask one of our emea cr's to ping him on topic12:50
devvesaor you can send a mail... (In case you have not already did it)12:50
sdakeya that will work [kuryr] is the tagline?12:51
devvesayup :)12:51
sdakecool will do12:52
*** vikasc has quit IRC12:52
sdakeemail away12:56
sdakedevvesa thanks for the help :)12:56
sdakethat should hae just been an automatic for me :)12:57
*** irenab has quit IRC12:57
sdakestruggling with m3 overload...12:57
*** yamamoto has joined #openstack-kuryr13:07
*** yamamoto has quit IRC13:07
*** vikasc has joined #openstack-kuryr13:08
*** dingboopt has quit IRC13:08
devvesasdake: anytime!13:10
*** diogogmt has joined #openstack-kuryr13:33
*** diogogmt has quit IRC13:42
*** banix has joined #openstack-kuryr13:58
*** devvesa has quit IRC14:02
*** devvesa has joined #openstack-kuryr14:06
*** yamamoto has joined #openstack-kuryr14:07
*** devvesa has quit IRC14:07
*** devvesa has joined #openstack-kuryr14:08
*** devvesa has quit IRC14:08
*** devvesa has joined #openstack-kuryr14:09
*** devvesa has quit IRC14:09
*** devvesa has joined #openstack-kuryr14:09
*** yamamoto has quit IRC14:13
*** devvesa has quit IRC14:14
*** devvesa has joined #openstack-kuryr14:14
*** hongbin has joined #openstack-kuryr14:25
*** banix has quit IRC14:30
*** banix has joined #openstack-kuryr14:32
*** devvesa has quit IRC14:34
*** diogogmt has joined #openstack-kuryr14:37
*** devvesa has joined #openstack-kuryr14:45
*** gsagie_ has joined #openstack-kuryr14:57
*** tonanhngo has joined #openstack-kuryr14:59
*** oanson has quit IRC15:24
*** janki|afk has quit IRC15:36
*** diga has quit IRC15:37
*** devvesa has quit IRC15:44
*** sdake has quit IRC16:09
*** pablochacin has quit IRC16:15
*** gsagie_ has quit IRC16:22
*** pablochacin has joined #openstack-kuryr16:44
*** oanson has joined #openstack-kuryr17:09
*** irenab has joined #openstack-kuryr17:09
*** diga__ has quit IRC17:11
*** shashank_hegde has joined #openstack-kuryr17:43
*** oanson has quit IRC17:52
*** sdake has joined #openstack-kuryr17:55
*** sdake_ has joined #openstack-kuryr19:14
*** sdake has quit IRC19:14
*** sdake_ has quit IRC19:15
*** sdake has joined #openstack-kuryr19:16
*** pablochacin has quit IRC19:42
apuimedosdake: I'm the one trying to get it finished20:48
sdakeapuimedo sweet20:49
sdakeapuimedo thanks for working on that ;)20:49
apuimedoI have the support for working with keystone v2 and v3 via env vars20:49
apuimedonow I need to fix the config based one20:49
apuimedoit comes from using mostly env vars myself when doing containers20:50
apuimedoinstead of config options20:50
*** yamamoto has joined #openstack-kuryr21:08
*** yamamoto has quit IRC21:15
sdakeapuimedo cool that sounds like nottoo painful a problem to fix then21:24
sdakesince the had part is done21:24
apuimedoright21:24
apuimedosdake: btw, the demonym for the people of the Kuryr land is Kuryrs :-)21:47
apuimedobut kuryrians sounds good too :-)21:47
*** tonanhngo has quit IRC22:26
*** sdake has quit IRC22:27
*** sdake has joined #openstack-kuryr22:29
*** banix has quit IRC22:31
sdakeapuimedo just had to make somethign up :)22:33
apuimedo;-)22:33
*** limao has joined #openstack-kuryr22:38
*** limao_ has joined #openstack-kuryr22:40
*** vikasc has quit IRC22:41
*** limao has quit IRC22:43
*** yamamoto has joined #openstack-kuryr22:46
*** vikasc has joined #openstack-kuryr22:54
*** limao_ has quit IRC23:03
*** limao has joined #openstack-kuryr23:04
*** hongbin has quit IRC23:05
openstackgerritLiping Mao proposed openstack/kuryr-libnetwork: RequestAddress when subnet has not been created  https://review.openstack.org/35276823:16
apuimedolimao: what distro do you test on?23:17
limaoRHEL23:17
apuimedolimao: you run devstack on that or you do a full installation?23:18
limao# cat /etc/redhat-release23:18
limaoCentOS Linux release 7.2.1511 (Core)23:18
apuimedocool23:18
limaoI use packstack to install Mitaka aoi23:18
limaoand test kuryr with Mitaka stable branch23:18
apuimedothat's saner than just going master on all :-)23:19
limao[root@limao-kuryr kuryr_libnetwork]# docker info23:19
limaoContainers: 923:19
limaoImages: 4623:19
limaoServer Version: 1.9.123:19
limaoStorage Driver: devicemapper23:19
limao Pool Name: docker-253:1-268569388-pool23:19
limao Pool Blocksize: 65.54 kB23:19
limao Base Device Size: 107.4 GB23:19
limao Backing Filesystem:23:19
limao Data file: /dev/loop023:19
limao Metadata file: /dev/loop123:19
limao Data Space Used: 783.7 MB23:19
limao Data Space Total: 107.4 GB23:19
limao Data Space Available: 81.4 GB23:19
limao Metadata Space Used: 1.987 MB23:20
limao Metadata Space Total: 2.147 GB23:20
limao Metadata Space Available: 2.145 GB23:20
limao Udev Sync Supported: true23:20
limao Deferred Removal Enabled: false23:20
limao Deferred Deletion Enabled: false23:20
limao Deferred Deleted Device Count: 023:20
limao Data loop file: /var/lib/docker/devicemapper/devicemapper/data23:20
limao Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata23:20
limao Library Version: 1.02.107-RHEL7 (2015-12-01)23:20
limaoExecution Driver: native-0.223:20
*** limao_ has joined #openstack-kuryr23:21
*** limao has quit IRC23:24
*** limao has joined #openstack-kuryr23:25
*** tonanhngo has joined #openstack-kuryr23:27
*** limao_ has quit IRC23:28
*** tonanhngo has quit IRC23:29
limaoapuimedo: Just a quick question, which mini docker version kuryr will support? 1.12?23:31
openstackgerritLiping Mao proposed openstack/kuryr-libnetwork: Fix hard-code listen ip "0.0.0.0"  https://review.openstack.org/35456323:38
apuimedolimao: mini docker?23:38
limaoI mean which version will be supported by kuryr, kuryr support from 1.9(which is first production ready for libnetwork) or we need latest version of docker?23:40
apuimedofor now the recommended version is 1.11.223:41
apuimedo1.12 comes with some changes and I was not able to test it yet23:41
*** limao_ has joined #openstack-kuryr23:43
*** limao__ has joined #openstack-kuryr23:43
*** limao_ has quit IRC23:44
*** limao has quit IRC23:45
*** limao__ is now known as limao23:45

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!