Thursday, 2017-08-31

*** thegreenhundred has joined #openstack-containers00:14
*** catintheroof has quit IRC00:35
*** yuanying has joined #openstack-containers01:00
*** yuanying_ has quit IRC01:04
*** yuanying_ has joined #openstack-containers01:04
*** yuanying has quit IRC01:05
*** ricolin has joined #openstack-containers01:05
openstackgerritWayne Chan proposed openstack/magnum master: Use https links for documentation  https://review.openstack.org/47365201:07
*** shu-mutou-AWAY is now known as shu-mutou01:36
*** hongbin has joined #openstack-containers01:50
*** askb has quit IRC01:53
*** jmlowe has quit IRC02:00
*** jmlowe has joined #openstack-containers02:01
*** yuanying_ has quit IRC02:01
*** catintheroof has joined #openstack-containers02:03
*** yuanying has joined #openstack-containers02:04
*** wangbo has joined #openstack-containers02:11
*** wangbo has quit IRC02:41
*** ykarel has joined #openstack-containers02:44
*** wangbo has joined #openstack-containers02:44
*** cliles has quit IRC02:54
*** catintheroof has quit IRC03:07
*** wangbo has quit IRC03:10
*** ykarel has quit IRC03:13
*** jmlowe has quit IRC03:13
*** jmlowe has joined #openstack-containers03:14
*** wangbo has joined #openstack-containers03:15
*** ramishra has joined #openstack-containers03:22
*** ykarel has joined #openstack-containers04:07
*** janki has joined #openstack-containers04:10
*** hongbin has quit IRC04:11
*** chhavi has joined #openstack-containers04:16
*** wangbo has quit IRC04:21
*** wangbo has joined #openstack-containers04:57
*** ianychoi__ has joined #openstack-containers05:12
*** ianychoi_ has quit IRC05:16
*** maestropandy has joined #openstack-containers05:19
*** maestropandy has left #openstack-containers05:25
*** trinaths has joined #openstack-containers05:28
*** dsariel has joined #openstack-containers05:29
*** mdnadeem has joined #openstack-containers05:50
*** mjura has joined #openstack-containers06:00
*** maestropandy1 has joined #openstack-containers06:02
*** Oku_OS-away is now known as Oku_OS06:10
*** maestropandy1 has left #openstack-containers06:12
*** pcaruana has joined #openstack-containers06:22
*** maestropandy has joined #openstack-containers06:26
*** maestropandy has left #openstack-containers06:27
*** mjura has quit IRC06:28
*** rcernin has joined #openstack-containers06:40
*** mjura has joined #openstack-containers06:41
*** dsariel has quit IRC06:45
*** livelace has joined #openstack-containers06:45
*** maestropandy has joined #openstack-containers06:51
*** maestropandy has left #openstack-containers06:51
*** trinaths has quit IRC07:10
*** magicboiz has quit IRC07:10
*** larizmen has joined #openstack-containers07:17
*** trinaths has joined #openstack-containers07:32
*** funzo_ has joined #openstack-containers07:43
*** funzo has quit IRC07:46
*** magicboiz has joined #openstack-containers07:59
*** mgoddard has joined #openstack-containers08:14
*** openstackgerrit has quit IRC08:17
*** danil has joined #openstack-containers08:29
*** mdnadeem has quit IRC08:29
strigaziDimGR: I'm here08:32
DimGRhi08:32
DimGRdo we still need coredns in pike?08:32
*** dsariel has joined #openstack-containers08:32
strigaziyeap08:32
*** mdnadeem has joined #openstack-containers08:33
DimGRstrigazi is there another coredns yaml other than the one we last used ?08:40
DimGRsomething more up to date ?08:44
strigaziwhat is out of date?08:45
strigaziyou can try using coredns:latest08:46
DimGRbecause strigazi the coredns.yaml we were using is giving these errors http://paste.openstack.org/show/619993/    and i can verify these deployments and services are not present08:46
strigaziwhere is this error?08:46
strigazilog of what?08:47
DimGRstrigazi sorry i  havent had coffee yet , i saw these with --all-namespaces08:48
DimGRsorry :P08:48
DimGRok coredns pod is running08:49
DimGRhttp://paste.openstack.org/show/619999/   does this means dns in the cluster is now finally working strigazi  ?08:55
strigaziPike should work with no problems08:55
strigaziI tried with fission.io and i was able to talk to the names of other services.08:56
DimGRill test fission if coredns is giving me issues08:59
strigaziok08:59
*** maestropandy has joined #openstack-containers09:01
*** maestropandy has left #openstack-containers09:01
DimGRhttp://paste.openstack.org/show/619999/   this was not working with the previous release09:01
strigazinow it is working right?09:03
DimGRnot really , i think i need to edit kubelet configs09:06
strigazito edit what?09:06
DimGR--cluster-dns09:06
DimGRlike we did in the past09:06
strigaziWhat was  missing?09:07
strigaziI don't remember09:07
DimGRin the past --cluster-dns was never set and we did it by hand but it never worked right . right now i try to attach to running pods and do nslookups , and the nslooksups fail09:10
strigaziDimGR: https://github.com/openstack/magnum/blob/stable/pike/magnum/drivers/common/templates/kubernetes/fragments/configure-kubernetes-master.sh#L7009:13
strigaziDimGR: https://github.com/openstack/magnum/blob/stable/pike/magnum/drivers/common/templates/kubernetes/fragments/configure-kubernetes-minion.sh#L5509:13
strigaziin pike, cluster_dns is there09:13
strigaziDimGR: How come and you needed to add the cluster-dns manually?09:14
DimGRnot now for pike strigazi  i was talking about the other time09:17
strigaziyes, in ocata it wan't there, but in pike it is.09:18
strigaziThe nslookup works in pike right?09:18
*** mdnadeem has quit IRC09:21
DimGRfrom http://paste.openstack.org/show/620005/ check this09:23
magicboizpike09:23
strigaziDimGR: so it works fine, right?09:23
DimGRi guess it is09:24
*** mdnadeem has joined #openstack-containers09:24
DimGRyes it is working . looking at logs from each running service and they talk to each other strigazi09:32
*** shu-mutou is now known as shu-mutou-AWAY09:42
*** wangbo has quit IRC09:52
*** wangbo has joined #openstack-containers10:02
*** mdnadeem has quit IRC10:03
*** mdnadeem has joined #openstack-containers10:18
*** wangbo has quit IRC10:25
*** maestropandy has joined #openstack-containers10:56
*** wangbo has joined #openstack-containers10:58
*** wangbo has quit IRC10:59
*** wangbo has joined #openstack-containers11:01
*** trinaths has left #openstack-containers11:03
*** maestropandy has quit IRC11:06
*** maestropandy has joined #openstack-containers11:09
*** maestropandy has left #openstack-containers11:11
*** maestropandy has joined #openstack-containers11:17
*** maestropandy has left #openstack-containers11:17
*** dave-mccowan has joined #openstack-containers11:18
*** thegreenhundred has quit IRC11:55
DimGRstrigazi ping11:59
strigazihere11:59
DimGRhttp://paste.openstack.org/show/620038/          do you remember how we had it fixed last time ?12:00
*** trinaths has joined #openstack-containers12:06
strigaziDimGR: when you specify a fixed_subnet use the ID not the name12:07
DimGRi did not specify anything at all , im looking to find the conf to make the change12:08
strigaziDimGR you need one12:08
strigazi/etc/kubernetes/kube_os_config in all nodes12:09
DimGRi have /etc/kubernetes/config12:09
DimGRhttp://paste.openstack.org/show/620041/   like so12:10
DimGRfrom master12:10
strigazils /etc/kubernetes ?12:11
DimGRhttp://paste.openstack.org/show/620043/12:11
strigaziyou must have this file /etc/kubernetes/kube_openstack_config12:12
strigaziyeap it is there12:12
DimGRi see it12:13
strigazisubnet-id= put id there and restart the controller mamaner or create a new cluster with fixed_subnet the subent id12:13
DimGRso replace the name with the id12:13
strigaziyes12:13
DimGRthis needs to be done in minions too ?12:13
*** trinaths has left #openstack-containers12:21
strigaziDimGRm I think not12:22
DimGRok only in master then12:24
DimGRgoogling how to restart compose manager12:25
DimGR:)12:25
strigazisystemctl restart kube-controller-manager12:25
DimGRok lets try the nginx again12:26
*** catintheroof has joined #openstack-containers12:30
DimGRstrigazi  http://paste.openstack.org/show/620050/12:31
*** thegreenhundred has joined #openstack-containers12:38
strigaziDimGR: looks good12:43
DimGRits not good12:43
DimGRthere is an error there12:43
strigazian id is wrong then12:44
*** wangbo has quit IRC12:46
DimGRchecking again12:46
*** wangbo has joined #openstack-containers12:48
DimGRunless im mistaken i think we had told last time now to have the lbass auto create a floating ip and that i would have to do it manually12:48
DimGRis this the case?12:48
strigaziFor the public ones you need to do it manually, it is how neutron-lbaas works12:49
*** wangbo has quit IRC12:57
*** dave-mccowan has quit IRC12:57
*** dave-mcc_ has joined #openstack-containers13:09
*** ianychoi__ is now known as ianychoi13:20
DimGRso i should comment out the floating-id from the .conf and retry strigazi13:24
strigaziyou can try, I don't know if it will work. The config that you showed me should work with the right IDs13:27
mnaserstrigazi gonna try the containerized 1.7.4 now :)13:30
strigazimnaser: great! it works for DimGR ^^13:30
strigaziof course he is a bit stuck with the cloud provider and lbaas like everyone13:31
mnaserDimGR floating-id will make k8s automatically assign a floating ip13:32
mnaserif you put that in your kube_openstack_config13:32
DimGRi know but it somehow doesnt like my floating id13:32
mnaserDimGR can you confirm that it is the id of the public floating network? (not subnet)13:33
mnaserDimGR to confirm, can you run `neutron floatingip-create FLOATING_NETWORK` and see if it says ok13:33
DimGRhold on13:34
DimGRi already have a floating subnet where i give out floating ip to instances mnaser13:35
mnaserbut you shouldnt provide a floating ip subnet13:35
mnaseryou provide a floating ip network13:35
*** mgoddard has quit IRC13:40
DimGRmnaser  http://paste.openstack.org/show/620069/13:43
*** danil has quit IRC13:43
*** mgoddard has joined #openstack-containers13:44
mnasersorry my bnc dropped13:45
DimGRmnaser  http://paste.openstack.org/show/620069/     i showed you that13:45
mnaseryour floating-network-id should be GATEWAY_NET DimGR13:45
mnaserthat should be the one publicly routable one13:46
mnaseraka external=true13:46
DimGRhere you are http://paste.openstack.org/show/620071/13:47
DimGRmnaser looking good ?13:50
mnaserso thats good, which means configure your floating-network-id to your gateway net uuid13:50
mnaserand then it should automatically allocate a floating ip13:50
DimGRok lets try13:51
DimGRhere is the error mnaser strigazi  http://paste.openstack.org/show/620074/           it picks up the wrong id .13:55
mnaserdid you make sure to restart the controller service13:56
mnaseronce you made that hcnage13:56
mnaserDimGR actually13:56
mnaseri think you're good13:56
mnaserthose error messages are 3m ago13:56
DimGRyes i restarted it13:56
mnaser35s ago created load balancer is okay13:56
mnaserno error after13:56
mnaserLoadBalancer Ingress:192.168.0.10, 88.99.176.11913:56
mnaser^ thats you ip :)13:57
mnaserunless you manually added that floating ip?13:57
DimGRcan you ping that 88.99 ?13:57
DimGRyou are right i did not notice the seconds13:57
mnaseri can see nginx page13:58
DimGRso thats good13:59
*** mdnadeem has quit IRC14:00
DimGRshould there be a loadblanacer named nginx in horizon under loadbalancers ? because there is not .14:05
*** hongbin has joined #openstack-containers14:07
*** marst has joined #openstack-containers14:08
mnaserno it will be called some odd name that k8s decided DimGR14:14
DimGRok so i guess we are good . i created a second lbass and it worked14:18
mnaserDimGR congrats :)14:18
DimGRneed to write down the whole procedure for future references :)14:18
mnaserDimGR and ill make sure you dont have to by making sure magnum does it for you ;)14:19
mnaserDimGR https://review.openstack.org/#/c/489676/14:19
mnasercc strigazi ^14:19
mnaseri had this whole thing automated14:19
mnaserit runs beautifully on our public cloud too14:20
mnaserso you dont have to deal with that mess :)14:20
DimGRmagnum pike so far seems excellent , magnum ocata was another story14:20
mnaserit'll keep getting better.  personally my goal is increasing integration with openstack14:20
mnaserso k8s is as integrated as it can be with the underlying cloud14:20
mnasermore plug and play, i dont want customers to ssh to servers to do things, ever14:21
mnaserif they have to: we're doing it wrong14:21
DimGRdns and cinder were not woking in magnum ocata , now they are without any hackery14:22
*** openstackgerrit has joined #openstack-containers14:25
openstackgerritMohammed Naser proposed openstack/magnum stable/pike: Clean-up server names in drivers  https://review.openstack.org/49961714:25
DimGRstrigazi and mnaser all is good now , you can sleep with no worries :)14:32
* mnaser is in EDT14:34
mnaser:p14:34
strigazimnaser: 489676 looks good to me, can you open a bug for this?14:37
mnaserstrigazi absolutely one sec14:39
DimGRstrigazi are you proud of me now  ? :)14:40
strigaziyeap ;)14:40
DimGRdidn't bug you a lot this time14:40
openstackgerritMohammed Naser proposed openstack/magnum master: Automatically assign floating IPs with LBaaS v2  https://review.openstack.org/48967614:40
mnaserstrigazi done14:41
strigazimnaser: have you seen this patch https://review.openstack.org/#/c/497223/ ?14:51
strigaziit can speed the deployment time of a cluster a lot14:52
*** janki has quit IRC14:55
mnaserstrigazi been playing catch up with upgrades to pike and some sahara stuff at the moment, ill review14:56
mnaserstrigazi this is super cool though.14:57
mnaserstrigazi we can leverage the docker image cache in the gate fyi14:57
*** ramishra has quit IRC14:58
strigazimnaser we have a docker images cache in ci? or setup one? I though about pulling the images in the node before creating cluster and serve them from there14:58
mnaserstrigazi https://review.openstack.org/#/c/453811/15:00
mnaserstrigazi old kolla patch when i was working on it how to set it up https://review.openstack.org/#/c/455378/15:01
mnaserbut i dont know if the atomic cli tool uses docker to download so i dont know if that will affect it15:01
strigazihttps://review.openstack.org/#/c/497223/5/magnum/drivers/common/templates/kubernetes/fragments/configure-kubernetes-master.sh@7 docker.io15:01
mnasernot sure if that can apply for it15:02
strigazithe images are here: https://hub.docker.com/r/openstackmagnum/kubernetes-kubelet/tags/15:02
mnaseri'll have a look15:02
mnaserstrigazi if you dont mind me asking, im checking which patches we can drop with 5.0.1 -- https://review.openstack.org/#/c/488511/15:02
mnaserservices arent deployed in k8s anymore right? so the volume mounts are not used?15:02
*** daidv has quit IRC15:03
strigazimnaser: what do you mean by drop patches?15:04
*** daidv has joined #openstack-containers15:04
mnaserstrigazi we were installing master and some patches on top (that were upstreamed)15:04
mnaserbut they were still pending approval etc15:04
mnaserso now that i'm going to 5.0.1 im seeing which patches are no longer necessary cause they merged15:04
strigaziwith pike you don't need 55378 is not needed anymore15:04
mnaserthats what i thought15:05
mnaserso the only two patches we're carrying is the floating network id (the one i linked earlier)15:05
strigazithis file is dropped: https://review.openstack.org/#/c/488511/6/magnum/drivers/common/templates/kubernetes/fragments/enable-kube-controller-manager-scheduler.sh15:05
mnaserand a backport of the cleaned up server name which i submitted earlier15:05
mnaserfor stable/pike (looks like it missed it)15:05
strigaziyou carry two patches?15:06
mnaseryes, both upstreamed.  one is pending merge (the floating ip) and one is pending backport (server name cleanup)15:06
mnaserwe just run them locally instead of waiting :)15:06
mnaserhttps://review.openstack.org/#/c/489676/ and https://review.openstack.org/#/c/499617/15:07
strigazimnaser: we carry 46 :)15:07
mnaseraha, we tend to stick closer to trunk so15:07
mnaseri try to rebase often15:07
strigazimnaser: I'll try to merge and release 5.0.215:08
strigazihave to go now15:08
strigazisee you15:08
mnaserlater, take care strigazi15:08
*** ramishra has joined #openstack-containers15:12
*** Oku_OS is now known as Oku_OS-away15:16
openstackgerritMohammed Naser proposed openstack/magnum master: Create hidden stacks for clusters  https://review.openstack.org/49596015:16
*** ramishra has quit IRC15:23
*** mjura has quit IRC15:32
*** pcaruana has quit IRC15:32
*** ykarel has quit IRC15:32
*** ykarel has joined #openstack-containers16:00
*** maestropandy has joined #openstack-containers16:25
*** maestropandy has left #openstack-containers16:25
*** larizmen has quit IRC16:31
*** dsariel has quit IRC16:45
*** dave-mcc_ is now known as dave-mccowan17:01
*** mgoddard has quit IRC17:04
*** rcernin has quit IRC17:12
*** trinaths has joined #openstack-containers17:13
openstackgerritMohammed Naser proposed openstack/magnum master: Avoid running periodic processes inside each worker process  https://review.openstack.org/49973917:17
*** mgoddard has joined #openstack-containers17:43
*** trinaths has left #openstack-containers17:53
*** ykarel_ has joined #openstack-containers17:59
*** ykarel has quit IRC18:01
*** dave-mccowan has quit IRC18:07
*** dave-mccowan has joined #openstack-containers18:14
*** livelace has quit IRC18:24
*** chhavi has quit IRC18:36
*** rcernin has joined #openstack-containers19:01
*** ykarel_ has quit IRC19:02
DimGRwhy is gerrit so stupid to set it up20:04
DimGRbeen trying to commit a patch20:04
*** yee379 has quit IRC20:09
*** yee379 has joined #openstack-containers20:09
*** askb has joined #openstack-containers21:04
openstackgerritdimitris proposed openstack/magnum master:  detailed guide how to upgrade magnum ocata to magnum pike  using openstack-ansible  https://review.openstack.org/49980221:06
*** askb has quit IRC21:09
*** penick has joined #openstack-containers21:10
*** askb has joined #openstack-containers21:10
*** mgoddard has quit IRC21:15
*** dave-mccowan has quit IRC21:20
*** catintheroof has quit IRC21:31
openstackgerritMohammed Naser proposed openstack/magnum master: Switch DNS service to Kube DNS  https://review.openstack.org/49981321:34
DimGRmnaser ping21:38
mnaserhey DimGR21:38
DimGRabout your commit21:38
DimGRis it for pike ?21:39
DimGRthe dns21:39
DimGRi had horrible experiences with dns in magnum ocata but not with pike21:40
mnaserDimGR it is on master but i hope to convince folks to backport it21:40
mnaserDimGR if you try using helm and install something like mongodb replicaset .. it doesnt work.21:40
*** penick has quit IRC21:41
DimGRwiith magnum ocata i never got coredns to work in the cluster , services would not talk to each other . coredns on pike is working excellent , mongodb replicasets all work etc etc21:41
mnaserDimGR can you confirm you can install mongodb replicaset using helm?21:42
mnaserhttps://github.com/kubernetes/charts/tree/master/stable/mongodb-replicaset21:43
mnaserthis one specifically!21:43
mnaserand with 5.0.1 i can't even create cinder volumes without adding the storage class which i'm doing right now in a new patch21:43
DimGRaha that helm chart was the first one i used21:44
DimGRnever worked and i ditched helm all together21:44
DimGRfew months ago21:44
DimGRbut i can give it a try again in the next few days21:45
mnaserDimGR it won't work :) unless you have that patch i just put up which uses kube-dns21:47
DimGRcinder volumes also working excellent21:47
mnaserDimGR 5.0.0 or 5.0.1 ?21:47
DimGRstable/pike21:47
mnaseri think 5.0.1 introduced a bug where cinder volumes don't work.  there is no default storage class so nothing happens21:47
mnaseri had to create a default storage class21:48
DimGRhttp://paste.openstack.org/show/620141/21:49
DimGRmnaser21:50
DimGRworked right away21:50
mnaseraaah21:50
mnaserDimGR what you're doing works fine21:50
mnaserthat's manually picking a volume21:50
mnaserim talking about k8s automatically provisioning volumes21:50
mnaserthat doesn't work (for me at least and ive tested it enough)21:50
DimGRoh i see now what you mean21:51
mnaserim testing the fix locally21:51
mnaseralso there is a problem with very high cpu usage21:52
mnaserthat i put out a fix for21:52
mnaserso there will be a few very nice roboust-ifing fixes soon21:52
DimGRnice21:54
openstackgerritMohammed Naser proposed openstack/magnum master: Install default storage class for Cinder  https://review.openstack.org/49984222:30
mnaserDimGR ^ voila22:30
mnaseronce those pass and get merged i'll backport to stable/pike and hopefully strigazi can cut a 5.0.222:30
*** rcernin has quit IRC22:30
DimGRcool  nice22:31
mnaserif they don't kill me for the coredns to kubedns change :)22:32
DimGRhaha22:34
*** thegreenhundred has quit IRC22:38
*** hongbin has quit IRC23:22
*** yamamoto_ has joined #openstack-containers23:31
*** yamamoto_ has quit IRC23:31
*** yamamot__ has joined #openstack-containers23:31
*** catintheroof has joined #openstack-containers23:55
*** marst has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!