| *** eaguilar has quit IRC | 00:04 | |
| *** caoyuan has joined #openstack-kolla | 00:06 | |
| *** manheim has joined #openstack-kolla | 00:07 | |
| *** caoyuan has quit IRC | 00:09 | |
| *** schwicht has quit IRC | 00:12 | |
| *** eaguilar has joined #openstack-kolla | 00:21 | |
| *** manheim has quit IRC | 00:26 | |
| *** yangyapeng has joined #openstack-kolla | 00:43 | |
| *** yangyape_ has joined #openstack-kolla | 00:44 | |
| *** yangyapeng has quit IRC | 00:45 | |
| eanylin | sdake: I just updated the Deployment Guide with the things that I tested today. Thanks. | 00:50 |
|---|---|---|
| *** Pavo has quit IRC | 00:55 | |
| *** lamt has quit IRC | 00:57 | |
| *** lamt has joined #openstack-kolla | 01:00 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 01:11 |
| openstackgerrit | Mohammed Naser proposed openstack/kolla stable/newton: Fix hardcoded path for custom configuration of Neutron VPNaaS https://review.openstack.org/449975 | 01:22 |
| *** yangyape_ has quit IRC | 02:00 | |
| *** yangyapeng has joined #openstack-kolla | 02:03 | |
| *** eaguilar has quit IRC | 02:15 | |
| *** dave-mccowan has joined #openstack-kolla | 02:18 | |
| *** eaguilar has joined #openstack-kolla | 02:34 | |
| *** sayantan_ has joined #openstack-kolla | 02:48 | |
| *** eaguilar has quit IRC | 02:58 | |
| *** dixiaoli has joined #openstack-kolla | 03:01 | |
| *** dixiaoli has quit IRC | 03:08 | |
| *** dave-mccowan has quit IRC | 03:24 | |
| sdake | eanylin thanks dude :) | 03:42 |
| *** jascott1_ has quit IRC | 03:44 | |
| *** jascott1_ has joined #openstack-kolla | 03:44 | |
| *** jascott1_ has quit IRC | 03:49 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 03:50 |
| *** iceyao has joined #openstack-kolla | 03:54 | |
| *** lamt has quit IRC | 04:14 | |
| *** iceyao has quit IRC | 04:25 | |
| *** iceyao has joined #openstack-kolla | 04:26 | |
| eanylin | sdake: np :) | 04:32 |
| *** lamt has joined #openstack-kolla | 04:34 | |
| spsurya | sup guys | 04:43 |
| *** sayanta__ has joined #openstack-kolla | 04:47 | |
| *** sayantan_ has quit IRC | 04:50 | |
| *** iceyao has quit IRC | 04:54 | |
| *** zhubingbing__ has joined #openstack-kolla | 05:03 | |
| *** lamt has quit IRC | 05:04 | |
| *** lamt has joined #openstack-kolla | 05:10 | |
| zhubingbing__ | hey sdake | 05:12 |
| zhubingbing__ | ;) | 05:13 |
| zhubingbing__ | hi guys | 05:13 |
| *** manheim has joined #openstack-kolla | 05:15 | |
| *** manheim has quit IRC | 05:19 | |
| spsurya | hi zhubingbing__ :) | 05:22 |
| zhubingbing__ | hi spsurya | 05:22 |
| *** lamt has quit IRC | 05:36 | |
| openstackgerrit | zhubingbing proposed openstack/kolla-kubernetes master: Test gate https://review.openstack.org/449979 | 05:51 |
| zhubingbing__ | ping sbezverk | 06:12 |
| zhubingbing__ | kolla-kubernetes gate look not good ;( | 06:13 |
| *** yangyapeng has quit IRC | 06:18 | |
| *** yangyapeng has joined #openstack-kolla | 06:18 | |
| *** iceyao has joined #openstack-kolla | 06:21 | |
| *** yangyape_ has joined #openstack-kolla | 06:30 | |
| *** yangyapeng has quit IRC | 06:30 | |
| *** furlongm has quit IRC | 06:37 | |
| *** furlongm has joined #openstack-kolla | 06:37 | |
| *** hieulq has quit IRC | 07:09 | |
| *** lamt has joined #openstack-kolla | 07:14 | |
| *** hieulq has joined #openstack-kolla | 07:25 | |
| *** iceyao has quit IRC | 08:06 | |
| *** lamt has quit IRC | 08:15 | |
| *** iceyao has joined #openstack-kolla | 08:15 | |
| *** sayanta__ has quit IRC | 08:20 | |
| *** manheim has joined #openstack-kolla | 09:03 | |
| *** manheim has quit IRC | 09:25 | |
| *** yangyape_ has quit IRC | 09:50 | |
| *** bmace has quit IRC | 10:01 | |
| *** bmace has joined #openstack-kolla | 10:02 | |
| *** manheim has joined #openstack-kolla | 10:20 | |
| *** pbourke has quit IRC | 10:21 | |
| *** pbourke has joined #openstack-kolla | 10:23 | |
| *** manheim has quit IRC | 10:24 | |
| *** zhurong has joined #openstack-kolla | 10:32 | |
| *** iceyao has quit IRC | 10:39 | |
| *** yangyapeng has joined #openstack-kolla | 10:42 | |
| *** jascott1_ has joined #openstack-kolla | 10:45 | |
| *** zhurong has quit IRC | 10:52 | |
| *** yangyapeng has quit IRC | 11:11 | |
| *** yangyapeng has joined #openstack-kolla | 11:12 | |
| *** yangyapeng has quit IRC | 11:16 | |
| *** iceyao has joined #openstack-kolla | 11:18 | |
| *** manheim has joined #openstack-kolla | 11:21 | |
| *** jascott1_ has quit IRC | 11:21 | |
| *** manheim has quit IRC | 11:26 | |
| *** deadnull_ has joined #openstack-kolla | 11:27 | |
| *** deadnull_ has quit IRC | 11:28 | |
| *** zhubingbing__ has quit IRC | 11:29 | |
| *** caoyuan has joined #openstack-kolla | 11:35 | |
| *** adil452100 has joined #openstack-kolla | 11:47 | |
| adil452100 | Hello Kola users/devs | 11:49 |
| adil452100 | I'm thinking of deploying Kolla on 3 bare metal servers | 11:50 |
| adil452100 | but in a hyperconverged manner with HA | 11:50 |
| adil452100 | so the 3 servers will act as ceph/controller/compute nodes in HA | 11:51 |
| adil452100 | That would be possible ? | 11:51 |
| *** iceyao has quit IRC | 12:23 | |
| *** iceyao has joined #openstack-kolla | 12:23 | |
| *** yangyapeng has joined #openstack-kolla | 12:26 | |
| adil452100 | Hello Kola users/devs | 12:35 |
| adil452100 | I'm thinking of deploying Kolla on 3 bare metal servers | 12:35 |
| adil452100 | but in a hyperconverged manner with HA | 12:35 |
| adil452100 | so the 3 servers will act as ceph/controller/compute nodes in HA | 12:35 |
| adil452100 | That would be possible ? | 12:35 |
| *** hachi has joined #openstack-kolla | 12:39 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 12:47 |
| *** zhurong has joined #openstack-kolla | 12:50 | |
| *** hachi has quit IRC | 12:54 | |
| *** hachi has joined #openstack-kolla | 12:55 | |
| *** manheim has joined #openstack-kolla | 13:09 | |
| *** Pavo has joined #openstack-kolla | 13:12 | |
| *** yangyapeng has quit IRC | 13:12 | |
| *** manheim has quit IRC | 13:14 | |
| *** sbezverk has quit IRC | 13:20 | |
| *** sayantan_ has joined #openstack-kolla | 13:21 | |
| *** sbezverk has joined #openstack-kolla | 13:23 | |
| *** yangyapeng has joined #openstack-kolla | 13:23 | |
| *** jascott1_ has joined #openstack-kolla | 13:24 | |
| *** sayantan_ has quit IRC | 13:26 | |
| *** caoyuan has quit IRC | 13:26 | |
| *** caoyuan has joined #openstack-kolla | 13:26 | |
| *** jascott1_ has quit IRC | 13:29 | |
| *** iceyao has quit IRC | 13:37 | |
| spsurya | adil452100: yes that should | 13:50 |
| spsurya | be possible | 13:50 |
| *** manheim has joined #openstack-kolla | 13:52 | |
| *** sbezverk has quit IRC | 13:52 | |
| *** sbezverk has joined #openstack-kolla | 13:54 | |
| *** zhurong has quit IRC | 13:55 | |
| *** caoyuan_ has joined #openstack-kolla | 13:58 | |
| *** caoyuan has quit IRC | 13:59 | |
| *** manheim has quit IRC | 14:13 | |
| *** eaguilar has joined #openstack-kolla | 14:16 | |
| sdake | morning folks | 14:42 |
| adil452100 | <spsurya> Thank you very much | 14:43 |
| adil452100 | <spsurya> Is there anyone you know tried this architecture in production | 14:44 |
| adil452100 | <spsurya> And what whould be its limitation | 14:44 |
| *** ccesario has quit IRC | 14:45 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 14:46 |
| *** caoyuan_ has quit IRC | 14:46 | |
| spsurya | sdake: morning | 14:47 |
| spsurya | adil452100: AFAIK many of the people tried with this https://docs.openstack.org/developer/kolla/newton/multinode.html | 14:47 |
| spsurya | I didnt tried on master | 14:48 |
| *** caoyuan has joined #openstack-kolla | 14:49 | |
| adil452100 | <spsurya> I see . Thx again | 14:50 |
| *** caoyuan has quit IRC | 14:50 | |
| *** caoyuan has joined #openstack-kolla | 14:50 | |
| *** financeMann has joined #openstack-kolla | 14:52 | |
| financeMann | GOOD MORNING | 14:52 |
| financeMann | I have become pretty proficient at creating windows 10 VMs to be used inside the kolla-openstack deployment i setup on my supercomputer | 14:52 |
| financeMann | Working great | 14:52 |
| financeMann | I’m trying to play Civ 6 on one of my instances but of course my super-computer has no graphics cards | 14:53 |
| financeMann | I’m using virtio drivers but was wondering if there were any drivers I could use that would emulate graphics such that I can run directx | 14:53 |
| financeMann | I don’t plan on using this as a gamin rig but wanna play some Civ 6 here or there maybe | 14:54 |
| Pavo | financeMann that would require you to have a GFX card and do GPU pass-through | 14:55 |
| Pavo | I have a RX 480 in my setup with GPU pass-through and stream my Steam games to my mac | 14:56 |
| Pavo | from my Windows instance | 14:56 |
| financeMann | how would I do it without a graphics card Pavo | 15:00 |
| financeMann | theres gotta be an emulator or something | 15:00 |
| Pavo | don't think you can | 15:00 |
| Pavo | you are technically already emulating since Windows instance is running on top of a Hypervisor | 15:01 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 15:19 |
| *** sayantan_ has joined #openstack-kolla | 15:22 | |
| *** sayantan_ has quit IRC | 15:26 | |
| *** zhubingbing_ has joined #openstack-kolla | 15:29 | |
| *** Pavo has quit IRC | 15:30 | |
| *** hrw has quit IRC | 15:34 | |
| *** hrw has joined #openstack-kolla | 15:37 | |
| *** kolla-slack has joined #openstack-kolla | 15:37 | |
| *** eaguilar has quit IRC | 15:38 | |
| sdake | sup financeMann | 15:38 |
| sdake | long time no talk | 15:38 |
| zhubingbing_ | hi sdake | 15:39 |
| sdake | sup zhubingbing_ | 15:39 |
| zhubingbing_ | kolla-k8s gate look so bad | 15:39 |
| sdake | zhubingbing_ i see that | 15:39 |
| sdake | sbezverk did a recheck on your patch and its still busted | 15:40 |
| zhubingbing_ | kubernetes cluster don't running | 15:40 |
| sdake | one thing to note - - calico upstream released a version of calico for kubernetes 1.6.0 | 15:40 |
| sdake | kfox1111 and sbezverk had believed they had fixed this | 15:40 |
| sdake | by pinning to a specific version of calico.yaml | 15:41 |
| sdake | this breakage happend roghly 30 hours ago | 15:41 |
| sdake | that patch merged | 15:41 |
| zhubingbing_ | so i must set kubernetes version | 15:41 |
| sdake | i dont know what the resolution is, I am giving you the histroy | 15:41 |
| sdake | calico.yaml pulls in docker containers | 15:41 |
| sdake | those could have been updated independently of the calico repository | 15:42 |
| sdake | and be the true cause of the problem | 15:42 |
| sdake | my specualtion is the gate fix was not correct | 15:42 |
| kolla-slack | <sbezverk> Curious how you explain then the fix we pushed, got merged?? If it had not fixed the issue then it would have never gotten merged. Simple logic here.. | 15:42 |
| kolla-slack | <sbezverk> I had several completely green runs this morning.. | 15:43 |
| sdake | sbezverk - pretty straightforward, the docker containers were modified after the gate test was run | 15:43 |
| sdake | sbezverk this is easy to verify | 15:43 |
| sdake | sbezverk check this repository: | 15:43 |
| zhubingbing_ | agree | 15:44 |
| kolla-slack | <sbezverk> I HAD green gate 1 hour ago | 15:44 |
| zhubingbing_ | I think it is kubernetes upgrade to 1.6 problem ,and calico should update | 15:44 |
| kolla-slack | <sbezverk> we do not have 1.6 kube | 15:45 |
| sdake | https://github.com/projectcalico/canal/tree/master/k8s-install/kubeadm | 15:46 |
| zhubingbing_ | i thinks we should limit the version of kubernetes | 15:46 |
| sdake | are we on kubernetes 1.6 now? | 15:46 |
| zhubingbing_ | yeah | 15:46 |
| sdake | moment | 15:46 |
| zhubingbing_ | default | 15:47 |
| sdake | https://github.com/projectcalico/canal/tree/master/k8s-install/kubeadm/1.6 | 15:47 |
| zhubingbing_ | https://review.openstack.org/#/c/449873/ | 15:47 |
| zhubingbing_ | but i set to 1.5.4 it look bad | 15:48 |
| sdake | here is what I suggest | 15:48 |
| sdake | 1.6.0 kubernetes + projectcalico 1.6 file | 15:48 |
| sdake | yaml manifest | 15:48 |
| zhubingbing_ | yes | 15:48 |
| zhubingbing_ | i test it | 15:49 |
| sdake | the fact that containers are versions independently of the manifests in these third party projects and not managed by ci | 15:49 |
| financeMann | yo sdake how do i run directx on my openstack when i dont have a video card :D Can I do any emulation more than my kvm? | 15:49 |
| sdake | is a real problem | 15:49 |
| sdake | financeMann no clue | 15:49 |
| sdake | the real problem there with these third party repos is that all of these containers may not be managed consistently | 15:49 |
| sdake | zhubingbing_ are you using my box to test at present | 15:50 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 15:50 |
| kfox1111 | we really need to cache our deps. :/ | 15:51 |
| sdake | kfox1111 that would be one solution - not sure how to do that however | 15:51 |
| sdake | kfox1111 we can barely cache our own containers ;) | 15:51 |
| sbezverk | sdake: the images we use were built at least a week ago: http://logs.openstack.org/03/449803/7/experimental/gate-kolla-kubernetes-deploy-ubuntu-source-3-ironic-nv/c204716/logs/docker_images.txt | 15:52 |
| kfox1111 | sdake: we save off the rpms the same way we are saving off our containers in the periodic gate. | 15:52 |
| sdake | kfox1111 and that fixes the gate, however, if an operator deploys they end up with a non-cached version - brekaage | 15:52 |
| kfox1111 | would take a bit of work though. | 15:52 |
| sdake | sbezverk I am talking about the calico images... | 15:52 |
| kfox1111 | sdake: the gate's important to have a reliable gate. | 15:52 |
| sbezverk | sdake: there is no calico images | 15:53 |
| sdake | kfox1111 agree twe need a reliable gate | 15:53 |
| kfox1111 | so we can fix it the next day rather then have to imediately fix it to continue to make progress. | 15:53 |
| sbezverk | sdake: there is canal.yaml with flannel and calico binaries | 15:53 |
| sdake | ok- when were the calico binaries last updated? | 15:53 |
| sdake | (and canal binaries) | 15:54 |
| sbezverk | sdake: we use old version of binaries | 15:54 |
| *** zhubingbing_ has quit IRC | 15:54 | |
| sbezverk | sdake: latest are 0.5.1 but we still use 0.3.0 | 15:54 |
| kfox1111 | did they actually cahnge the images? | 15:54 |
| sdake | sbezverk can you prove they are old - i owuld have thought they come in containers | 15:54 |
| sbezverk | whcih are in the repo | 15:54 |
| sdake | kfox1111 no idea - that is what we need to validate | 15:54 |
| *** zhubingbing_ has joined #openstack-kolla | 15:54 | |
| sdake | kfox1111 i'd do it myself - although strugglign to come up to speedo n even getting kubernetes deployed as calico is busted | 15:55 |
| sdake | or canal | 15:55 |
| sdake | or wahtever it is we are using | 15:55 |
| sdake | CNI is broken | 15:55 |
| sbezverk | quay.io/coreos/flannel v0.7.0 63cee19df39c 10 weeks ago 73.75 MB | 15:55 |
| sbezverk | quay.io/calico/cni v1.5.5 ada87b3276f3 3 months ago 67.13 MB | 15:55 |
| zhubingbing_ | why don;t use fennel | 15:55 |
| sbezverk | I think it is not CNI | 15:55 |
| kfox1111 | have a log of the failure? | 15:55 |
| sbezverk | it is the host where it runs | 15:55 |
| kfox1111 | I can dig in a bit. | 15:56 |
| sbezverk | because as I said I had this morning completely green runs | 15:56 |
| sbezverk | kfox1111: the failure looked kind of similar to what we saw | 15:56 |
| sbezverk | before | 15:56 |
| sdake | dnsmask not going to 4/4? | 15:56 |
| zhubingbing_ | i am testing kubernetes v1.5.4 | 15:57 |
| sbezverk | canal is failing to find endpoint for etcd | 15:57 |
| sdake | if you do a pod describe in that state on dns-kube, it shows CNI failure | 15:57 |
| kfox1111 | 1.6 went from etc2 to 3 I think. | 15:57 |
| sbezverk | I think one of the reason they started using new cnala yaml is to use new etcd | 15:57 |
| sdake | kfox1111 yup that is correct | 15:57 |
| kfox1111 | anyone have a full gate log set? | 15:57 |
| kfox1111 | or should I submit a new test one? | 15:57 |
| sbezverk | so maube even though 1.5.5 is still "old" | 15:57 |
| sdake | kfox1111 i just woke up - i dont have one | 15:57 |
| sbezverk | it might expect to use "new" etcd | 15:57 |
| zhubingbing_ | kfox1111 yes | 15:58 |
| sdake | zhubingbing_ i am taking over my box for awhile | 15:58 |
| sdake | i shut down your vagrant vm | 15:58 |
| sdake | vagrant vms | 15:58 |
| zhubingbing_ | ok | 15:58 |
| kfox1111 | zhubingbing_: got a link? | 15:58 |
| sbezverk | kfox1111: on my last run there was no hard all red, there were several green | 15:59 |
| zhubingbing_ | https://github.com/kubernetes/features/blob/master/release-1.6/release-1.6.md | 15:59 |
| zhubingbing_ | but i usually flannel | 16:00 |
| kfox1111 | sbezverk: interesting. | 16:00 |
| *** eaguilar has joined #openstack-kolla | 16:00 | |
| sbezverk | that is why I think it not 100% kube or cni is related | 16:00 |
| sbezverk | I think the issue is on osic cloud | 16:00 |
| kfox1111 | could be a race, or a change in infra. | 16:00 |
| kfox1111 | we sould submit a bunch and see if there is a pattern. | 16:00 |
| sbezverk | we need to compare providers where we saw success and failure | 16:00 |
| kfox1111 | could just be a coincidence that it happend so closely to the canal change yesterday. | 16:01 |
| kfox1111 | +1 | 16:01 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Test gate https://review.openstack.org/450018 | 16:01 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Test gate2 https://review.openstack.org/450019 | 16:02 |
| sbezverk | kfox1111: just sent you email with complete green run from 10;46 am est | 16:02 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes master: WIP: Test gate3 https://review.openstack.org/450020 | 16:03 |
| kfox1111 | k. thakns. | 16:03 |
| zhubingbing_ | yesterday there was a problem with gate | 16:03 |
| zhubingbing_ | kfox1111 | 16:03 |
| kfox1111 | zhubingbing_: yeah. that was: https://review.openstack.org/#/c/449843/ | 16:04 |
| sdake | kfox1111 - calico has a kubernetes 1.6 yaml file now | 16:04 |
| kfox1111 | sdake: we pinned it back | 16:04 |
| sdake | kfox1111 yes but kubernetes is 1.6 in the gate... | 16:04 |
| kfox1111 | they cut 1.6? | 16:05 |
| sdake | if you read zhubingbing_ 's release schedule above, march 22 is 1.6.0 release | 16:05 |
| zhubingbing_ | no 1.5.5 | 16:05 |
| zhubingbing_ | yeah sdake +！ | 16:05 |
| zhubingbing_ | +1 | 16:05 |
| *** caoyuan has quit IRC | 16:05 | |
| sdake | whether 1.6.0 has been releaesd or is being used by kbueadm I dont' know | 16:05 |
| sdake | this is why i kicked zhubingbing_ off my box - so i could bring up kubeadm manually to find out :) | 16:06 |
| kfox1111 | it wasn't yesterday | 16:06 |
| zhubingbing_ | i know sdake | 16:06 |
| zhubingbing_ | ;) | 16:06 |
| sdake | there is some lag in a downstream (kubeadm) implementing changes from usptream (kubernetes 1.6.0 release) | 16:06 |
| sdake | best not to think about yesterday in open source because of this reason :) | 16:06 |
| kfox1111 | still only seing 1.5.x in the repo | 16:07 |
| sbezverk | where do you see 1.6?????? | 16:07 |
| zhubingbing_ | gate still user 1.5.5 | 16:07 |
| sbezverk | all logs are for 1.5.5 | 16:07 |
| zhubingbing_ | use | 16:07 |
| sdake | zhubingbing_ said 1.6.0 is released | 16:07 |
| zhubingbing_ | 1.60 have released | 16:07 |
| sdake | calico has 1.6.0 files | 16:07 |
| zhubingbing_ | yeah | 16:07 |
| kfox1111 | 1.6.0 isn't released yet. there's an rc1 | 16:07 |
| kfox1111 | we're not using calico. | 16:08 |
| kfox1111 | canal. | 16:08 |
| sdake | rather canal has a 1.6.0 release | 16:08 |
| sbezverk | sdake: you guys are confused ;) | 16:08 |
| sdake | sbezverk no shit | 16:08 |
| kfox1111 | for canal, they have a 1.5 and a 1.6 release set. | 16:08 |
| zhubingbing_ | you can testing kubeadm, it default deploy 1.6.0 | 16:08 |
| kfox1111 | they broke things by copying 1.6 to 1.5's yaml, | 16:08 |
| kfox1111 | but we pinned that back. | 16:08 |
| sdake | [09:08:34] <zhubingbing_>you can testing kubeadm, it default deploy 1.6.0 | 16:08 |
| sdake | zhubingbing_ please expand | 16:08 |
| zhubingbing_ | wait ;) | 16:09 |
| kfox1111 | 'yum clean all' | 16:09 |
| sbezverk | sdake: zhubingbing_: please ignore kubeadm version and kubectl version | 16:09 |
| sdake | 1.6.0 of what | 16:09 |
| kfox1111 | sudo yum list | grep kubeadm | 16:09 |
| kfox1111 | kubeadm.x86_64 1.6.0-0.alpha.0.2074.a092d8e0f95f52 | 16:09 |
| kfox1111 | rpm -qa | grep 1.6.0-0.alpha.0.2074.a092d8e0f95f52 | 16:09 |
| kfox1111 | kubeadm-1.6.0-0.alpha.0.2074.a092d8e0f95f52.x86_64 | 16:09 |
| zhubingbing_ | emm | 16:09 |
| kfox1111 | still the same kubeadm from yesterday. | 16:09 |
| sbezverk | they ahve been using 1.6 versions for these two for weeks if not month | 16:09 |
| kfox1111 | not sure why it would go from deploying 1.5 to 1.6 with the same binary. | 16:09 |
| sdake | kubeadm is on a different version track then kubernetese | 16:09 |
| kfox1111 | right. | 16:09 |
| zhubingbing_ | +! | 16:09 |
| kfox1111 | I don't think its actually using 1.6. | 16:10 |
| kfox1111 | lets see what pops out of the gate checks. | 16:10 |
| sdake | kfox1111 it is possible tha tthe same binary does deploy 1.6.0 - hard to say | 16:10 |
| kfox1111 | I think sbezverk's on to something. | 16:10 |
| kfox1111 | sdake: it can, but it should default to 1.5. it always has in the past. | 16:10 |
| kfox1111 | unless it prefers release candidtates. | 16:10 |
| zhubingbing_ | waiting gate log | 16:10 |
| zhubingbing_ | https://review.openstack.org/#/c/449979/ | 16:12 |
| sdake | https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/filelists.xml | 16:12 |
| sbezverk | sdake: whjere do you see 1.6 gets deployed?!?!?!? | 16:12 |
| sdake | sbezverk i haven't seen it - zhubingbing_ said it | 16:12 |
| sbezverk | sdake: but it is not correct | 16:12 |
| zhubingbing_ | i have been testing gate in 3 hours ago | 16:13 |
| sbezverk | why keep repeating what is not true?? | 16:13 |
| sbezverk | zhubingbing_: where do you see 1.6 other than kubeadm? | 16:13 |
| zhubingbing_ | i mean kubeadm use kubernetes version 1.5.5 but cni version is 1.6.0 | 16:14 |
| sdake | there we go. | 16:14 |
| sbezverk | zhubingbing_: kubeadm does not install any new software other than you already have installed with yup or appt-get | 16:14 |
| sbezverk | zhubingbing_: where do you see that cni is 1.6? | 16:14 |
| sdake | curls from all over the internet = winning | 16:15 |
| zhubingbing_ | wait gate log | 16:15 |
| sbezverk | Package kubernetes-cni.x86_64 0:0.3.0.1-0.07a8a2 | 16:15 |
| sbezverk | this is what we install from repo and it has been for a while now | 16:15 |
| sdake | <package pkgid="0c923b191423caccc65c550ef07ce61b97f991ad54785dab70dc07a5763f4222" name="kubernetes-cni" arch="x86_64"> | 16:16 |
| sdake | <version epoch="0" ver="0.3.0.1" rel="0.07a8a2"/> | 16:16 |
| sdake | that matches what is in the repofile | 16:16 |
| sbezverk | here is osic host where I see sucess run ubuntu-xenial-osic-cloud1-s3700 | 16:17 |
| sbezverk | ubuntu-xenial-osic-cloud1-s3500 failures | 16:17 |
| sdake | s3700 succeds then s35000 fails? | 16:18 |
| zhubingbing_ | http://status.openstack.org/zuul/ | 16:18 |
| kfox1111 | lets calm down, start from scratch, and see what fresh gate tests say. then debug from the ground up. | 16:19 |
| kfox1111 | somethings broken. not sure what. we'll find it. | 16:19 |
| zhubingbing_ | https://review.openstack.org/#/c/449803/ | 16:19 |
| zhubingbing_ | this patch testing look so good | 16:19 |
| kfox1111 | so far, 3 of the tests in the non changed gate tests passed. | 16:20 |
| zhubingbing_ | http://logs.openstack.org/18/450018/1/check/gate-kolla-kubernetes-deploy-ubuntu-binary-2-iscsi-nv/4cd7e35/console.html#_2017-03-26_16_04_24_154886 | 16:22 |
| zhubingbing_ | we still use 1.5.5 | 16:22 |
| sbezverk | sdake: that is what I see, but it would require more runs which kfox1111 has initiated.. | 16:23 |
| kfox1111 | another success | 16:24 |
| kfox1111 | and two more | 16:25 |
| zhubingbing_ | Gate now looks good | 16:25 |
| sbezverk | oh well, it must be a rising sun unfroze , a couple of frozen bits on osic cloud | 16:27 |
| *** manheim has joined #openstack-kolla | 16:28 | |
| zhubingbing_ | sbezverk what;s happened ? | 16:30 |
| sbezverk | zhubingbing_: it was an attempt of a joke | 16:31 |
| *** eaguilar has quit IRC | 16:31 | |
| zhubingbing_ | ;) | 16:32 |
| zhubingbing_ | now looks very good, I continue to do things | 16:33 |
| zhubingbing_ | https://review.openstack.org/#/c/449904/1 | 16:36 |
| zhubingbing_ | can u review it ? | 16:36 |
| sbezverk | 2017-03-25 09:59:47.830197 I | etcdmain: etcd Version: 3.1.4 | 16:39 |
| sbezverk | this is not normal | 16:39 |
| sbezverk | have you rebased your PS after kfox1111 pushed the change? | 16:40 |
| sbezverk | for the gate | 16:40 |
| sbezverk | it looks like the change we pushed is not in your ps | 16:40 |
| sdake | sbezverk you dont need to rebase dude - a recheck does the job | 16:41 |
| kolla-slack | <sbezverk> how then you explain the kfox1111 fix is not in his ps ;) ?? | 16:41 |
| kfox1111 | sbezverk: sdake's right. if you 'recheck', it does the recheck with a rebase. | 16:43 |
| kfox1111 | the recheck was done just recently though, so wasn't tested in his ps yet. | 16:43 |
| sdake | i can confirm running [sdake@kolla ~]$ kubeadm init --use-kubernetes-version v1.6.0-beta.1 followed with: [sdake@kolla ~]$ kubectl apply -f https://raw.githubusercontent.com/projectcalico/canal/master/k8s-install/kubeadm/1.6/canal.yaml results in this: http://paste.openstack.org/show/604196/ | 16:43 |
| sbezverk | url="https://raw.githubusercontent.com/projectcalico/canal/7deb07cda04147ab49115f437151705c747d0374" | 16:44 |
| sbezverk | url="$url/k8s-install/kubeadm/canal.yaml" | 16:44 |
| sdake | sbezverk that version of canal will definately not work with kubernetes 1.6.0 | 16:44 |
| sbezverk | zhubingbing_: make sure in your PS /tests/bin/setup_canal.sh | 16:44 |
| sbezverk | these two lines are present | 16:44 |
| sdake | vhosakot indicated he got canal running with kubernetes 1.6.0-beta1 iiuc | 16:44 |
| zhubingbing_ | ok | 16:44 |
| sbezverk | sdake: we do not use kube 1.6 | 16:44 |
| sbezverk | at the gate | 16:45 |
| sdake | sbezverk i get that | 16:45 |
| sdake | sbezverk I need an enviornment that works LOCALLY | 16:45 |
| sbezverk | then why keeping it up, we are fing gate now | 16:45 |
| sdake | at present the fact that at he gate is broken is being worked on by 4 people - why be a 5th wheel? :) | 16:45 |
| sdake | because I need an env that works locally | 16:45 |
| sdake | and i thought the information may be valuable when we do go to 1.6.0 | 16:45 |
| sdake | which will happen soon i suspect | 16:45 |
| kfox1111 | sdake: everyone that's trying to do things not like the gate, are surprised when it doesn't work. try to do things as much like the gate as possibloe, and you'll have much more chacne of success. | 16:45 |
| sdake | kfox1111 i've tried doing things like the gate does, which doesn't work | 16:46 |
| sbezverk | kfox1111: +1 | 16:46 |
| sdake | kfox1111 i've identified part of the problem - flannel depends on vxlan, which I don't have hardware in my personal lab for | 16:46 |
| kfox1111 | sdake: not things that t he gate does. everything the gate does. ;) | 16:46 |
| kfox1111 | sdake: vxlan is virtual. you don't need hardware for it. | 16:46 |
| sdake | kfox1111 i did *everything* the gate does, however, I don't have vxlan in my network | 16:46 |
| sbezverk | zhubingbing_: have you checked your setup_canal? | 16:46 |
| kfox1111 | vxlan support in hardware is just accelleration. | 16:47 |
| sdake | kfox1111 portdirect indicated vxlan is required in that default flannel setup | 16:47 |
| sdake | (hardware vxlan) | 16:47 |
| kfox1111 | sdake: it works in the gate. no hardware there. ;) | 16:47 |
| sbezverk | sdake: that is BS | 16:47 |
| sdake | kfox1111 you are wrong - the gate has hardware vxlan switches | 16:47 |
| sbezverk | hardware vxlan cannot be required | 16:47 |
| sdake | sbezverk well i dont know what is or isn't bs | 16:47 |
| sbezverk | in this case it would brake millions of installations | 16:47 |
| zhubingbing_ | sbezverk my environment is a bit more problematic | 16:48 |
| sdake | I am attempting to get something going locally | 16:48 |
| kolla-slack | <portdirect> @sdake you indicated that your switch would block xvlan? | 16:48 |
| sdake | and everything fails | 16:48 |
| kfox1111 | sdake: yah, but flannel cant use them. | 16:48 |
| sdake | portdirect not block, lacks | 16:48 |
| kfox1111 | sdake: trust me. vxlan's are workign software only. | 16:48 |
| kfox1111 | I'm doing it ipoib too. | 16:48 |
| kolla-slack | <portdirect> ^^ +2 | 16:48 |
| sbezverk | portdirect: have you really said that hardware vxlan is required?? | 16:48 |
| kolla-slack | <portdirect> lol no | 16:48 |
| kfox1111 | definitely no vxlan support there. | 16:48 |
| sbezverk | portdirect: thank god :) | 16:49 |
| zhubingbing_ | I have a problem if kubernetes update version , are we going to guarantee a stable version ？or continue update new version | 16:49 |
| sdake | portdirect said to try calico direclty instead of canal as canal has a bunch of complexity | 16:49 |
| kolla-slack | <portdirect> @sdake have you tried provisoning a new host - I really think that your machine may have been so through the mill thats its in a super weird state | 16:49 |
| kolla-slack | <portdirect> the fact that you cannot start any network underlay on just a single node indicated that somthing is really wrong | 16:50 |
| sdake | portdirect i'd like to understand what is broken not nuke and replace | 16:50 |
| sdake | portdirect other people will run into the same problem - need to explain to them how to recover their env - and most people can't typically reprovision their hardware | 16:51 |
| kfox1111 | sdake: I don't think cni's that safe yet. you install one and stick with it. otherwise, can get very snarled up. | 16:51 |
| sdake | here is my cleanup script: http://paste.openstack.org/show/604197/ | 16:52 |
| kolla-slack | <portdirect> @sdake considering all you've done to that thing it could be anywhere, and I'd recommend getting familiar with the technology before trying to deploy it on fubared stuff | 16:52 |
| kolla-slack | <portdirect> @sdake ther will also be stuff in /opt/cni and others | 16:53 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 16:57 |
| zhubingbing_ | check experimental what's mean ? | 16:58 |
| zhubingbing_ | who know this command mean ? | 16:59 |
| zhubingbing_ | sbezverk | 16:59 |
| sbezverk | zhubingbing_: the change I am playing is related to ironic experimental gate jobs | 16:59 |
| zhubingbing_ | ok | 16:59 |
| sbezverk | zhubingbing_: check experimental forces gate to run experimental jobs as well | 17:00 |
| zhubingbing_ | roger | 17:00 |
| zhubingbing_ | thanks sbezverk | 17:01 |
| zhubingbing_ | i going to sleep ，good byes guys | 17:03 |
| *** zhubingbing_ has quit IRC | 17:04 | |
| *** jascott1_ has joined #openstack-kolla | 17:06 | |
| sbezverk | zhubingbing_: good night | 17:07 |
| *** manheim has quit IRC | 17:07 | |
| *** yangyapeng has quit IRC | 17:08 | |
| *** Pavo has joined #openstack-kolla | 17:08 | |
| *** yangyapeng has joined #openstack-kolla | 17:09 | |
| *** jascott1_ has quit IRC | 17:12 | |
| sdake | portdirect /opt/cni was it | 17:12 |
| sdake | portdirect THNAKS | 17:12 |
| sdake | kube-dns-2924299975-6q14v 4/4 Running 0 3m | 17:13 |
| *** yangyapeng has quit IRC | 17:13 | |
| *** zhubingbing_ has joined #openstack-kolla | 17:14 | |
| *** manheim has joined #openstack-kolla | 17:19 | |
| *** lucasxu has joined #openstack-kolla | 17:22 | |
| *** lucasxu has quit IRC | 17:26 | |
| *** lamt has joined #openstack-kolla | 17:34 | |
| sdake | kfox1111 sbezverk i can confirm that canal 1.5 from that pinned version + mimicing the gate works on my local gear | 17:37 |
| kfox1111 | sdake: ok. cool. | 17:38 |
| sdake | i think that leads us to the conclusion that something in the infra has changed | 17:38 |
| sdake | I just deployed about 20 minutes ago | 17:38 |
| portdirect | sdake: whoop! We'll get you that 'I can set up K8s' sticker soon :) | 17:39 |
| sdake | portdirect i've got k8s setup | 17:39 |
| * portdirect makes sticker for fridge ;) | 17:39 | |
| sdake | portdirect i have had it setup for 3 months in an internal lab - just not on my external gear | 17:39 |
| kfox1111 | sdake: infra's working. | 17:41 |
| sdake | my internal gear uses ironic to provision nodes | 17:41 |
| kfox1111 | gates fine I mean. | 17:41 |
| kfox1111 | did 3 paralel sets at once. | 17:41 |
| kfox1111 | all 3 worked. | 17:41 |
| sdake | kfox1111 may have been some temporary thing | 17:41 |
| sdake | kfox1111 computers are hard ;) | 17:41 |
| kfox1111 | yeah. | 17:41 |
| kfox1111 | with that many clouds, ones bound to fail occationally. | 17:42 |
| kfox1111 | but seems fixed now. | 17:42 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 17:47 |
| sbezverk | kfox1111: man that ubuntu libvirt issue is really something :( | 17:47 |
| kfox1111 | sbezverk: :/ | 17:48 |
| kfox1111 | I struggled with ceph and it for a while too. :/ | 17:48 |
| sbezverk | kfox1111: I even added root into nova group inside of libvirt container, it became a bit better but still failing | 17:48 |
| sbezverk | kfox1111: going to get complete libvirtd.log and file an issue with libvirtd and/or ubuntu.. | 17:49 |
| kfox1111 | +1 | 17:49 |
| sbezverk | because it does not make sense when a user is in the group configured in qemu.conf | 17:50 |
| sbezverk | and it still gets permission failure | 17:50 |
| *** lamt has quit IRC | 17:50 | |
| sbezverk | kfox1111: also it looks like config merge of kolla-ansible in case of qemu.conf is not working.. | 17:51 |
| sbezverk | I will chat about it with Jeffrey4l tomorrow | 17:51 |
| kfox1111 | sbezverk: did we ever turn off apparmor in ubuntu? | 17:51 |
| kfox1111 | maybe its a problem? | 17:52 |
| sbezverk | kfox1111: not ubuntu expert, not sure how it is done | 17:52 |
| kfox1111 | me neither. dont know anything about it, other then its similar to selinux. | 17:52 |
| sbezverk | sudo systemctl stop apparmor.service | 17:53 |
| sbezverk | will give it ia try | 17:53 |
| kfox1111 | k | 17:54 |
| *** Pavo has quit IRC | 17:55 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 17:55 |
| *** unicell has joined #openstack-kolla | 17:57 | |
| *** unicell has quit IRC | 18:00 | |
| *** manheim has quit IRC | 18:02 | |
| *** eaguilar has joined #openstack-kolla | 18:02 | |
| adil452100 | Hello Kola users/devs .. Sorry to ask again | 18:02 |
| adil452100 | I'm thinking of deploying Kolla on 3 bare metal servers | 18:02 |
| adil452100 | but in a hyperconverged manner with HA | 18:03 |
| adil452100 | so the 3 servers will act as ceph/controller/compute nodes in the same time and in HA | 18:03 |
| adil452100 | That would be possible ? | 18:03 |
| *** manheim has joined #openstack-kolla | 18:03 | |
| sdake | adil452100 yup that is how i deploy kolla-ansible | 18:03 |
| *** zhubingbing_ has quit IRC | 18:06 | |
| *** unicell has joined #openstack-kolla | 18:07 | |
| *** jemcevoy has quit IRC | 18:08 | |
| *** manheim has quit IRC | 18:11 | |
| *** lucasxu has joined #openstack-kolla | 18:20 | |
| adil452100 | sdake> in production ? | 18:26 |
| adil452100 | sdake> any script or docs to share ;) | 18:27 |
| *** adil452100_ has joined #openstack-kolla | 18:30 | |
| *** adil452100 has quit IRC | 18:32 | |
| *** adil452100__ has joined #openstack-kolla | 18:45 | |
| *** adil452100_ has quit IRC | 18:48 | |
| *** eaguilar has quit IRC | 18:52 | |
| *** matrohon has joined #openstack-kolla | 19:03 | |
| openstackgerrit | Mohammed Naser proposed openstack/kolla-ansible master: Remove hardcoded signing_dir in keystone_authtoken https://review.openstack.org/450037 | 19:06 |
| mnaser | does anyone know how to show a bug as affecting other versions | 19:08 |
| mnaser | (ex affecting ocata/newton/etc) | 19:08 |
| *** dave-mccowan has joined #openstack-kolla | 19:18 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 19:20 |
| *** sayantan_ has joined #openstack-kolla | 19:22 | |
| *** fooliouno has quit IRC | 19:23 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla-kubernetes master: Fixing ironic deployment failure on Ubuntu https://review.openstack.org/449803 | 19:23 |
| *** sayanta__ has joined #openstack-kolla | 19:27 | |
| *** sayantan_ has quit IRC | 19:27 | |
| *** sayanta__ has quit IRC | 19:31 | |
| *** manheim has joined #openstack-kolla | 19:34 | |
| *** manheim has quit IRC | 19:36 | |
| *** adil452100 has joined #openstack-kolla | 19:41 | |
| adil452100 | sdake> in production ? | 19:42 |
| adil452100 | sdake> any script or docs to share ;) | 19:42 |
| adil452100 | thx in advance | 19:42 |
| *** adil452100__ has quit IRC | 19:44 | |
| *** lamt has joined #openstack-kolla | 20:06 | |
| adil452100 | Should I do it using Kolla Kubernetes or Kolla Ansible ? | 20:23 |
| adil452100 | Anyone ? | 20:23 |
| adil452100 | Kolla-Kubernetes multi-node deployment doesn't support HA ! | 20:24 |
| adil452100 | or use the native self-healing and auto-wiring benefits that a Kubernetes cluster has to offer | 20:25 |
| *** manheim has joined #openstack-kolla | 20:39 | |
| *** bjolo has joined #openstack-kolla | 20:44 | |
| bjolo | howdy | 20:45 |
| *** openstack has joined #openstack-kolla | 21:00 | |
| *** bjolo has quit IRC | 21:08 | |
| *** lamt has quit IRC | 21:11 | |
| *** manheim has quit IRC | 21:12 | |
| *** jascott1_ has joined #openstack-kolla | 21:18 | |
| *** bjolo has joined #openstack-kolla | 21:20 | |
| sdake | hey adil452100 | 21:21 |
| sdake | adil452100 at this piont in time if you want what youw ant, I'd recommend kolla-ansible deliverable - some people have put kolla-kubernetes in production although I feel it is a bit preamture as we haven't released 1.0.0 of that deliverable | 21:22 |
| *** jascott1_ has quit IRC | 21:23 | |
| *** lucasxu has quit IRC | 21:23 | |
| adil452100 | <sdake> I'm currently fighting with Kolla-Kubernetes :) The documentation doesn't really help cause I didn't play enough with kuberentes in the past. | 21:24 |
| sdake | adil452100 as to how to do kolla-ansible witht he approach you mentioned, that is pretty straightforard - documentedd at docs.openstack/openstack-dev/kolla-ansible | 21:24 |
| sdake | rahter docs.openstack.org/developer/kolla-ansible | 21:24 |
| adil452100 | <sdake> but I installed with success many times with kolla-ansible | 21:24 |
| sdake | adil452100 yup - i am a dev not an op - i have installed hundreds of times | 21:25 |
| *** bjolo has quit IRC | 21:25 | |
| sdake | if not thousands | 21:25 |
| sdake | https://docs.openstack.org/developer/kolla-ansible/quickstart.html | 21:25 |
| sdake | adil452100 my hundreds of times of experience + communities hundreds of times of experience is distilled into tht quickstart | 21:26 |
| sdake | and the rest of the docs on that site | 21:26 |
| *** jascott1_ has joined #openstack-kolla | 21:26 | |
| *** matrohon has quit IRC | 21:27 | |
| adil452100 | <sdake> great job | 21:28 |
| adil452100 | <sdake> please keep up :) | 21:28 |
| sdake | adil452100 thanks the community | 21:28 |
| sdake | strugglyg to keep up:) | 21:28 |
| adil452100 | A big thank to everyone contributing to this work ;) | 21:29 |
| *** lamt has joined #openstack-kolla | 21:31 | |
| *** adil452100 has quit IRC | 21:48 | |
| *** lamt has quit IRC | 21:53 | |
| *** lamt has joined #openstack-kolla | 22:09 | |
| *** lamt has quit IRC | 22:29 | |
| *** masber has left #openstack-kolla | 23:11 | |
| *** masber has joined #openstack-kolla | 23:11 | |
| *** yangyapeng has joined #openstack-kolla | 23:15 | |
| *** yangyapeng has quit IRC | 23:26 | |
| spsurya | morning all | 23:35 |
| *** manheim has joined #openstack-kolla | 23:40 | |
| *** Pavo has joined #openstack-kolla | 23:42 | |
| *** jrobinson has joined #openstack-kolla | 23:43 | |
| *** Pavo has quit IRC | 23:46 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!