Wednesday, 2017-04-19

*** eaguilar has quit IRC00:00
kfox1111inc0: what do you think of this:00:02
kfox1111 which pip 2>/dev/null && echo pip && pip freeze | sort; which dpkg 2>/dev/null && echo dpkg && dpkg -l | awk '{print $1, $2, $3, $4}' | sort -k 2; which rpm 2>/dev/null && echo rpm && rpm -qa | sort00:02
* inc0 parsing shell00:02
kfox1111if container versions without revisions match, compare those two to see if containers are diffrent and revision needs incremented.00:03
inc0so one thing00:03
inc0for source builds - I think it's pointless00:03
inc0commits go in every day00:03
kfox1111to pip?00:03
inc0we don't use pip during source build00:03
inc0not for main servie anyway00:03
kfox1111in ocata/newton/mitaka?00:04
kfox1111Trunk, yeah for sure.00:04
inc0mitaka is eol00:04
jascott1kfox1111 thanks00:04
jascott1oh that issue00:04
inc0so we maybe just build last tag of mitaka and leave it there00:04
kfox1111jascott1: yeah... a big long issue. :)00:04
jascott1the 1.6 explosion catch all00:05
kfox1111jascott1: yeah. :/ but a lot of good detail on what went wrong.00:05
jascott1was this not more correct than our approach? https://github.com/kubernetes/kubernetes/issues/43815#issuecomment-29234190200:05
kfox1111inc0: do you think the stable releases really get updated that frequently?00:05
inc0kfox1111: https://github.com/openstack/neutron/commits/stable/newton00:06
kfox1111jascott1: for the permission thing?00:06
inc0every day00:07
jascott1ya00:07
jascott1AllowsAllow over the unauthenticated00:07
jascott1guess its minor issue for dev env00:07
kfox1111inc0: interesting.... so really, source should be tracking stable release tags, not the tip of the branch?00:07
inc0I think we should have option for both00:07
inc0security patches..00:08
jascott1wouldnt want to propagate a less secure anything if there is a better solution within reason00:08
kfox1111jascott1: yeah. we put the hack in just to get things going again in the gate. totally believe its not the right solution. :)00:08
inc0tags are every few weeks, you don't want to wait that long00:08
*** chas has joined #openstack-kolla00:08
kfox1111inc0: +1 to having both. a -source-stable, and a -source-tip or something?00:08
kfox1111every few weeks would track with binary packages.00:09
kfox1111agreed that we might not want to deal with revisions and stable-tip's.00:09
*** eaguilar has joined #openstack-kolla00:10
inc0speaking of which, this need fixing https://github.com/openstack/kolla/blob/stable/ocata/kolla/common/config.py#L44200:11
inc0also I took a look at barbican, and 4.0.0 tag is newest from Feb00:11
*** rwellum has quit IRC00:11
inc0damn our version management needs a lot of work00:12
*** chas has quit IRC00:13
inc0so we have few things to handle here00:16
inc01- tip of branch, pretty straighforward00:17
kfox1111inc0: +1.00:17
jascott1hey kfox1111 what about docker diff on the containers?00:17
kfox1111the version stuff really hurts. :/00:17
inc02- latest released tag from repo00:17
inc0harder stuff :(00:17
kfox1111jascott1: the concern is, there could be a lot of random things that change that are not important.00:17
jascott1but we could ignore00:18
kfox1111yeah.00:18
jascott1yea im saying add that to what you are doing00:18
jascott1haha00:18
jascott1do more work! twwfpt00:18
kfox1111indeed. :)00:18
kfox1111"more's always better"  :)00:18
kfox1111hmm.. docker diff does look like it only shows changes between layers?00:19
kfox1111not sure if it even can be used to compare two differently rooted containers, which would be always the case.00:19
jascott1ah00:19
jascott1 i forget the complex image building00:20
kfox1111inc0: so, not considering the stable-tip case, do you think that fingerprinting command would gather what info we would need?00:20
inc0what I'm concerned with this is that lots of things get changed00:21
inc0each service has a lot of deps00:21
kfox1111yeah. but if anyu of them change, it really should be a different container.00:21
kfox1111tag00:21
inc0we share base image so if something changes in openstack-base, which holds a lot of deps00:22
inc0everything gets rebuilt00:22
jascott1contents hash and date and some diff along the lines kfox1111 is doing might be workable00:22
kfox1111inc0: ah. yeah, that could probably use a thining out.00:22
inc0what I'm saying is, if we rebuild and nothing changes, nothing gets uploaded00:22
inc0because layers in docker are checksums00:22
kfox1111inc0: yeah. and thats what we need the fingerprinting for.00:22
inc0so we have "fingerprinting" already00:23
jascott1hash says its different. date helps figure out what world liked like wehn it was built. and then the change long to say which packages changed00:23
kfox1111so that if a log file or install thingy or whatever drops a 'date > foo' then it doesn't trigger the container to be different.00:23
kfox1111inc0: how so?00:23
kfox1111no a checksum is not enough for a fingerprint.00:23
kfox1111its way way too touchy.00:23
kfox1111the container didn't really change if all it did was have date > /var/log/whatever.00:24
kfox1111we only care if software changed in it.00:24
inc0well still, base is rebuilt - we rebuild whole set00:24
kfox1111the docker checksum changes if any byte on any file changes, including stuff we really don't care about.00:24
kfox1111inc0: true. but how often do we think that will happen?00:25
kfox1111Its less then ideal, for sure. that can be fixed seperately though.00:25
inc0not sure, I'd bet on quite often00:25
kfox1111I'd bet no more frequently then once a week.00:25
inc0but that's researchable00:25
kfox1111I'd guess its more like once a month.00:25
inc0kfox1111: if you can crack out rudimentary manifesto generating shellscript00:26
inc0we could put it into gates and log it after gate build00:26
kfox1111inc0: yeah. thats the shell snippet from above.00:26
inc0gate run00:26
kfox1111ah. that would be interesting. yeah.00:26
kfox1111does kolla have a periodic gate for binary container builds yet?00:27
inc0let's add this to deploy gates run in kolla gates00:27
inc0every commit also gates on binary00:27
kfox1111I could stick image building in the kolla-kubernetes periodic gate.00:27
inc0it's just not voting that's all00:27
kfox1111well, we need to gather the fingerprints somehow too to mine it.00:28
kfox1111if it was in a periodic gate, we could upload them to tarballs.o.o.00:28
*** eanylin has joined #openstack-kolla00:28
inc0infra keeps logs from changes for quite a while00:28
kfox1111inc0: true. just not sure how to mine it out of review's.00:28
inc0we just need to look at them and download file output every now and then00:28
*** chas has joined #openstack-kolla00:29
inc0same as we publish logs...00:29
kfox1111hmm... yeah, that might work.00:29
inc0or even dump it to stdout00:29
kfox1111put it in $LOGS/container_prints or something.00:29
*** duonghq has joined #openstack-kolla00:29
inc0exactly00:29
kfox1111then can easily fetch them by filename.00:29
duonghqkfox1111, are you there?00:29
inc0yup00:29
kfox1111inc0: k. I'll post a ps for that.00:29
inc0thanks00:29
kfox1111duonghq: yeah. whats up?00:29
duonghqhi inc0 kfox111100:29
inc0you are better at gates than myself00:30
inc0good afternoon Duong00:30
kfox1111I know nothing about the kolla gate. but can take a stab at it.00:30
duonghqkfox1111, I have a small question about my ps, let me know when you free00:30
*** huzhengchuan has joined #openstack-kolla00:30
kfox1111duonghq: nows a good time. whats up?00:30
inc0well I gtg now, but I'll be happy to help tomororrow00:30
kfox1111inc0: uh... one issue...00:31
inc0go ahead00:31
duonghqkfox1111, https://review.openstack.org/#/c/437252/9/tests/bin/basic_tests.sh@181 about the last line, what is it for?00:31
kfox1111that would be against trunk builds. and we know those will change a lot?00:31
kfox1111or is this just on base?00:31
inc0kfox1111: good point, we'd need that change in stable branch I guess00:31
inc0but I think we can do it for ocata at least00:32
inc0will give us idea anyway00:32
kfox1111duonghq: it belongs to the test right before your changes. what that test does, is00:32
kfox1111create a cinder volume on vm1, write a file to it. moves the cinder volume to vm2, and verify the data in it is the same as was written.00:32
kfox1111the diff does the actual compare of the data written to what was read back out.00:33
inc0for glory of science, I'll allow this backport:P00:33
duonghqkfox1111, understood, thanks00:33
kfox1111duonghq: np. :)00:34
*** chas has quit IRC00:34
*** tervancovan has joined #openstack-kolla00:34
inc0ok, I'm going off00:34
inc0talk to y'all tomorrow00:35
duonghqinc0, cya00:35
*** inc0 has quit IRC00:35
*** huzhengchuan has quit IRC00:37
spsuryakfox1111: small item for you https://review.openstack.org/#/c/457614/00:37
*** sayantan_ has joined #openstack-kolla00:38
spsuryaduonghq  one more https://review.openstack.org/#/c/451654/00:38
kfox1111spsurya: looks good. :)00:39
spsuryakfox1111 one more https://review.openstack.org/#/c/451654/00:39
kfox1111done. thanks for fixing that. :)00:39
kfox1111gotta head out for a while. bbl. have a good one. :)00:39
spsuryakfox1111 thanks for review00:40
kfox1111np. thanks for the ps. :)00:41
*** openstackgerrit has joined #openstack-kolla00:43
openstackgerritMerged openstack/kolla-kubernetes master: Deployment guide updated with few more information  https://review.openstack.org/45761400:43
openstackgerritMerged openstack/kolla-kubernetes master: README.rst character improvement  https://review.openstack.org/45165400:44
*** david-lyle has joined #openstack-kolla00:44
*** huzhengchuan has joined #openstack-kolla00:49
*** sayantan_ has quit IRC00:50
*** chas has joined #openstack-kolla00:50
*** cuongnv has joined #openstack-kolla00:51
*** chas has quit IRC00:54
openstackgerritDuong Ha-Quang proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725200:57
tervancovanHi01:04
*** StephenWang1991 has joined #openstack-kolla01:10
*** yangyapeng has joined #openstack-kolla01:15
openstackgerritChen proposed openstack/kolla master: Use LABEL instead of MAINTAINER (deprecated) in all Dockerfile.j2  https://review.openstack.org/45751301:18
tervancovanCould somebody tell me your own dev environment include kubernetes  , in  my case I am trying to  openstack-helm's kubeadm-aio for kolla-kubernetes01:20
*** caowei has joined #openstack-kolla01:21
openstackgerritTaejeoung Kim proposed openstack/kolla-kubernetes master: fix error on document  https://review.openstack.org/45782901:24
*** hrw has quit IRC01:31
*** MarginHu2017 has joined #openstack-kolla01:31
*** iceyao has joined #openstack-kolla01:31
*** chas has joined #openstack-kolla01:31
*** hrw has joined #openstack-kolla01:32
*** zhubingbing has joined #openstack-kolla01:34
*** goldyfruit has joined #openstack-kolla01:36
*** chas has quit IRC01:36
*** japestinho has joined #openstack-kolla01:40
*** iceyao has quit IRC01:45
*** zhubingbing has quit IRC01:48
spsuryasdake  http://paste.openstack.org/show/607093/01:49
spsuryahelm installation01:49
spsuryastep 301:49
sdakespsurya your running asa the root user01:49
sdake /usr/local/bin is notin the root user's path for security reasons01:50
spsuryaohhh01:50
sdakespsurya when i create a vagrant box using your vagrant scipt it logs me in not as teh root user01:50
spsuryayou can do that01:51
spsuryasudo su01:51
spsuryaone more step to be root01:51
spsuryajust after logged in01:51
*** chas has joined #openstack-kolla01:52
sdakespsurya the guide doesn't say sudo -i :)01:53
sdakeor suod su01:53
sdakethe guide is meant to be run as a local user01:53
sdakewith passwordless sudo01:53
*** chas has quit IRC01:57
spsuryasdake  r0ger01:57
spsuryagot thhis time01:57
*** iceyao has joined #openstack-kolla02:01
*** zhubingbing_ has joined #openstack-kolla02:03
*** ssurana has quit IRC02:03
spsuryabut tiller heads up :(02:04
spsuryahttps://www.irccloud.com/pastebin/1eHLocWA/02:04
spsuryasdake02:04
spsuryathough helm is configured02:04
sdakehmm02:04
sdakei ahve actually never looksd at the get script02:04
sdakecan you run kubeadm get pods -n kube-system02:05
spsuryayes ran02:05
spsuryarunning fine02:05
sdake-> paste02:05
sdaketiller is running?02:05
sdakehow about helm version02:05
spsuryahttps://www.irccloud.com/pastebin/yas2ZX5i/02:06
sdakeno tiller running02:06
sdakerun helm version -> paste02:07
spsuryaHelm v2.3.102:08
sdakerun helm init02:08
spsuryaran connection refused02:09
spsuryahttps://www.irccloud.com/pastebin/MsRZucwZ/02:09
sdakeyou are no long errunning in the root creds02:09
spsuryayes02:10
sdakeso you need to do the step which copies over the creds fiel in /etc/kubernetes/config02:10
spsuryano root02:10
sdakeadmin.conf that is02:10
sdakemkdir -p $HOME/.kube02:10
sdakesudo -H cp /etc/kubernetes/admin.conf $HOME/.kube/config02:10
sdakesudo -H chown $(id -u):$(id -g) $HOME/.kube/config02:11
spsuryaalready followed in Step 102:12
sdakeyou ran it as root02:12
spsuryayes02:12
sdakeit copied it to /root/.kube.config02:12
sdakenot /spurya/.kube/config02:12
sdakeor wahtever your login is02:12
spsuryaohhh02:12
sdakecould probably use a doc change that these instructions are meant to be run as the non-root user and explain why02:13
sdakebecause tiller cannot be found in /usr/local/bin02:13
*** chas has joined #openstack-kolla02:13
spsuryanow worked when copied as non root as well02:14
spsuryaand ran02:14
spsuryahelm init02:14
spsuryanow tiller got the tiller too02:15
spsuryahttps://www.irccloud.com/pastebin/BWaXaTss/02:15
spsuryanoted to updated the doc for this non root running and the reason02:16
*** unicell has quit IRC02:17
spsuryaI think i should run again02:17
spsuryadestroy and up02:17
sdakenah your good to go02:18
*** chas has quit IRC02:18
spsuryak8s setup02:18
sdakejust proceed from here02:18
spsuryaok02:18
*** huzhengchuan has quit IRC02:18
*** yangyapeng has quit IRC02:21
*** jayahn has joined #openstack-kolla02:26
*** schwicht has quit IRC02:29
spsuryasdake as i ran into error unable to execute gcc02:30
spsuryashould i include it into guide02:30
spsuryaso that user go02:31
*** huzhengchuan has joined #openstack-kolla02:31
spsuryauniterupted02:31
sdakespsurya yum groupinstall "Development Tools"02:32
sdakespsurya yum -y groupinstall "Development Tools"02:33
spsuryathat needs to be in doc too02:34
*** chas has joined #openstack-kolla02:34
spsuryasdake right ?02:34
portdirectHey tervancovan, i wrote the kubeadm-aio container in openstack-helm, there's nothing special about the k8s it installs: so you should be able to run kolla-k8s ontop of it no problem unless it has specific requirements above and beyond vanilla k8s.02:34
sdakespsurya indeed02:37
sdakespsurya probably separate patches02:37
sdakeon efor that and one for root02:37
sdakeportdirect you sure read far back in the scrollback :)02:38
portdirectLol, only an hour:20.02:38
*** chas has quit IRC02:38
portdirectThough be good to discuss strategy for Dev/poc moving forward02:39
portdirectI'm pretty sure we can come up with something pretty awesome together :)02:40
spsuryaok02:40
*** unicell has joined #openstack-kolla02:40
sdakeportdirect that may be - i'd like new contribs to come up with the approach rather then the core reviewers defining a strategy02:40
sdakestrategy fails for many reasons02:41
*** huzhengchuan has quit IRC02:41
openstackgerritDuong Ha-Quang proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725202:42
portdirectYour telling me ;) organic growth wins every time02:42
sdakei guess it depends on your definition of organic :)02:43
sdakeanyway busy helping others have no time for debate02:43
portdirectAll good brother, if there's anything I can help with just holler :)02:44
*** dave-mccowan has quit IRC02:47
sdakespsurya i'm headed to bed soon02:52
sdakespsurya where ya at in the instructions :)02:52
spsuryaok02:52
spsuryaupdating the guide02:53
sdakedid you get through the deploy guide sucessfullY?02:53
spsuryathen proceed further02:54
spsuryajust copied the  kolla and k8s files toetc02:54
spsuryaetc/kolla and etc/k8s02:54
sdakeok well get throug hthe guide02:54
sdakei'll wait up02:54
sdakeand submit patches after :)02:55
sdakethat way I can help ya out02:55
spsuryaok02:55
*** chas has joined #openstack-kolla02:55
sdakelegimately help you out02:55
spsurya:)02:56
*** huzhengchuan has joined #openstack-kolla02:56
*** lamt has joined #openstack-kolla02:59
*** chas has quit IRC02:59
*** unicell has quit IRC03:04
spsuryasdake just want to be precaucious... As in my case vm has ip 192.168.121.163 at eth0. So i should put kolla internal ip  192.168.121.163, network interface as eth0 and neutron external interface to eth1 as per below $ip addr03:09
spsuryahttps://www.irccloud.com/pastebin/9va4X5zK/03:09
*** iceyao has quit IRC03:10
spsuryain globals.yml03:10
*** MasterOfBugs has quit IRC03:11
sdakespsurya where is your eth1? :)03:12
spsuryano eth1 ;)03:12
sdakeright03:12
sdakeneutron needs an eth1 to operate03:12
sdakethere is probably a way to handle that - I think the easiest is to change your vagrant file to export a second interface03:13
sdakespsurya as to your quesiton, network_iterface hsould be 192.168.121.6303:13
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Add fwaas extension in l3 agent when neutron fwaas is enabled  https://review.openstack.org/45784603:14
*** shasha___ has joined #openstack-kolla03:15
spsuryasdake: or i can create eth103:15
*** chas has joined #openstack-kolla03:16
sdakespsurya how ?03:16
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Cleanup /run/netns host directory volume mount  https://review.openstack.org/45784803:17
*** shasha___ has quit IRC03:18
*** kencjohnston_ has quit IRC03:19
spsuryasdake: with  system-config-network03:19
sdakeok if that works03:19
sdakehaven't used that toolbefore03:19
spsuryavia creating new device and assigning new address03:19
*** kencjohnston has joined #openstack-kolla03:19
spsuryai will check let you know03:20
*** chas has quit IRC03:20
portdirectspsurya: a dummy device should be ok for dev/testing03:24
spsuryaportdirect yes, it should03:26
portdirecthttps://www.irccloud.com/pastebin/ATHFWoTq/03:28
portdirectshould be all you need i think?03:28
spsuryayes03:32
spsuryathanks03:32
*** lamt has quit IRC03:34
spsuryaportdirect:  i  think it will goes away once get down03:35
spsuryai.e vm reboot03:35
portdirectyup03:35
spsuryaneed to keep it in file03:36
spsuryamore reliable03:36
spsuryafor now your one is good03:36
*** chas has joined #openstack-kolla03:37
*** iceyao has joined #openstack-kolla03:37
portdirectspsurya: makes sense, though I'm not sure if theres a PS for the vagrant env your working on up atm, so just spitballing options :)03:39
spsurya:)03:40
spsuryanice03:41
*** yangyapeng has joined #openstack-kolla03:41
*** chas has quit IRC03:41
*** lamt has joined #openstack-kolla03:41
*** iceyao has quit IRC03:42
*** hieulq_ has joined #openstack-kolla03:42
*** gkadam has joined #openstack-kolla03:43
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Bump OpenStack Service Version  https://review.openstack.org/45785103:43
*** gkadam is now known as Guest7939403:43
*** Guest79394 has quit IRC03:44
*** gkadam_ has joined #openstack-kolla03:44
*** gkadam_ is now known as gkadam03:45
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Bump OpenStack service version  https://review.openstack.org/45785103:47
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Bump OpenStack service version  https://review.openstack.org/45785103:52
openstackgerritzhubingbing proposed openstack/kolla-ansible stable/ocata: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong  https://review.openstack.org/45785603:54
*** bmace has quit IRC03:54
openstackgerritMerged openstack/kolla-ansible master: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong  https://review.openstack.org/45750503:57
*** chas has joined #openstack-kolla03:58
*** eaguilar has quit IRC03:58
*** chas has quit IRC04:02
*** goldyfruit has quit IRC04:03
sdakespsurya i've hit the wall dude - headed to bed :)04:07
spsuryanp04:07
sdakespsurya I think you should be set now - you have worked thorugh all the deployment guide issues you ahve found04:07
spsuryasdake04:07
spsuryai will update in guide04:08
sdakespsurya not sure how to handle the second interface - I am sure it is solveable in your environment - google probely holds the answer :)04:08
sdakethanks04:08
sdakenight all04:08
spsuryagot the eth1 with no ip that what i required04:08
spsuryathanks04:08
spsuryanight04:09
*** yangyapeng has quit IRC04:16
tervancovanThanks portdirect04:17
*** yangyapeng has joined #openstack-kolla04:17
*** chas has joined #openstack-kolla04:18
tervancovanBut to use kolla-k8s We have to set CIDR in kubeadm-aio04:19
*** iceyao has joined #openstack-kolla04:21
*** salv-orlando has joined #openstack-kolla04:23
*** chas has quit IRC04:23
openstackgerritMerged openstack/kolla-kubernetes master: Version 0.7.0  https://review.openstack.org/45765704:25
*** iceyao has quit IRC04:26
*** lamt has quit IRC04:26
*** salv-orlando has quit IRC04:28
*** sudipto has joined #openstack-kolla04:39
*** chas has joined #openstack-kolla04:39
*** hieulq_ has quit IRC04:40
*** chas has quit IRC04:44
*** skramaja has joined #openstack-kolla04:46
openstackgerritDuong Ha-Quang proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725204:47
*** shasha___ has joined #openstack-kolla04:54
*** salv-orlando has joined #openstack-kolla04:56
*** hieulq_ has joined #openstack-kolla04:57
*** shasha___ has quit IRC04:58
*** hieulq_ has quit IRC04:58
*** chas has joined #openstack-kolla05:00
*** lamt has joined #openstack-kolla05:01
*** iceyao has joined #openstack-kolla05:01
*** unicell has joined #openstack-kolla05:04
*** chas has quit IRC05:05
*** lamt has quit IRC05:05
*** iceyao has quit IRC05:06
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Deployment guide updation for better followup  https://review.openstack.org/45787005:14
*** tovin07_ has joined #openstack-kolla05:26
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Development Tools package installation added in deployment guide  https://review.openstack.org/45787105:28
portdirecttervancovan: though undocumented you can set the default device/ip to bind to when launching the container using the KUBE_BIND_DEV env var. I'll try to add customisation of the CNI CIDR in the next few days. :)05:28
tervancovanthanks portdirect05:34
tervancovanI just check that env var05:35
*** zhurong has joined #openstack-kolla05:42
*** chas has joined #openstack-kolla05:42
tervancovanCan you share your CIDR  configuration for running kolla-k8s in kubeadm-aio container?05:44
tervancovanportdirect?05:44
*** iceyao has joined #openstack-kolla05:46
*** chas has quit IRC05:46
*** iceyao has quit IRC05:51
*** Serlex has joined #openstack-kolla06:03
*** unicell1 has joined #openstack-kolla06:03
*** unicell has quit IRC06:03
*** salv-orlando has quit IRC06:04
openstackgerritzhubingbing proposed openstack/kolla-ansible stable/ocata: neutron_vpnaas_agent: kolla_docker parameter 'volumes' is wrong  https://review.openstack.org/45787906:10
*** yangyapeng has quit IRC06:15
*** dvx76 has quit IRC06:15
*** chas has joined #openstack-kolla06:24
tervancovanportdirect After apply "canal CNI driver" in kubeadm-aio container -> Canal pods were in "ClashLoopBackoff" -> https://thepasteb.in/p/lOhOx4lpROJFB06:28
*** iceyao has joined #openstack-kolla06:28
*** chas has quit IRC06:28
*** StephenWang1991 has quit IRC06:33
*** pcaruana has joined #openstack-kolla06:33
*** iceyao has quit IRC06:33
Jeffrey4lduonghq, could u review https://review.openstack.org/45784806:34
duonghqJeffrey4l, mins06:37
Jeffrey4lthanks :)06:37
duonghqdone ;)06:37
duonghqhope that some day we can get more stable gate06:38
Jeffrey4laha, ;) gate is hard06:40
duonghqyes06:40
openstackgerritMerged openstack/kolla-ansible stable/ocata: Cleanup /run/netns host directory volume mount  https://review.openstack.org/45784806:41
*** lamt has joined #openstack-kolla06:44
*** lamt has quit IRC06:44
*** chas has joined #openstack-kolla06:44
Jeffrey4lzhubingbing_, please review https://review.openstack.org/45785106:47
*** chas has quit IRC06:49
*** yangyapeng has joined #openstack-kolla06:49
*** jaosorior has joined #openstack-kolla06:53
*** chas has joined #openstack-kolla06:56
*** jayahn has quit IRC07:00
*** matrohon has joined #openstack-kolla07:02
*** MasterOfBugs has joined #openstack-kolla07:03
*** manheim has joined #openstack-kolla07:08
*** manheim has quit IRC07:08
*** manheim has joined #openstack-kolla07:08
*** matrohon has quit IRC07:08
*** athomas has joined #openstack-kolla07:09
*** StephenWang1991 has joined #openstack-kolla07:11
*** iceyao has joined #openstack-kolla07:12
*** tovin07 has joined #openstack-kolla07:12
*** shardy has joined #openstack-kolla07:14
zhubingbing_ok07:17
*** salv-orlando has joined #openstack-kolla07:17
zhubingbing_done07:18
*** iniazi has quit IRC07:29
*** egonzalez has joined #openstack-kolla07:38
*** mgoddard has joined #openstack-kolla07:43
*** blallau has joined #openstack-kolla07:46
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible stable/ocata: Set fluentd output to elasticsearch to use elasticsearch_address  https://review.openstack.org/45790707:52
*** matrohon has joined #openstack-kolla07:53
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible stable/ocata: Fix panko mongo idempotency  https://review.openstack.org/45790807:54
*** jascott1 has quit IRC07:56
*** iceyao has quit IRC07:56
*** jascott1 has joined #openstack-kolla07:57
sbezverkJeffrey4l: ping07:58
*** magicboiz has joined #openstack-kolla07:59
Jeffrey4lsbezverk, pong.08:01
*** athomas has quit IRC08:02
*** jascott1 has quit IRC08:02
*** matrohon_ has joined #openstack-kolla08:03
sbezverkJeffrey4l: thanks but I found my misconfig :) sorry to bug you08:04
Jeffrey4lnp ;)08:05
*** matrohon has quit IRC08:05
*** mgoddard has quit IRC08:07
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: [TEST][OCATA][KOLLA-ANSIBLE] TEST HEAT RELATED SERVICES  https://review.openstack.org/43463608:08
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: [TEST][OCATA][KOLLA-ANSIBLE] TEST CEILOMETER RELATED SERVICE  https://review.openstack.org/43463708:08
*** zhubingbing_ has quit IRC08:14
*** zhubingbing_ has joined #openstack-kolla08:23
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: By default neutron interface corrected in guide  https://review.openstack.org/45792208:26
*** gfidente has joined #openstack-kolla08:29
openstackgerritMerged openstack/kolla stable/ocata: Bump OpenStack service version  https://review.openstack.org/45785108:34
*** MasterOfBugs has quit IRC08:37
openstackgerritChen proposed openstack/kolla master: Use LABEL instead of MAINTAINER (deprecated) in all Dockerfile.j2  https://review.openstack.org/45751308:39
*** mgoddard has joined #openstack-kolla08:42
*** sambetts|afk is now known as sambetts08:52
*** zhubingbing_ has quit IRC08:54
*** zhubingbing has joined #openstack-kolla08:57
*** g3ek has quit IRC09:00
*** britthouser has quit IRC09:01
*** britthouser has joined #openstack-kolla09:01
hrwzhubingbing: which distros are supported by grafana image? Ubuntu one fails terribly09:04
*** g3ek has joined #openstack-kolla09:09
*** zhugaoxiao has joined #openstack-kolla09:13
*** rmart04 has joined #openstack-kolla09:16
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: grafana: disable for non-x86  https://review.openstack.org/45794609:18
hrwsbezverk: can you update helm to 2.2.0? it would then be buildable for non-x86 targets too09:22
sbezverkhrw: we are currently using 2.3.0 and we cannot downgrade it to 2.2.009:23
sbezverkdue to some features we use09:23
hrwsbezverk: hm have to update probably as Dockerfile.j2 here says 2.1.309:23
sbezverkhrw: helm does not come in docker, well at least we do not use one09:25
hrwsbezverk: ok, I meant helm-repository one. srry09:25
sbezverkhrw: we install helm as a part of deployment process onto kubernetes cluster09:25
sbezverkhrw: oh, that one for sure09:26
sbezverkhrw: please use version 2.3.0 to be more consistent09:26
hrwsbezverk: so far I only build so no idea how to test 2.1.3->2.3.0 update for being ok09:27
hrwsbezverk: I can send one patch which will bump to 2.3.0 and then on top of it one to make it buildable on non-x86. ok?09:29
sbezverkhrw: sounds good09:31
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: helm-repository: update to v2.3.0  https://review.openstack.org/45796709:34
hrwand now let's try that on aarch64 ;D09:36
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Reparent kolla-toolbox from openstack-base  https://review.openstack.org/43502309:42
*** salv-orl_ has joined #openstack-kolla09:48
hrwINFO:kolla.image.build.helm-repository:Happy Helming!09:48
hrw;D09:49
hrwon aarch6409:49
*** salv-orlando has quit IRC09:51
japestinhoHey folks, I am able to deploy kolla-k8s AIO on my centos-7 desktop machine. the problem is I can't ping or ssh VM address although I can assign floating IP to VM. My AIO host address 10.10.10.13/24, pod-network-cidr=10.11.0.0/16 service-cidr=10.13.3.0/24, public net openstack address is same with AIO host network 10.10.10.0/24, and private openstack address09:51
japestinho10.10.11.0/2409:51
japestinhois that correct I use same address for public net on openstack with AIO host network address?09:52
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix cinder-api ubuntu binary  https://review.openstack.org/45798209:52
sbezverkhrw: so now you can helm on your phone ;) ??09:53
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: helm-repository: make it work on non-x86  https://review.openstack.org/45798309:53
hrwsbezverk: My arm64 box is normal server09:53
hrwsbezverk: 8 cores, few sata disks, pci express, 10GbE network ports09:53
sbezverkhrw: ok got it..09:53
hrwsbezverk: there are servers with 48cores/cpu (up to 2 cpus per board), 1TB ram per cpu...09:54
*** zhubingbing has quit IRC09:57
*** jascott1 has joined #openstack-kolla09:59
*** Seyong has quit IRC10:00
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy  https://review.openstack.org/45799310:01
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy  https://review.openstack.org/45799310:03
hrwsbezverk: you convinced me to finally write a post on my blog ;D10:04
*** jascott1 has quit IRC10:05
sbezverkhrw: sure, good idea, until you mentioned I was under impression arm processors were meant for something really small10:05
gemasbezverk: they are, but there are also server chips10:06
hrwsbezverk: 32bit arm had some desktops in past. There were servers as well. Some high density ones too (64 quad core systems in 2-4U). But it was never popular.10:06
gemasbezverk: so we are working on bringing the new AArch64 servers up there in terms of software supported with the x86_64 ones10:08
gemasbezverk: it really feels no different than an intel machine10:08
hrwand as a side effect I get kolla working on ppc64le as well10:08
hrwhave access to some power8 machine with 128GB ram and 80 cpu threads10:09
sbezverkgema: curious is MS supports this arch as well?10:09
hrwsbezverk: yes10:09
sbezverkhrw: 80 cpu threads sounds nice..10:10
hrwsbezverk: they presented MS Windows running Azure load10:10
hrwsbezverk: no idea will they make desktop version10:10
hrwsbezverk: x86-64 feels limited when you work with other architectures ;d10:10
hrw8/16 systems (cpu/threads) compared to 10/80 or 8-48-240 cores10:11
*** MarginHu2017 has quit IRC10:12
*** cuongnv has quit IRC10:13
sbezverkhrw: I have quad server each with 36 threads so I would not say it is too limited ;)10:14
hrw;D10:14
*** yangyapeng has quit IRC10:16
hrwsbezverk: I spoke with sdake recently about which aarch64 server to buy. The main problem was to find one with 1GbE port as it usually means extra cost because SoC does not support anything <10GbE so you need extra pcie controller10:16
*** tovin07_ has quit IRC10:19
sbezverkhrw: I see, always some gotchas ;)10:21
*** zhurong has quit IRC10:34
openstackgerritMerged openstack/kolla-ansible stable/ocata: Add fwaas extension in l3 agent when neutron fwaas is enabled  https://review.openstack.org/45784610:35
*** zhubingbing has joined #openstack-kolla10:39
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Reparent kolla-toolbox from openstack-base  https://review.openstack.org/43502310:39
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Reparent kolla-toolbox from openstack-base  https://review.openstack.org/43502310:41
*** jayahn has joined #openstack-kolla10:44
egonzalezwhen is ocata 4.0.1 released?10:45
*** StephenWang1991 has quit IRC10:47
*** salv-orl_ has quit IRC10:52
*** tervancovan has quit IRC10:54
openstackgerritcaoyuan proposed openstack/kolla master: Install pankoclient in cloudkitty base container  https://review.openstack.org/45802410:59
*** dmk0202 has joined #openstack-kolla11:02
*** sudipto has quit IRC11:03
*** sudipto has joined #openstack-kolla11:03
*** sudipto has quit IRC11:03
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy  https://review.openstack.org/45799311:04
*** magicboiz has quit IRC11:04
*** tervancovan has joined #openstack-kolla11:04
egonzalezcan I get reviews on https://review.openstack.org/#/c/457993/ and https://review.openstack.org/#/c/457982/11:05
egonzalezatm cinder binary is broken in ocata11:05
egonzalez*ubuntu cinder binary11:05
*** jascott1 has joined #openstack-kolla11:06
hrwegonzalez: "USER cinder" line is not used then on debian/ubuntu?11:07
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy  https://review.openstack.org/45799311:07
*** StephenWang1991 has joined #openstack-kolla11:08
egonzalezhrw, ubuntu removed support for cinder-api system service, only works with apache11:08
hrwok11:08
egonzalezhrw, debian uses same packages as ubuntu right?11:08
hrwegonzalez: Debian lacks ocata packages11:09
hrwegonzalez: ubuntu has own packages11:09
hrwthey are based on Debian ones but done only for ubuntu11:10
*** jascott1 has quit IRC11:11
*** StephenWang1991 has quit IRC11:12
sdakemorning folks11:15
sdaketervancovan if you are in need of help let me know11:15
*** rwallner has joined #openstack-kolla11:16
openstackgerritcaoyuan proposed openstack/kolla master: Install pankoclient in cloudkitty base container  https://review.openstack.org/45802411:16
hrwhi sdake11:16
*** caowei has quit IRC11:16
egonzalezhrw, thanks for the explanation11:18
sdakehey hrw11:19
sdakehrw  actually what I wa safter with an arm server is something that had 10gig ports that were copper :)11:19
sdakehrw I looked around quite a bit, and really couldn't find anything which means I'd need a 6k 10gig QSFP+ switch11:19
*** rwallner has quit IRC11:20
*** dmk0202 has quit IRC11:21
*** dmk0202 has joined #openstack-kolla11:21
*** mgoddard has quit IRC11:27
*** rwallner has joined #openstack-kolla11:29
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Move Cloudkitty options into globals.yml  https://review.openstack.org/45803211:35
portdirectmorning o/11:38
egonzalezmorning portdirect11:39
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Move Cloudkitty options into globals.yml  https://review.openstack.org/45803211:40
*** salv-orlando has joined #openstack-kolla11:42
*** salv-orlando has quit IRC11:48
openstackgerritMerged openstack/kolla-ansible master: Allow use of any backup backend for any volume backend  https://review.openstack.org/45755411:51
portdirecttervancovan: the crash loop you were seeing is because calico is already running, you shouldnt need to install any cni or play with the cidr11:52
portdirect(unless it clashes with your network)11:52
tervancovanYes I saw the code portdirect11:53
tervancovanbut it is not same as kolla-k8s guide documents11:53
*** yangyapeng has joined #openstack-kolla11:54
hrwhttps://marcin.juszkiewicz.com.pl/2017/04/19/so-you-run-openstack-on-your-phone/11:54
tervancovanthis is kolla-k8s documents  --->  https://thepasteb.in/p/Anhr4q4xAQoFv11:55
tervancovanand this code is from kubeadm-aio -> https://thepasteb.in/p/P1hvWVW8gXOul11:56
tervancovanhi sdake11:56
portdirecttervancovan: if you want to use canal you'll need to modify the container - just swapping out the bits you have in those pastes should do it11:57
sdaketervancovan kubeadm-aio is not cimpatible iwth the deploy documentation11:57
sdakeportdirect also the deployment guide recomends setting the service and pod cidr11:57
tervancovanThanks sdake , so I just fix it a little bit11:59
tervancovanSo I applied canal for kolla-k8s in the container12:00
sdaketervancovan how did you run kubeadm init?12:01
tervancovanjust add some code12:02
hrwsdake, sbezverk: I hope that you enjoy post ;D12:02
portdirectsdake: you dont need to do that, but its great to show people how they can (change the cidr)12:02
sdakeportdirect i recognize it is not mandatory for many people12:02
sdakeportdirect however to really understand how to work on kolla - i find it was necessary persoanlly to understand how kubernetes deploys (atleast with kubeadm)12:03
tervancovanthis is snip code which i fixed it ->  https://thepasteb.in/p/xGhmkNk087juM  in kubeadm-aio12:04
sdaketervancovan there is more needed12:04
sdaketervancovan kubeadm doesn't do correct operations when the pod and service cidr are changed12:04
*** srwilkers has joined #openstack-kolla12:05
*** satyar has joined #openstack-kolla12:06
satyarHi Kolla12:06
sdaketervancovan specifically: sudo sed -i 's/10.96.0.10/10.3.3.10/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf12:06
sdakesup satyar12:06
tervancovansdake yes right it was not working when  cider was changed12:06
satyarhey sdake12:06
satyarhow are you?12:06
egonzalezsup satyar12:06
sdakesatyar good12:06
satyarit seems the upgrade is breaking the whole system12:06
sdaketervancovan i'd recommend just starting fresh without trying the kubeadm-aio container12:07
sdaketervancovan this will let you learn how kubeadm works12:07
satyarthis time i removed the code for jumbo frame support12:07
satyarall MTU is setup to 150012:07
satyarcreated a VM12:07
sdaketervancovan understanding how kubeadm works is crucial to doinganything with kuberntes imo :)12:07
satyarthe VM got 1450 MTU and able to ssh the VM12:07
tervancovanYes i12:08
satyarupgraded the system to stable/ocata12:08
tervancovanI thinks so12:08
satyarthen not able to access the VM any more12:08
satyarrestarted the VM and saw VM is having 1500 MTU12:08
sdaketervancovan kubeadm-aio is really a nice peice of tech, however, it is pretty much a black box12:08
satyari do have DVR and L3 HA12:08
sdaketervancovan much like any container it is declrative, eitherit works or it doesn't12:09
*** sudipto has joined #openstack-kolla12:09
satyarany help?12:09
sdaketervancovan if it does't work - it is very difficult to debug12:09
sdakejust like the reest of the continer world :)12:10
sdaketervancovan tell me what kind of hardware you ahve12:10
sdaketervancovan linux machine?12:10
egonzalezhrw, interesting article12:10
hrwegonzalez: thx12:10
*** Pavo has joined #openstack-kolla12:11
tervancovanyes right and our team is going to try testing on baremetal for production12:14
portdirectsdake: I think its not much of a black-box (but here we differ) by being a cuut down environment it shows exactly how things like kubeadm actually work12:14
portdirectthats one of the great things i see about docker style containers12:15
sdaketervancovan if your after an eval of kolla-kubernetes, I'd recommend using a) baremental with two nics b) a vm with two nics12:16
*** jayahn has quit IRC12:16
tervancovanI think a is better12:17
sdaketervancovan yup - either will work for an AIO eval12:17
sdaketervancovan I use both :)12:17
sdaketervancovan if you ahve linux baremetal to work with, the guide works flawlessly12:17
sdaketervancovan once you learn how kubeadm works - you wlil find kubeadm prints out a big "this is alpha, do not use in production" warning12:18
sdaketervancovan there are other options for production deployments of kubernetes - if your end  goal is to go to production, you will need in great detail to understand how kubernetes works12:18
sdaketervancovan also note kolla-kubernetes has not released a 1.0.0 - while good for eval, kolla-kubernetes is currently unsuitable for produciton12:19
tervancovanMy boss should know that this issue ....12:19
sdaketervancovan which one - I listed out two :)12:20
sdaketervancovan by the time you get done evaluating, I think you will have an idea how to actually use kolla-kubernetes in production12:20
sdaketervancovan first step there is learning how to use kubernetes in production12:21
tervancovanthanks for information sdake12:22
sdaketervancovan when is your deployment target month?12:22
tervancovanMaybe end of this year?12:23
sdakeya - thats no problem for kolla-kubernetes12:23
sdakeit wll take you that long to learn how all this stuff works :)12:23
sdakeby then i expect kubeadm to be out of alpha12:23
sdakeor many of the other kubernetes deployment options12:23
sdakekolla doesn't really care how you deploy kubernetes - the motive for the  guide is to help you understand how to get to a kolla deployment12:24
sdakeif the kolla community could take out steps 1 and step 2 from the guide, we would do so - however the upstream docs on deployment of kubernetes are lacking12:24
tervancovansdake thanks for informations12:26
*** mgoddard has joined #openstack-kolla12:29
sdakesup mgoddard12:29
manheimdoes anyone have problems building centos images?12:31
*** zhurong has joined #openstack-kolla12:31
manheimopenstack-base fails to build as it fails to download some packages12:31
*** iceyao has joined #openstack-kolla12:31
sdakemanheim master or a specific tag or branch?12:32
manheimpermission denied on some rpm12:32
manheimstable/ocata12:32
sdakemanheim moment let me give it a spin12:32
manheimhmmm, or master, let me check12:33
sdakemanheim which distro/type are you building?12:34
manheimsudo kolla-build neutron-server -n kolla --base centos --base-image centos --base-tag 712:34
manheimthe exact command12:34
manheim4.0.012:35
manheimi'm on master, yes12:36
sdakemanheim you just said you were on stable/ocata :)12:42
*** Pavo has quit IRC12:42
manheimsorry, my mistake12:42
sdakemanheim stable/ocata wfm12:44
sdakemanheim checking out master - moment12:44
manheimok12:44
sdakemanheim please run pip show kolla -> paste12:45
*** zhurong has quit IRC12:45
manheimhttp://paste.ubuntu.com/24413607/12:46
*** kbaegis has joined #openstack-kolla12:46
manheimi was behind on my fork  a few days, rebased now12:46
*** lamt has joined #openstack-kolla12:47
sdakemanheim - master wfm12:51
manheimnice12:51
sdakemanheim i'd recommend trying master - I believe a recent change fixed a yum repo location - grafana iirc12:52
manheimok, i'll reinstall kolla after rebase and see, thanks12:52
*** Pavo has joined #openstack-kolla12:53
sdakepbourke ++12:54
*** goldyfruit has joined #openstack-kolla12:56
*** lamt has quit IRC12:58
*** rwellum has joined #openstack-kolla13:00
manheimsdake it built with latest commits, thanks again13:01
*** lamt has joined #openstack-kolla13:01
sdakemanheim enjoy13:01
*** huzhengchuan has quit IRC13:03
*** srwilkers has quit IRC13:04
mgoddardhey sdake, sorry for the slow reply :)13:04
sdakemgoddard no sweat - just a greeting13:04
sdakemorning rwellum13:05
rwellummorning sdake13:05
sdakerwellum have you had an opportunity to run through the latest deployment guide?13:06
rwellumNo - but definitely this week.13:06
sdakerwellum it has undergone a few changes - may need to refresh your browser13:06
rwellumOk will do - looking at it now13:07
sdakerwellum cool - i am starting up a twice weekly "Demystifying the gate" webex session for 30 minutes13:07
*** jascott1 has joined #openstack-kolla13:07
sdakerwellum I think the gate is difficult for people to comprehend as jascott1 pointed out - and having such an effort will be beneficial for many13:08
rwellumyesterday I was previewing the Upstream University latest slides for Boston summit with one of the authors. I bet he'd be interested to see if there was any overlap they could include.13:09
*** sean-k-mooney has joined #openstack-kolla13:09
*** calbers has quit IRC13:11
*** jascott1 has quit IRC13:12
rwellumsdake: you'll send out invites?13:12
sdakerwellum first will vote on a time13:12
rwellumsgtm13:13
*** lamt has quit IRC13:14
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix heat ec2 keystone auth  https://review.openstack.org/44495313:15
*** lamt has joined #openstack-kolla13:16
*** gkadam has quit IRC13:17
*** schwicht has joined #openstack-kolla13:18
*** rwallner has quit IRC13:19
*** calbers has joined #openstack-kolla13:20
*** rwallner has joined #openstack-kolla13:20
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Move Cloudkitty options into globals.yml  https://review.openstack.org/45803213:21
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Comment designate options at globals.yml  https://review.openstack.org/45807413:21
*** magicboiz has joined #openstack-kolla13:22
*** magicboiz has quit IRC13:26
*** srwilkers has joined #openstack-kolla13:27
*** super has quit IRC13:28
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix neutron agents restarted on ml2 config change  https://review.openstack.org/44799213:34
*** magicboiz has joined #openstack-kolla13:37
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: kubetoolbox: mark as x86-64 only  https://review.openstack.org/45807813:45
*** eanylin has quit IRC13:45
*** huzhengchuan has joined #openstack-kolla13:46
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: [TEST][OCATA][KOLLA-ANSIBLE] TEST CEILOMETER RELATED SERVICE  https://review.openstack.org/43463713:46
sdakerwellum can you make today's team meeting?13:46
sdakerwellum 16:00 UTC13:47
hrw~curse current development methods13:50
hrwthere are few images which fetch kubectl binary from network. x86-64 binary. for Debian it can be installed from package. but not for ubuntu.13:51
hrwcentos has package too13:51
rwellumsdake: I should be able to make that - have a meeting I run that finishes when that starts13:52
*** lamt__ has joined #openstack-kolla13:52
*** lamt has quit IRC13:53
sdakerwellum good to hear13:55
sdakespsurya hopefully you can make our team meeting as well13:55
openstackgerrithosup choi proposed openstack/kolla-kubernetes master: Update deployment-guide.rst document for ubuntu  https://review.openstack.org/45808313:55
hrwanyway. another image to disable13:56
sdakequery hrw13:56
hrwquery sdake13:56
*** al498u has joined #openstack-kolla13:57
sdakewinning13:57
*** eanylin has joined #openstack-kolla13:57
*** ksumit has joined #openstack-kolla13:57
*** al498u_ has joined #openstack-kolla13:58
spsuryasdake: hi..13:59
spsuryameeting ??13:59
spsuryadidn't get you13:59
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: magnum-conductor: mark as x86-64 only  https://review.openstack.org/45808514:00
hrwspsurya: #openstack-meeting-4 in 2h from now - Kolla meeting14:01
*** al498u has quit IRC14:02
*** jtriley has joined #openstack-kolla14:02
*** david-lyle has quit IRC14:04
*** tervancovan has quit IRC14:07
*** Pavo has quit IRC14:08
*** skramaja has quit IRC14:12
*** iceyao has quit IRC14:13
*** david-lyle has joined #openstack-kolla14:14
*** satyar has quit IRC14:19
*** eaguilar has joined #openstack-kolla14:19
*** satyar has joined #openstack-kolla14:28
*** satyar has quit IRC14:29
*** MarginHu2017 has joined #openstack-kolla14:29
*** satyar has joined #openstack-kolla14:29
sdakespiette meeting at 16:00 UTC we have every wednesday on irc14:29
sdakespsurya ^^14:29
sdakespsurya contributors generally attend the meeting14:29
*** Pavo has joined #openstack-kolla14:29
*** zhubingbing has quit IRC14:29
spsuryasdake i always attend14:30
spsuryagot that14:30
sdakespsurya cool :)14:30
spsuryasdake on doodle now14:31
*** iceyao has joined #openstack-kolla14:31
*** satyar has quit IRC14:31
spsuryachoosing the slot14:31
*** satyar has joined #openstack-kolla14:31
*** StephenWang1991 has joined #openstack-kolla14:31
*** tomatopeel has left #openstack-kolla14:34
*** lamt__ has quit IRC14:35
*** iceyao has quit IRC14:35
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Allow different configuration of iSCSI helper  https://review.openstack.org/45811314:36
*** MarginHu2017 has quit IRC14:38
sdakerwellum do you have an email address you can share14:38
*** MarginHu2017 has joined #openstack-kolla14:39
*** eaguilar has quit IRC14:44
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix troubleshooting guide debugging section  https://review.openstack.org/45811514:50
*** eaguilar has joined #openstack-kolla14:50
*** magicboiz has quit IRC14:50
*** MarginHu2017 has quit IRC14:51
*** magicboiz has joined #openstack-kolla14:53
*** duonghq_ has joined #openstack-kolla15:00
*** rmart04 has quit IRC15:00
*** zhubingbing has joined #openstack-kolla15:04
rwellumsdake: sent it to you private message15:06
duonghq_evening guys15:06
sdakerwellum ack getting to it :)15:06
sdakesup duonghq_15:06
*** duonghq has quit IRC15:06
*** duonghq_ is now known as duonghq15:06
*** duonghq_ has joined #openstack-kolla15:07
*** lamt has joined #openstack-kolla15:07
*** jascott1 has joined #openstack-kolla15:08
openstackgerritMerged openstack/kolla-ansible master: Add panko.conf file in ceilometer container  https://review.openstack.org/44471915:08
*** iceyao has joined #openstack-kolla15:10
*** jascott1 has quit IRC15:13
openstackgerritMerged openstack/kolla-ansible master: Allow different configuration of iSCSI helper  https://review.openstack.org/45811315:14
spsuryaduonghq: sup15:14
duonghqhi spsurya15:14
*** iceyao has quit IRC15:15
*** manheim has quit IRC15:16
*** manheim has joined #openstack-kolla15:17
*** satyar has quit IRC15:17
*** satyar has joined #openstack-kolla15:18
*** manheim has quit IRC15:21
egonzalezcan I get reviews on https://review.openstack.org/#/c/457993/ and https://review.openstack.org/#/c/457982/ are critical bugs blocking cinder in ubuntu binary15:22
*** pcaruana has quit IRC15:28
*** iceyao has joined #openstack-kolla15:30
pbourkeegonzalez: is the if/else at the end of the dockerfile necessary?15:31
egonzalezpbourke, yes for the user15:31
egonzalezpbourke, with cinder user cannot start apache process15:32
pbourkewill we not be fixing cinder for source in ocata?15:32
egonzalezsource works15:32
egonzalezis just ubuntu package who changed to apache only15:32
pbourkeah I see15:33
egonzalezis just to no backport features, only the fix necessary15:33
*** iceyao has quit IRC15:35
*** inc0 has joined #openstack-kolla15:36
inc0good morning15:36
zhubingbingHI GUYS15:37
zhubingbinghi guys15:37
srwilkerso/15:38
duonghqmorning inc015:39
duonghqhi zhubingbing15:39
duonghqhi srwilkers15:39
duonghqnice to see you guys15:39
zhubingbinghi15:39
zhubingbing;)15:39
srwilkershey duonghq nice to see you too15:40
duonghqsrwilkers, will you go to Boston?15:40
*** david-lyle has quit IRC15:41
kfox1111morning.15:44
srwilkersduonghq, yeah ill be there. you?15:45
srwilkerskfox1111, morning man15:45
sbezverkgood morning15:45
duonghqsrwilkers, me too15:45
srwilkersduonghq, awesome :)15:45
*** yee37916 has joined #openstack-kolla15:46
*** schwicht has quit IRC15:47
*** yee379 has quit IRC15:47
*** shardy has quit IRC15:48
sbezverkanybody willing to get fluentd/eleasticsearch/kibana going in kube? ;) my experience with installing 5 node kolla-kube cluster strongly suggests we really need it :)15:49
*** shardy has joined #openstack-kolla15:49
kfox1111sbezverk: been working on it a bit in a way.15:50
kfox1111but kind of in a different way then we discussed.15:50
kfox1111I found fluent-bit and have been playing around with it.15:50
kfox1111then submitted this:15:50
kfox1111https://github.com/kubernetes/charts/pull/89515:50
sbezverkkfox1111: ok will you be committing it to kolla?15:51
kfox1111was waiting till all the ansible config's merged (done I think) and then was going to be submitting patches to15:51
srwilkerssbezverk, kfox1111 ive been working on that as well15:51
kfox1111support configing to log out to stdout instead of log files.15:51
kfox1111with that in place, it will properly support log rolling/shipping with whatever k8s has installed.15:51
kfox1111I tested log rolling with fluent-bit and k8s 1.6 and it seemed to work ok.15:52
*** matrohon_ has quit IRC15:52
sbezverkkfox1111: so fluent-bit runs as a side car ?15:52
kfox1111sbezverk: daemonset. we could run it as a side car instead too, but would have to add a log roller too. :/15:53
kfox1111docker has a log roller built in apparently.15:53
*** shardy has quit IRC15:53
rwellumThis is a bit out there - but does anyone have any advice/experience with configuring RHEL as a base OS for a server? Or point me to the correct place to ask?15:53
kfox1111for dealing with stdout in its logs.15:53
sbezverkkfox1111: another thing do we have kolla container for it?15:53
sdakerwellum you might try #centos15:54
kfox1111rwellum: what do you mean?15:54
kfox1111rwellum: like, you want to do file serving, or whatever using RHEL?15:54
kfox1111sbezverk: yeah, either we put it in kolla, or we use upstreams container/chart.15:55
sbezverkkfox1111: if it does not run asa side car, how do you plan to access openstack generated log files?15:55
kfox1111I think there's other upstream charts we may want to use.15:55
kfox1111prometheus/grafana too.15:55
kfox1111been usign that for a while.15:55
kfox1111sbezverk: the plan is to disable openstack log files. just push them directly to stdout.15:55
rwellumkfox1111: I mean a new server, no OS installed, I've used both Ubuntu and Centos in the past - wanting to try RHEL - had some issues with basic stuff like how it partitions, licensing post bringup etc.15:56
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Deployment guide updation for better followup  https://review.openstack.org/45787015:56
rwellumsdake: Centos I can do :)15:56
*** jascott1 has joined #openstack-kolla15:56
kfox1111rwellum: its very similar to centos. really only just the extra licence registration step.15:57
zhubingbingmeeting15:57
kfox1111we've got a fair amount of rhel servers.15:57
sbezverkkfox1111: that would be a major change in behavior15:57
zhubingbingkfox1111  why user fluent-bit ?15:58
kfox1111sbezverk: yeah. thats why I haven't talked about it too much yet. still experimenting.15:58
inc0meeting peeps15:58
kfox1111zhubingbing: its much lighter weight then fluentd, since it needs to be on all nodes.15:58
kfox1111fluentd and logstash are basically equivilent.15:58
rwellumkfox1111: I stupidly chose default partition, and it put everything on /home. Pretty sure Centos doesn't do that. At least not the version I've been using...15:58
kfox1111fluent-bit and filebeat are basically equivilent.15:58
zhubingbinghmm i konw15:59
kfox1111rwellum: are you using the workstation edition or server edition?15:59
kfox1111I think the defaults are a bit different. but they are otherwise essentially the same.16:00
sdakekfox1111 we have blueprints on the agenda today16:00
sdakewould be nice to have you there #openstack-meeting-416:00
kfox1111sdake: you started early. ;)16:01
rwellumkfox1111: I'll continue after the meeting!16:01
*** unicell has joined #openstack-kolla16:01
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy  https://review.openstack.org/45799316:02
*** unicell1 has quit IRC16:04
*** Pavo has quit IRC16:04
*** schwicht has joined #openstack-kolla16:06
*** dave-mccowan has joined #openstack-kolla16:06
*** dmk0202 has quit IRC16:09
openstackgerritMerged openstack/kolla master: Fix cinder-api ubuntu binary  https://review.openstack.org/45798216:12
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/ocata: Fix cinder-api ubuntu binary  https://review.openstack.org/45815216:12
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix cinder-api ubuntu binary deploy  https://review.openstack.org/45799316:15
*** huzhengchuan has quit IRC16:16
*** david-lyle has joined #openstack-kolla16:19
*** manheim has joined #openstack-kolla16:21
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Add panko.conf file in ceilometer container  https://review.openstack.org/44472016:21
*** manheim has quit IRC16:25
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: WIP: set kibana default index pattern  https://review.openstack.org/45815616:27
* hrw drops from meeting - will leave session running16:27
*** blallau has quit IRC16:29
*** rwallner has quit IRC16:34
*** rwallner has joined #openstack-kolla16:35
*** eaguilar has quit IRC16:35
*** jheroux has joined #openstack-kolla16:38
*** daidv has joined #openstack-kolla16:41
*** unicell has quit IRC16:43
*** schwicht has quit IRC16:45
*** daidv has quit IRC16:45
*** zhubingbing has quit IRC16:49
*** daidv has joined #openstack-kolla16:49
*** daidv_ has joined #openstack-kolla16:50
*** daidv_ has quit IRC16:50
*** daidv_ has joined #openstack-kolla16:51
*** david-lyle has quit IRC16:51
*** Pavo has joined #openstack-kolla16:53
*** daidv_ has quit IRC16:53
*** daidv has quit IRC16:54
openstackgerritDuong Ha-Quang proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725216:54
*** salv-orlando has joined #openstack-kolla16:54
*** ksumit has quit IRC16:54
*** newmember has joined #openstack-kolla16:59
*** egonzalez has quit IRC17:00
*** inc0 has quit IRC17:00
*** inc0 has joined #openstack-kolla17:00
*** sambetts is now known as sambetts|afk17:00
duonghqkfox1111, can you take a look at my ps: https://review.openstack.org/#/c/437252/ can you give me advice on error at http://logs.openstack.org/52/437252/12/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-nv/923733d/console.html#_2017-04-19_07_02_14_82557417:00
*** rwallner has quit IRC17:00
jascott1relocatin' bbiab17:00
kfox1111duonghq: sure. sec.17:01
*** kbaegis has quit IRC17:01
inc0duonghq: we'll take your topics next week17:01
duonghqinc0, thank you17:01
kfox1111dosaboy: oh. I know what that is.17:02
*** rwallner has joined #openstack-kolla17:02
kfox1111duonghq: oh. I know what that is.17:02
*** mgoddard has quit IRC17:02
kfox1111duonghq: trunk just got reversioned to 0.7.0.17:02
*** rwallner has quit IRC17:02
*** magicboiz has quit IRC17:03
*** rwallner has joined #openstack-kolla17:03
kfox1111duonghq: but your ps is new, so didn't get the version update. please change all 0.6.0 refreences to 0.7.0.17:03
duonghqkfox1111, ah, so I should bump all heat-related microservice and service to 0.7.0?17:03
kfox1111yeah. that should fix it.17:03
duonghqkfox1111, thanks17:03
*** jascott1 has quit IRC17:03
kfox1111np.17:03
duonghqkfox1111, will you go to Boston?17:04
*** jascott1 has joined #openstack-kolla17:04
kfox1111yup. will be there. you?17:04
duonghqme too17:04
*** rwallner has quit IRC17:04
inc0duonghq: bring something tasty from your area:P17:04
sdakeinc0 any chance you can form a signup list of who will actually be at summit since over half our contribs wont be because the US govt is crazy?17:04
kfox1111duonghq: nice. looking forward to seeing you there. :)17:05
rwellumkfox1111 - when you get a chance look at this for RHEL. Is it normal to pack out /home like this? https://www.irccloud.com/pastebin/HqTwvVVY/17:05
sdakeinc0 lots of contribs/cores don't have visa clearance17:05
duonghqinc0, :P17:05
inc0sdake: I already bubbled up visa issues to foundation17:05
*** kbaegis has joined #openstack-kolla17:05
duonghqI need to sleep, cya guys17:05
sdakeinc0 sounds good17:05
sdakenight duonghq17:05
kfox1111rwellum: for a workstation edition of rhel, yeah, I think thats right. for a server edition, I think they have different, more centos like defaults.17:06
kfox1111duonghq: have a good one.17:06
*** duonghq has quit IRC17:06
rwellumkfox1111: ok off to recheck wth I installed....17:06
sbezverkrwellum: wow that is really wierd to have /home stretched accross two file systems17:07
sbezverkI mean two disks17:07
*** newmember has quit IRC17:07
kfox1111sbezverk: yeah. for a work station, most disk space needs to be fore the user.17:07
spsuryanight all17:07
kfox1111spsurya: nite17:07
*** newmember has joined #openstack-kolla17:07
sbezverkkfox1111: for centos rhel I use the simplest and which I think most relable approch /boot - 500mb /swap 4096mb / - all the rest and it is using lvm17:08
*** jascott1 has quit IRC17:09
kfox1111sbezverk: yeah. centos I think is based off of the server edition of rhel. so has all the server defaults.17:09
sbezverkneed more space, add disk to lvm and then extend17:09
*** unicell has joined #openstack-kolla17:09
*** hieulq_ has joined #openstack-kolla17:10
*** harlowja_ has quit IRC17:13
*** rwallner has joined #openstack-kolla17:13
*** rwallner has quit IRC17:13
*** rwallner has joined #openstack-kolla17:14
*** harlowja has joined #openstack-kolla17:15
*** StephenWang1991 has quit IRC17:15
*** harlowja has quit IRC17:17
*** harlowja has joined #openstack-kolla17:17
*** tonanhngo has joined #openstack-kolla17:20
sdakefor those of you that have not voted for the tc election, pleae do vote17:24
*** tonanhngo has quit IRC17:24
openstackgerritKevin Fox proposed openstack/kolla master: Fingerprint containers.  https://review.openstack.org/45817417:24
kfox1111inc0: -----^17:24
rwellumsbezverk: I know right? Makes sense I guess for desktop - but still not convinced.17:24
inc0kfox1111: have you seen ML discussion this morning?17:25
kfox1111inc0: not yet.17:25
inc0I think pbourke has great idea17:25
rwellumsbezverk: so for centos - you set that up manually correct? ( /boot - 500mb /swap 4096mb / - all the rest and it is using lvm)17:25
sbezverkrwellum: absolutely17:26
sbezverkonly manual setup17:26
sbezverkand software packages (minimal server)17:26
kfox1111inc0: not sure you can use a label for image pulling though.17:26
kfox1111at least not in kubernetes.17:26
rwellumsbezverk: thanks - going to try that then :)17:26
kfox1111though for k8s I don't think we want that anyway.17:26
kfox1111so maybe its ok.17:27
inc0kfox1111: you can't but what you can do is to use it to easily compare images17:27
kfox1111yeah. we could store the fingerprint there.17:27
sbezverkrwellum: sure, I am confident it will work well as I use this approach for years17:27
kfox1111then it gets shipped with the container.17:27
rwellumsbezverk: yes I've run into problems not using minimal server - especially something seems to mess up docker17:27
kfox1111I do really like that idea.17:27
inc0you can pull image to local17:27
kfox1111but pbourke's intention was to avoid needing revisions in the tags. which I think we still need.17:28
inc0and compare it with existing17:28
kfox1111I think thats the place where it could shine though.17:28
*** tonanhngo has joined #openstack-kolla17:28
kfox1111if yuou can get the labels without pulling the image, then a big win.17:28
sbezverkrwellum: right you let setup install some packages by default it will turn into a nightmare17:28
kfox1111if you ahve to pull the image, computing the fingerprint is relatively light weight to do from scratch then.17:28
sbezverkdue to versions collisions with packages you will need to install later17:29
inc0not sure if docker supports that, but hmm17:29
sbezverkkfox1111: are you talking about adding md5 to the image name?17:30
rwellumsbezverk: yup - learned this the hard way17:30
rwellumBut never understood exactly why.17:30
kfox1111sbezverk: no. a revision number.17:30
sbezverkrwellum: different repos keeps different versions17:30
kfox1111centos-binary-nova-api:4.0.0-1, centos-binary-nova-api:4.0.0-2, etc.17:31
kfox1111so if its the same exact container, but has gotten rpm upgrades, its different.17:31
sbezverkexample rhel repo might keep one version of rpm but epel another17:31
sbezverkso here is the collision17:31
sbezverkkfox1111: ah ok it will be used just at the tagging time17:32
sbezverkto see if new version needs to be published17:32
kfox1111sbezverk: yeah.17:32
kfox1111exactly.17:33
kfox1111I want to be able to:17:33
kfox1111helm upgrade nova-api kolla/nova-api17:33
kfox1111done. :)17:33
*** schwicht has joined #openstack-kolla17:33
*** gianpietro has joined #openstack-kolla17:33
*** gianpietro has quit IRC17:35
rwellumkfox1111: This is what I installed: rhel-server-7.3-x86_64-dvd.iso17:36
*** iceyao has joined #openstack-kolla17:37
kfox1111rwellum: hmm... weird. not sure then why it would default to that layout.17:37
kfox1111lets see...17:37
kfox1111oh... one other thought.17:37
kfox1111I think rhel registration codes can be workstation/server related?17:38
kfox1111its been a while since I played with them tough, so don't remember if that happens before/after partitioning. I would guess after.17:38
kfox1111though if it was provided the code before partitioning, it might be able to figure it out.17:39
sbezverkkfox1111: yes, but rhel has WS flavor of image17:39
rwellumIt happens well after partitioning :)17:39
sbezverkI used exactly the same image for my installation17:39
sbezverkrwellum: my rule of thumb, do not use defaults unless you established them ;)17:40
rwellumWell I am going to re-image and do exactly that :)17:40
*** mgoddard has joined #openstack-kolla17:41
portdirectsbezverk: i like your styles17:41
rwellumI'll be careful on the subscription as well - just in case kfox1111 has a point17:41
portdirecthave you played with centos atomic host yet, if not I'm pretty sure you'd like it17:41
*** iceyao has quit IRC17:41
portdirect(and/or rhel atomic)17:42
kfox1111I generally use centos and cobbler with a kickstart so always specify my partions manually anyway.17:42
kfox1111well, manually but scripted.17:42
kfox1111more magicical. :)17:42
rwellumIs Atomic stable portdirect ?17:42
portdirectrock solid17:43
*** sudipto has quit IRC17:43
*** sudipto has joined #openstack-kolla17:43
*** sudipto has quit IRC17:44
*** hieulq_ has quit IRC17:44
*** gianpietro has joined #openstack-kolla17:44
*** gfidente is now known as gfidente|afk17:45
rwellumportdirect: interesting. So does the default partitioning work? :)17:45
rwellumFor containerization etc.17:45
portdirect100% though I'd still reommend setting it up for your own needs17:46
kfox1111inc0: responded to a bunch of the thread.17:46
kfox1111inc0: I dont think the label idea is fully thought through.17:47
portdirectI run a 4gb root and put logs on their own part, set an appropriate swap up and then use the rest for docker.17:47
kfox1111inc0: reguardless, what do you think of the ps I posted? we will need that either way.17:47
*** eaguilar has joined #openstack-kolla17:48
rwellumThanks portdirect17:49
inc0change with fingerprint in gate is good17:50
openstackgerritMerged openstack/kolla master: Add OpenDaylight Container to kolla  https://review.openstack.org/41636917:50
inc0regardless17:50
inc0mgkwill ^ yay17:50
openstackgerritKevin Fox proposed openstack/kolla master: Fingerprint containers.  https://review.openstack.org/45817417:51
kolla-slack<jascott1> woah helm install tgz is missing all of a sudden https://kubernetes-helm.storage.googleapis.com/helm--linux-amd64.tar.gz17:51
mgkwillinc0: awesome!17:53
*** jaosorior is now known as jaosorior_away17:54
kolla-slack<jascott1> false alarm its just the IP im coming out of is rate limited at github17:59
kfox1111ah. you hit it too hard. :)18:00
kolla-slack<jascott1> lol18:00
kolla-slack<jascott1> so much for the non-proxy network that makes life livable and work doable18:00
kfox1111;)18:01
kolla-slack<jascott1> why do I come here? its a series of work speed bumps :S18:01
kolla-slack<jascott1> (here being the office)18:01
kfox1111just need to tunnel back home. ;)18:01
kfox1111and uncheck the yearly review "do I have everything nessisary to do my job" checkbox. :)18:02
inc0I set up my nuc for dev that sits nicely on my desk at home18:02
kolla-slack<jascott1> hm i live 2 minutes away I could probably construct a tunnel in a few years ;)18:02
kfox1111heh. thats one way to do it. ;)18:02
kolla-slack<jascott1> inc0 cool got it all dialed in?18:02
openstackgerritDavid Moreau Simard proposed openstack/kolla-ansible master: Do not merge: Test ARA 0.12.5rc1  https://review.openstack.org/43996318:03
openstackgerritDavid Moreau Simard proposed openstack/kolla-ansible master: Do not merge: Test ARA 0.12.5rc1  https://review.openstack.org/43996318:03
inc0pretty much yeah18:03
kfox1111ssh socks proxies make life sooo much easier. :)18:05
*** jascott1 has joined #openstack-kolla18:10
*** kbaegis has quit IRC18:10
*** kbaegis has joined #openstack-kolla18:15
inc0kfox1111: tell me this, how would you define whether to pull from dockerhub or not?18:16
*** gianpietro has quit IRC18:21
*** eaguilar has quit IRC18:23
*** unicell has quit IRC18:23
*** rwallner has quit IRC18:24
*** rwallner has joined #openstack-kolla18:24
*** satyar has quit IRC18:25
*** unicell has joined #openstack-kolla18:27
inc0kfox1111: I think for devenv we should have setup_gate_emptydir:)18:32
*** rwallner has quit IRC18:32
*** Pavo has quit IRC18:44
*** lucasxu has joined #openstack-kolla18:45
*** rwallner has joined #openstack-kolla18:53
*** eaguilar has joined #openstack-kolla18:55
*** jmguzmanc has joined #openstack-kolla19:02
*** MasterOfBugs has joined #openstack-kolla19:03
*** david-lyle has joined #openstack-kolla19:06
*** lucasxu has quit IRC19:08
*** lucasxu has joined #openstack-kolla19:09
kfox1111inc0: say we have centos-binary-nova-api:4.0.0-1.19:10
kfox1111user:19:10
kfox1111helm install --name nova-api  kolla/nova-api-deployment19:10
kfox1111it pulls kolla/nova-api-deployment-4.0.0-1.tgz and installs it.19:11
kfox1111it points to dockerhub/kolla/centos-binary-nova-api:4.0.0-1. now, redhat releases openssl that fixes a security update.19:11
kfox1111the build system detects openssl changed, and pushes dockerhub/kolla/centos-binary-nova-api:4.0.0-2,19:12
kfox1111and a helm package updated to point to it.19:12
inc0are we going to publish new helm chart every time we publish image?19:12
kfox1111for the updated package then to land on the users system, the user simply has to:19:13
kfox1111helm upgrade nova-api kolla/nova-api-deployment19:13
kfox1111I think so.19:13
kfox1111we maintain a kolla-kubernetes helm repo with packages people can pull from, the same way we manage containers they can pull rom.19:14
kfox1111same kind of use case.19:14
inc0if you run helm upgrade nova-api while image and tag stays the same, but image has different sha1, will it upgrade?19:14
kfox1111there are two modes of configuration.19:14
kfox1111in the default, it will skip the upgrade, as we have imagePullPolicy=IfNotPresent19:14
kfox1111it could be set to Always.19:15
inc0so way I'd run it locally19:15
kfox1111but actualy in that mode, nothing will change in the final k8s objects and k8s will fail to do the upgrade.19:15
inc0(regardless of scale)19:15
inc0ehh19:15
kfox1111you would have to set it to Always, and skip the helm upgrade, and delete all the active pods.19:16
inc0ok, so :latest tag will never work properly with helm+k8s?19:16
kfox1111right.19:16
inc0how fun19:16
kfox1111it will work the first time.19:16
kfox1111it won't support ugprades.19:16
inc0-.-19:16
kfox1111thats why I've been talking about tags so much.19:16
kfox1111k8s is all about atomic images.19:16
*** david-lyle_ has joined #openstack-kolla19:17
kfox1111that only works if you can distinguish containers, and thats done with tags.19:17
*** david-lyle has quit IRC19:17
inc0so what I don't see happening then is constantly-upgrading use case19:18
inc0I mean publishing helm charts alongside of images every day19:18
inc0especially if we add caveat of publish only if something changes19:18
inc0we create chain of co-dependency that's going to be hard to maintain19:19
kfox1111I don't think its htat hard to maintain.19:20
kfox1111we look at containers. and previously published helm packages.19:20
kfox1111if they don't point to the newest, we update the values file and push to tarballs.o.o.19:20
inc0but tags are in code right?19:21
kfox1111values.yaml.19:21
inc0which everyone will have on their own19:21
kfox1111should be simple to poke in the values and helm package.19:21
kfox1111so, that may need further integrations, yeah.19:22
inc0I assume people will run their own values.yaml, no 2 openstack are exactlu the same19:22
kfox1111but, this is in a catagory of, you want to use kolla-kubernetes and build your own helm packages,19:22
kfox1111but don't want to build your own containers.19:22
kfox1111think that will be common?19:23
kfox1111(other then maybe for dev)19:23
kfox1111there are multiple sets of values.yaml.19:23
inc0for dev ironically I'd assume local registry19:23
kfox1111there are the ones inside the packages, and there are users values.yaml19:23
kfox1111I'm talking about the defaults in the packages.19:23
inc0but values.yaml are for deploy phase not chart build right?19:23
kfox1111yes/no.19:24
inc0as in, if I want to do enable_neutron_lbaas, I still use kolla-prebuilt charts?19:24
kfox1111the package has a values.yaml that has defaults.19:24
kfox1111the user can provide a vaulues.yaml as well.19:24
kfox1111at install/upgrade time, the values.yaml from the package is taken, then anything in the users one overrides that.19:25
inc0yeah and I assume that will be the case most of the time19:25
*** eaguilar has quit IRC19:25
inc0and defaults are in repo so I don't want to edit these automagically every time we update containers19:25
kfox1111so if the user specified version 4.0.0-75 and the package 4.0.0-5, the users version would be taken.19:25
kfox1111so if they are vbuilding their own stuff, and want their own revision numbers they are free to do so.19:25
kfox1111in the case of prebuilt packages,19:26
kfox1111they would point to 4.0.0-5, and if the user didn't specify, they woudl get the "most up to date version"19:26
kfox1111and on upgrade, it would point to a newer version, and then do the right thing.19:26
inc0how about defaults?19:27
kfox1111defaults for what?19:27
*** eaguilar has joined #openstack-kolla19:28
inc0https://github.com/openstack/kolla-kubernetes/blob/master/helm/microservice/cinder-api-deployment/templates/cinder-api.yaml#L519:28
kfox1111right now, that comes from helm/all_values.yaml.19:28
inc0or rather that19:28
inc0https://github.com/openstack/kolla-kubernetes/blob/master/helm/all_values.yaml19:28
inc0yeah19:28
kfox1111yeah.19:28
inc0so we publish image daily19:28
inc0do we edit this file daily?19:29
inc0to up a revision on image19:29
kfox1111I guess the question is, do we push it back to git, or just do it in the prebuilt packages we ship?19:29
kfox1111we up the revision only when it chagnes.19:29
kfox1111no reason to make it look like updates are available when nothing changes.19:30
inc0so one thing I'd like to have is full and easy repeatibility of helm build locally19:30
inc0so no-no for automagical values.yaml19:30
kfox1111we have that now.19:30
inc0not officially really19:30
kfox1111but are you talking pointing at local built containers, or for hub ones?19:30
inc0mechanism will be the same reall19:30
inc0y19:30
kfox1111I think some folks will want to run this kind of process at their own sites with their own ci/cd.19:31
inc0ok, I think your use case will be waaaay too complex to have a quick win:/19:31
kfox1111so the plumbing should be in tree.19:31
inc0ofc19:31
kfox1111I don't think it will take too long to implement really.19:31
inc0so how about scenerio I described?19:31
inc0you retag your own images19:31
inc0and we'll provide tools to make it easier19:31
inc0kolla-build pull-changed19:32
inc0or sth like that19:32
kfox1111I think maybe we have a tool that takes a yaml of container -> version's, and the tool tweaks the values.yaml as appropriate to build?19:32
kfox1111you don't need to run it if you dont want.19:32
kfox1111if you update and build revisions for things, then your tooling can call that and update the things, then build the packages?19:33
kfox1111ideally, the file even would be spit out of the kolla build process.19:33
inc0uhh19:34
inc0we're talking about really complex system:/19:34
inc0in my mind at leasy19:34
kfox1111maybe we make a kolla build mode that incorperates the fingerprinting.19:34
kfox1111it does what it does today, but with a :latest tag.19:34
inc0yeah, and LABELs19:34
kfox1111then compares the containers it just built with a repo specified.19:35
inc0or do something like19:35
kfox1111it then tags the containers that changed with a rev+1,19:35
inc0pull changed, retag locally, push to local registry19:35
kfox1111and writes that to a yaml manifiest file.19:35
inc0and we'll host tip of branch on dockerhub19:35
kfox1111or even I guess the whole list of freshest revisions.19:35
inc0so everyone will install from local registry19:35
inc0but we'll provide tools to mainain it19:35
kfox1111then kolla-kubernetes can take that file, update values.yaml with it.19:36
kfox1111yeah.19:36
inc0well if we keep all local, we can render values.yaml for kolla-k8s from kolla19:36
*** pbourke has quit IRC19:36
inc0kolla-build pull-changes -o values.yaml19:36
inc0or sth like that19:36
*** pbourke has joined #openstack-kolla19:37
kfox1111yeah.19:37
kfox1111then that values.yaml goes through something like kolla-kubernetes-update-values.py values.yaml19:37
inc0kolla-build pull-changes -o values.yaml --repush my-registry:5000 --increment-revision19:37
*** StephenWang1991 has joined #openstack-kolla19:37
kfox1111yeah.19:37
inc0that thing could be run in cron19:38
kfox1111yeah. :)19:38
inc0values.yaml it outputs can be formated for helm19:38
inc0so we won't need 2 separate tools19:39
kfox1111or just kept very generic. key/value.19:39
kfox1111then it could be used some day by kolla-ansible too if it wanted this kind of upgrade support.19:39
inc0we'll figure this one out19:39
inc0kolla-ansible is easier, you don't need to change tags;)19:39
inc0but yeah19:39
kfox1111the tool from generic yaml to helm specific is very small.19:39
inc0anyway, that's detail19:40
inc0so to summarize19:40
inc01. we provide automatic pusher for dockerhub for tip of branch19:40
inc02. we create tool to fingerprint versions, ideally in LABEL19:40
inc03. we create tool to repush/retag changed containers to local registry19:41
inc0it will output freshest list of image:tag19:41
kfox1111the tags gotta get pushed to the hub too.19:41
kfox1111local/remote registry.19:41
inc0so that's where I don't really agree19:42
inc0I don't want to maintain all that using dockerhub, not yet anyway19:42
*** StephenWang1991 has quit IRC19:42
kfox1111whats to maintain?19:42
kfox1111we keep the most recent tag.19:42
inc0for one, kolla-ansible defaults19:42
kfox1111we delete all older ones.19:42
kfox1111keep 4.0.0, which is a link to the 4.0.0-N.19:42
kfox1111when a new one comes in, 4.0.0 points to 4.0.0-N+1, and 4.0.0-N is deleted.19:43
inc0if we keep just freshest, thats better19:43
inc0we'll effectively just mirror on dockerhub some local registry somewhere19:43
kfox1111it costs no extra space then we store now.19:43
kfox1111I'm not arguing we should keep around 100's of old versions.19:43
inc0we'll need to prep kolla-ansible for that tho19:43
kfox1111just that the way we tag things allows for change detection.19:43
inc0as today it doesn't support per-image tagging19:43
kfox1111it would just point to the generically tagged 4.0.0 as it does today.19:44
*** iceyao has joined #openstack-kolla19:44
kfox1111it would still work without changes.19:44
kfox11114.0.0 is "the freshest", (latest).19:44
kfox11114.0.0-1 is the specific build.19:44
inc0no, it will be ocata-119:44
inc04.0.0 is 4.0.0 tag in github19:44
kfox1111pike-1 maybe.19:45
inc0yeah or pike19:45
kfox11114.0.0's already out there.19:45
inc0yup19:45
kfox1111well... lets think this through...19:45
inc0that's what we're doing:P19:45
kfox1111how would you handle changes to kolla code? would you just bump the revision again?19:45
kfox1111so, using the current convention + revision,19:46
kfox1111we'd have,19:46
inc0yeah, I assume so19:46
*** tonanhngo has quit IRC19:46
kfox11114.0.0-1, 4.0.0-2, 4.0.1-1, 4.0.1-2, etc.19:46
inc0if we fix bug in kolla container19:46
*** eaguilar has quit IRC19:46
kfox1111with pike,19:46
inc0you want this bug to be fixed too19:46
kfox1111those would be pike-1, pike-2, pike-3, pike-4?19:46
inc0so I'm nto sure if we'll ever have 4.0.0-119:47
inc0we'll have 4.0.0 and that's it19:47
kfox1111Its our current "stable". we should have revisions of our current stable.19:47
*** dmk0202 has joined #openstack-kolla19:47
inc0we'll have tip of the stable branch19:47
kfox1111assuming this build system is completed while ocata is still stable.19:47
inc0and we'll have tip-revision too19:47
kfox1111if it takes till pile to implement, then ok.19:48
*** eaguilar has joined #openstack-kolla19:48
inc0(I still want :latest like tag)19:48
kfox11114.0.0 is the equiv of :latest today.19:48
*** iceyao has quit IRC19:48
kfox1111agreed, we do want some kind of equiv of :latest for stable branches.19:48
inc0well only because we don't have daily pushes19:48
inc0so we'll have daily push ocata19:48
inc0that will also be tagged ocata-119:48
inc0same image, 2 tags19:49
kfox1111k.19:49
inc0then next one will be :ocata and :ocata-219:49
kfox1111that would cover the use case.19:49
kfox1111right.19:49
kfox1111and we ignore kolla's versioning entirely?19:49
inc0we'll push tags but that's going to be one time thing I'd assume19:49
sbezverkinc0: what -2 means in the image label?19:49
inc0at least for now19:49
inc0-2 means get away from this change;)19:50
kfox1111sbezverk: container revision number.19:50
inc0-2 is revision sbezverk19:50
kfox1111container ocata-2 is a newer build then ocata-119:50
kfox1111so its an "upgrade"19:50
sbezverkinteresting, got it..19:50
*** tonanhngo has joined #openstack-kolla19:50
sbezverkwhich will be the latest then?19:51
inc0btw it's still going to be bitch to implement with docker becasue afaik docker registry currently doesn't allow you to quert all the tags available for image19:51
inc0sbezverk: latest would be :ocata19:51
inc0and :master19:51
openstackgerritKevin Fox proposed openstack/kolla master: Fingerprint containers.  https://review.openstack.org/45817419:51
*** tonanhngo has quit IRC19:51
inc0and I don't think we'll revision :master19:51
kfox1111sbezverk: just ocata19:51
inc0as it will always change19:51
sbezverkkfox1111: cool push down stack kind of things19:52
*** david-lyle_ has quit IRC19:52
kfox1111sbezverk: yeah.19:52
kfox1111sbezverk: if you see you have ocata-2 and there's a ocata-4 out on the hub, you know you need to do some updateing. :)19:52
*** tonanhngo has joined #openstack-kolla19:52
inc0I'll sumarize this on mailing list19:53
inc0and you guys check if we understood each other19:53
*** spsurya has quit IRC19:53
kfox1111k.19:53
*** mgoddard has quit IRC19:56
*** manheim has joined #openstack-kolla19:58
*** tonanhngo has quit IRC19:59
jascott1kfox1111 PR for simple --net-host was just merged. yay19:59
*** eaguilar has quit IRC20:01
kfox1111jascott1: cool.20:02
*** manheim has quit IRC20:02
*** eaguilar has joined #openstack-kolla20:02
kfox1111so I think the next couple things for that is the DaemonSet support and nodeSelector.20:02
*** eaguilar has quit IRC20:02
kfox1111with those two, I think the flannel helm chart will work.20:03
kfox1111with 1.6.20:03
*** iceyao has joined #openstack-kolla20:03
kfox1111the patch that just merged should work fine with k8s 1.5.20:03
jascott1i might just ask technosophos about daemonset before starting a thread20:03
kfox1111yeah.20:03
*** tonanhngo has joined #openstack-kolla20:05
jascott1PSA: #helm is now #helm-users and a new channel #helm-dev has been created20:07
kfox1111in slack too, or just irc?20:07
*** iceyao has quit IRC20:08
jascott1slack afaik20:08
kfox1111ok. thanks for the info. :)20:09
jascott1we are talking about it now if you have a min20:12
inc0kfox1111: mail sent, comment plz:) I'm going off for a few, need to clear my mind20:13
inc0bbiaf20:13
kfox1111k.20:13
inc0I like this solution, seems flexible and elegant20:13
*** lucasxu has quit IRC20:13
*** lucasxu has joined #openstack-kolla20:14
inc0in fact lately I started to think that we create so much awesome tools around container builds20:16
inc0that it would be great to allow other people to use it outside openstack too20:16
kfox1111+120:16
inc0something to think about:)20:16
kfox1111keeping fresh containers is a very unsolved problem in docker community outside of those containers that are purely ci/cd and are daily recieving commits.20:16
inc0keeping up to date containers that aren't equivalent to curl | sh is even harder20:17
kfox1111as the latter just punts on the problem and assumes frequency is good enough.20:17
inc0I mean you know what's inside20:17
kfox1111eyah.20:17
kfox1111yeah.20:17
inc0ok bbiaf20:18
jascott1kfox1111 well the initial idea of daemonset was not shot down so that is something. I will get something going with it20:20
kfox1111jascott1: cool. :)20:20
kfox1111when I tried it in gate, I only had to do two things. kind=Deployment to kind=DaemonSet and removing .spec.replicas.20:21
kfox1111for multinode we need --node-selector helm=true thingy too.20:21
kfox1111so we can tag only one node with helm=true.20:21
jascott1dont they run in all pods by default? so the flip is to make sure it is restricted right?20:22
kfox1111with a daemonset, it will run on all selected nodes.20:22
kfox1111I think tiller will malfunction if there is more then one though.20:22
jascott1right just thinking about safe by default (not launching a million tillers)20:23
kfox1111so being able to set a node selector and pin it to just one node ensures only one will ever run at a  time.20:23
jascott1so require a label or use default label20:23
kfox1111safest thing is probably to automatically add --node-selector helm=true.20:23
kfox1111yeah.20:23
jascott1cool th20:24
jascott1+x20:24
*** iceyao has joined #openstack-kolla20:24
*** srwilkers has quit IRC20:25
*** sayantan_ has joined #openstack-kolla20:26
kfox1111jascott1: no, thank you. :)20:26
*** iceyao has quit IRC20:29
*** rwallner has quit IRC20:29
*** srwilkers has joined #openstack-kolla20:30
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: Test out flannel helm package.  https://review.openstack.org/45696420:35
*** david-lyle_ has joined #openstack-kolla20:38
*** Serlex has quit IRC20:41
*** mgoddard has joined #openstack-kolla20:44
openstackgerritKevin Fox proposed openstack/kolla master: Fingerprint containers.  https://review.openstack.org/45817420:50
*** david-lyle_ has quit IRC20:53
*** jascott1 has quit IRC20:57
*** jascott1 has joined #openstack-kolla20:57
inc0back20:58
kfox1111wb20:58
inc0kfox1111: so, shall we start making kolla-registry toolset?20:58
kfox1111+120:59
kfox1111I just updated the fingerprinting code. one minor issue on ubuntu.20:59
inc0good! python coding20:59
kfox1111I think it will work though.20:59
kfox1111+120:59
inc0well, at the end of the day we need to figure out LABEL thingy20:59
inc0or some other way to get it out of container and put it into metadata21:00
kfox1111was wondering if there was a second option there...21:00
kfox1111we're making a yaml file with teh manifest of the most recent stuff to pass to kolla-kubernetes.21:00
kfox1111should we just post that to tarballs.o.o?21:00
inc0file inside container is good idea, but hard to get out from inside container21:00
*** jrobinson has joined #openstack-kolla21:00
*** jascott1 has quit IRC21:01
inc0so I want to make thing that's fully universal and based on 2 registries, no matter what they are21:01
inc0so you can have cicd-fed registry in your infra21:01
kfox1111yeah. its about as hard to extract from inside the container then it is to just finger print the container in place.21:01
inc0and prod registry that hosts containers deployed21:01
kfox1111fully universal good.21:01
kfox1111whould work with either hub or local repo.21:01
inc0that's the idea21:01
inc0I kinda have idea how to do the labeling21:02
*** kbaegis has quit IRC21:02
inc0we can output some well structured fingerprint to stdout21:02
inc0during build21:02
kfox1111for kolla + kolla-kubernetes, helm's the only tool the user shoudl have to interact with if they want.21:02
inc0and kolla-build will screen-scrape it and put on image21:02
kfox1111that would work.21:02
kfox1111but if you wanted to make the tooling more generic, like we talked about,21:02
kfox1111the aproach I took in the ps woudl work better.21:03
inc0well labels are generic tho21:03
inc0they're really good because you can always docker inspect running contanier21:03
kfox1111no, the idea that you run the fingerprint script on a built container, then place the fingerprint in a label.21:03
kfox1111rather then have the fingerprint spit out during build.21:03
kfox1111the former works with all containers. the latter, only ones you built.21:04
kfox1111and possibly modified the build of.21:04
*** gianpietro has joined #openstack-kolla21:04
inc0hmm, well, let's see21:04
*** jrobinson has quit IRC21:04
*** jrobinson has joined #openstack-kolla21:04
inc0that's implementation detail, I don't care that much for contianers not built by us21:04
inc0I mean it's nice if we can make it this generic21:04
kfox1111inc0: have a look at: https://review.openstack.org/#/c/458174/3/tools/gate_run.sh21:05
kfox1111it should work on any container.21:05
inc0kolla-build manifesto image-name:tag21:05
kfox1111we just change the > logs/fingerprints.txt to docker label $line $fingerprint in the loop.21:05
inc0yeah I guess21:05
inc0ok, we can do both really21:05
inc0I'll keep that in mind21:06
kfox1111probably really should split that fingerprint thingy out to its own script even.21:06
kfox1111then move the forloop outside.21:06
sdakekfox1111 - hey I know your in a conversation atm21:06
inc0it's own cli call21:06
kfox1111yeah.21:06
sdakekfox1111 but I want to intrude on a different topic21:06
sdakekfox1111 perhapas you can answere with a  yes or no21:06
kfox1111sdake: sure. whats up?21:06
sdaketo do multinode kolla-kubernetes, you need keepalived only?21:06
sdakeor keepalivd and haproxy?21:07
kfox1111sdake: heh. well...21:07
kfox1111keepalived to route traffic into the system.21:07
kfox1111haproxy is currently being used in all the api pods to do connection tracking for safe rolling upgrades and some day ssl termination.21:07
kfox1111but its not used in the way you would think of using haproxy as a load balancer.21:08
sdakehaproxy is implemented already?21:08
*** goldyfruit has quit IRC21:08
kfox1111sdake: sort of... see helm/kolla-common/templates/_common_api_python_deployment.yaml21:08
kfox1111its not used as an l7 lb.21:08
sdakekfox1111 thanks - i'll take a look21:08
inc0guys...am I understanding this correctly? docker is being renamed to moby? oO21:09
kfox1111keepalived provides a vip. kube-proxy provides an l4 lb to the pods.21:09
sdakeinc0 who woulda guessed :)21:09
kfox1111inc0: docker's being renamed essentially to containerd I thought.21:09
inc0scrach that, it uses containerd21:09
sdakebbiaf - need to collect child21:09
inc0omfg21:09
kfox1111sdake: then once the traffic is at the pod, it goes through haproxy to do connection tracking/ssl termination, then through 127.0.0.1 to the python service.21:10
inc0they made openstack for containers21:10
inc0each piece is swappable and such21:11
*** jascott1 has joined #openstack-kolla21:11
inc0which is not a bad idea tbh21:11
* kfox1111 looks up moby....21:11
jascott1have we looked for 3rd party container comparison/introspection tools?21:12
* inc0 grabs popcorn watching politics around containers21:12
jascott1man i would figure there was something `comprehensive` out there already21:12
inc0we had some notes from summit21:12
jascott1iirc there was security tool but dont recall comparison tool for 2 containers21:12
*** chas has quit IRC21:14
*** chas has joined #openstack-kolla21:14
kfox1111this sounds like its more of a modular swarm.21:14
kfox1111this is an attempt to un-upset all the parners they alientated with swarm I think.21:15
kfox1111further atemts to try and get market share away from kubernetes.21:16
jascott1know your role Docker!21:16
jascott1j/k21:16
*** jtriley has quit IRC21:16
kfox1111they as a company know what their role is. thye just don't wanto play with the emerging community.21:16
kfox1111they want to try and own a community.21:17
jascott1they do have that Canonical partnership21:17
kfox1111last I looked they were alieneting redhat/ubuntu/coreos/google/etc.21:18
kfox1111the 'partnerships' when from,21:18
kfox1111company X is working on docker,21:18
kfox1111to company X isn't blocking docker inc's stuff from working on their supported distro.21:18
*** lucasxu has quit IRC21:19
jascott1i was refering to https://www.docker.com/docker-news-and-press/docker-and-canonical-partner-cs-docker-engine-millions-ubuntu-users21:19
*** chas has quit IRC21:19
kfox1111and docker inc is doing the legwork to making docker inc;s version supported on distro X21:19
kfox1111jascott1: exactly.21:19
kfox1111all that means, is, canonical won't block docker inc's version from running well on ubuntu.21:20
kfox1111and canonical tech support has an extra page in the playbook for answering very simple questions, then forwarding more complicated stuff back to docker inc.21:20
sbezverkinc0: naming docker moby is dangerous, green peace will be after them ;)21:21
kfox1111thats very different I think then actively having sales people going out and marketing docker swarm like they do kubernetes.21:21
jascott1and that pop music guy ive never listened to ;)21:21
sbezverklike some people forced kubernetes to rename petsets :)21:21
kfox1111it looks like moby is to docker what fedora is to rhel potentially.21:21
jascott1ugh21:22
kfox1111but k8s doesn't really care about either moby or docker.21:22
kfox1111it only cares about containerd now.21:22
inc0what dick thought of renaming docker to moby...21:22
inc0:P21:22
kfox1111cause both "moby" and "docker" really are about coe functionality.21:23
kfox1111its not a rename. its kind of more like a distro.21:23
*** jheroux has quit IRC21:23
jascott1moby... dick?21:23
kfox1111look at http://mobyproject.org/ and see Audience.21:23
kfox1111its a place to prototype changes in their coe.21:23
jascott1always thought they shoulda named the whale "Lexi" as an homage to LXC roots21:24
sbezverkkfox1111: alternative name :) MOBER21:24
kfox1111:)21:24
inc0I like to moby moby \\o o//21:25
inc0sorry I had to..21:25
kfox1111sbezverk: we probably should go back through all the k8s templates and move all the non node specific config changy bits to the templates now that we manage them.21:25
*** lucasxu has joined #openstack-kolla21:26
*** kbaegis has joined #openstack-kolla21:26
*** kbaegis has quit IRC21:26
sdakekfox1111 one thing just occured to me - does kolla-kubernetes work at all on multinode kubernetes?21:27
sbezverkI have running kolla-kube on 5 node kube nodes, works like a charm21:29
sdakesbezverk cool - are you using keepalived to handle the API endpoints?21:30
sbezverkbtw sdake I will not be needing your hardware for demo21:30
sdakesbezverk roger21:30
sbezverkkeepalived21:30
sdakesbezverk thanks - only keepalived is needed?21:30
sbezverkyes21:31
sdakesbezverk ++21:31
*** david-lyle_ has joined #openstack-kolla21:33
inc0kfox1111: so, any chances you can take over segregation of gate code <> non-gate code from me? I'll get to image mgmt tool21:34
inc0will be more efficient this way imho21:34
inc0basically lets turn our gate code to series of shellscripts which, when run in correct order, will deploy kolla-k8s21:35
*** chas has joined #openstack-kolla21:35
inc0sdake: sbezverk ^21:35
jmguzmancGuys, sorry to interrupt... but we have been looking for some support with Kolla. Specifically, trying 4.0.0 and 4.0.1, we cannot see any styles in Horizon... I imagine it is something trivial to fix, but have not succeeded.... Any suggestions?21:36
inc0jmguzmanc: fresh deploy of 4.0.0?21:37
jmguzmancyes, and from stable/ocata for 4.0.121:37
inc0dockerhub containers or built by you?21:37
jmguzmancin 4.0.0 fetched containers. 4.0.1 built by me21:38
jmguzmancI tried 4.0.1 expecting to have different results, but got stuck at the same point21:38
inc0hmm21:38
inc0ok, when you try to do http request for css stylesheet, what error do you get?21:39
*** david-lyle_ has quit IRC21:39
inc0just stylesheet21:39
*** chas has quit IRC21:40
jmguzmancweird thing, is that I don´t see any css code in the html retrieved.. I already fixed an issue with "Invalid Time Zone", that prevented to serve images, so static/ is fine.. but not css are being refered from html21:41
*** sayantan_ has quit IRC21:41
inc0uhh21:41
inc0strange21:41
inc0you can find logs in /var/lib/docker/volumes/kolla_logs21:41
inc0on host21:41
inc0see if there is anything strange out there21:42
sbezverkinc0: I was under impression that we actually do exactly what you said, we runs a series of scripts to bring up the environment, I guess for me to understand what you want would require a specific example..21:44
inc0sbezverk: ok, so I ahve ubuntu machine, which scripts I run in which order to setup kolla-k8s?21:44
inc0or centos machine21:44
jmguzmancFound the logs, horizon.log with errors like Not Found: /manager/html21:44
inc0jmguzmanc: never seen anything like this...21:45
sdake@inc0 I am hopeful that segration will occur  as a result of the workshop i am holding upstream21:45
inc0sdake: it won't occur on it's own21:46
jmguzmancinc0: Ok, i will revert to 4.0.0.. Would that be easier to troubleshoot?21:46
inc0we still need to have correct howto from shellscripts21:46
*** tonanhngo has quit IRC21:46
inc0jmguzmanc: unlikely21:46
inc0hmm21:46
sdakeinc0 the workshop ends on may 4th21:46
inc0Force will be with us that day21:46
sdakeroger21:46
sbezverkinc0: you need to check the main gate setup to select which actual job you are interested in and run that scripts21:46
sbezverkwe do have bunch of hard coded things, since it is after all gate environment21:47
jmguzmancinc0: if you or somebody else wants to take a look by yourself, I can provide access to the EC2 instance where is running.. if it helps to fix something for the rest21:47
inc0that's my point21:47
sbezverkbut with minor modification you can use then without gate21:47
inc0remove hardcoes to separate files21:47
inc0so we can just run non-hardcoded pieces alone21:47
inc0without any modifications21:48
inc0jmguzmanc: if you allow me to log into it, I can take a look21:48
inc0but I'm not horizon expert21:48
jmguzmancSure, send me your pubkey, and will allow you21:48
*** salv-orl_ has joined #openstack-kolla21:49
*** tonanhngo has joined #openstack-kolla21:49
*** lucasxu has quit IRC21:49
jmguzmancinc0: jmguzman@whitestack.com21:49
inc0jmguzmanc: http://paste.ubuntu.com/24416276/21:50
*** rwallner has joined #openstack-kolla21:50
kfox1111inc0: I can put it on the list, but unfortuately, time is scarce and can't be top priority I think. :/ but maybe I can focus on it during the summit.21:51
*** tonanhngo has quit IRC21:51
*** salv-orlando has quit IRC21:52
jmguzmancinc: Try ubuntu@34.198.102.126, then sudo21:52
inc0I'll start tmux session21:53
*** lucasxu has joined #openstack-kolla21:54
inc0ok so these not-found things are from google crawler21:55
*** rwallner has quit IRC21:55
jmguzmancinc0: yeah, unrelated to the main issue.21:56
inc0jmguzmanc: fun fact...I can't see any notion of css in page source21:56
jmguzmancinc0: Exactly.. it is weird.21:56
jmguzmancinc0: http://34.198.102.126/21:56
inc0no css after login page too:/21:57
*** tonanhngo has joined #openstack-kolla21:58
jmguzmancinc0: yeap... It is my first kolla deploy, so no idea how to proceed21:58
kfox1111sdake: cool.21:59
inc0jmguzmanc: attach to tmux, you'll see me moving around;)21:59
inc0but this is strange, I can't see any notion of style in html code21:59
inc0looks like horizon issue...22:00
sdakeanyone ever read moby dick?22:00
inc0never got to22:01
jascott1nope22:01
*** mgoddard has quit IRC22:01
*** tonanhngo_ has joined #openstack-kolla22:01
sdakethe cliff notes version: https://en.wikipedia.org/wiki/Moby-Dick22:02
jmguzmancinc0: FYI, I had to edit the static/ alias, to serve images22:03
jmguzmancAlias /static /var/lib/kolla/venv/lib/python2.7/site-packages/openstack_dashboard/static22:03
*** tonanhngo has quit IRC22:03
inc0hmm22:04
jascott1bug?22:04
inc0so issue I'm seeing is not static images22:04
inc0I can't see any notion of styles alltogether in horizon22:05
*** tonanhngo_ has quit IRC22:05
jascott1sdake all my free reading time goes to playing music... to my detriment im sure22:05
jascott1oh and cat videos22:06
sdakejascott1 all my free reading time goes to openstack - definately to my detriment :)22:06
jascott1maybe its after reading all day/night just cant really get into it22:06
jascott1usually read on plane22:06
jascott1i have started some really good books :)22:07
*** lucasxu has quit IRC22:08
inc0lol when I turned on debug it worked22:09
inc0jmguzmanc:22:09
jascott1heisen bug!22:10
jmguzmancinc0: :)22:10
inc0jmguzmanc: well enjoy working css22:11
*** tonanhngo has joined #openstack-kolla22:11
*** newmember has quit IRC22:11
inc0I guess by changing config I forced django to refresh cache22:11
inc0or whatever happened22:11
*** newmember has joined #openstack-kolla22:11
jmguzmancI will try to disable debug... and see what happens22:12
inc0already done22:12
jmguzmancI REALLY appreciate your help22:12
inc0np22:12
jmguzmancI owe you a beer...:) if you go to boston!!22:12
inc0indeed I be there22:12
inc0and beers are welcome22:13
inc0jmguzmanc: are you coming to our hands on workshops for kolla-ansible?22:13
jmguzmancGreat! I hope to see you there.. traveling from Chile (south america)...22:13
inc0interesting:) I've never been to south america, but Chile is high on my list22:14
jmguzmancI expect to join as much as possible the Kolla activities.. Will check the sched for the workshop22:14
inc0see you there then, make sure to say hello:)22:15
jmguzmancWell, someday we will host a Openstack summit here... I hope:)22:15
jmguzmancThanks again for your magic touch... I saved me a lot of time.. see ya22:15
inc0I don't know how much community OpenStack has in south america22:15
inc0one of our cores, Mauricio is from Brazil22:16
inc0other than that, I'm not sure if I know anyone22:16
gianpietroinc0: there are only a few of us :) (live from Lima, Peru)22:16
inc0ha!22:16
*** chas has joined #openstack-kolla22:16
inc0definetly need to visit then;)22:17
inc0maybe openstack days at least?22:17
*** dmk0202 has quit IRC22:17
inc0awesome, makes me happy to see how diverse and global our community is!22:17
jmguzmancLet´s plan for these LATAM Openstack Days!!22:19
inc0if you do, I'll try my best to visit and preach about Kolla;)22:20
*** chas has quit IRC22:21
gianpietrojmguzmanc: yours may be the first kolla-powered Ocata deployment "down here" :')22:21
kfox1111so, will zuul3 support pipelines?22:35
jascott1weird when I join nodes it breaks tiller even if its scheduled on master22:36
kfox1111jascott1: in daemonset or deployment?22:36
jascott1network teardown, some conntainer not found22:36
kfox1111I'd expect daemonset to break. deployment to work.22:36
jascott1still deployment, ive got my own ansible/ubuntu dev playbook going22:37
kfox1111weird.22:37
jascott1if I dont join the nodes everything works22:37
kfox1111maybe the taint on master likes to push pods away if at all possible?22:37
kfox1111maybe pull the taint before adding another node and see what happens?22:37
jascott1good ideas22:37
jascott1alright going to cut up that proposal ;)22:38
*** chas has joined #openstack-kolla22:38
*** MarginHu2017 has joined #openstack-kolla22:41
*** manheim has joined #openstack-kolla22:42
*** chas has quit IRC22:43
*** salv-orl_ has quit IRC22:44
*** david-lyle has joined #openstack-kolla22:45
*** srwilkers has quit IRC22:50
*** chas has joined #openstack-kolla22:59
jascott1kfox1111 we still need tolerations tho right?23:00
*** lamt has quit IRC23:01
kfox1111k8s 1.6.2. :)23:01
kfox1111jascott1: ah. for single node, it would help.23:01
kfox1111well.... actually for single node, you really gotta drop the taint to get anything to work.23:01
jascott1but can you tolerate away NoSchedule?23:02
jascott1yeah ok i was going to say23:02
kfox1111no. :/23:02
jascott1so its nice but required?23:02
kfox1111they want to make NoSchedule a proper taint, but its a funny inbetween state tat the moment.23:02
kfox1111yeah. I think its a nice to have but not required for the use case.23:02
jascott1prob easier to go ahead and do it. ok23:02
kfox1111yeah, basically the same code as nodeSelector I think.23:03
*** chas has quit IRC23:03
*** goldyfruit has joined #openstack-kolla23:03
*** schwicht has quit IRC23:04
jascott1hmmm --tolerations or --tolerate23:04
jascott1guess im for the former although it is longer23:04
*** gfidente|afk has quit IRC23:04
kfox1111yeah, consistency probably best.23:05
*** MarginHu2017 has quit IRC23:06
*** MarginHu2017 has joined #openstack-kolla23:06
*** newmember has quit IRC23:11
jascott1https://github.com/kubernetes/helm/issues/229923:15
jascott1kfox1111 ^23:15
jascott1oops forgot to add about default label23:16
kfox1111jascott1: tolerations/node-selectors are key/value in k8s.23:16
jascott1its key or key and value right?23:17
jascott1hmm23:17
kfox1111so would be like helm init --node-selector type=helm or something.23:17
jascott1ooh right23:17
jascott1i was thnking taints23:18
jascott1you can do key:- or key:whatever-23:18
kfox1111taints are also key/value?23:18
jascott1key-23:18
jascott1ya but weird23:18
kfox1111yeah, matching whatever kubectl does would be good.23:18
jascott1right!23:18
jascott1;)23:18
kfox1111:)23:18
jascott1might need to be key:value23:19
kfox1111how kubectl run does node selectors (if it even supports them.23:19
jascott1id23:19
jascott1+k23:19
*** chas has joined #openstack-kolla23:19
kfox1111[--env="key=value"]23:20
jascott1yeah I will follow k8s23:20
*** chas has quit IRC23:24
*** manheim has quit IRC23:25
*** Margin has joined #openstack-kolla23:32
*** MarginHu2017 has quit IRC23:32
*** Margin has quit IRC23:32
openstackgerritKevin Fox proposed openstack/kolla master: Fingerprint containers.  https://review.openstack.org/45817423:33
kfox1111inc0: the fingerprinting job I think is working.23:35
inc0kfox1111: I just got to python to get it done23:37
inc0still figuring out how to do it exactly23:37
*** eaguilar has joined #openstack-kolla23:39
kfox1111inc0: its basically 2 lines of shell, and a simple shell script for runing in the container? maybe it doesn't need python?23:39
kfox1111we make a fingerprint.sh?23:39
inc0kfox1111: so 1. no for running in container23:39
inc0it has to be done on image23:40
inc0without running23:40
kfox1111why?23:40
inc0because we wont be running containers on push23:40
kfox1111that makes it very kolla specific.23:40
inc0too much of a hassle23:40
inc0well, just let me figure it out23:40
kfox1111docker run --rm -v /tmp/fingerprint.sh.$$:/fingerprint.sh $line /bin/bash /fingerprint.sh is a hastle?23:40
kfox1111k.23:40
*** chas has joined #openstack-kolla23:41
kfox1111just thinking this could be very useful outside of kolla. like, fingerprinting all sorts of containers in ci/cd.23:41
*** chas has quit IRC23:45
*** gianpietro has quit IRC23:47

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!