Tuesday, 2017-01-17

*** sacharya has quit IRC00:00
*** sacharya has joined #openstack-kolla00:00
kfox1111sdake_: thats one of the binifits of the gate. you can switch it out and see what falls out. :)00:02
kfox1111I'm a bit skeptical myself. but if it works, is stable, and faster, I'm good with it. :)00:02
portdirectcause I'm a hyper geek I've been reading the release notes, and it seems that its supported now(overlay). Overlay2 is in tech preview (at least on atomic) News to me-cause it used to be a mess, they must have updated xfs-tools(or at least changed the defaults to ftype=1) as well as backporting the kernel.00:03
sdake_portdirect cool00:03
sdake_still, I know how to sausage is made at red hat - suspect kernel even though supported for overlay - may not be supported :)00:03
kfox1111interesting.00:04
portdirectTime will tell if its preview the new awesome, or burnt fingers time again :)00:04
kfox1111right. if its a tech preview, its questionalbe if it will work or not.00:04
sdake_right tech preview means not supported :)00:04
kfox1111"supported" and "supported"00:05
sdake_tech preview in Red Hat speak = "we think it works but it hasn't undergone extensive field testing and internal testing is light"00:05
kfox1111centos isn't exactly "supported" either. :)00:05
kfox1111right.00:05
sdake_that is RHEL policy00:05
sdake_for centos, i imagine its even looser00:05
kfox1111they usually just mirror what redhat does.00:06
sdake_another way to frame tech preview is "it will eat your babies"00:06
kfox1111so it has the same level of testing redhat gave it.00:06
kfox1111"it might eat your babies" :)00:06
sdake_there ya go :)00:06
portdirectYeah, I always think of centos was the release that fell of the back of the truck :)00:06
kfox1111but in the gate, it doesn't matter if it eats the babies. just if its unstable. :)00:06
sdake_portdirect  centos is actually a midstream00:06
sdake_portdirect si its an upstream of rhel in some hard to explain way00:07
sdake_mcuh like rdo is a midstream of RHEL OSP00:07
kfox1111its a whirlpool.00:07
*** liyifeng has joined #openstack-kolla00:07
kfox1111downstream and upstream. :)00:07
portdirectWell someone's gotta file bug reports ;)00:07
kfox1111its a downstream of rhel mostly, but some stuff feeds back up.00:08
sdake_CentOS is a midstream of rhel because it feeds big chunks of mid-leve lsoftwre (like libvirt for eample, or ceph)00:08
sdake_into RHEL later00:08
sdake_whereas fedora is an upstream of RHEL because all the low level stuff happens there00:09
sdake_when red hat acquired centos, it was a rough thing for me to understand for years on end00:09
sdake_i really like where centos has gone under red hat's financial umberall :)00:09
kfox1111yeah.00:09
sdake_the upstream needs to move fast, the midstream can move slowly except for the big mid-leve lcomponents that need updating00:10
sdake_the kernel comes from RHEL maintainers for centos00:10
kfox1111centos is a thing that needs to be. there were several varients of it in the past. including whitebox linux and scientific linux is still a thing.00:11
sdake_centos IIRC doesnt' have its own kernel maint team00:11
kfox1111by owning it, redhat gets a lot of the same benifits from owning fedora.00:11
sdake_kfox1111 red hat really doesn't own centos00:11
kfox1111yeah. centos users don't want it to.00:11
sdake_or even the trademarks iirc00:11
sdake_its a community project00:11
kfox1111sure.00:11
sdake_rht aquired the main contribs as employees00:11
kfox1111thats as much as any real opensource project can be owned. ;)00:12
portdirectrh own all the trademarks00:12
sdake_rht mostly funds it now tho - vs donations which red hat can't process00:12
sdake_for legal reasons its really hard to have a non-profit in a public company at the same time :)00:13
sdake_fedora was a non-profit for a brief hot second00:13
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move iscsi precheck into its own role  https://review.openstack.org/41772500:13
sdake_they had a stamp and everything :)00:13
sdake_the main reason to be a non-profit is to be able to process donatoins00:14
sdake_without 10 tax guys sittting in a room figuring out what to do with them00:15
kfox1111openstack really needs to figure out how to share post big tent. :/00:15
sdake_how do you mean share00:15
sbezverkkfox1111: could you ack keystone ps?00:15
kfox1111sbezverk: https://review.openstack.org/#/c/412239/3600:15
kfox1111looking00:16
*** david-lyle has joined #openstack-kolla00:19
*** hfu has joined #openstack-kolla00:19
sbezverkkfox1111: neutron looks really good00:20
kfox1111sdake_: they are trying so very hard not to have dependencies on each other.00:20
kfox1111and creating workarounds in every project to avoid so, rather then work on the common parts together.00:20
sbezverkthere are a couple of things I do not understand, once you clarified, will ack it00:20
kfox1111k00:20
sdake_ok - have an errand to run - bbiaf :)00:21
*** sdake_ has quit IRC00:21
portdirectkfox1111: I'm in same boat as sbezverk - otherwise looks great00:21
kfox1111it was kind of a caryover from when neutron was the only service package being deployed.00:22
kfox1111with those false, you didn't have to have it depend on any other package.00:22
kfox1111it will still deploy.00:22
portdirectof course - but should it not be true now?00:23
kfox1111I guess I could just remove them though.00:23
kfox1111yeah. I guess so. let me fix it....00:23
*** schwicht has quit IRC00:24
sbezverkkfox1111: sounds good, otherwise I was scratching my head try to see what trick kfox1111 is acomplishing with it ;)00:24
portdirectsnap :)00:24
kfox1111:)00:24
kfox1111don't stop questioning me. most of the time I have a method to my madness. sometimes I'm just crazy. :)00:25
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Neutron service package  https://review.openstack.org/41223900:25
portdirectlol - i dont think you are the only one00:25
sbezverkyeah, I think it is common sickness, just with different level00:26
kfox1111:)00:29
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855000:35
inc0let see if it indeed uses overlay00:35
inc0btw kfox1111 I think we don't need insecure 127 after all00:36
inc0tested locally00:36
*** duonghq_ is now known as duonghq00:37
duonghqmorinng guys00:37
* portdirect tosses coin and wonders00:37
inc0portdirect, go to sleep!00:38
inc0or drink alcohol00:38
inc0or drink alcohol and go to sleep drunk00:38
kfox1111inc0: nice.00:39
portdirectThe latter is preferable :)00:39
kfox1111thats weird....00:40
kfox1111inc0: on your ubuntu box, does this work:00:40
kfox1111 docker ps -a --format 'table {{.ID}}\t{{.Names}}'00:40
inc0kfox1111, y00:41
inc0well00:41
inc0CONTAINER ID        NAMES00:41
inc034fd2e8054af        registry00:41
inc0that was output00:41
kfox1111hmm... seems to have worked.. strange.00:42
inc0why?00:42
inc0Docker version 1.12.6, build 78d180200:42
kfox1111http://logs.openstack.org/93/413193/29/experimental/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-reboot-nv/f0845a7/console.html00:43
*** david-lyle has quit IRC00:43
inc0wierd indeed00:43
kfox1111ah. I think ssh is eating the -c00:44
kfox1111arg...00:46
kfox1111lets see if this works.00:46
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: test rebooting in the gate  https://review.openstack.org/41319300:46
inc0question tho...why?00:46
inc0kfox1111, portdirect false alarm, overlay wasnt running in gates after all00:48
portdirectyeah - you'll need to clear out var lib docker for it to change00:49
portdirectso has it been running in auf/loopback all this time?00:50
portdirect*aufs00:50
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855000:51
inc0aufs on buntu, dm on centos00:52
inc0thats how we run our gates in k8s today00:52
inc0I vote for making it btrfs00:52
sbezverkbtrfs +100:52
portdirectsounds good00:52
inc0I'll play with it tomorrow then00:53
portdirectif you put that dropin before starting/installing docker it'll stick without any extra effort00:53
inc0I think we still need to prep btrfs device00:53
inc0so create loopback and shit00:54
sbezverkkfox1111: really strange but after enabling external port for keystone mariadb started crashing00:54
inc0and mount it to /var/lib/docker before docker installation00:54
inc0moreover, I think it will understand our intentions and use btfrs on it's own then00:54
inc0going home00:55
inc0talk to you guys later00:55
portdirectroger - or just curl in/copy the kolla setup - anyway time for nightcap - laters boss ;)00:55
*** inc0 has quit IRC00:56
kfox1111I'm not sure btrfs will be more stable.00:57
*** schwicht has joined #openstack-kolla00:57
kfox1111as there is not an easy way to repartition the vm's storage to slip in btrfs.00:57
kfox1111sbezverk: thats odd. maybe a network error? try again?00:58
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Keystone service deployment with kubernetes entrypoint  https://review.openstack.org/42079100:59
*** schwicht has quit IRC01:02
openstackgerritJeffrey Zhang proposed openstack/kolla: install ansible from pypi in bifrost  https://review.openstack.org/42073201:07
*** schwicht has joined #openstack-kolla01:10
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator  https://review.openstack.org/41807301:11
*** zhubingbing has joined #openstack-kolla01:12
*** YuYangWang has quit IRC01:14
*** schwicht has quit IRC01:14
*** sayantani01 has joined #openstack-kolla01:16
*** tonanhngo has quit IRC01:20
*** sayantani01 has quit IRC01:21
sbezverkkfox1111: gate is behaving strange tonight, some randon falures :(01:24
*** salv-orlando has joined #openstack-kolla01:25
kfox1111yeah. I've seen a few too.01:27
kfox1111wonder if infra's having some known issues.01:27
*** tonanhngo has joined #openstack-kolla01:27
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Keystone service deployment with kubernetes entrypoint  https://review.openstack.org/42079101:28
sbezverkkfox1111: last run was success for mariadb, but I had external_port: false in all.yaml01:29
*** sdake has joined #openstack-kolla01:29
sbezverknow I reverted back true, will see if mariadb is still running01:29
kfox1111should be false for mariadb?01:30
kfox1111its not a public service.01:30
sbezverkyeah, I was talking about keystone public01:30
kfox1111oh. ok.01:30
sbezverkit does not make sense but if it fails again now, it seems there is unknown relation..01:31
kfox1111yeah.01:31
kfox1111I'm still puzzled why 3.0.2 is failing in the gate.01:31
kfox1111everything looks like it should be working ok.01:31
sbezverkwhere is it failing?01:31
sbezverkspecific service?01:32
kfox1111http://logs.openstack.org/09/420009/5/experimental/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/c213cfc/01:32
kfox1111makes it almost to the very end.01:32
kfox1111vm's not contactable though.01:32
*** tonanhngo has quit IRC01:32
sbezverkah that wierd ssh issue01:32
kfox1111its very consistent though.01:33
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: test rebooting in the gate  https://review.openstack.org/41319301:33
openstackgerritJack Ning proposed openstack/kolla-ansible: Render ceph.conf using merge_configs  https://review.openstack.org/41973701:33
sbezverkkfox1111: could you add a couple of ip netns commands? for basic connectivity checking?01:35
kfox1111they only work in the container though?01:35
kfox1111if so, would be really tricky to get to the right container.01:35
sbezverkkfox1111: right I forgot :(01:35
kfox1111the really weird thing is I reverted just neutron to 2.0.2 and it still failed.01:36
kfox1111I'm going to try and pin back nova next.01:36
sbezverkkfox1111: on the host when VM is up can you run ifconfig we should see veth interfaces created for vm01:37
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: Support 3.0.2 in the gate  https://review.openstack.org/42000901:38
kfox1111thats here: http://logs.openstack.org/09/420009/5/experimental/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/c213cfc/logs/ip.txt01:38
sbezverkif default route is not set, which I suspect the case here, absence of interface in that subnet would trigger this error01:38
kfox1111I don't see vifs there.01:39
kfox1111but would they only show in container?01:39
kfox1111default route on what?01:39
kfox1111host routes are here: http://logs.openstack.org/09/420009/5/experimental/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/c213cfc/logs/routes.txt01:40
*** unicell has joined #openstack-kolla01:40
kfox1111172.18 ip is on br-ex, and the route is through it.01:40
kfox1111so that part at least looks ok.01:40
sbezverkright01:40
kfox1111really need to see the br-int I guess.01:41
sbezverkdo you have arp -a01:41
kfox1111I coudl try and get that.01:41
kfox1111I could also try and switch over the external ovs test over.01:41
*** zhurong has joined #openstack-kolla01:41
*** unicell has quit IRC01:41
kfox1111then the ovs should be on the host, which might be easier to inspect....01:41
sbezverkkfox1111: ovs on the host would be really good. then we could check all plugs of veth01:42
kfox1111ok. will try that.01:43
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: Support 3.0.2 in the gate  https://review.openstack.org/42000901:43
kfox1111k. gota head out. thanks for the help.01:43
*** mdnadeem has joined #openstack-kolla01:44
*** mdnadeem has quit IRC01:46
*** salv-orlando has quit IRC01:46
*** schwicht has joined #openstack-kolla01:47
sbezverkhave a good one, I am off too.01:48
*** sacharya has quit IRC01:59
*** sacharya has joined #openstack-kolla02:00
*** schwicht has quit IRC02:00
*** tonanhngo has joined #openstack-kolla02:01
*** tonanhngo has quit IRC02:02
Jeffrey4lany core around? please review https://review.openstack.org/420732  which unblock the master gate.02:08
*** tonanhngo has joined #openstack-kolla02:09
*** tonanhngo has quit IRC02:11
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: Support 3.0.2 in the gate  https://review.openstack.org/42000902:13
*** salv-orlando has joined #openstack-kolla02:13
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: test rebooting in the gate  https://review.openstack.org/41319302:15
duonghqportdirect, ping02:23
openstackgerritJeffrey Zhang proposed openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42048402:27
*** salv-orlando has quit IRC02:32
openstackgerritZeyu Zhu proposed openstack/kolla-ansible: Modify the repo in multinode.rst file  https://review.openstack.org/42103102:47
*** tonanhngo has joined #openstack-kolla02:50
*** tonanhngo has quit IRC02:51
openstackgerritZeyu Zhu proposed openstack/kolla: Remove Ansible and Globals.yml in diag script.  https://review.openstack.org/42055202:53
*** prameswar has joined #openstack-kolla02:54
*** salv-orlando has joined #openstack-kolla03:00
*** yuanying has quit IRC03:01
*** yuanying has joined #openstack-kolla03:01
*** yuanying has quit IRC03:01
*** yuanying has joined #openstack-kolla03:02
*** sayantani01 has joined #openstack-kolla03:04
*** salv-orlando has quit IRC03:05
*** yuanying has quit IRC03:06
*** sayantani01 has quit IRC03:08
openstackgerritZeyu Zhu proposed openstack/kolla-kubernetes: FIx the issue in multi-node.rst  https://review.openstack.org/42105403:29
*** jrich523 has quit IRC03:37
*** sdake has quit IRC03:38
*** zhurong has quit IRC03:40
*** mdnadeem has joined #openstack-kolla03:56
*** celo_r has quit IRC03:59
*** sayantani01 has joined #openstack-kolla04:01
*** yuanying has joined #openstack-kolla04:02
*** sayantani01 has quit IRC04:06
*** v1k0d3n has joined #openstack-kolla04:07
*** schwicht has joined #openstack-kolla04:11
openstackgerritSurya Prakash Singh proposed openstack/kolla-kubernetes: Use upper constraints for all the jobs in tox.ini  https://review.openstack.org/42106304:11
*** v1k0d3n has quit IRC04:11
*** schwicht has quit IRC04:15
*** tonanhngo has joined #openstack-kolla04:19
duonghqsrwilkers_, are you around?04:21
*** schwicht has joined #openstack-kolla04:22
*** tonanhngo has quit IRC04:22
*** schwicht has quit IRC04:25
*** jrist has quit IRC04:27
*** tonanhngo has joined #openstack-kolla04:29
*** tonanhngo has quit IRC04:31
*** jrist has joined #openstack-kolla04:39
*** sayantani01 has joined #openstack-kolla04:43
*** sayantani01 has quit IRC04:47
*** yuanying has quit IRC04:55
*** salv-orlando has joined #openstack-kolla05:01
*** yuanying has joined #openstack-kolla05:03
*** salv-orlando has quit IRC05:06
*** MasterOfBugs has joined #openstack-kolla05:09
*** prameswar has quit IRC05:13
*** unicell has joined #openstack-kolla05:14
*** v1k0d3n has joined #openstack-kolla05:14
*** MasterOfBugs has quit IRC05:15
*** skramaja has quit IRC05:15
*** pramodrj07 has joined #openstack-kolla05:15
*** skramaja has joined #openstack-kolla05:15
openstackgerritSurya Prakash Singh proposed openstack/kolla: Coverage tool package additon to test-requirements.txt  https://review.openstack.org/42107505:17
*** satyar has joined #openstack-kolla05:22
openstackgerritSurya Prakash Singh proposed openstack/kolla: Coverage tool package additon to test-requirements.txt  https://review.openstack.org/42107505:22
sp_duonghq: please review the above patch. Now we can find the code coverage report in kolla.05:23
sp_duonghq: you just need to run *tox -ecover* command after checkout the kolla repo, but prior to that you have to install coverage tool.05:25
*** v1k0d3n has quit IRC05:25
*** tonanhngo has joined #openstack-kolla05:30
*** david-lyle has joined #openstack-kolla05:33
*** sayantani01 has joined #openstack-kolla05:37
*** sayantani01 has quit IRC05:41
*** prameswar has joined #openstack-kolla05:41
*** david-lyle has quit IRC05:44
*** yuanying has quit IRC05:46
*** yuanying has joined #openstack-kolla05:47
*** yuanying has quit IRC05:51
*** david-lyle has joined #openstack-kolla05:53
*** cliles has joined #openstack-kolla06:00
*** sayantani01 has joined #openstack-kolla06:01
*** salv-orlando has joined #openstack-kolla06:02
*** pc_m has quit IRC06:03
*** sayantani01 has quit IRC06:05
sp_duonghq: ping..06:06
*** salv-orlando has quit IRC06:07
duonghqsp_, here06:14
sp_duonghq: I have updated the BP https://blueprints.launchpad.net/kolla/+spec/support-python-35 please at once06:15
sp_should I change its title too.06:15
sp_please suggest.06:16
duonghqsp_, nice, but I guess that your work deserve separate bp, (like improve unit test in kolla), but iirc there is bp like this06:16
duonghqmoment06:16
duonghqseem that nothing like this in kolla yet06:17
sp_duonghq: yes, I have just added this format of representation of coverage statistics,06:18
duonghqsp_, already read06:19
duonghqbut gate is busted (if I inspect correctly)06:19
sp_duonghq: I think We should modify the title of current BP.06:20
duonghqbifrost-base is busted06:20
sp_duonghq: yes gate is busted06:20
sp_duonghq: still investigating but not getting the root cause of this06:20
sp_since yesterday06:20
duonghqsp_, sure, but by modifying the bp's title the work items should update to reflect new goal06:21
duonghqcan you do this?06:21
duonghqIMO06:21
duonghqthere should be separate bp for your work06:21
duonghqand my bp can be depend on your06:21
sp_duonghq: So in your opinion, I should create the separate one right ?06:22
duonghqyup06:23
sp_duonghq: and need to,link the new BP to the current one06:23
duonghqsure, just my opinion06:23
duonghqlike the current one to the new bp06:23
duonghqdue to the new bp should be completed first06:23
*** david-lyle has quit IRC06:24
sp_Jeffrey4l: ping... need your opinion too on this.06:24
Jeffrey4lsp_, yep?06:24
Jeffrey4lgate red?06:25
sp_Jeffrey4l: about the creation of new BP for coverage increase.06:25
sp_in kolla06:25
sp_Jeffrey4l: yes gate is red06:26
Jeffrey4li fixed the fate. not it is not merged. https://review.openstack.org/42073206:26
Jeffrey4lre coverage: just file a new bp about this and work on the bp.06:26
*** markmcclain has quit IRC06:26
Jeffrey4lbtw, any core review around? please review https://review.openstack.org/420732 which unblock the gate.06:27
Jeffrey4lpbourke ^^06:27
sp_Jeffrey4l: thanks for making this green06:27
Jeffrey4lnp ;)06:27
Jeffrey4lactually bifrost crashes the kolla's gate. there are some bug in bifrost.06:28
*** sacharya has quit IRC06:28
*** sacharya has joined #openstack-kolla06:29
*** liyifeng has quit IRC06:29
sp_Jeffrey4l: re new BP creation:  ok, I will create and starting work right away on this.06:30
*** Jeffrey4l_ has joined #openstack-kolla06:30
*** l4yerffeJ_ has joined #openstack-kolla06:31
*** sudipto has joined #openstack-kolla06:34
*** sudipto_ has joined #openstack-kolla06:34
*** Jeffrey4l has quit IRC06:34
*** l4yerffeJ has quit IRC06:34
*** Jeffrey4l_ is now known as Jeffrey4l06:34
sudiptohi there, has anyone tried backing up the mysql container in a deployment done through koala?06:35
*** saneax-_-|AFK is now known as saneax06:35
*** zhurong has joined #openstack-kolla06:36
sp_Jeffrey4l: created the BP for same. Please approve set the target for this. https://blueprints.launchpad.net/kolla/+spec/coverage-increment-for-kolla06:39
sp_Jeffrey4l: duonghq: thanks for having discussion and making the gate green :)06:40
Jeffrey4lsp_, done ;)06:40
sp_Jeffrey4l: thanks :)06:41
Jeffrey4lnp06:41
duonghqsp_, np06:41
portdirectduonghq: you still about?06:51
duonghqportdirect, hi06:51
portdirectHey, just booting up for the day :) sup?06:52
duonghqmorning portdirect, the halcyon finished without any error but tiler still do not get up06:53
portdirectHmm :(06:53
duonghqI'm start working on memcached service, too06:53
duonghq[vagrant@kube1 ~]$ helm ls06:53
duonghqError: could not find tiller06:53
portdirectCan you download images from gcr.io?06:54
duonghqkube1                      : ok=62   changed=37   unreachable=0    failed=006:54
duonghqkube2                      : ok=39   changed=28   unreachable=0    failed=006:54
duonghqkube3                      : ok=39   changed=28   unreachable=0    failed=006:54
duonghqkube4                      : ok=39   changed=28   unreachable=0    failed=006:54
duonghqhttp://paste.openstack.org/show/595124/06:54
portdirectOK first did you run helm init?06:55
duonghqsure06:55
portdirect:(06:55
duonghq[vagrant@kube1 ~]$ helm init --client-only06:55
duonghq$HELM_HOME has been configured at $HOME/.helm.06:55
duonghqNot installing tiller due to 'client-only' flag having been set06:55
duonghqHappy Helming!06:55
portdirectTry without client only06:56
duonghq$HELM_HOME has been configured at $HOME/.helm.06:56
duonghqWarning: Tiller is already installed in the cluster. (Use --client-only to suppress this message.)06:56
duonghqHappy Helming!06:56
duonghqI run on this problem many time06:56
duonghqsometime tiler is fine06:56
portdirectOK, so your problem looks related to image pulling06:57
duonghqbut I cannot find any titler-related pod06:57
portdirectCan you vagrant ssh into a node and try to pull the controller manager image manually?06:57
duonghqsure06:58
portdirectLet's come onto tiller in a bit06:58
portdirectThe paste you showed above shows that even if it was running it would not work :(06:58
duonghqportdirect, I don't know exactly why but it seem that my setting still use 1.4.8, which it should not06:59
duonghqI passed 1.5.1 in setup-halcyon07:00
duonghqdo I need destroy and up vagrant again, portdirect ?07:01
portdirect2 mins (I'm on my phone - so will go to PC)07:01
duonghqk07:01
*** pc_m has joined #openstack-kolla07:02
portdirectduonghq: hey :)07:06
portdirectright havein 1.4.8 is weird07:06
duonghqI think you should destroy cluster and up it again?07:06
portdirectas i dont think any version of halycon has used that by default :/07:06
portdirectlets do that in a minute07:06
portdirectbut first can you ssh in kube1?07:07
portdirect I want to investivate: 'kube-system   kube-controller-manager-172.16.35.11   0/1       ImagePullBackOff    0          1h'07:07
duonghqya07:08
portdirectimagepull backoff is not a good sign - like your repo issues yesterday it looks like somthing is behaving stangly with your proxy setup07:08
duonghqI try pulling the image manually07:08
portdirectso once in kube1 sudo to root07:08
duonghqfound that v1.4.8 does not existed07:08
portdirectoh cool - your ahead of me then07:08
portdirectcan you pull v1.5.107:09
duonghqyup07:09
portdirectand it pulls?07:09
duonghqv1.5.1 is fine07:09
portdirectok destroy cluster07:09
duonghq ./setup-halcyon.sh  --k8s-config kolla  --guest-os centos07:09
portdirectbefore you start again i want to check some things07:09
duonghqit's ok?07:09
portdirectshould be07:10
*** msimonin has quit IRC07:10
portdirectbut can you paste config.rb07:10
portdirectand halcyon-kubernetes/kube-deploy/group_vars/all.yml07:10
duonghqhttp://pastebin.com/tq6CNZnL07:11
duonghqhttp://pastebin.com/g2vcJrBW07:12
duonghqguess that I should fix the no_proxy?07:12
portdirectok now i cant help there :(07:14
portdirectbut the rest looks fine07:14
duonghqok, so guess that I should try to bring up the cluster again :)07:15
portdirectyeah - thoughh if you can id give the nodes some more ram :)07:15
portdirectbut that should work07:15
duonghqportdirect, I think I can give it around 3GiB07:18
duonghqbut 2GiB slow down my pc little bit07:18
*** yuanying has joined #openstack-kolla07:23
*** salv-orlando has joined #openstack-kolla07:40
*** sdake has joined #openstack-kolla07:46
*** sdake has quit IRC07:47
*** matrohon has joined #openstack-kolla07:47
*** sdake has joined #openstack-kolla07:48
sdakeJeffrey4l morning07:48
Jeffrey4lmorning sdake07:49
sdakeJeffrey4l thanks for releasing kolla-k8s07:49
*** tonanhngo_ has joined #openstack-kolla07:49
Jeffrey4lnp. ;)07:49
Jeffrey4lcould u ack this https://review.openstack.org/420732 which block kolla's gate.07:49
*** tonanhngo_ has quit IRC07:49
*** tonanhngo has quit IRC07:51
sdakeJeffrey4l note i am not a fan of this hange https://review.openstack.org/#/c/420484/507:52
sdakeJeffrey4l do you need help finding the latest version in teh centos mirrors?07:52
openstackgerritSurya Prakash Singh proposed openstack/kolla: Code coverage improvement for kolla  https://review.openstack.org/42111307:52
Jeffrey4lsdake, epel-latest is the same with install epel-release from centos base repo.07:55
Jeffrey4li just compared epel.repo provide by centos base repo and EPEL07:55
Jeffrey4lthey are both using "mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch"07:55
Jeffrey4lurl.07:56
*** tonanhngo has joined #openstack-kolla07:56
sdakea mirrorlist is different from a cdn07:56
sdakea mirrorlist is like a cdn on the cheap07:57
Jeffrey4lepel-release from centos repo is using this mirrorlist too.07:57
Jeffrey4lepel-release from centos baser repo is not using a cdn.07:57
sdakeit is not, epel-release from entos uses their cdn07:57
Jeffrey4lno.07:57
*** tonanhngo has quit IRC07:57
sdakethis is cdn: http://mirror.centos.org/centos-7/7/07:57
sdakecentos has a cdn and also has a mirrorlist which relies on fedora07:58
Jeffrey4lyep. centos base/extras/cloud repo are on cdn. but epel-release not.07:58
sdakeoh - didn't know that07:58
Jeffrey4lcheck the folder names in the page you provides, there is no epel.07:59
sdakehttp://mirror.centos.org/centos-7/7/extras/x86_64/Packages/epel-release-7-9.noarch.rpm07:59
sdakebut you siad that doesn't use the cdn it uses the mirrorlist07:59
sdake(internally)07:59
sdakemy point is dl.fedoraproject.org is not on a cdn08:00
sdakeits a SPOF08:00
Jeffrey4lsdake, check the epel.repo file content here http://paste.openstack.org/show/595139/08:00
sdakethe epel repo itself is not on a cdn08:01
Jeffrey4lit is using mirrors.fedoraproject.org too.08:01
sdakethe file that is installed is on a cdn08:01
duonghqportdirect, everything seem working now08:01
*** tonanhngo has joined #openstack-kolla08:01
sdakeepel-release-7-9.noarch.rpm is not on a cdn if its on dl.fedoraproject.org08:01
sdakedl.fedoraproject.org should be used08:01
Jeffrey4lsdake, yep. but epel-release is updated now and then, then the gate broken. this is more critical than download epel-release from dl.fedoraproject.org.08:01
sdakemirrorlist.fedoraproject.org seems ok to me08:01
openstackgerritzhubingbing proposed openstack/kolla-ansible: itest  https://review.openstack.org/42111508:02
*** jmontleon has quit IRC08:02
Jeffrey4lsdake, check "dig dl.fedoraproject.org"08:02
Jeffrey4li think it should be on a kinda cdn. at least it is multi node rather then SPOF.08:03
sdakeJeffrey4l looking moment08:03
sdakeJeffrey4l this may be a new change08:03
Jeffrey4l209.132.181.25 belong to redhat?  hrm..08:04
sdakelooks like they have a 5 node cluster08:04
sdakethat should be good enough unless there is a dc outage08:05
*** jmontleon has joined #openstack-kolla08:05
Jeffrey4lyep. i got "209.132.181.24-27"08:05
*** skramaja_ has joined #openstack-kolla08:05
*** skramaja has quit IRC08:05
sdakeJeffrey4l ok - i'll remove my -108:06
Jeffrey4lthanks ;)08:06
sdakeJeffrey4l the way we download it is from the cdn not from mirrorlist08:08
sdakeoops wrong reply :)08:09
*** saneax is now known as saneax-_-|AFK08:09
Jeffrey4llol08:09
*** salv-orlando has quit IRC08:09
*** msimonin has joined #openstack-kolla08:14
sdakesurprised they have the frames i want for my glassses on amzn: https://www.amazon.com/Berlin-XM0067-Alwin-Black-Eyeglasses/dp/B01A91RK2U/ref=sr_1_69?ie=UTF8&qid=1484640870&sr=8-69&keywords=IC+BERLIN08:15
sdakei am ordering new frames monday08:15
*** saneax-_-|AFK is now known as saneax08:16
sdakezeiss lens looks like what i'm getting08:16
sdakenext monday i mean08:16
sdakenew glasses - have eyeglass exam08:16
*** sdake has quit IRC08:17
portdirectduonghq: WHOOP :D thats great to hear08:18
*** sdake has joined #openstack-kolla08:18
duonghqportdirect, thanks08:19
portdirectnp :) - so now you have got there, do we need to update the documentation? or were all the issues specific to your site?08:20
portdirectif the docs need updated I'd be happy to do it - just drop me a list of things and I'll get them up to scratch08:21
duonghqportdirect, I think it's about my environment08:21
duonghqbut it's great if you can add note about local_repo08:22
portdirectI'll get that into the proxy box on the docs later today :)08:22
portdirectthanks duonghq08:22
duonghqlocal repository helps much when Internet connection is not good08:22
portdirectyeah - i often forget how lucky I am on that front08:23
*** tonanhngo has quit IRC08:23
portdirectthe limiting factor for my connections are my 1gb Nics08:23
openstackgerritMerged openstack/kolla: install ansible from pypi in bifrost  https://review.openstack.org/42073208:24
duonghqportdirect, nice08:26
duonghqalways wish that I can have such kind of connection08:26
openstackgerritZeyu Zhu proposed openstack/kolla: Remove Ansible and Globals.yml in diag script  https://review.openstack.org/42055208:28
sdakeduonghq arey ou unblocked ?08:31
sdakewhy did i wake up at 1am08:31
* sdake groans08:31
sdakeprobably because i selpt all day monday and sunday :)08:32
*** prameswar has quit IRC08:33
duonghqsdake, yup, the kubernetes w/ helm is running08:34
sdakeduonghq sweet :)08:34
duonghqafter local repo trick, not so good for me08:34
*** magnumbonum has quit IRC08:34
duonghqbut it's running at least08:34
sdakei am lucky to have a raw internet connection08:34
duonghqwish that it still running to weekend08:34
sdakei am unlucky in that i have no ip addresses to use externally08:35
duonghqraw internet connection w/o public ip addresses?08:35
sdakeyup08:35
*** athomas has joined #openstack-kolla08:35
sdakei work at home08:35
sdakemy isp won't give an external ip address /26 or /2808:35
sdakerather they will at a high cost08:36
duonghqunderstood08:36
sdakemy gige to the internet is 99$/mo08:36
sdakerather 69$/mo08:36
duonghqdoes it common in the US?08:36
*** matrohon_ has joined #openstack-kolla08:36
sdakeno in the us gige is not common yet08:36
sdakeespecially at that price point08:36
duonghqsure08:37
*** matrohon has quit IRC08:38
duonghqiirc, Google fiber also offer some good p/p plans?08:38
*** egonzalez has joined #openstack-kolla08:47
duonghqsdake, you want to change you glasses?08:50
openstackgerritjimmygc proposed openstack/kolla: Use install-pip macro in most source images  https://review.openstack.org/41386108:50
sdakeduonghq yes - these glasses i bought 2 years ago are not ideal08:51
sdakeduonghq my glasses are my most prized posession of all08:51
sdakeas they help me see :)08:51
duonghqya,08:51
sdakeits a bit like having a wheelchair08:51
sdakeif I needed one, I'd buy the best damn wheelchair made08:52
sdakeits a physical dependency i have on a phyical object08:52
*** l4yerffeJ_ has quit IRC08:52
duonghqdoes the glasses has any special features08:52
sdakethey need to be absolutely perfect - or vision gets worse over time08:52
sdakethe frame i want is really rockin08:53
sdakethe lens i want is zeiss08:53
duonghqexpect optical precision?08:53
duonghq*except08:53
*** l4yerffeJ_ has joined #openstack-kolla08:53
sdakei am getting blue light coating08:53
sdakewhich is supposed to block blue light from leds08:53
sdakeleds are in every device i use around the clock -  tv - phone - laptop -etc08:53
sdakealso getting some good anti-reflective coating08:54
sdakehopefully something that doesn't magnify my eyes08:54
sdakezeiss has products for all of these problems08:54
sdakewhe nI was a kid my glasses were literally 1/4inch thick08:54
duonghqhow much does the zeiss cost?08:54
*** tovin07 has quit IRC08:55
sdakeno idea - my last pair of ic berlin frames and lens that is a competitor to zeiss was 1200 or so08:55
sdakeinsurance covers about 400 of the cost08:56
duonghqwhat kind of insurance?08:56
sdakeVSP08:56
sdakeviwsion service plan i think is what it is called08:56
*** rmart04 has joined #openstack-kolla08:56
sdakein the US VSP is pretty much the only game in town08:56
duonghqhmm,08:57
duonghqso you need take new prescription?08:58
sdakenah i hd those 4 years ago08:58
sdakeand went to a new place who sold me glasses08:59
sdaketwo years ago08:59
sdaketotally disassistified with the glasses they produced08:59
sdakeits not just hte script08:59
sdakeits who fits it and the quality of the materials08:59
duonghqsure09:00
sdakeit takes a pro to fit properly and also use quality materials and also hve a good optometrist to do an eye exam09:00
duonghqit's never easy to have proper eyeglass09:01
duonghqI haven't need eyeglass till now, quite lucky09:01
sdakehad em since i was 309:03
sdakeor 209:03
duonghqreally?09:03
sdakereally09:03
sdakemy corrected vision is about 20/3009:03
sdakewithout correctoin is terrible09:03
sdake3.25+astigmastism in both eyes09:03
*** skramaja has joined #openstack-kolla09:04
*** skramaja_ has quit IRC09:04
sdakeplus a muscle control problem (forget the name)09:04
*** Serlex has joined #openstack-kolla09:05
duonghqso, good eyeglass help you much09:05
sdakeyes - if my glasses are incorrect the muscle control problem gets worse09:07
sdakeit has gotten worse09:07
sdakefor example i look more out of my left eye then righteye09:07
sdakewhen i was a kid i was trained to use both eyes and rebuild my muscle control in my right eye09:08
sdakepart of keeping that in check is proper diags of my eyes09:08
sdakeotherwise I see double as an adult09:08
sdakehttp://www.precisioneyecareaz.com/Content/EyeConditions/strabismus2.aspx09:08
sdakeone of my doctors said my parents did a rockin job on working on my eyes when i was a kid after she examined them :)09:09
*** sdake has quit IRC09:10
*** sdake has joined #openstack-kolla09:13
sp_duonghq: I did just an improvement in kolla-k8s tox.ini. It would be great if you take a look on this for review. https://review.openstack.org/#/c/421063/09:14
*** mgoddard has joined #openstack-kolla09:16
duonghqsdake, my eyes just tell me I need to take a break09:20
sdakemy eyes tell me its bedtime again :009:20
sdakenight09:20
duonghqsp_, LMTO09:21
duonghqnight sdake09:21
sp_duonghq: i didn't get you properly :(09:23
duonghqsorry, typo, LGTM09:24
*** apuimedo|away is now known as apuimedo09:24
duonghq:)09:24
*** saneax is now known as saneax-_-|AFK09:24
sp_duonghq: :)09:25
*** gfidente has joined #openstack-kolla09:28
*** gfidente has joined #openstack-kolla09:28
openstackgerritMerged openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42048409:28
*** saneax-_-|AFK is now known as saneax09:34
openstackgerritzhubingbing proposed openstack/kolla-ansible: Optimize reconfiguration for mariadb  https://review.openstack.org/42117109:34
sdakeghostbusters - everyone has 3 mortgages these days09:36
sdakelol09:36
*** matrohon__ has joined #openstack-kolla09:38
*** matrohon_ has quit IRC09:40
duonghqsdake, I think you slept yet?09:42
sdakeya getting ready for bed - night :)09:42
*** matrohon_ has joined #openstack-kolla09:42
duonghqnight09:42
*** zhurong has quit IRC09:44
*** matrohon__ has quit IRC09:44
openstackgerritcaoyuan proposed openstack/kolla-ansible: Optimize reconfiguration for heat  https://review.openstack.org/42118409:49
openstackgerritMerged openstack/kolla-ansible: Move keystone precheck into its own role  https://review.openstack.org/41772309:52
*** prameswar has joined #openstack-kolla09:55
*** duonghq has quit IRC09:58
*** narasimha_SV_ has joined #openstack-kolla09:58
narasimha_SV_https://bugs.launchpad.net/kolla-ansible/+bug/165692909:58
openstackLaunchpad bug 1656929 in kolla-ansible "ceilometer bootstrap fails" [Undecided,New]09:58
narasimha_SV_anyone facing issue with ceilometer deployment09:59
narasimha_SV_with gnocchi as its backend09:59
egonzalezany error in gnocchi logs?10:01
egonzalezJeffrey4l: i'm looking into reconfigure optimization, as I see, deploy and reconfigure will be the same right?10:02
*** sdake has quit IRC10:03
openstackgerritzhubingbing proposed openstack/kolla: Fix designate dockerfile  https://review.openstack.org/42118810:03
openstackgerritcaoyuan proposed openstack/kolla-ansible: Move iscsi precheck into its own role  https://review.openstack.org/41772510:04
*** magnumbonum has joined #openstack-kolla10:04
*** hfu has quit IRC10:05
openstackgerritzhubingbing proposed openstack/kolla: Fix designate dockerfile  https://review.openstack.org/42118810:05
magnumbonumi have a problem with a PoC environment running kolla stable/newton. I cannot start the docker service. It has been hanging for several hours in status "starting (activating)"10:05
magnumbonumcan anybody help10:05
egonzalezdo you have too many images?10:06
egonzalezmaybe is checking md510:06
egonzalezmagnumbonum: try with systemctl status -l docker, will see messages about what is doing10:07
*** sdake has joined #openstack-kolla10:07
narasimha_SV_no no errors in gnocchi logs ?10:08
*** salv-orlando has joined #openstack-kolla10:10
*** matrohon__ has joined #openstack-kolla10:11
egonzaleznarasimha_SV_: have ceilometer_database_type = gnocchi?10:12
narasimha_SV_yes10:12
narasimha_SV_gnocchi containers are deployed properly10:12
narasimha_SV_and this issue occuring while doing resource update code in gnocchi_client.py10:13
narasimha_SV_if I remove that code this is working properly10:13
narasimha_SV_this issue is occuring from the master branch code of ceilometer and gnocchi10:14
*** matrohon_ has quit IRC10:14
*** salv-orlando has quit IRC10:14
*** sdake has quit IRC10:15
openstackgerritzhubingbing proposed openstack/kolla-ansible: Optimize reconfiguration for mariadb  https://review.openstack.org/42117110:16
narasimha_SV_http://paste.openstack.org/show/595161/ last line in gnocchi_client.py file is throwing this error10:18
narasimha_SV_but the same is working in devstack10:18
*** haplo37_ has quit IRC10:19
*** haplo37_ has joined #openstack-kolla10:19
Jeffrey4legonzalez, yep. i hope we can remove reconfigure in the future.10:19
*** sdake has joined #openstack-kolla10:21
egonzalezJeffrey4l: people need reconfigure http://lists.openstack.org/pipermail/openstack-dev/2017-January/109917.html10:21
egonzalezbut in a different way as current one10:22
*** sdake has quit IRC10:22
egonzaleznarasimha_SV_: that code snippet was changed recently, maybe we have to do something in our side https://review.openstack.org/#/c/417370/7/ceilometer/gnocchi_client.py10:22
*** tonanhngo has joined #openstack-kolla10:24
*** satyar has quit IRC10:24
narasimha_SV_egonzalez: i have the same code base10:25
narasimha_SV_what exactly need to be updated I am confused10:25
egonzaleznarasimha_SV_: dunno, is just a guess10:26
narasimha_SV_and even --skip-event-database option has been removed from ceilometer-upgrade command10:26
magnumbonumegonzalez : currently i am running dockerd in the foreground, so systemctl shows "docker.service stop-sigterm timed out. Killing."10:26
magnumbonumegonzalez : dockerd currently shows Create container failed with error: containerd: container did not start before the specified timeout10:26
*** tonanhngo has quit IRC10:29
Jeffrey4legonzalez, which part did u mean in the ml.10:29
openstackgerritzhubingbing proposed openstack/kolla-ansible: Revert "Move keystone precheck into its own role"  https://review.openstack.org/42119710:31
openstackgerrityuhui proposed openstack/kolla: Added oslo_log replacing logging library  https://review.openstack.org/42119910:32
magnumbonumegonzalez : the host is a controller, network and horizon node10:33
*** sdake has joined #openstack-kolla10:36
openstackgerritQiLiang proposed openstack/kolla-kubernetes: Fix typo in loopback file setup  https://review.openstack.org/42120410:38
egonzalezmagnumbonum: which OS you have in the host?10:38
magnumbonumcentos 7.210:38
*** zhubingbing has quit IRC10:38
magnumbonumegonzalez :i am currently using devicemapper in loop mode but in process of trying to migrate to OverlayFS10:38
Jeffrey4legonzalez, does curl can continue to download when some thing wrong happen?10:39
egonzalezmagnumbonum: there are some  bugs related to that error, could you try adding --start-timeout=2m to the start script10:39
magnumbonumegonzalez: wilco10:40
*** sdake has quit IRC10:40
Jeffrey4ltarballs.o.o site may be interrupt by some unknown issue.10:40
narasimha_SV_Jeffrey4l: any idea on the ceilometer bug https://bugs.launchpad.net/kolla-ansible/+bug/165692910:40
openstackLaunchpad bug 1656929 in kolla-ansible "ceilometer bootstrap fails" [Undecided,New]10:40
Jeffrey4lnarasimha_SV_, in my env, it works.10:41
narasimha_SV_i am doing this in Cent OS10:42
narasimha_SV_are you using the master branch code ??10:42
Jeffrey4lnarasimha_SV_, yep. master+centos+source10:43
narasimha_SV_403 forbidden means like it is unable to connect to gnocchi or other service10:43
narasimha_SV_right ?10:43
egonzalezJeffrey4l: seems curl does not support broken connections http://askubuntu.com/questions/678404/what-is-the-difference-between-curl-and-wget10:43
Jeffrey4lnarasimha_SV_, no. it may the credential is wrong.10:43
narasimha_SV_can you please send me your ceilometer.conf and gnocchi.conf10:43
egonzalezJeffrey4l: so, yeah, for that amount of data is better use wget10:43
Jeffrey4legonzalez, nice link ;)10:44
*** matrohon_ has joined #openstack-kolla10:45
Jeffrey4lnarasimha_SV_, i haven't changed the configuration file.10:45
narasimha_SV_ok let me show you mine10:45
narasimha_SV_gnocchi.conf - http://paste.openstack.org/show/595167/10:46
Jeffrey4lnarasimha_SV_, have u changed anything?10:46
narasimha_SV_sorry ceilometer.conf10:46
Jeffrey4ldoes you gnocchi works?10:46
narasimha_SV_yes gnocchi containers are up and running10:46
egonzaleznarasimha_SV_: controller hostname is resolvable?10:47
*** nrado has joined #openstack-kolla10:47
*** matrohon__ has quit IRC10:47
Jeffrey4lnarasimha_SV_, check with gnocchiclient.10:47
narasimha_SV_yes10:47
*** sdake has joined #openstack-kolla10:47
narasimha_SV_gnocchiclient ?10:47
*** pbourke has quit IRC10:48
narasimha_SV_i didn't understand10:48
*** pbourke has joined #openstack-kolla10:48
Jeffrey4lnarasimha_SV_, i haven't running env now.10:48
Jeffrey4lbut you can check https://review.openstack.org/412278  patchset 510:48
egonzaleznarasimha_SV_: http://docs.openstack.org/cli-reference/gnocchi.html10:49
egonzaleztest if gnocchi is fine doing stuff directly to it, to find if the root cause is from ceilometer or gnocchi service10:50
Jeffrey4lnarasimha_SV_, try "gnocchi metric list"10:50
egonzalezJeffrey4l: registry images in tarballs.o.o are built when tagging versions or with every change?10:52
narasimha_SV_that command output is empty10:52
Jeffrey4legonzalez, when one commit is merged.10:52
narasimha_SV_in api.log file request is success with 200 respnse code10:52
Jeffrey4lthe pack registry job is run in post pipeline in zuul.10:52
egonzalezcool, no more 5 hours waiting to build :)10:52
*** shardy has joined #openstack-kolla10:53
openstackgerritQiLiang proposed openstack/kolla-kubernetes: Improve docker images pulling  https://review.openstack.org/42121010:53
Jeffrey4legonzalez, why 5 hours?10:53
Jeffrey4lactually, it do not decrease much time for jobs in kolla-ansible.10:53
egonzalezbuild in my laptop ;)10:54
*** sdake has quit IRC10:54
Jeffrey4llol10:54
egonzalezi don't have a docker tunned (one day will do)10:54
Jeffrey4lbtw, it should work when creating tag ( need change someting in project-config)10:55
Jeffrey4lnarasimha_SV_, hrm gnocchi works.10:55
Jeffrey4lnarasimha_SV_, have u tried again?10:55
narasimha_SV_yes10:55
narasimha_SV_yes10:55
Jeffrey4lnarasimha_SV_, the same error?10:56
narasimha_SV_yes I am deploying with everyday lastest code10:56
narasimha_SV_from past 3 days I am getting the same issue10:56
narasimha_SV_but the same code is working with devstack10:57
*** sdake has joined #openstack-kolla10:57
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121210:58
Jeffrey4lnarasimha_SV_, could u post your globals.yml file and passwords.yml file out? add the link into bu descript?10:59
narasimha_SV_ok10:59
narasimha_SV_sure10:59
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix ironic inspector  https://review.openstack.org/38077410:59
Jeffrey4lthanks.11:00
*** dougs1 has quit IRC11:00
*** dougs1 has joined #openstack-kolla11:01
openstackgerritJeffrey Zhang proposed openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42121311:01
*** Serlex has quit IRC11:02
*** sudipto_ has quit IRC11:02
*** sudipto has quit IRC11:02
openstackgerritJeffrey Zhang proposed openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42121411:02
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121211:03
*** sdake has quit IRC11:03
*** g3ek has quit IRC11:03
*** Serlex has joined #openstack-kolla11:06
*** sdake has joined #openstack-kolla11:06
*** sdake has quit IRC11:07
*** sdake has joined #openstack-kolla11:09
narasimha_SV_Jeffrey4l: added both files11:11
*** sdake has quit IRC11:11
*** g3ek has joined #openstack-kolla11:12
*** sdake has joined #openstack-kolla11:15
*** matrohon__ has joined #openstack-kolla11:15
*** gfidente has quit IRC11:17
*** matrohon_ has quit IRC11:17
*** matrohon_ has joined #openstack-kolla11:18
Jeffrey4lnarasimha_SV_, what is kolla_host_mount_directory: "/var/lib/kolla"11:18
narasimha_SV_I am using local docker registry to store images11:20
Jeffrey4lnarasimha_SV_, so?11:20
*** matrohon__ has quit IRC11:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121211:21
narasimha_SV_i am storing all the images built by kolla into that local registry and my registry is mounted to /var/lib/kolla11:21
Jeffrey4lok..11:22
Jeffrey4lseems fine. i will try to deploy again to test.11:22
narasimha_SV_ok11:22
*** sdake has quit IRC11:23
SerlexHi, is it safe to test kolla-kubernetes 0.4 on a single machine?11:23
Serlexis the following doc still relevant or is there more recent one? https://github.com/openstack/kolla-kubernetes/blob/master/doc/source/development-environment.rst11:24
*** gfidente has joined #openstack-kolla11:26
*** gfidente has joined #openstack-kolla11:26
*** matrohon_ has quit IRC11:30
*** sdake has joined #openstack-kolla11:31
mliimamorning guys11:32
*** sdake has quit IRC11:32
*** sdake has joined #openstack-kolla11:34
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121211:39
*** sayantani01 has joined #openstack-kolla11:39
Jeffrey4lmorning mliima11:39
Jeffrey4lhave u tested ironic inspector? mliima11:40
*** skramaja has quit IRC11:41
*** skramaja has joined #openstack-kolla11:41
*** saneax is now known as saneax-_-|AFK11:42
*** JoseMello has joined #openstack-kolla11:43
*** saneax-_-|AFK is now known as saneax11:43
*** sayantani01 has quit IRC11:44
mliimano Jeffrey4l, I just fix merge conflict11:45
Jeffrey4lroger. but thanks mliima ;)11:45
mliimawe have many ps with merge conflict11:45
*** sdake has quit IRC11:45
Jeffrey4li will test and fix the issue.11:45
Jeffrey4lyep..11:45
*** gfidente has quit IRC11:45
mliimathis ps for example11:46
mliimahttps://review.openstack.org/#/c/420899/11:46
mliimai did it to avoid merge conflict11:46
mliimaand it have a merge conflict11:46
mliimahahahaha11:46
Jeffrey4llol11:46
*** tonanhngo has joined #openstack-kolla11:49
*** tonanhngo has quit IRC11:50
*** sdake has joined #openstack-kolla11:50
*** masuberu has joined #openstack-kolla11:51
*** gfidente has joined #openstack-kolla11:52
*** gfidente has joined #openstack-kolla11:52
openstackgerritMauricio Lima proposed openstack/kolla-ansible: Move some prechecks into its own role  https://review.openstack.org/42089911:54
*** liyifeng has joined #openstack-kolla11:54
*** sdake has quit IRC11:55
sp_portdirect: please review this. https://review.openstack.org/#/c/421063/11:55
*** sdake has joined #openstack-kolla11:58
*** gfidente^2nd has joined #openstack-kolla11:59
*** gfidente has quit IRC11:59
*** gfidente^2nd is now known as gfidente12:00
*** gfidente has quit IRC12:00
*** gfidente has joined #openstack-kolla12:00
openstackgerritMauricio Lima proposed openstack/kolla: use static uid/gid in images  https://review.openstack.org/41223112:01
openstackgerritMauricio Lima proposed openstack/kolla: use static uid/gid in images  https://review.openstack.org/41223112:02
magnumbonumegonzalez : no luck for me. same type of errors. However, I am unsure on how to set the --start-timeout=2m in the startup script12:04
magnumbonumegonzalez : i set ExecStart=/usr/bin/dockerd in /usr/lib/systemd/system/docker.service12:06
magnumbonumegonzalez also tried dockerd --exec-opt start-timeout=2m in command line12:07
*** tonanhngo has joined #openstack-kolla12:09
*** tonanhngo has quit IRC12:11
openstackgerritJeffrey Zhang proposed openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42121312:12
openstackgerritJeffrey Zhang proposed openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42121412:13
openstackgerritMerged openstack/kolla-ansible: Modify the repo in multinode.rst file  https://review.openstack.org/42103112:13
mliima+2 Jeffrey4l12:14
Jeffrey4lthanks.12:14
mliimai don't know all details behind cherry pick12:15
mliima:/12:15
*** alteriks has joined #openstack-kolla12:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121212:21
*** srwilkers has joined #openstack-kolla12:22
Jeffrey4lmliima, the core of cherry-pick is: share the same change-id in gerrit. this is required in gerrit. i think the "cherry picked xxx" message is only used for human ;)12:24
mliimahehe12:25
mliimaOk Jeffrey4l12:25
mliima:)12:25
mliimathank you12:25
Jeffrey4lnp12:25
*** srwilkers has quit IRC12:26
*** mkoderer has quit IRC12:26
*** srwilkers has joined #openstack-kolla12:27
mliimaJeffrey4l, can you review https://review.openstack.org/#/c/420899/ ?12:40
*** l4yerffeJ_ has quit IRC12:40
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121212:40
*** sbezverk_ has joined #openstack-kolla12:40
*** l4yerffeJ_ has joined #openstack-kolla12:40
*** lamt has quit IRC12:41
Jeffrey4lmliima, done.12:41
*** sbezverk has quit IRC12:42
*** athomas has quit IRC12:42
openstackgerritMerged openstack/kolla: Remove Ansible and Globals.yml in diag script  https://review.openstack.org/42055212:42
openstackgerritJeffrey Zhang proposed openstack/kolla: Move openvswitch db file into docker volume  https://review.openstack.org/41287312:45
openstackgerritMerged openstack/kolla-ansible: Move some prechecks into its own role  https://review.openstack.org/42089912:46
*** athomas has joined #openstack-kolla12:52
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla: Reduce size of images built from git repos  https://review.openstack.org/42125212:53
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla: Reduce size of images built from git repos  https://review.openstack.org/42125212:54
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Pull kolla images from tarballs.o.o  https://review.openstack.org/41372012:54
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121212:56
*** Masber_080 has joined #openstack-kolla12:56
Jeffrey4legonzalez, could check this again? https://review.openstack.org/408933  reconfigure.yml file is a soft link. gerrit show it in wrong.12:57
egonzalezJeffrey4l: sorry, forgot to remove -112:58
egonzalezis tested?12:58
Jeffrey4legonzalez, yep.12:58
*** masuberu has quit IRC12:59
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla: Add basic docs for neutron-vpnaas  https://review.openstack.org/40199612:59
Jeffrey4lthanks ;)12:59
Jeffrey4legonzalez, and this one please https://review.openstack.org/415504  :)13:00
openstackgerritMerged openstack/kolla-ansible: Optimize reconfiguration for keystone  https://review.openstack.org/40893313:01
mliimaegonzalez, Jeffrey4l https://review.openstack.org/#/c/412231/13:02
mliima:)13:02
*** dave-mccowan has joined #openstack-kolla13:02
egonzalezJeffrey4l: pbourke found that passwords are show in the output13:02
Jeffrey4legonzalez, in my patch?13:03
Jeffrey4lor in kolla-ansible?13:04
egonzalezJeffrey4l: yup, "login_password": "pass1"13:04
Jeffrey4li think we have such issue in whole kolla-ansible ;(13:04
Jeffrey4lhrm, let me check this first .13:06
Jeffrey4lthere is some way to avoid this in ansible.13:06
egonzalezJeffrey4l: yeah, but in a different way http://logs.openstack.org/31/412231/33/check/gate-kolla-dsvm-deploy-centos-binary-centos-7-nv/c90f967/console.html#_2017-01-17_12_24_24_63770813:06
egonzalezusing no_log: true to that task13:07
Jeffrey4legonzalez, 1. the issue exist all the time, right?   2. no_log for module_args in my patch?13:08
egonzalez1, yes, I though it was included with the patch but not13:10
*** jheroux has joined #openstack-kolla13:10
*** matrohon_ has joined #openstack-kolla13:10
Jeffrey4l2 should work. let me check what it will be like.  if not, i have no idea about this ;(13:10
*** lamt has joined #openstack-kolla13:11
egonzalezshould work, if not we can add it to the ansible task directly. don't know how is called in the python code13:11
Jeffrey4lwhat it mean by " add it to the ansible task directly"13:12
egonzalezhere https://review.openstack.org/#/c/415504/7/ansible/roles/barbican/tasks/bootstrap.yml13:12
*** skramaja has quit IRC13:12
Jeffrey4legonzalez, still do not get your point ;(13:14
egonzalezif we cannot add it to python module(that probably yes) we can use no_log in the roles only applying to the tasks who need to hide passwords13:15
*** saneax is now known as saneax-_-|AFK13:17
Jeffrey4legonzalez, seems not that bad http://paste.openstack.org/show/595193/13:20
Jeffrey4lcheck this first task is using no_log in ansible module, the second is using no_log in tasks. http://paste.openstack.org/show/595194/13:22
Jeffrey4legonzalez, ^^13:22
*** schwicht has joined #openstack-kolla13:24
*** cristicalin has joined #openstack-kolla13:25
*** sayantani01 has joined #openstack-kolla13:27
*** sayantani01 has quit IRC13:32
*** schwicht has quit IRC13:32
*** schwicht has joined #openstack-kolla13:35
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121213:35
*** matrohon__ has joined #openstack-kolla13:36
*** sdake has joined #openstack-kolla13:37
*** matrohon_ has quit IRC13:38
*** sdake_ has joined #openstack-kolla13:41
*** sdake has quit IRC13:42
*** sudipto has joined #openstack-kolla13:45
*** sudipto_ has joined #openstack-kolla13:45
*** zhubingbing has joined #openstack-kolla13:45
*** schwicht has quit IRC13:49
*** rhallisey has joined #openstack-kolla13:54
*** sdake_ is now known as sdake13:55
sdakemorning peeps13:55
sdakewatched ghost busters last night in 4k13:56
sdakesurprised i could sit through that as a child without getting afraid :)13:56
*** matrohon_ has joined #openstack-kolla13:58
*** tonanhngo has joined #openstack-kolla13:59
openstackgerritJeffrey Zhang proposed openstack/kolla: Install the required packages for ceph-base image  https://review.openstack.org/41480513:59
*** tonanhngo has quit IRC13:59
*** krtaylor has joined #openstack-kolla14:00
*** mdnadeem has quit IRC14:01
*** matrohon__ has quit IRC14:01
*** schwicht has joined #openstack-kolla14:02
srwilkersyo sdake14:04
sdakemorning srwilkers14:04
sdakebtw don't think i ever asked, what is yoru location14:04
sdakei'm in AZ os 7am for me14:04
srwilkersSTL14:05
srwilkershoping to change that in the next few years14:05
sdakenever been to saint loius I don't think14:05
sdakewhere you want ot move to?14:05
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121214:05
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Pull kolla images from tarballs.o.o  https://review.openstack.org/41372014:05
srwilkerslooking at Charlotte currently14:06
sdakegiven the fact that at&t is huge, I'd expect you could work out of any office you wanted14:06
sdakeNC is pretty sweet14:06
sdakeI can pretty much never move14:06
sdakeunless both I and my wife were desparate for jobs14:06
srwilkersyeah, climates a big thing for me. our weather here is awful14:06
sdakesnow and whatnot?14:06
sdakeNC is super humid in the summer times14:07
srwilkersnot even that. the saying is "dont like the weather? give it 24 hours".  had snow a few weeks ago. the very next day, was over 60 degrees14:07
sdakeis that bad? :)14:07
sdakethat melts the snow - so no digging :)14:07
sdakelots of weather swings?14:08
*** cristicalin has quit IRC14:08
srwilkersyeah -- we've got ridiculously short springs and falls14:08
srwilkersoutside of that, the weather swings like mad14:08
sdaketell me about it14:08
sdakeAZ is mostly desert, (there are parts that are mountains where it snows)14:08
sdakein the desert, the swing is like 40 degrees to 80 degrees during winter14:08
sdakeand 115 to 100 in summer14:09
sdakenighttime/daytime first one14:09
sdakedaytime/nightnighttime second one14:09
sdakeI can handle the heat pretty well because I am a vampire14:09
srwilkersyeah, i can handle heat too14:09
srwilkerscold is a different story14:09
sdakechained to my desk :)14:09
sdakei lived in flagstaff for roughly 12 years of my life14:10
sdakethe second to last year i lived there, we had the most snow of any city in the united states14:10
*** salv-orlando has joined #openstack-kolla14:10
sdakethe city was shutdown for most of winter - not designed to handle that much snow14:10
sdakeover 1 foot of snow, flagtaff shuts down until it melts14:10
srwilkersthats crazy14:10
sdakeya - considering its arizona :)14:10
sdakemost people think of arizona as blisteringly hot14:11
sdakealhtough flagstaff is full 4 seasons environment with nice stable climate14:11
*** celo_r has joined #openstack-kolla14:11
sdakei'd like to live there fulltime - however schools aren't hat good and my wife needs close access to an airport14:11
sdakethat is a real airport not the flagstaff airport14:11
sdakeonce she started working again, we moved to phoenix14:11
sdakeabout 5-6 years ago14:12
sdakeif our kids ever manage to move out, might consider living in flagstaff again14:12
sdakehave a 1ksqfoot condo there14:12
sdakeworth almost as much as my house14:12
sdakeand its paid off14:12
sdakehousing in flagstaff is rediculously expensive14:12
srwilkersthats awesome14:14
srwilkerswell, awesome youve got a space there thats paid for14:14
openstackgerritMauricio Lima proposed openstack/kolla-ansible: Unify haproxy prechecks condition order  https://review.openstack.org/41551014:14
srwilkersmy wife and i thought about moving to chicago to be closer to her family, but we didn't want to pay the housing there.  we get twice the space for less than half the price here in STL14:15
srwilkerstradeoff being that theres nothing to do in STL though14:15
celo_rHi, I'm experiencing an issue launching instances. Nova compute log says: ibvirtError: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.14:21
sbezverk_good morning14:22
celo_rI have the full logs here http://paste.openstack.org/show/595212/ This is centos with kolla 3.0.3dev14:22
sbezverk_sdake: srwilkers: can you check please this one: https://review.openstack.org/#/c/420791/14:23
*** liyifeng has quit IRC14:25
sbezverk_celo_r: you need to check also libvirtd logs14:26
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121214:26
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Pull kolla images from tarballs.o.o  https://review.openstack.org/41372014:26
sdakesbezverk_ you got it14:26
openstackgerritMerged openstack/kolla: use static uid/gid in images  https://review.openstack.org/41223114:27
celo_rHi sbezverk_ I added libvirtd logs to the paste link as well, it's just a single entry at the end14:28
srwilkerssbezverk_, looks good to me14:28
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible: Add basic docs for neutron-vpnaas  https://review.openstack.org/42129414:28
*** jtriley has joined #openstack-kolla14:32
sdakesbezverk_ not criticizing your patch - just want ot learn here14:38
*** srwilkers has quit IRC14:38
sdakehttps://review.openstack.org/#/c/420791/12/helm/service/keystone/values.yaml14:38
sdakeline 28-3014:38
sdakehow does entrypoint process that?14:38
sbezverk_sdake: entrypoint will make sure all these services are instantiated14:39
sdakeso it just checks helm for them?14:39
sdakewhat does it poll to determine whether they are enabled?14:39
sbezverk_sdake: no kubernetes cluster14:39
sdakecool - so how do we avoid race conditions on startup14:39
sbezverk_it check for kube objects14:39
sdakegot that14:40
sdakethanks :)14:40
sdakeI really like the microservice/service model we came up with :)14:40
sbezverk_sdake: it is question to entrypoint developer14:40
sdakeits exactly as I envisioned it14:40
sdakesbezverk_ roger14:40
sdakesbezverk_ i think inc0 knows some of those folks - maybe he can ask for us14:40
*** cristicalin has joined #openstack-kolla14:41
sdakesbezverk_ i think that can mostly be stamped out now14:42
sdakesbezverk_ now that you ahve written 3 of the services :)14:43
sbezverk_sdake: I am back to my operator battle14:43
sdakesbezverk_ sounds like fun :)14:43
openstackgerritLi Yingjun proposed openstack/kolla-kubernetes: Fix different repo issue for documentation  https://review.openstack.org/40505414:43
*** satyar has joined #openstack-kolla14:44
openstackgerritMerged openstack/kolla-kubernetes: Keystone service deployment with kubernetes entrypoint  https://review.openstack.org/42079114:46
*** cristicalin has quit IRC14:47
*** salv-orlando has quit IRC14:50
*** lrensing has joined #openstack-kolla14:52
*** lamt has quit IRC14:52
sdakeis the portdirect around?14:53
*** lamt has joined #openstack-kolla14:57
* portdirect is here :)14:58
sdakesbezverk_ portdirect summit planning :)14:58
openstackgerritzhubingbing proposed openstack/kolla: Fix designate dockerfile  https://review.openstack.org/42118814:59
*** saneax-_-|AFK is now known as saneax14:59
*** tovin07 has joined #openstack-kolla15:00
*** tonanhngo has joined #openstack-kolla15:01
*** tonanhngo has quit IRC15:01
*** v1k0d3n has joined #openstack-kolla15:03
*** JoseMello has quit IRC15:05
zhubingbingping mliima15:05
openstackgerritzhubingbing proposed openstack/kolla: Fix kibana dockerfile format  https://review.openstack.org/42038715:07
*** TxGirlGeek has joined #openstack-kolla15:08
*** sayantani01 has joined #openstack-kolla15:13
Jeffrey4legonzalez, re password issue, can we merge my patch first and try to solve the issue in the later. because this issue exist all the time. add no_log in the python module is hard to debug in some, imo.15:15
*** pcaruana has quit IRC15:15
*** sayantani01 has quit IRC15:18
*** tonanhngo has joined #openstack-kolla15:19
*** tonanhngo has quit IRC15:19
*** saneax is now known as saneax-_-|AFK15:20
*** sayantani01 has joined #openstack-kolla15:21
*** fguillot has joined #openstack-kolla15:21
*** Masber_080 has quit IRC15:22
*** dturner has joined #openstack-kolla15:23
*** dturner has joined #openstack-kolla15:23
*** david-lyle has joined #openstack-kolla15:24
*** david-lyle has quit IRC15:29
*** david-lyle has joined #openstack-kolla15:29
*** lrensing has quit IRC15:31
*** alteriks has quit IRC15:31
*** zhubingbing has quit IRC15:31
*** lrensing has joined #openstack-kolla15:32
*** david-lyle has quit IRC15:34
*** mgiles has joined #openstack-kolla15:36
*** tonanhngo has joined #openstack-kolla15:39
*** tonanhngo has quit IRC15:42
*** david-lyle has joined #openstack-kolla15:42
openstackgerritJiri Stransky proposed openstack/kolla: Reslient bootstrapping for MariaDB  https://review.openstack.org/42132315:43
*** sdake_ has joined #openstack-kolla15:44
*** sdake has quit IRC15:46
*** gfidente has quit IRC15:46
*** huikang has joined #openstack-kolla15:48
*** matrohon__ has joined #openstack-kolla15:49
openstackgerritcaoyuan proposed openstack/kolla-ansible: Optimize reconfiguration for heat  https://review.openstack.org/42118415:50
*** tonanhngo has joined #openstack-kolla15:50
*** matrohon_ has quit IRC15:51
*** nrado has quit IRC15:51
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible: Fix cleanup-containers script  https://review.openstack.org/42077515:52
*** shardy has quit IRC15:53
openstackgerritcaoyuan proposed openstack/kolla-ansible: Optimize reconfiguration for heat  https://review.openstack.org/42118415:53
*** shardy has joined #openstack-kolla15:54
*** adrian_otto has joined #openstack-kolla15:55
*** david-lyle has quit IRC15:55
*** Masber_080 has joined #openstack-kolla15:56
*** mattmceuen has joined #openstack-kolla15:57
*** erhudy has joined #openstack-kolla15:58
*** gfidente has joined #openstack-kolla16:00
*** gfidente has joined #openstack-kolla16:00
*** msimonin has left #openstack-kolla16:01
*** tovin07 has left #openstack-kolla16:06
*** tovin07 has joined #openstack-kolla16:07
*** david-lyle has joined #openstack-kolla16:07
*** absubram has joined #openstack-kolla16:09
openstackgerritMerged openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42121316:10
openstackgerritMerged openstack/kolla: Install epel-release-latest rather than certain version  https://review.openstack.org/42121416:10
openstackgerritMerged openstack/kolla: Move openvswitch db file into docker volume  https://review.openstack.org/41287316:11
openstackgerritMerged openstack/kolla: Install the required packages for ceph-base image  https://review.openstack.org/41480516:11
openstackgerritcaoyuan proposed openstack/kolla-ansible: Optimize reconfiguration for heat  https://review.openstack.org/42118416:15
*** nrado has joined #openstack-kolla16:15
*** hieulq_ has joined #openstack-kolla16:16
openstackgerritEdmund Rhudy proposed openstack/kolla: Add blocks for Ubuntu apt setup and package installation  https://review.openstack.org/41961016:16
*** breitz has joined #openstack-kolla16:17
*** h16mara_ has joined #openstack-kolla16:21
*** hachi has quit IRC16:23
*** sbezverk_ has quit IRC16:24
*** kencjohnston has joined #openstack-kolla16:24
*** sbezverk has joined #openstack-kolla16:25
*** cdent has joined #openstack-kolla16:25
cdentsdake_ (or anyone else): Has anyone started looking into kolla+placement api?16:27
*** rhallisey has quit IRC16:31
*** inc0 has joined #openstack-kolla16:31
inc0good morning16:31
erhudyo/16:31
openstackgerritMauricio Lima proposed openstack/kolla-ansible: Optimize reconfigure action for manila  https://review.openstack.org/42136516:35
*** saneax-_-|AFK is now known as saneax16:35
kfox1111morning.16:35
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix gate in iax-ord node when using libvirt 2.0  https://review.openstack.org/42048016:37
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix the variable name mismatch  https://review.openstack.org/41480416:37
*** vhosakot has joined #openstack-kolla16:38
*** mattmceuen has quit IRC16:38
*** kbaegis has joined #openstack-kolla16:38
openstackgerritJeffrey Zhang proposed openstack/kolla: Add neutron vpnaas code into neutron-server container  https://review.openstack.org/41997716:38
kbaegisHi all. Kolla 3.0.2 is failing to deploy the log volume: AttributeError: Client object has no attribute 'volumes'16:38
*** rhallisey has joined #openstack-kolla16:39
openstackgerritMerged openstack/kolla-ansible: Render ceph.conf using merge_configs  https://review.openstack.org/41973716:40
kfox1111mgiles: sorry that took so long to fix.16:44
mgilesNo problem, but glad to have it working now.16:44
kfox1111+1 :)16:45
*** kbaegis has quit IRC16:46
*** lrensing has quit IRC16:48
*** salv-orlando has joined #openstack-kolla16:51
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: TEST depends on  https://review.openstack.org/42121216:52
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Pull kolla images from tarballs.o.o  https://review.openstack.org/41372016:52
pbourkekbyrne: make sure you have the correct dependency versions as outlined in the docs, in particular docker-py16:53
*** huikang has quit IRC16:54
*** kbaegis has joined #openstack-kolla16:54
*** salv-orlando has quit IRC16:54
*** huikang has joined #openstack-kolla16:55
*** salv-orlando has joined #openstack-kolla16:55
*** huikang has quit IRC16:55
*** huikang has joined #openstack-kolla16:55
*** jrich523 has joined #openstack-kolla16:56
*** inc0_ has joined #openstack-kolla16:58
*** egonzalez has quit IRC17:07
*** jrist has quit IRC17:08
*** shardy has quit IRC17:09
*** shardy has joined #openstack-kolla17:10
*** cdent has quit IRC17:10
jascott1good morning17:11
*** saneax is now known as saneax-_-|AFK17:12
*** jrist has joined #openstack-kolla17:13
openstackgerritMauricio Lima proposed openstack/kolla-ansible: Fix octavia image names in start.yml  https://review.openstack.org/42138817:14
*** hieulq_ has quit IRC17:17
*** Serlex has quit IRC17:17
*** matrohon_ has joined #openstack-kolla17:18
*** rmart04 has quit IRC17:19
*** bmace has quit IRC17:21
*** matrohon__ has quit IRC17:21
*** bmace has joined #openstack-kolla17:21
narasimha_SV_Jeffrey4l: this time I am getting 401 unauthorized issue for ceilometer + gnocchi17:22
narasimha_SV_i pulled the latest code in master17:22
*** tovin07 has quit IRC17:22
*** sudipto has quit IRC17:24
*** sudipto_ has quit IRC17:24
*** v1k0d3n has quit IRC17:25
*** shardy has quit IRC17:25
*** v1k0d3n has joined #openstack-kolla17:25
*** david-lyle_ has joined #openstack-kolla17:26
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Neutron service package  https://review.openstack.org/41223917:27
kfox1111kolla-kube cores: we need to try and not merge service packages with --debug still enabled.17:27
kfox1111they add a LOT of stuff to the logs.17:27
*** unicell has quit IRC17:28
*** matrohon__ has joined #openstack-kolla17:29
*** inc0_ has quit IRC17:30
*** rhallisey has quit IRC17:30
*** matrohon_ has quit IRC17:31
*** inc0_ has joined #openstack-kolla17:32
*** matrohon_ has joined #openstack-kolla17:32
*** v1k0d3n has quit IRC17:32
*** rhallisey has joined #openstack-kolla17:33
sbezverkkfox1111: morning, how about the clean up ps after most of services are done?17:33
*** sacharya has quit IRC17:33
*** sacharya has joined #openstack-kolla17:33
kfox1111I slipped the cleanup into the neutron one. hoping that will merge soon.17:33
kfox1111brb. got a meeting.17:33
*** david-lyle_ has quit IRC17:34
*** athomas has quit IRC17:35
*** matrohon__ has quit IRC17:35
openstackgerritMarcus Williams proposed openstack/kolla: Add OpenDaylight Container to kolla  https://review.openstack.org/41636917:36
*** chopmann has joined #openstack-kolla17:39
*** matrohon_ has quit IRC17:42
openstackgerritMerged openstack/kolla: Fix kibana dockerfile format  https://review.openstack.org/42038717:46
*** tonanhngo has quit IRC17:46
*** v1k0d3n has joined #openstack-kolla17:48
*** nrado has quit IRC17:49
*** nrado has joined #openstack-kolla17:50
*** nrado has left #openstack-kolla17:50
*** huikang has quit IRC17:52
*** huikang has joined #openstack-kolla17:52
*** absubram has quit IRC17:55
*** huikang has quit IRC17:56
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator  https://review.openstack.org/41807317:57
chopmannwhat is the precedure for enabling tempest. Im unsure about "tempest_image_id", if this is the first deployment i dont have such an ID.17:59
kfox1111back17:59
chopmannshould i deploy, push image and reconfigure?17:59
*** prameswar has quit IRC18:00
*** david-lyle_ has joined #openstack-kolla18:00
*** mgoddard has quit IRC18:02
*** inc0_ has quit IRC18:03
sdake_cdent pinged me earlier about a placement api - no idea what that is :)18:04
sdake_mgiles morning fine sir18:04
sdake_mgiles we released 0.4.0 over the weekend18:05
jrich523pssh i see what you think of me, i never get address in such a loving way18:05
jrich523im jk ;)18:05
jrich523how are you guys this fine day18:05
sdake_mgiles now we are ready for 0.5.0 - I think most of the team is focused on this blueprint: https://launchpad.net/kolla-kubernetes/+milestone/0.5.018:05
sdake_mgiles specifically this blueprint -> https://blueprints.launchpad.net/kolla-kubernetes/+spec/helm-services18:06
sdake_sup jrich523 :)18:06
sdake_jrich523 didn't know you were alive :)18:06
jrich523its questionable :)18:06
sdake_jrich523 back to back meetings for the rest of th week - FUN18:06
jrich523oh geesh18:06
jrich523big happenings at Cisco?18:06
kfox1111hmmm....18:06
sdake_jrich523 nah - cisco culture ithink18:06
kfox1111I reverted nova to 2.0.2 and the network started working...18:07
sdake_jrich523 i just ignore half of them18:07
sdake_;-)18:07
jrich523ahh, well start of the year, im sure it gets a little heavier around this time18:07
kfox1111so its something about nova maybe.18:07
*** absubram has joined #openstack-kolla18:07
jrich523lol i had a 7:30am meeting i was suppose to be on, forget that!18:07
sdake_kfox1111 2.0.2 does not use the virt sig's work output - instead it is using a different version of libvirt18:07
jrich523if a client isnt involved its fairly rare i'll show up18:07
sdake_jrich523 for me 7am meetings are get the kids to school thing18:07
sdake_i just block 7-9 out of my calendar18:08
jrich523you work from home?18:08
sdake_although i am usually up by 6 doing a little work18:08
kfox1111sdake_: interesting. maybe I'll try just reverting libvirt. that should still work maybe.18:08
sdake_jrich523 for about 12 years18:08
jrich523lol i struggle to get out of bed by 8am18:08
sdake_kfox1111 ya it may18:08
jrich523wow, thats a long time, cisco the whole time?18:08
sdake_jrich523 no red hat fo r9 years18:08
sdake_cisco for218:08
sdake_i guess only 11 years18:08
sdake_anyway - long time hiding out :)18:09
sdake_jrich523 i worked for pmyers for a big chunk of it at red hat18:09
*** kbaegis has quit IRC18:09
sdake_had another manager kevin anderson prior who left the co18:09
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: Support 3.0.2 in the gate  https://review.openstack.org/42000918:09
jrich523pmyers? whats that, name rights a bell18:10
jrich523rings18:10
sdake_perry myers18:10
sdake_jrich523 aren't you a red hatter?18:10
jrich523hmm i dont even know what that is :)18:12
jrich523as in, a RHEL user?18:12
jrich523and if so, not really, im not even a nix guy, just working with a client who has a lot of RHEL systems... i work for a VAR, so i do whatever is currently needed18:13
*** kbaegis has joined #openstack-kolla18:16
*** prameswar has joined #openstack-kolla18:17
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: test rebooting in the gate  https://review.openstack.org/41319318:19
jrich523mostly im just a powershell guy, but when you write scripts to automate things people just expect you to do it all and understand the tech you're automating (not really how it works :)18:20
*** huikang has joined #openstack-kolla18:23
*** unicell has joined #openstack-kolla18:26
*** vhosakot has quit IRC18:26
*** SP__ has joined #openstack-kolla18:26
mgilessdake_ good morning. Thanks for the update.  I added my name to one of the work items.18:27
sdake_mgiles cool thanks :)18:27
sdake_jrich523 wierd for some reaosn i thought you workdd at red hat - oh well my bd :)18:28
sdake_bd/bad18:28
jrich523they would be ashamed to have me :)18:29
*** kbaegis has quit IRC18:30
SP__mliima: it would be great if you review it again  https://review.openstack.org/#/c/414813/18:31
*** inc0_ has joined #openstack-kolla18:31
SP__mliima: coverage report will be after this patch merging in kolla. Please review this too     https://review.openstack.org/#/c/421113/18:33
*** kbaegis has joined #openstack-kolla18:33
SP__mliima: *will be ready*18:34
*** kbaegis has quit IRC18:35
*** krtaylor has quit IRC18:36
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855018:39
sdake_jrich523 they had me - so dobut it :)18:40
jrich523lol geesh who havent you worked for?18:40
jrich523microsoft of course lol18:40
sdake_intel, montavista, redhat, then cisco18:41
sdake_only 4 jobs in 18 years18:41
jrich523those in order?18:41
*** huikang has quit IRC18:42
jrich523Intel right out of college, or are we skipping the jr level jobs? :)18:42
*** lrensing has joined #openstack-kolla18:42
*** huikang has joined #openstack-kolla18:42
sdake_in order18:43
sdake_i also was a lab director at university18:43
sdake_before that all18:43
sdake_for a dorm18:43
sdake_but that was barely a job18:43
jrich523yeah i was a 'lab monitor' back in college too, got to pay for it some how!18:43
jrich523mostly it was paid time to do homework :)18:43
sdake_i made 6 bucks an hour or something18:43
sdake_what at the time was decent18:44
jrich523yeah i got something horribly similar18:44
jrich523it was probably min wage at the time18:44
sdake_i also was a lab manager at our engineering college for a couple years, grading papers, helping students, etc18:44
jrich523which i think is whati got, maybe a tad more18:44
sdake_nah min wage was 4.2518:44
jrich523lol showing age :)18:44
sdake_prior to that i worked for my uncle rebuilding motors for a couple weeks on a contract gig :)18:45
jrich523i think i was like 16 when that was the min wage, maybe even younger18:45
sdake_i determined the electric motor business wasn't for me18:45
jrich523lol yeah, labor based jobs are def not for me :)18:45
sdake_even though my uncle and my cousin retired with about 40 and 20 mil each respectively18:45
*** mgoddard has joined #openstack-kolla18:46
sdake_those are all the gigs i've had18:46
sdake_i don't list the stuff prior to graduation18:46
sdake_oh for a couple summers i  worked at a co called infomagic18:46
sdake_or maybe it was during the school year - forget which18:46
sdake_doing tech support18:46
*** huikang has quit IRC18:47
sdake_"how would you like your xwindows today?"18:47
jrich523lol18:47
sdake_"how would you like your partitions setup today?"18:47
sdake_60/40 split on those two questions18:47
jrich523the first 'real' job i had was tech support at an ISP, back when i was like 1518:47
*** david-lyle has quit IRC18:47
sdake_ya infomagic was also an isp18:47
sdake_but they sold red hat cds18:47
sdake_in fact, the first 300 cds ordered were from infomagic18:48
jrich523lol the most common conversation was "are you SURE you're typing your password correctly, because im almost certain you arent"18:48
sdake_I tried to talk joel into going big18:48
sdake_bug he wasn't interested sadly18:48
SP__sdake_:  seems like mliima is not around. It will be helpful if review these patches of corresponding BPs. one for coverage tool ready for coverage report generation and other for better python coding style respectively https://review.openstack.org/#/c/421113/   https://review.openstack.org/#/c/414813/18:48
*** schwicht has quit IRC18:48
sdake_SP__ i've got 8 minutes - will get thru what i can moment18:48
*** prameswar has quit IRC18:48
*** schwicht has joined #openstack-kolla18:49
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: Support 3.0.2 in the gate  https://review.openstack.org/42000918:50
*** david-lyle_ has quit IRC18:50
*** inc0_ has quit IRC18:51
*** tonanhngo has joined #openstack-kolla18:52
*** tonanhngo has quit IRC18:52
*** tonanhngo has joined #openstack-kolla18:53
sdake_SP__ nice work there18:56
sdake_ok gotta jet for awhile18:57
sdake_enjoy18:57
*** pramodrj07 has quit IRC18:57
SP__sdake_: thanks :)  though still lot of work is there to do18:58
sdake_SP__ agreed - coverage for kolla-kubernetes would be helpful18:58
sdake_SP__ as far as python3.5 goes, we don't have to tackle that immediately on the kolla-kubernetes deliverable18:58
SP__sdake_: i have eyes on that too18:58
sdake_SP__ cool thanks :)18:58
mliimai'm back SP__19:01
SP__mliima: done by sdake_ . thank for reply19:02
SP__mliima: you are Mauricio Lima right ? if i am correct19:03
*** lrensing has quit IRC19:07
jrich523you scared him off by outting his secret!19:07
*** kbaegis has joined #openstack-kolla19:08
mliimayes19:10
mliima:)19:10
mliimaSP__,19:10
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855019:10
SP__mliima: ok19:10
openstackgerritMark Giles proposed openstack/kolla-kubernetes: WIP - Nova service deployment  https://review.openstack.org/42145719:11
openstackgerritMerged openstack/kolla-ansible: Allow operators to use 'fallback mode' for Ceph disks  https://review.openstack.org/39837319:11
*** l4yerffeJ_ has quit IRC19:13
*** l4yerffeJ_ has joined #openstack-kolla19:13
*** satyar has quit IRC19:14
kfox1111sdake_: confirmed.19:16
sdake_kfox1111 its libvirt version?19:16
kfox1111reverting just libvirt to 2.0.2 fixes the network issue, but it fails cinder as its too old.19:16
sdake_kfox1111 how about 3.0.1 of libvirt images?19:16
sdake_kfox1111 can you try that  there is a regression in libvirt in the 3.0.2 images19:17
kfox1111when I moved everything to 3.0.1 it didn't help. so I don't think so.19:17
kfox1111though I'm a little fuzzier on that one.19:17
sdake_so your running 3.0.2 except for libvirt 3.0.1?19:17
sdake_rather 2.0.2 of libvirt?19:17
inc0ehh...do I need to tell you how ugly this is?19:17
kfox1111sdake_: https://review.openstack.org/#/c/420009/10/helm/all_values.yaml19:17
kfox1111inc0: just trying to narrow down why 3.x isn't working.19:18
kfox1111inc0: isolated it down to the libvirt 3.x container.19:18
kfox1111now to figure out why.19:18
*** MasterOfBugs has joined #openstack-kolla19:18
inc0try checking version of libvirt in both19:18
inc0kfox1111, also, both buntu and centos fails?19:20
jascott1inc0 https://github.com/jascott1/helm19:20
kfox1111havent been trying ubuntu yet.19:21
kfox1111http://paste.openstack.org/show/NhPAOqdHKlH47KuaiRiJ/19:22
kfox1111quite a bit of version difference.19:22
*** schwicht has quit IRC19:22
inc0something happened in qemu 2.6...hmm hmm19:23
inc0I think19:23
*** schwicht has joined #openstack-kolla19:23
inc0the bell does ring19:23
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: Support 3.0.2 in the gate  https://review.openstack.org/42000919:24
*** unicell has quit IRC19:24
kfox1111hmm..... so I set libvirt to explicit 3.0.1.19:25
kfox1111lets double check that.19:25
kfox1111rpm -qf on it shows:19:25
kfox1111qemu-kvm-ev-2.3.0-31.el7.16.1.x86_6419:26
kfox1111which is much closer to the 2.0.2's qemu-kvm-ev-2.3.0-31.el7_2.10.1.x86_6419:26
kfox1111though the 3.0.1 version seems older??19:26
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855019:27
*** lamt has quit IRC19:27
inc0oO19:28
kfox11112.0.2's build date was:19:28
kfox1111Fri Apr  8 11:55:36 201619:28
kfox11113.0.1's:19:28
*** kbaegis has quit IRC19:28
kfox1111Fri Jul  8 12:49:17 201619:28
kfox1111ok. so just a weird looking revision.19:29
*** kbaegis has joined #openstack-kolla19:29
kfox1111sbezverk: portdirect: srwilkers_: https://review.openstack.org/#/c/42145719:32
kfox1111a conversation going on in that review.19:32
portdirectcheers kfox111119:33
*** matrohon_ has joined #openstack-kolla19:34
*** mgoddard has quit IRC19:35
*** diablo_rojo has joined #openstack-kolla19:38
diablo_rojoHello, just wanted to give you all a reminder that the period for the Travel Support Program for the PTG ends today. We would love to get more people from kolla to the PTG. Here is the link to the TSP Application: https://openstackfoundation.formstack.com/forms/travelsupportptg_atlanta19:38
kfox1111diablo_rojo: thanks for the reminder.19:39
diablo_rojokfox1111, No problem :)19:39
openstackgerritMerged openstack/kolla: Code coverage improvement for kolla  https://review.openstack.org/42111319:40
portdirectdiablo_rojo: application lodged, fingers crossed, thanks for the reminder :)19:44
diablo_rojoportdirect, No problem :)19:44
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: WIP: test rebooting in the gate  https://review.openstack.org/41319319:44
sbezverkportdirect: how broken is entrypoint for daemonset dependency?19:46
kfox1111sbezverk: yeah, we may want to add an explicit dep on libvirt from nova, like we did for ovs?19:46
kfox1111a socket dep?19:46
portdirectaccording to them its ok, provided we set the pod name env var19:46
portdirectcan get that  from downward api - 2 sec and I'll check19:46
kfox1111not daemonset. socket19:46
sbezverkkfox1111: is it always sokcet used for communicating between libvirtd and compute?19:47
kfox1111sbezverk: yeah.19:47
sbezverkkfox1111: in this case, sure19:47
portdirectcould be ip, but then youd be really weird..19:48
kfox1111probably should have a socket dep on ovs as well.19:48
kbaegisHey guys. If I have multiple lv groups in cinder, should I code that into the /etc/kolla/config/ overrides?19:48
portdirectnot tested socket with k8s-entrypoing19:48
kfox1111portdirect: one of the neutorn services has it working I think.19:49
*** unicell has joined #openstack-kolla19:49
portdirectoh - nice :)19:49
kfox1111the advantage of socket is it doens't have to reach back to the controler just to check on locally running stuff.19:49
portdirect+10019:49
*** msimonin has joined #openstack-kolla19:50
des_consoladoholyshit, finally I got internet connectivity from an instance!!!19:50
sbezverkkfox1111: we already have dependency for ovs19:50
kfox1111des_consolado: congrats. :)  thats uaually the hardest part. :)19:50
*** david-lyle has joined #openstack-kolla19:50
portdirectdes_consolado: welcome to the i got openstack running club :)19:50
kfox1111sbezverk: in libvirt/nova-compute container?19:50
sbezverkdo you mean nova also needs19:50
kfox1111yeah.19:50
des_consoladothe main problem was that friggin br-ex and br-tun needed to be manually put up19:50
portdirectdes_consolado: took me weeks to get there the first time :)19:50
des_consoladohowcome our kolla-ansible isn't taking charge of this and setting that shit to come up on boot?19:50
kfox1111portdirect: yeah, neutron may benifit from the same treatment.19:51
kfox1111splitting the network node from the control nodes.19:51
sbezverkkfox1111: I am not sure, because if in future we add something instead of ovs, then we need to change code19:51
des_consoladoyeah me too lol... I'm lucky that my manager is pretty cool but I still feel like he might suspect I'm a bit of a moron to take so long just to get instance connectivity on some cloud infrastructure :P19:51
*** v1k0d3n has quit IRC19:51
kfox1111sbezverk: yeah. would maybe need some conditional deps in that case.19:51
des_consolado(doing R&D PoC stuff)19:51
des_consoladoalso egonzalez was the most helpful motherfucker ever19:52
portdirectkfox1111: yeah - I'm on the fence there - I'm not sure if it may create more problems than it solves, but thought we should consider it19:52
*** v1k0d3n has joined #openstack-kolla19:52
sbezverkkfox1111: with condition would be great19:52
kfox1111des_consolado: I'll vouch for you. network connectivity in openstack is by far the hardest part. takes a long time. :)19:52
mgileskfox1111 Did you say socket dep is already working somewhere?  Do you have a pointer?19:52
portdirectkfox1111: i think we may find a lot of people wanting calic, romana and ovn in k8s19:52
*** gfidente has quit IRC19:53
kfox1111mgiles: helm/microservice/openvswitch-vswitchd-daemonset/templates/openvswitch-vswitchd-daemonset.yaml19:53
mgilesthanks19:53
openstackgerritMerged openstack/kolla: Py3 and py2 implementation for kolla/image module  https://review.openstack.org/41481319:53
kfox1111portdirect: that could be.19:53
openstackgerritMerged openstack/kolla: Fix the variable name mismatch  https://review.openstack.org/41480419:53
portdirectkfox1111: at least speaking personally I'm not going to be using reference neutron :)19:54
*** chopmann has quit IRC19:55
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855019:55
sbezverkkfox1111: maybe we should hold on with ovs dependency, especially that it is noce to have and not really required..19:55
kfox1111portdirect: ok.19:56
kfox1111portdirect: would be nice to see what you have in mind. :)19:56
kfox1111sbezverk: well, I have the reboot gate job in progress. I may very well see it hit there too.19:56
des_consoladoany input on whether br-ex/br-tun should be set to come up automatically by kolla-ansible (incl. on boot)? they're not right now, having to do it manually, wondering what I should do about this19:57
kfox1111so we may need a solution sooner or later.19:57
des_consoladothis is on Ubuntu I will test on CentOS tomorrow19:57
portdirectkfox1111: probably ovn, and running a common networking layer between k8s and openstack - the same as i did for harbor19:57
kfox1111portdirect: you really want to expose undercloud stuff to the overcloud?19:58
kfox1111do they have good support for some kind of security?19:58
*** narasimha_SV_ has quit IRC19:58
kfox1111I've been meaning to look closer at canal's security rules.19:59
*** unicell has quit IRC19:59
portdirectkfox1111: canal is just calico and flannel19:59
*** v1k0d3n has quit IRC19:59
sbezverkkfox1111: is it not calico which is security rule provided in canal plugin?19:59
portdirect^^ yes, its just a way of getting fanel to support calicos security logic20:00
kfox1111portdirect: right. but calico has some "securitygroup" like rules I think in k8s.20:00
kfox1111sbezverk: they have support for it, but you have to use it. by default its wide open I think.20:01
kfox1111ok.... so.20:01
portdirectyes they are all wide open by default20:01
kfox11113.0.2 everything but 3.0.1 libvirt worked on the external ovs.20:01
portdirectbut using kuryr-k8s (or raven whick i am currently using) you can apply neutron security groups to the pod - and with ovn this is a big step up from most solutions20:02
kfox1111so, yeah. 3.0.2 seems broken'er then 3.0.1 for libvirt somehow.20:02
kfox1111portdirect: yeah. been meaning to lookin to that. but would it work well having neutron hosted out of the same cluster using it?20:03
kfox1111or do you build out a second k8s for using kuryr?20:03
portdirectthe soltion to this was quite simple once i cracked it20:03
portdirecti run some keystones and neutron-servers in the hosts network ns20:04
portdirectand these are the ones that k8s uses20:04
kfox1111outside of k8s?20:05
portdirectno - just with net=host20:05
*** adrian_otto has quit IRC20:06
kfox1111and no lb? or some other solution there?20:06
kfox1111(svc)20:06
portdirectevery service is headless20:06
portdirectand i use ingress controllers20:06
kfox1111but ingress would require the network connectivity, so wouldn't work for that neutron?20:07
portdirectthe ingress controlers also run in in the network ns20:07
kfox1111net=host?20:07
portdirectyes20:08
portdirecti run them as daemonsets20:08
kfox1111so they can bind to the headless service for the local neutron?20:08
kfox1111interesting.20:08
portdirectyup20:08
portdirectI also run kube-dns with net-host20:09
portdirectand then use freeipa's bind to provide all cluster dns20:09
*** sambetts|afk has joined #openstack-kolla20:09
kfox1111hmm. interesting.20:09
kfox1111do you build out an external facing keystone/neutron on top of that k8s as well?20:10
portdirectyes20:10
kfox1111I guess you could scale out the neutron/keystone further.20:10
kfox1111the mariadb would be the only tricky thing.20:10
portdirectexactly20:10
portdirectthats the weak point with this setup20:11
kfox1111yeah.20:11
kfox1111interesting though.20:11
*** salv-orlando has quit IRC20:11
portdirectbut i run a seperate one for each service, so its only keystone/neutron that require special care20:11
sambetts|afkportdirect: Hi, I'm working on setting up a kube cluster for kolla on centos, and I wonder if you had any thoughts about flanneld vs CNI, guides using kube 1.3.0 seem to use flannel, but 1.5.0 use CNI20:12
portdirectid use CNI for sure20:12
kfox1111cni's the newer way of doing things, yeah. but usually you often will use flannel on top of cni.20:12
portdirect^^20:12
portdirectcalico is probably the easyest one to get going with kubeadm at the moment20:13
kfox1111canal?20:14
sambetts|afkOh interesting, /me is very new to kube, I followed this guide https://kubernetes.io/docs/getting-started-guides/centos/centos_manual_config/ and got a cluster going, but I want to try to be more up to date20:14
kfox1111do they have a pure flannel yet?20:14
portdirectkfox1111: yeah - but it requires a few extra steps20:14
kfox1111really? weird.20:15
kfox1111looks like there is a flannel object now: https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml20:16
portdirectkfox1111: yeah but you need to add a few optons to kubeadm, i forget what off the top of my head20:16
kfox1111so should be as easy as canal?20:16
kfox1111the docs say launch kubeadm,20:17
kfox1111then create the networking plugin by hand.20:17
kfox1111so should be the same instructions whichever plugin you use?20:17
kfox1111https://kubernetes.io/docs/getting-started-guides/kubeadm/20:17
kfox1111step 3.20:17
portdirectkfox1111: flannel is not20:17
portdirectkfox1111: looks like they have just got it down to '--pod-network-cidr=10.244.0.0/16' on the end of kubeadm command now though20:18
kfox1111should be able to just: kubectl create -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml ?20:18
kfox1111I think that uses it in non cni mode though?20:18
portdirectkfox1111: please read step 2 in that doc20:18
kfox1111ah.20:19
kfox1111so you ahve to use pod-network-cidr, then launch flannel from the resource.20:19
*** huikang has joined #openstack-kolla20:19
kfox1111not sure why that would matter, unless its jut the first starting to use a new feature?20:20
portdirectno its the last to make the shift i think20:20
portdirectas it was the first sdn layer :)20:20
kfox1111yeah, but I thought it was late to the daemonset party.20:21
kfox1111it wasn't available when I deployed canal from daemonset last time.20:21
portdirectsorry we are cross puposes but in agreement, flannelw as the first k8s networking layer20:21
kfox1111anyway. not really important. :)20:21
kfox1111right. yeah.20:21
*** huikang_ has joined #openstack-kolla20:22
portdirectans so the mist prmative of them, I expect it to be essentially retired soon20:22
kfox1111do you know if any of the other plugins suppor the networkpolicy api yet?20:22
portdirectas canal takes over, as its essentially the same thing with some iptables security groups on top20:22
kfox1111yeah.20:23
portdirectyes, calico, romana (which was the first) and kuryr do20:23
*** unicell has joined #openstack-kolla20:23
kfox1111cool.20:23
*** absubram has quit IRC20:23
kfox1111maybe we should start looking into providing objects for kolla-kubernetes then.20:23
*** huikang has quit IRC20:23
*** pramodrj07 has joined #openstack-kolla20:24
kfox1111hows ingress been treating you?20:24
kfox1111last time I looked, it was pretty unstable. but that was a while ago.20:24
portdirectyeah - at scale its had issues - but is ok now20:24
kfox1111how long has it been fixed?20:25
portdirectstill no where near as fit and forget as kube-proxys iptables tricks20:25
kfox1111bummer.20:25
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855020:25
kfox1111I was seeing issues with like, no load on it, and it wasn't applying new rules in a timely manner.20:26
*** krtaylor has joined #openstack-kolla20:26
*** msimonin1 has joined #openstack-kolla20:26
portdirectright - its just a driver interface for a lb20:26
sbezverkkfox1111: can you beleive kube people still has not fixed that dns issue. it is crazy.. now it might not even make to 1.620:26
*** l4yerffeJ_ has quit IRC20:26
portdirectso will not react in the same way20:26
kfox1111sbezverk: yeah and the shared namespace issue too.20:27
portdirectmuch more equvilant to lbaas20:27
*** l4yerffeJ_ has joined #openstack-kolla20:27
kfox1111sbezverk: not too surprised really.20:27
*** msimonin has quit IRC20:27
kfox1111portdirect: yeah. they need a good solid reference implementation though. :/20:27
portdirectsbezverk: kube-dns is worse now than it used to be :)20:27
portdirectkfox1111: the nginx one is solid20:27
kfox1111ok. I'll have to retry it then. I think nginx was the one I tried before.20:28
portdirectsbezverk: moving to configmaps rather than running a seperate etcd is *interesting* under really high load20:28
*** msimonin has joined #openstack-kolla20:28
sbezverkkfox1111: they really need to react more quickly on review request :-) in some cases it takes weeks to get review.. how they surviving?!?!20:29
kfox1111sbezverk: heh. don't get me started on the keystone token review. ;)20:29
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes: Updated from global requirements  https://review.openstack.org/41934320:30
kfox1111sbezverk: they have management issues, just like openstaack. just not as bad. ;)20:30
*** msimonin1 has quit IRC20:31
sbezverkkfox1111: I really hope they will get better20:33
sbezverkfor our sake20:33
kfox1111+1. that. or, we need to push harder.20:34
*** msimonin1 has joined #openstack-kolla20:35
*** msimonin has quit IRC20:37
kfox1111I'm kind of ok with the resolv.conf workaround. its pretty stable.20:39
kfox1111I'm much less fond of the no shared option, as that is just broken.20:40
kfox1111only real way to recover from a failure is a reboot.20:40
portdirectwhy not just edit hosts resolv.conf and setup a route to k8s pod cidr?20:41
kfox1111cuase the system will have issues until the dns system comes up.20:41
*** gema has quit IRC20:41
kfox1111just overwriting the resolv.conf with a configmap avoids that problem.20:41
kfox1111the workaround works. its just not as easy as it coudl be if k8s just did the right thing.20:42
portdirectthat where i think we will always have a fundimental disconnect :)20:42
portdirecttheygenerally want to keep the hosts and containers in seperate worlds20:43
portdirectwhich makes total sense, unless you want to do domthing crazy, like20:44
portdirect..20:44
portdirectrun openstack :)20:44
*** lrensing has joined #openstack-kolla20:44
*** v1k0d3n has joined #openstack-kolla20:45
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855020:46
inc0I hate working with gates..20:46
sbezverkkfox1111: works for me too.. btw looks like neutron is ready20:47
inc0allright...let see how horribly will I fail trying to deploy galera20:47
jascott1inc0 want to talk about condition in charts?20:47
sbezverkinc0: look at it from different perspective, you do not need to setup anything :) just shoot a job and wait20:47
inc0jascott1, sure20:47
inc0jascott1, mind giving a bried desc of this here?20:48
kfox1111portdirect: plz? :) https://review.openstack.org/#/c/412239/20:48
jascott1so following an issue from helm queue, i implemented optional field 'condition' in chart and requirements. example: condition: glance.enabled20:49
*** SP__ has quit IRC20:49
inc0or...I'll become ugly person and instead of contributing galera to kolla, I'll do a chart for k8s charts themselves mwhahaha, and then kolla can use it;)20:49
jascott1if you add condition to chart or requirements file, and that value exists and returns false, the chart will not be loaded20:49
inc0kfox1111, ^20:50
*** lrensing has quit IRC20:50
jascott1chart doesnt seem as useful as requirements20:50
kfox1111inc0: I have no preference there. but, I do really like the idea of operators for galera.20:50
kfox1111I'm not sure k8s will get native support for galera any time soon.20:50
jascott1from what ive read of helm philosophy requirements is supposed to capture more of operator intent20:51
inc0kfox1111, but correct me if I'm wrong - nothing stops us of writing operators and publishing it to charts right?20:51
kfox1111right. operators are containers that need to be instantiated.20:51
inc0we can host operators in images20:51
kfox1111containers that need to be instantiated can go into helm packages.20:51
*** severion has joined #openstack-kolla20:51
portdirectinc0: if you do that I'd looking to k8s support for leader election, there were some thing interesting things going on there i think - but I kinda lost touch.20:51
portdirectinc0: there is an etcd operator in charts atm.20:52
kfox1111yeah. the statefulset has some election stuff.20:52
inc0quick change of topic20:52
kfox1111but last I looked needed sidecar stuff to get to work.20:52
inc0jascott1 wrote some things to forked helm to enable conditional requirements20:52
kfox1111run it by the helm folks yet?20:53
portdirectyay jascott1 !20:53
kfox1111would be curious their take on things.20:53
inc0we wanted to run it here before20:53
portdirectjascott1/inc0: what does it do?20:53
*** absubram has joined #openstack-kolla20:54
inc0jascott1 explained couple lines above;)20:54
inc0I really don't know much, he wrote the thing20:54
kfox1111gota head to a meeting.20:54
kfox1111bbiab.20:54
jascott1portdirect if you have 'condition' field in a chart or entry in requirements.yaml, e.g. condition: glance.enabled20:54
openstackgerritMerged openstack/kolla-kubernetes: Neutron service package  https://review.openstack.org/41223920:54
jascott1and that resolves to a false from your values, then chart will not be loaded20:54
*** schwicht has quit IRC20:55
portdirectjascott1: so it cannot check environment state? Its simply checking the contents of values?20:55
*** schwicht has joined #openstack-kolla20:55
portdirect(not that thats a bad thing - just trying to clarify)20:55
jascott1right. this is for helm install/upgrade20:55
jascott1helm chart install/uprade20:55
jascott1so we could have a requirements.yaml for neutron and have ALL plugins listed and then in all_values, specify which are enabled20:56
jascott1so we dont have to wrap every tpl20:56
jascott1or other craziness20:56
jascott1or hand edit requirements20:57
*** sacharya_ has joined #openstack-kolla20:57
inc0looking at code, it doesn't seem to be too big of a change too20:57
*** athomas has joined #openstack-kolla20:57
jascott1yeah I tried to find best entrypoint to keep it simple20:58
portdirectthink I've just parsed what you said20:58
jascott1one thing is tho I had to modify a couple of structs, one of which gets grpc'd to tiller so that might have bigger impact20:59
portdirectso in pusdotalk, its essentially adding conditionals to reauiements.yaml20:59
*** sacharya has quit IRC20:59
inc0pretty much portdirect20:59
portdirectnice20:59
jascott1from what I understand about protobuf tho its kinda made so you ignore extra fields20:59
*** schwicht has quit IRC21:00
jascott1i think the neutron use case is good example of usefulness21:00
jascott1maybe I should mock all that up (neutron chart) to make it easier to understand?21:00
portdirectjascott1: how could this be of benifit to the wider community?21:00
portdirectjascott1: using wordpress as an exmple, could it be used to allow you to use an existing db over having to have the one in the chart?21:01
*** schwicht has joined #openstack-kolla21:01
jascott1portdirect here is the issue i got the idea from https://github.com/kubernetes/helm/issues/156821:01
*** adrian_otto has joined #openstack-kolla21:02
portdirectnice -i think it sounds pretty cool21:02
jascott1possibly21:02
jascott1you could maybe have in requirements, psql, sqlite and mysql charts and then control which one is use via values21:03
*** klindgren has quit IRC21:03
portdirectgetting an example would be awesome(neutron). my q about wordpress was just thinking that acceptance is much easier if we can demonstrate a use case that's not only specific to us21:04
jascott1ah good idea21:05
sbezverkinc0: I am working currently on memcached operator21:06
*** adrian_otto1 has joined #openstack-kolla21:06
sbezverkand it sits in the specail image which can be instantiated by a chart..21:06
inc0yeah, that'd be the plan21:07
*** msimonin1 has quit IRC21:07
*** msimonin has joined #openstack-kolla21:07
*** klindgren has joined #openstack-kolla21:07
*** dave-mccowan has quit IRC21:09
*** adrian_otto has quit IRC21:09
*** absubram has quit IRC21:11
inc0sooo local registry is working21:18
inc0https://review.openstack.org/#/c/418550/21:18
inc02 gates failed for 2 different reasons I don't understand yet21:18
*** lrensing has joined #openstack-kolla21:18
inc0but we have trunk images access in gates \o/21:18
*** lamt has joined #openstack-kolla21:21
*** chopmann has joined #openstack-kolla21:21
*** lrensing has quit IRC21:23
sbezverkinc0: registry-1 name tells you anything? Get https://registry-1.docker.io/v2/kolla21:28
inc0nothing21:28
sbezverkinc0: cool, thanks never seen before this url was used, must be kfox1111 is playing with different registries21:30
*** portdirect is now known as portdirect_away21:30
inc0anyway, we have trunk access in gates!21:31
sbezverkinc0: nice, now we can have fun with bleeding edge ;)21:32
inc0not that bleeding hopefully21:32
*** l4yerffeJ has joined #openstack-kolla21:34
*** l4yerffeJ_ has quit IRC21:34
*** Jeffrey4l has quit IRC21:35
sdake_sup peeps21:37
*** v1k0d3n has quit IRC21:37
jrich523gates and trunks21:37
sdake_nice job inc021:37
*** lrensing has joined #openstack-kolla21:38
*** severion has quit IRC21:39
inc0well, still have to make it actually use this21:44
inc0but so far so good21:44
*** Jeffrey4l has joined #openstack-kolla21:48
*** huikang_ has quit IRC21:51
*** huikang has joined #openstack-kolla21:52
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855021:53
*** diablo_rojo has quit IRC21:54
*** rhallisey has quit IRC21:54
*** huikang has quit IRC21:56
*** huikang has joined #openstack-kolla21:59
*** krtaylor has quit IRC21:59
jascott1inc0 portdirect do you think this explains the idea well enough for discussion?22:01
jascott1https://github.com/jascott1/testchart22:01
*** harlowja has quit IRC22:01
inc0looks good22:01
jascott1i already had that for testing, can make a wp (or whatever) specific one if the idea gets traction22:03
*** adrian_otto1 has quit IRC22:04
*** adrian_otto has joined #openstack-kolla22:06
*** sacharya_ has quit IRC22:06
*** jtriley has quit IRC22:06
*** sacharya has joined #openstack-kolla22:06
*** schwicht has quit IRC22:06
*** portdirect_away is now known as portdirect22:07
*** schwicht has joined #openstack-kolla22:07
portdirectjascott1: id put that in my carpet bag and take it to the Spouter Inn, looks good dude22:08
jascott1lol22:08
portdirectsee some people read the tests :)22:09
*** huikang has quit IRC22:09
*** jheroux has quit IRC22:10
*** huikang has joined #openstack-kolla22:10
*** fguillot has quit IRC22:10
sbezverkjascott1: qq what makes it different if you just do not include this chart in the requirement list?22:13
*** huikang has quit IRC22:13
*** huikang has joined #openstack-kolla22:14
*** huikang has quit IRC22:14
inc0sbezverk, it will remove need of meddling with requirements22:14
jascott1sbezverk you can control from values and never touch requirements22:14
jascott1so for neutron, we could have all charts in requirements and just enable/disable from values22:14
sbezverkjascott1: so .enabled can be rendered based on some variables?22:14
jascott1that is the yaml path22:15
jascott1it can be anything22:15
jascott1acutally helm should consider some default22:15
jascott1(imo)22:15
sbezverkjascott1: cool I did not get the part that it is dynamic, based on variable defined somewhere else22:15
jascott1i have it working for install but havent tried upgrade yet22:16
jascott1think it should work tho since Im in chartutil.load mostly22:16
*** ipsecguy_ has joined #openstack-kolla22:17
sbezverkjascott1: where variable defining whether a chart enabled/disable resides?22:18
jascott1values22:18
jascott1values from the parent chart22:18
jascott1so helm loads the first chart and then starts adding depdendencies22:18
jascott1first = parent22:19
sbezverkjascott1: nice.. so now requirement.yaml gets rendered :) hopefully soon we will see Chart.yaml gets rendered22:19
jascott1no its not that22:19
jascott1sorry!22:19
jascott1in requirements entry, you are specifying a field 'condition' with value of yaml path, "subchart1.enabled"22:20
jascott1then you edit your parent's values file to have subchart: enabled: false22:20
jascott1when you run 'helm install parent' it sees that you have that value set to false and does not load that specific subchart22:21
jascott1where parent is just the top level chart22:22
sbezverkjascott1: ok got it, thanks for exaplanation22:23
jascott1np. thoughts?22:23
openstackgerritMark Giles proposed openstack/kolla-kubernetes: WIP - Nova service deployment  https://review.openstack.org/42145722:23
*** ipsecguy_ has quit IRC22:24
*** matrohon_ has quit IRC22:26
kfox1111back for a few.22:27
jascott1hey kfox1111 would love to hear your thoughts as well22:27
sbezverkjascott1: when you build parent package, disabling subchart will excluded it from it?22:28
sbezverkor it will impact "install: action22:28
jascott1this is on install, and comes down to values at install time. doesnt matter where the chart comes from as long as it has condition field22:28
kfox1111on?22:28
*** salv-orlando has joined #openstack-kolla22:28
jascott1kfox1111 this conditional idea https://github.com/jascott1/testchart22:29
kfox1111if dynamic subcharts is a good idea?22:29
jascott1yeah22:30
jascott1and the approach22:30
kfox1111+1 to the idea. haven't looked too closely at the implementation to know.22:30
*** lrensing has quit IRC22:30
kfox1111the example though looks good.22:31
kfox1111have you run it by technosophos?22:31
jascott1i have not but I got the idea from an issue he commented on22:31
jascott1he did not shoot it down and said he would investigate22:31
kfox1111I'd run it by him and see what he thinks. if he blocks it, that would be a problem. so better to find out sooner rather then later on it.22:32
jascott1so I implemented it for helm install and was about to ask him22:32
kfox1111I haven't worked with him on code, so not sure how that will go.22:32
jascott1well its golang so I expect to go great :D22:32
kfox1111heh.22:32
jascott1he wrote a book on it so hoping I learn somethin22:32
jascott1ha didnt even know I made a pun until it was to late22:33
kfox1111:)22:33
kfox1111just skimmed through the chart. it looks good to me.22:34
kfox1111would be really useful. :)22:34
jascott1thanks I think it would be great for neutron where we could have all the subcharts in requirements and just enabled/disable from values22:35
kfox1111yeah. could do all the pv/pvc/bootstrap jobs that way.22:36
kfox1111was going to do them with a conditional otherwise.22:36
kfox1111would be much nicer if helm just had support for the use case.22:37
jascott1right and maybe even just support a default convention of <chartname>.enabled22:37
kfox1111yeah. memcached.enabled, mariadb.enabled.22:38
kfox1111thought with the global api kind of thing we've started doing, (which wont work for this kind of engine),22:39
kfox1111you could do global.kolla.all.service_mariadb=true/false22:39
kfox1111and hit them all.22:39
*** lrensing has joined #openstack-kolla22:40
kfox1111weird.... http://logs.openstack.org/09/420009/11/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-nv/536ca40/logs/openstack/kolla-neutron-openvswitch-agent-network-5q5zx.txt22:42
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-kubernetes: WIP - local registry in gates  https://review.openstack.org/41855022:43
inc0kfox1111, ^ registry working22:44
sbezverkkfox1111: I have seen it, it is when kernel moduleis not loaded, data path then broken22:44
sbezverkor at least look very similar :022:45
kfox1111inc0: darn. you submitted a new one, so didn't see it run.22:47
kfox1111great that its working though.22:47
kfox1111we just need to get it into the conditional logic now.22:47
inc0well, I made mistake before, this one will run through I hope22:47
kfox1111ah. ok.22:48
kfox1111sbezverk: thanks. I'll check for t22:48
kfox1111that22:48
*** adrian_otto has quit IRC22:49
*** lrensing has quit IRC22:50
*** david-lyle has quit IRC22:51
kfox1111gota head out. be back in a bit.22:55
*** david-lyle has joined #openstack-kolla23:03
*** david-lyle has quit IRC23:10
*** msimonin has quit IRC23:11
*** lamt has quit IRC23:15
*** salv-orlando has quit IRC23:21
*** mgiles has quit IRC23:28
kfox1111back.23:31
*** mattmceuen has joined #openstack-kolla23:33
*** TxGirlGeek has quit IRC23:39
*** mattmceuen has quit IRC23:40
sbezverkkfox1111: I got operator deploying svc :), soon I will be able to do a demo23:42
kfox1111this is a weird error: http://logs.openstack.org/09/420009/11/check/gate-kolla-kubernetes-deploy-ubuntu-binary-2-ceph-nv/9dc58bc/logs/pods/kolla-nova-conductor-0-main.txt23:44
kfox1111sbezverk: nice. :)23:44
sbezverkkfox1111: maybe it expects trailing '/'?23:46
kfox1111or a dot?23:46
kfox1111its strange though. only seems to be on ubuntu 3.0.2 container.23:47
sbezverkdoubt that, in the log above, there were other urls all ended with '/'23:47
kfox1111hmm.... could be.23:47
kfox1111well, it seems like a bug in kolla-ansible genconfig.23:47
kfox1111inc0: still around?23:48
*** sayantani01 has quit IRC23:50
*** dave-mccowan has joined #openstack-kolla23:51
sdake_discount tire repaired my flat for free \o/ :)23:51
kfox1111nice. :)23:52
*** sdake_ is now known as sdake23:55
kfox1111this one's really weird.. http://logs.openstack.org/93/413193/34/experimental/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-reboot-nv/141c01e/logs/container/k8s_etcd.c323986f_etcd-centos-7-2-node-rax-ord-6727849-379167_kube-system_3a26566bb004c61cd05382212e3f978f_337cb237.txt23:55
kfox1111etcd fails to come up after reboot...23:55
kfox1111but the docker log is permision denied?23:55
kfox1111hmm...23:55
kfox1111I wonder if selinux is reenabled..23:56
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: WIP memcached operator  https://review.openstack.org/41807323:56
sdakekfox1111 default selinux in the gate is enable23:56
sdakehave to sed /etc/selinux/config23:57
sbezverkkfox1111: you probably need to edit it23:57
sdakeif you want it t obe persistent23:57
kfox1111yeah.23:57
kfox1111looks like all its doing now is setenforce 0, which wont persist.23:57
kfox1111easily fixable....23:57
inc0here kfox111123:58
sdakeselinux ideally should be enabled in teh gates23:58
sdakehowever we can fix tht late r for kolla-kubernetes23:58
kfox1111yeah. the existing rules though break badly with kolla-kubernetes.23:58
kfox1111probably a lot of work to resolve.23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!