Sunday, 2016-11-20

*** huikang has quit IRC00:05
*** huikang has joined #openstack-kolla00:11
*** Pavo has quit IRC00:21
*** tonanhngo has joined #openstack-kolla00:25
*** Pavo has joined #openstack-kolla00:26
*** tonanhngo has quit IRC00:26
*** hfu has joined #openstack-kolla00:28
*** huikang has quit IRC00:37
*** v1k0d3n has joined #openstack-kolla00:37
*** severion has joined #openstack-kolla00:39
*** v1k0d3n has quit IRC00:43
*** tonanhngo has joined #openstack-kolla00:43
*** tonanhngo has quit IRC00:44
*** v1k0d3n has joined #openstack-kolla00:45
*** severion has quit IRC00:48
*** tonanhngo has joined #openstack-kolla01:04
*** tonanhngo has quit IRC01:05
*** huikang has joined #openstack-kolla01:09
openstackgerritJeffrey Zhang proposed openstack/kolla: Add octavia docker image  https://review.openstack.org/39989601:12
kollabot1Gerrit Code Review01:12
*** zhubingbing has joined #openstack-kolla01:16
*** ccesario has quit IRC01:26
openstackgerritJeffrey Zhang proposed openstack/kolla: DO_NOT_MERGE: TEST MASTER BRANCH  https://review.openstack.org/39989701:57
kollabot1Gerrit Code Review01:57
openstackgerritJeffrey Zhang proposed openstack/kolla: Run init-once in kolla-ansible folder  https://review.openstack.org/39989802:01
kollabot1Gerrit Code Review02:01
*** tonanhngo has joined #openstack-kolla02:03
openstackgerritJeffrey Zhang proposed openstack/kolla: Add octavia docker image  https://review.openstack.org/39989602:04
kollabot1Gerrit Code Review02:04
*** tonanhngo has quit IRC02:04
*** ccesario has joined #openstack-kolla02:17
*** Pavo has quit IRC02:21
*** tonanhngo has joined #openstack-kolla02:24
*** Pavo has joined #openstack-kolla02:25
*** tonanhngo has quit IRC02:26
*** rhallisey has quit IRC02:33
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible: Enable cinder backup tab in horizon  https://review.openstack.org/39990002:41
kollabot1Gerrit Code Review02:41
*** ccesario has quit IRC02:44
*** tonanhngo has joined #openstack-kolla02:44
*** tonanhngo has quit IRC02:46
*** Jeffrey4l has quit IRC02:53
*** ccesario has joined #openstack-kolla02:56
*** tonanhngo has joined #openstack-kolla03:04
*** tonanhngo has quit IRC03:06
*** tonanhngo has joined #openstack-kolla03:24
*** tonanhngo has quit IRC03:26
*** fragatina has joined #openstack-kolla03:32
*** dave-mccowan has joined #openstack-kolla03:48
*** dave-mccowan has quit IRC03:52
*** tonanhngo has joined #openstack-kolla04:11
*** tonanhngo has quit IRC04:13
*** Pavo has quit IRC04:21
*** eaguilar has quit IRC04:23
*** Pavo has joined #openstack-kolla04:26
*** tonanhngo has joined #openstack-kolla04:26
*** tonanhngo has quit IRC04:29
*** tonanhngo has joined #openstack-kolla04:45
*** huikang has quit IRC04:46
*** tonanhngo has quit IRC04:46
*** tonanhngo has joined #openstack-kolla04:58
*** tonanhngo has quit IRC04:59
*** Jeffrey4l has joined #openstack-kolla05:01
*** tonanhngo has joined #openstack-kolla05:15
*** huikang has joined #openstack-kolla05:17
*** tonanhngo has quit IRC05:17
*** huikang has quit IRC05:18
*** zhubingbing has quit IRC05:29
*** unicell1 has joined #openstack-kolla05:59
*** unicell has quit IRC05:59
*** unicell has joined #openstack-kolla06:02
*** unicell1 has quit IRC06:04
*** AnswerGuy has quit IRC06:05
*** unicell1 has joined #openstack-kolla06:10
*** unicell has quit IRC06:11
*** tonanhngo has joined #openstack-kolla06:14
*** tonanhngo has quit IRC06:17
*** Pavo has quit IRC06:21
*** Pavo has joined #openstack-kolla06:25
*** tonanhngo has joined #openstack-kolla06:35
*** tonanhngo has quit IRC06:38
*** tonanhngo has joined #openstack-kolla06:54
*** tonanhngo has quit IRC06:55
*** tonanhngo has joined #openstack-kolla07:15
*** tonanhngo has quit IRC07:16
*** tonanhngo has joined #openstack-kolla07:35
*** tonanhngo has quit IRC07:38
*** senk has joined #openstack-kolla07:49
*** newmember has joined #openstack-kolla07:54
*** tonanhngo has joined #openstack-kolla07:54
*** tonanhngo has quit IRC07:57
*** x220_ has joined #openstack-kolla07:59
*** newmember has quit IRC08:01
*** sbezverk_ has joined #openstack-kolla08:03
*** sbezverk has quit IRC08:03
*** tonanhngo has joined #openstack-kolla08:09
*** tonanhngo has quit IRC08:11
*** Pavo has quit IRC08:21
*** Pavo has joined #openstack-kolla08:25
*** tonanhngo has joined #openstack-kolla08:27
*** tonanhngo has quit IRC08:27
*** nihilifer has joined #openstack-kolla08:44
*** tonanhngo has joined #openstack-kolla08:45
*** tonanhngo has quit IRC08:47
*** tonanhngo has joined #openstack-kolla09:04
*** tonanhngo has quit IRC09:05
*** senk has quit IRC09:06
*** x220_ has quit IRC09:10
*** tonanhngo has joined #openstack-kolla09:24
*** tonanhngo has quit IRC09:27
*** tonanhngo has joined #openstack-kolla09:45
*** tonanhngo has quit IRC09:45
*** mgoddard has joined #openstack-kolla09:50
*** papacz has joined #openstack-kolla09:54
*** mgoddard has quit IRC09:54
*** senk has joined #openstack-kolla09:59
*** magicboiz has joined #openstack-kolla10:01
*** tonanhngo has joined #openstack-kolla10:04
*** tonanhngo has quit IRC10:05
*** magicboiz has quit IRC10:08
*** oxkipo has joined #openstack-kolla10:15
oxkipoHi I cannot attach a cinder lvm volume to a instance. Also kolla/ubuntu-source-iscsid:3.0.1 this containers status is only created. Please may someone help me?10:16
*** Pavo has quit IRC10:21
*** tonanhngo has joined #openstack-kolla10:25
*** Pavo has joined #openstack-kolla10:26
*** tonanhngo has quit IRC10:26
oxkipoplease may someone help me with this?10:28
*** tonanhngo has joined #openstack-kolla10:45
*** tonanhngo has quit IRC10:47
oxkipoplease may someone help me with cinder?10:52
*** papacz1 has joined #openstack-kolla11:01
*** papacz2 has joined #openstack-kolla11:04
*** papacz has quit IRC11:04
*** tonanhngo has joined #openstack-kolla11:05
*** papacz1 has quit IRC11:06
*** tonanhngo has quit IRC11:06
*** hfu has quit IRC11:13
*** hfu has joined #openstack-kolla11:15
*** hfu has quit IRC11:16
*** oxkipo has quit IRC11:16
*** hfu has joined #openstack-kolla11:16
*** hfu has quit IRC11:16
*** hfu has joined #openstack-kolla11:17
*** hfu has quit IRC11:17
*** tonanhngo has joined #openstack-kolla11:35
*** tonanhngo has quit IRC11:38
*** kristian2709 has joined #openstack-kolla11:43
kristian2709Hi please may someone help me fix cinder not assigning volumes to an instance?11:45
*** tonanhngo has joined #openstack-kolla11:55
*** tonanhngo has quit IRC11:58
*** senk has quit IRC12:02
*** portdirect_away is now known as portdirect12:11
portdirectkristian2709: sup?12:11
*** david-lyle has joined #openstack-kolla12:12
kristian2709portdirect: I cannot attach a volume to the instance. It doesnt give me any error12:12
portdirectkristian2709: ok - at what point do you think its failing? can you access the logs for cinder (what backend are you using - my experience with ceph is very limited :( ) and nova-compute for the node running the vm?12:14
*** tonanhngo has joined #openstack-kolla12:15
*** david-lyle has quit IRC12:17
*** tonanhngo has quit IRC12:17
kristian2709portdirect: my kolla installation is aio on a bare metal node with cinder and lvm configured on host12:20
*** Pavo has quit IRC12:21
portdirectkristian2709: ok, can you look at the logs for cinder volume, and nova-compute and see at what point it is failing?12:22
kristian2709ok12:22
*** hfu has joined #openstack-kolla12:24
*** Pavo has joined #openstack-kolla12:26
kristian2709portdirect: strange now it works12:26
kristian2709dunno how12:26
kristian2709portdirect: please may you help me fix this issue?12:40
kristian2709vhttps://bugs.launchpad.net/nova/+bug/164241912:40
openstackLaunchpad bug 1642419 in OpenStack Compute (nova) "GPU Passthrough isn't working" [Medium,New]12:40
kollabot1Bug #1642419 “GPU Passthrough isn't working” : Bugs : OpenStack Compute (nova)12:40
*** tonanhngo has joined #openstack-kolla12:44
*** tonanhngo has quit IRC12:47
portdirectkristian2709: whoop! thats great - I'll have a look at the bug now12:56
portdirectkristian2709: K - I've only done GPU passthrough with nova-docker before (and not with kolla) any possibility you could get the logs from nova-compute in a gist to look at as well - whihc may be able to give me a better idea about whether this is an issue at the kolla or nova end?13:00
portdirectkristian2709: you running this on a gaming rig!?13:03
kristian2709portdirect: yes13:04
*** tonanhngo has joined #openstack-kolla13:04
kristian2709also the logs should be there13:04
portdirectwhoops - missed that - 2 sec13:06
*** tonanhngo has quit IRC13:06
kristian2709np13:07
portdirectkristian2709: k, a few things I'd try:13:09
kristian2709ok13:09
kristian2709tell mw13:09
kristian2709*me13:09
portdirectget nova_compute running in the host-pid namespace: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/tasks/start_compute.yml#L4913:10
kollabot1kolla-ansible/start_compute.yml at master · openstack/kolla-ansible · GitHub13:10
portdirectalso try both libvirt and nova_compute in the host ipc namespace13:10
*** skramaja has quit IRC13:11
*** uclug has joined #openstack-kolla13:16
openstackgerritJeffrey Zhang proposed openstack/kolla: Install mkfs.vfat tool in ironic-conductor image  https://review.openstack.org/39995713:16
kollabot1Gerrit Code Review13:16
*** hfu has quit IRC13:19
*** hfu has joined #openstack-kolla13:20
*** hfu has quit IRC13:20
*** hfu has joined #openstack-kolla13:21
openstackgerritJeffrey Zhang proposed openstack/kolla: DO_NOT_MERGE: TEST NEWTON BRANCH  https://review.openstack.org/38698113:21
kollabot1Gerrit Code Review13:21
*** hfu has quit IRC13:21
*** hfu has joined #openstack-kolla13:21
*** uclug has quit IRC13:22
*** hfu has quit IRC13:22
*** hfu has joined #openstack-kolla13:22
*** hfu has quit IRC13:23
*** hfu has joined #openstack-kolla13:23
*** hfu has quit IRC13:23
*** zhubingbing has joined #openstack-kolla13:26
*** eaguilar has joined #openstack-kolla13:29
*** sp__ has joined #openstack-kolla13:29
openstackgerritMerged openstack/kolla: Use upper constraints for all jobs in tox.ini  https://review.openstack.org/38422113:35
kollabot1Gerrit Code Review13:35
*** sdake has joined #openstack-kolla13:48
*** sdake_ has joined #openstack-kolla13:52
*** hfu has joined #openstack-kolla13:52
*** sdake has quit IRC13:53
*** sp__ has quit IRC13:58
*** imcsk8_ has quit IRC14:08
*** imcsk8 has joined #openstack-kolla14:08
*** zhubingbing has quit IRC14:10
*** rhallisey has joined #openstack-kolla14:11
*** david-lyle has joined #openstack-kolla14:14
*** tonanhngo has joined #openstack-kolla14:14
*** tonanhngo has quit IRC14:16
*** david-lyle has quit IRC14:18
*** Pavo has quit IRC14:21
*** Pavo has joined #openstack-kolla14:26
*** tonanhngo has joined #openstack-kolla14:34
*** tonanhngo has quit IRC14:36
*** senk has joined #openstack-kolla14:42
*** sdake has joined #openstack-kolla14:52
*** sdake has quit IRC14:54
*** sdake__ has joined #openstack-kolla14:54
*** tonanhngo has joined #openstack-kolla14:54
openstackgerritMerged openstack/kolla-ansible: typo error of keyring spelling  https://review.openstack.org/39945514:55
kollabot1Gerrit Code Review14:55
*** sdake__ has quit IRC14:55
*** tonanhngo has quit IRC14:55
*** sdake has joined #openstack-kolla14:55
*** sdake_ has quit IRC14:55
openstackgerritMerged openstack/kolla: Remove Ansible references from tox.ini  https://review.openstack.org/39959715:02
kollabot1Gerrit Code Review15:02
*** sdake_ has joined #openstack-kolla15:06
*** sdake has quit IRC15:10
*** hfu has quit IRC15:15
*** tonanhngo has joined #openstack-kolla15:16
*** hfu has joined #openstack-kolla15:16
*** hfu has quit IRC15:17
*** tonanhngo has quit IRC15:17
*** senk has quit IRC15:17
*** magicboiz has joined #openstack-kolla15:20
*** eaguilar has quit IRC15:28
*** tyrola has joined #openstack-kolla15:29
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Mariadb Kubernetes Operator  https://review.openstack.org/39926315:32
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Kubernetes Operator base  https://review.openstack.org/39926215:32
kollabot1Gerrit Code Review15:32
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Change services dir to templates dir  https://review.openstack.org/39997115:32
openstackgerritRyan Hallisey proposed openstack/kolla-kubernetes: Service and Mariadb operator template  https://review.openstack.org/39997215:32
kollabot1Gerrit Code Review15:32
kollabot1Gerrit Code Review15:32
kollabot1Gerrit Code Review15:32
*** eaguilar has joined #openstack-kolla15:38
*** eaguilar has quit IRC15:40
*** v1k0d3n has quit IRC15:45
*** v1k0d3n has joined #openstack-kolla15:45
*** eaguilar has joined #openstack-kolla15:48
*** eaguilar has quit IRC16:00
*** senk has joined #openstack-kolla16:06
portdirectrhallisey: ping16:13
*** tonanhngo has joined #openstack-kolla16:15
rhalliseyportdirect, hey16:15
portdirectsunday workers :/16:15
portdirectlooking at your operator work - can I play with any of it atm?16:16
sbezverk_portdirect: with this weather on east coast, what else can you do ;-)16:16
*** tonanhngo has quit IRC16:16
portdirectcome to scotland - I'll show you reall rain and misery :)16:17
rhalliseyportdirect, ya sure16:17
portdirectany pointers?16:18
sbezverk_portdirect: visiting Scotland is definetly on my to do list :-)16:18
*** jrist has quit IRC16:19
rhalliseyportdirect, do you  want to try it or develop it osme?16:20
rhalliseysome*16:20
portdirectsbezverk_: hit me up anytime your through man, though I'd strongly recommed august for the Edinburgh Festival16:20
rhalliseythe process to try it is a little annoying16:20
*** jrist has joined #openstack-kolla16:20
rhalliseyyou have to build the base16:20
rhalliseythen the mariadb-op16:20
rhalliseythen create the resources16:20
rhalliseyI haven't fully tried every piece yet16:20
rhalliseybecause I don't think I have all of them done yet16:20
rhalliseyI think I need a deployment for the operator16:21
portdirectrhallisey: figured :)16:21
*** Pavo has quit IRC16:21
*** sdake_ has quit IRC16:21
*** Pavo has joined #openstack-kolla16:21
rhalliseyya it's super rough atm16:21
portdirectthat the early stages of a major WIP for you :)16:22
rhalliseyya16:22
rhalliseysbezverk_, how come we can't change the dir name?16:22
rhalliseyhelm patches haven't merged yet so I figured we'd make the change now16:22
sbezverk_rhallisey: I am not saying we cannot, but if you check Kevin's PS for helm16:23
portdirectmostly interested to make sure that I know whats going on so when I charge into development I'm going the right direction.16:23
rhalliseysbezverk_, right he uses it16:23
sbezverk_Kevin uses completely different approach16:23
rhalliseyI'll check out what he has16:23
Pavogood morning everyone16:24
sbezverk_rhallisey: and even that structure is not even finalized.. if changing folder is not a show stopper for you, maybe we should hold a bit..16:25
rhalliseyit's not the biggest deal, but it makes more sense to me in terms of structure16:26
portdirectrhallisey: I agree with that - as it's what they are - but makes it a bit tricky atm16:28
rhalliseysure16:28
portdirectPossibly the most usfull thing i could do right now is get kolla-build updated so we can build the images?16:30
rhalliseysome things we need revolve around making sure the repos are good to work with both projects16:31
portdirectyup: I made a first stab here: https://review.openstack.org/#/c/395676/, but got some feedback from inc0 and kfox (which I agree with), though will leave tthe option of just building a single template dir for tings like koll-k8s only container builds16:33
kollabot1Gerrit Code Review16:33
*** tonanhngo has joined #openstack-kolla16:35
*** tonanhngo has quit IRC16:38
sbezverk_rhallisey: are you going to use helm to deploy a specific chart for a service example mariadb or some other form of template?16:42
v1k0d3nhey portdirect16:42
v1k0d3nis that halcyon stuff working out for you? do you think that would be good to write up in kolla docs as a decent dev solution for kolla?16:43
v1k0d3nsorry...kolla-k8s i mean16:43
portdirectv1k0d3n: halcyon's working great for me atm :)16:46
*** g3ek has quit IRC16:46
*** haplo37 has quit IRC16:46
portdirectthough I've only really tried with with weave as that worked first time out of the box for me - i think using it was a basis flow local dev docs would be awesome16:47
portdirect*for not flow16:48
rhalliseysbezverk_, ya an operator will trigger a helm install16:53
*** g3ek has joined #openstack-kolla16:55
*** haplo37 has joined #openstack-kolla16:56
v1k0d3noperator as in person or operator container rhallisey ?16:56
rhalliseyugh that name16:56
v1k0d3nexactly :)16:56
v1k0d3ni am starting to wish i never brought up the subject.16:56
portdirectv1k0d3n: stop trolling :D16:56
rhalliseyv1k0d3n, from what I understood about the context of sbezverk_, I'm referring to a kubernetes operator16:57
v1k0d3ni'm not. i'm seriously wondering, that's all.16:57
rhalliseywe should re name it to admiral16:57
v1k0d3nok16:57
v1k0d3njust checking16:57
v1k0d3nour workflow will be 100% developer driven.16:57
v1k0d3nso developer will push code. container registry sees commit tag, and build new release tag. helm is updated, and triggers CI. once CI completes, queues up for human to push-button deploy the update.16:58
v1k0d3nall developer-driven.16:58
v1k0d3nmay even use some jira component to approach CD changes.16:59
v1k0d3nstill thinking through these steps, but that's a general logic.16:59
v1k0d3nwhy i asked anyway.16:59
portdirectv1k0d3n: The only thing 'missing' from halcyon in my opionion at the moment is a helper script to set up creds for kubectl and helm on the machine running vagrant - I could knock up a quick bash script for that if your up for it?16:59
rhalliseygitcha17:00
rhalliseygotcha*17:00
v1k0d3nportdirect: absolutely.17:00
v1k0d3nwould definitely accept that.17:00
portdirect5 mins :)17:01
kristian2709portdirect: still the same error with CPUID17:03
sbezverk_rhallisey: then it make sense to sync up with Kevin about templates..17:09
rhalliseyya I'll study his patch17:09
rhalliseyI haven't yet17:09
rhalliseyjust a little, but I need to more17:09
*** tyrola has quit IRC17:11
*** bmace has quit IRC17:11
*** bmace has joined #openstack-kolla17:11
*** senk has quit IRC17:12
*** tonanhngo has joined #openstack-kolla17:14
v1k0d3nso can anyone help me to understand an AIO deployment i am trying to stand up.17:16
*** tonanhngo has quit IRC17:16
v1k0d3ni guess it's been a little while (and new version) since my last attempt.17:16
v1k0d3ni have two interfaces: em1 and em2 (ubuntu). em1 is the network_interface and em2 is the neutron_external_interface.17:17
v1k0d3nem1 is 192.168.70.25/24.17:17
v1k0d3nem2 does not have an ip of course, but is attached to a "public" subnet of 192.168.4.x/22.17:17
v1k0d3nafter deployment, i configure a flat interface to physnet1. all good there.17:18
v1k0d3nthen i want to create my pub-sub and create 192.168.4.0 net with a dhcp pool. hosts get an ip just fine...but for some reason i can't communicate to them. something i'm missing here?17:19
v1k0d3n192.168.4 and 192.168.70 are attached to the same router. should be fine there.17:20
v1k0d3nto be clear...can't ping 192.168.4.x hosts in the same broadcast domain.17:21
v1k0d3ntell me i'm doing something dumb here, please? :)17:21
*** tonanhngo has joined #openstack-kolla17:24
*** tonanhngo has quit IRC17:26
*** magicboiz has quit IRC17:33
*** tonanhngo has joined #openstack-kolla17:36
* v1k0d3n realizes that it's a lazy sunday17:37
portdirectkristian2709: sorry dude, I'm not familiar enough with IOMMU to help you any further in a constructive manner :(17:40
v1k0d3nsbezverk_: are you around, or is it too late for you?17:44
*** kwazar has quit IRC17:55
*** timss has quit IRC17:55
*** kwazar has joined #openstack-kolla17:56
*** timss has joined #openstack-kolla17:56
sbezverk_v1k0d3n: hey, still around.. it is only 1pm  here..18:08
kristian2709ok np18:08
v1k0d3ndo you have any ideas or thoughts on above?18:10
v1k0d3nsbezverk_:18:10
sbezverk_v1k0d3n: reading18:11
v1k0d3nthing that is strange....i see the mac address for the instances on the 192.168.4.x subnet.18:11
*** david-lyle has joined #openstack-kolla18:11
sbezverk_v1k0d3n: how about security rules?18:12
*** pbourke has quit IRC18:13
*** kristian2709 has quit IRC18:13
*** pbourke has joined #openstack-kolla18:13
v1k0d3nyeah, that's always the first thing i look at. all good there.18:13
sbezverk_v1k0d3n: can you configm that your public interface is pluged to ovs bridge18:14
sbezverk_I meant confirm18:14
sbezverk_v1k0d3n: usually in this situation I just go step by step with tcpdmp and see where packets gets dropped..18:15
sbezverk_v1k0d3n: instance tap interface, linux bridge interface in/out, ovs interface, external interface, somewhere you will see packets stop crossing..18:16
v1k0d3nah! got it.18:16
v1k0d3nall good sbezverk_18:17
sbezverk_v1k0d3n: what was it?18:17
v1k0d3nsbezverk_: really simple fix. had to add a route on the AIO host to send all 192.168.4.x traffic to em2.18:18
v1k0d3nin other words...really simple bonehead error :)18:19
v1k0d3nprobably from lack of sleep or busy week... :)18:19
sbezverk_v1k0d3n: good catch!18:19
portdirectThats caught me out before18:19
v1k0d3nsbezverk_: do you know if AIO deployments go haywire on reboots?18:19
v1k0d3nor is there any special considerations to make for an AIO development host?18:20
v1k0d3nlike there is with OSA18:20
*** Pavo has quit IRC18:21
*** Pavo has joined #openstack-kolla18:21
sbezverk_v1k0d3n: Have not seen many issues with aio on reboot.18:26
sbezverk_in fact I do not recall even a single one18:26
v1k0d3nsbezverk_: so everything just comes up, or do i have to restart the containers?18:26
sbezverk_v1k0d3n: yep, if your docker autostarts, then everything should just come up18:27
v1k0d3nyeah that's great.18:27
v1k0d3nthanks sbezverk_18:27
sbezverk_v1k0d3n: np18:28
*** hogepodge has quit IRC18:34
*** sdake has joined #openstack-kolla18:41
sdakerhallisey we need operators for our operators - yo dawg!18:42
rhallisey:/18:54
*** senk has joined #openstack-kolla19:10
*** sdake has quit IRC19:18
v1k0d3nprediction for 2017: operators will be the most overused and abused term of the year :)19:20
v1k0d3nbtw...can anyone riddle me this...?19:22
portdirecthot off the press: kubernetes 1.5 introduces 'users'.19:22
v1k0d3ni have an AIO without TLS, and i want to keep all configuration options (including running machihne) and just add TLS using ansible. possible?19:22
v1k0d3nportdirect: ha!19:22
v1k0d3ni'm going to have our team build the "consultant"  container. it will claim to magically "improve" your CI/CD pipline, leave you with more questions than answers, and it bills you at the rate of $400 per hour automatically.19:26
v1k0d3nand it can scale without anyone's knowledge.19:27
v1k0d3ni.e. bring in more "consulting containers".19:27
v1k0d3nit can even bring in 3rd party resources when it can't get the job done.19:27
portdirectI know a few VC's who may be interested in that..19:28
*** senk has quit IRC19:31
v1k0d3nrhallisey: can i just change globals to include TLS and redeploy without ansible messing up any of my running workloads or config?19:32
v1k0d3nagain, this is for that aio i am building for development.19:32
rhalliseythat's tricky19:36
rhalliseyidk if you reconfigure will work with enabling tls19:37
sbezverk_portdirect: I am more concerned with this in 1.5 Rename PetSet to StatefulSet19:38
portdirectsbezverk_: i saw that; you know what logic's behind that?19:39
rhalliseylet me think19:40
rhallisey..19:40
*** dave-mccowan has joined #openstack-kolla19:42
sbezverk_portdirect: nope19:43
rhalliseyv1k0d3n, I think you will be ok19:44
rhalliseynova will get restarted19:44
sbezverk_sbezverk_: I guess one way to verify is to load 1.5.0 beta 1 and test spec with kind: petset19:44
rhalliseybut I don't think it will destroy infrastructure you have19:45
v1k0d3nrhallisey: so fyi...that completely killed my host :(20:07
v1k0d3ncan't access anymore.20:07
rhalliseyO.o20:07
*** dave-mccowan has quit IRC20:08
rhalliseythe vms? or the bm machine?20:08
v1k0d3nwell, i guess there's a reliance on haproxy?20:08
rhalliseyyes, endpoints get reset20:09
v1k0d3nhorizon. i have aio with haproxy turned off.20:09
v1k0d3njust pointing to the internet20:09
v1k0d3ninterface, sorry.20:09
v1k0d3nso i guess maybe removing it may correct? anyway...trying it just in case. don't want to have to rebuild or anything.20:09
v1k0d3ntotally should've thought about that when i was blowing through the changes.20:10
* v1k0d3n feels like a bonehead. rtfm.20:12
rhalliseyhehe20:12
v1k0d3nto be fair...this is the first time i chose not to use haproxy.20:13
v1k0d3ntrying to toy around with various options and reconfigure options. i want to rely on it more, and break it often. so this is good anyway.20:14
*** Pavo has quit IRC20:21
v1k0d3nrhallisey: yeah that wacked everything out. i think i'm forced to rebuild :(20:22
*** hogepodge has joined #openstack-kolla20:22
rhalliseyugh20:22
rhalliseydarn20:22
*** hogepodge has quit IRC20:22
v1k0d3nnova errors...errors all over the place.20:23
*** hogepodge has joined #openstack-kolla20:24
*** Pavo has joined #openstack-kolla20:26
*** dave-mccowan has joined #openstack-kolla20:46
*** kristian2709 has joined #openstack-kolla20:46
kristian2709Hi is anyone familiar with gpu passthrough?20:51
*** Jeffrey4l has quit IRC21:35
kristian2709please may someone help me with pci passthrough?21:36
v1k0d3nkristian2709: sundays are really slow. the group will be more responsive tomorrow.21:52
kristian2709ok we will see21:52
v1k0d3nsbezverk_: you still around?21:53
v1k0d3nmore questions if you're still hanging out.21:53
*** dave-mccowan has quit IRC21:59
*** portdirect is now known as portdirect_away22:09
*** Pavo has quit IRC22:21
*** rhallisey has quit IRC22:23
*** Pavo has joined #openstack-kolla22:26
*** sdake_ has joined #openstack-kolla22:28
*** eaguilar has joined #openstack-kolla22:30
*** eaguilar has quit IRC22:38
v1k0d3ndoes anyone know how i might be able to redeploy kolla on the same host without reinstalling everything?22:38
v1k0d3ni tried TLS and that failed...tried to go back and nova stopped working.22:38
v1k0d3ni'd like to clean and redeploy, but that's causing issues.22:39
sdake_v1k0d3n do you mean kolla-ansible?22:39
v1k0d3nthere's got to be some cruft left over somewhere.22:39
v1k0d3nyes22:39
sdake_v1k0d3n did you run kolla-ansible destroy inbetween?22:39
sdake_note kolla-ansible takes a flag which does a super cleanup22:40
v1k0d3nanother thing, i marked magnum and lbaas to be installed, but didn't seem to be enabled.22:40
v1k0d3nsdake_: i have done that yes.22:40
sdake_also tls does work22:40
sdake_;)22:40
v1k0d3nnot sure what you mean. i know it does.22:40
sdake_so if you use destroy your system should be back in a pristine state22:40
sdake_i mean in our immplementation22:41
sdake_you said you tried tls and it failed22:41
v1k0d3njust didn't do it the first time of deploy, thihnking i could redeploy...22:41
v1k0d3nand that bombed22:41
sdake_oh right reconfigure of tls22:41
sdake_ya - reconfigure not meant for globals.yaml22:41
v1k0d3ni did try tls and it failed. right.22:41
sdake_bit of a misnomer22:41
sdake_reconfigure is limited to reconfiguring /etc/kolla/config22:41
v1k0d3ni asked ahead of time. was told should work.22:41
v1k0d3nso there is nothing here currently...destroyed.22:42
sdake_v1k0d3n the answer is some things work some of the times22:42
v1k0d3nwhen i redeploy no love.22:42
v1k0d3nlol22:42
v1k0d3nok22:42
sdake_v1k0d3n so my answer is "globals.yaml cannot be reconfigured"22:42
v1k0d3neven after a destroy?22:42
sdake_nope a destroy should do a ttal cleanup22:42
sdake_so lets fi that22:42
sdake_did you pass the inventory file when you destroyed?22:43
v1k0d3nAIO22:43
v1k0d3nAIO for development22:43
sdake_got it22:43
sdake_ok so can you tell me how redeploy is failling?22:43
v1k0d3ncould of things...in my statement.22:44
v1k0d3nLBaaS never seemed to deploy.22:44
v1k0d3nso no magnum.22:44
v1k0d3nthat's a fail.22:44
sdake_upstream keeps doing funny things with lbaas22:44
v1k0d3nthen when i redeploy, horizon fails, nova fails. endpoints are not working.22:44
sdake_so there is no "redeploy" do you mean after a destroy you deploy and everything craters?22:45
v1k0d3nso i am hoping that when i pick 3.0.1 it should be stable.22:45
v1k0d3ni will be clear.22:45
v1k0d3nthis is ALWAYS after a destroy.22:45
v1k0d3nyes22:45
sdake_mind showing me the log of where it craters during deploy - or does deploy work correctly22:46
sdake_work/finish22:46
v1k0d3ncurrently destroyed.22:46
sdake_cool run script22:46
sdake_this will create a file called typescript in your cwd22:46
sdake_then run kolla-ansible deploy22:46
sdake_run with -vvv22:46
sdake_once that craters or finishes, type exit22:47
sdake_this will save a file called typescript22:47
sdake_then paste the contets of typescript somewhere22:47
sdake_if your on centos, you can yum install fpaste and fpaste typescript22:47
sdake_if your on some other distro not sure how to past efrom cli but i know its possible ;)22:47
v1k0d3ni'm not sure that's making sense to me....22:48
sdake_there is a command called script22:48
v1k0d3ncwd?22:48
sdake_cwd = current working directory22:48
v1k0d3noh...yeah gotcha22:48
sdake_typescript will contain the contents of your shell run22:48
sdake_this is how i capture and share runs22:49
v1k0d3nand run script for kolla pull && kolla deploy?22:49
sdake_some other people use tee and stuff22:49
sdake_you run script once, it createa a subshell22:49
v1k0d3nalso, what needs to be done to ensure lbaas and magnum?22:49
v1k0d3nthat's sort of what i want to test.22:49
sdake_lets tackle that next22:49
sdake_i want to see where your at curently22:49
sdake_i get conflicting reports on magnum working vs not working22:50
v1k0d3nok, so i removed all containers and images from the box and rebooted.22:50
sdake_it also wokrs for me22:50
sdake_aer you using ceph per chance?22:50
v1k0d3nyou're saying do a kolla-ansible pull22:50
v1k0d3nthen kolla-ansible deploy -vvv22:50
v1k0d3ncorrect?22:50
v1k0d3nno ceph22:50
sdake_did you run the script command first?22:50
v1k0d3nfigured didn't need on an aio host22:50
v1k0d3nscript?22:51
sdake_type "script"22:51
sdake_into your shell22:51
sdake_then type echo "hello world"22:51
sdake_then type exit22:51
sdake_then type cat typescript22:51
sdake_rather cat tyescript22:51
v1k0d3ni gotcha now. yup got it.22:52
sdake_ok22:52
v1k0d3nso do a kolla-ansible pull and then deploy, correct?22:52
sdake_so we are just giong to do the above exercise, except create a typescript for kolla-ansible deploy -vvv22:52
v1k0d3nwell i want to get tls so need to check globals, and deploy certs22:52
v1k0d3none sec22:52
v1k0d3nk pulling22:54
v1k0d3nas this is pulling and running, i am going to grab something to eat sdake_ you gonna be around in 30 mins? prob gonna take about 15-20 mins or so anyway.22:55
sdake_if my batteries dont run out22:56
sdake_24% atm22:56
v1k0d3npower outlet?22:58
*** sdake has joined #openstack-kolla22:58
v1k0d3nwell, wife wants to go with to get food apparently. changes my whole plan....22:58
v1k0d3nso i will probably be faster and can keep troubleshooting since shes on phone.22:58
v1k0d3nok, running deploy with -vvv now.22:59
*** sdake_ has quit IRC23:02
*** eaguilar has joined #openstack-kolla23:04
*** tonanhngo has quit IRC23:05
*** bjolo has quit IRC23:07
*** bjolo has joined #openstack-kolla23:08
v1k0d3nok, so sdake same thing...23:11
v1k0d3nran just fine...23:12
v1k0d3ncannot access horizon23:12
v1k0d3nredirects to TLS.23:12
sdakeok so that typescript file.. is where?23:15
sdakeif you type exit23:15
sdakeit will save it23:16
sdakethe nyou can paste it :)23:16
v1k0d3nin cdir but that contains a lot23:16
sdakeya its a busy file23:16
sdakedid it complete ?23:16
sdakeif it completed the firs tthing i want to look at is the last 100 or so lines23:17
sdakeso just paste that first if you like23:17
v1k0d3nare there any things you're looking for in particular?23:17
sdakewhat went wrong23:17
v1k0d3nfirst, the file is really large. the other thing is that passwords are in there.23:17
sdakeok well if yoru passwords are secure then thats not going to work23:18
sdakei had thought you used genpwd23:18
sdakeso how about pasting the last 100 lines23:18
v1k0d3nhorizon redirecting to SSL and fail23:18
sdakether eshouldn't be anything there secure23:18
sdakei can't help with that information - i need full logs :)23:18
sdakebut mayben ot even full23:18
*** tonanhngo has joined #openstack-kolla23:18
sdakei need to see if any ansible jobs are "failed"23:19
sdakeuse tail to display the last parts of typescript23:19
Pavowhat you doing23:19
sdakepavo walking v1k0d3n through a kolla-ansible tls deploy23:20
sdakepavo your using tls right?23:21
Pavoyeah23:21
v1k0d3nsdake: ...23:21
v1k0d3nso horizon is ignoring TLS23:22
sdakepavo cool maybe you can help v1k0d3n when my power drops out :)23:22
sdakeat 8% atm23:22
sdakev1k0d3n i hear that, i need more information23:22
v1k0d3nok23:22
v1k0d3nhey if you're at 8% don't worry about it.23:22
sdakei reserve it for you :)23:23
sdakeshow me the line that says pass / fail / etc23:23
sdakeat the end of the ansible output23:23
v1k0d3ni will torch this. or try something else.23:23
sdakenothing secure about it23:23
v1k0d3nthere is no fail in any of the deploy logs23:23
sdakethe last bit of output23:23
sdakeit says number of jobs completed etc23:23
sdakerather tasks23:23
*** tonanhngo has quit IRC23:23
v1k0d3nsomething is cached. the endpoint is still there. when i curl the endpoint....23:24
sdakeyou jsut deployed23:24
v1k0d3n{"version": {"status": "stable", "updated": "2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.7", "links": [{"href": "http://192.168.70.25:5000/v3/", "rel": "self"}]}}23:24
sdakeare you running a regisry?23:25
v1k0d3n...no?23:25
sdakecool23:25
sdakeso i would expect the endpoint to respond unles you do a destroy23:25
*** tonanhngo has joined #openstack-kolla23:26
v1k0d3nyou said you need full log. anything from the log in particular?23:26
v1k0d3nthe log looks like a clean deployment.23:26
sdakev1k0d3n the last 3 lines after kolla-ansible deploy23:26
sdakeit says TASKS: xyz23:26
*** eaguilar has quit IRC23:27
sdakethere are two lines i'd take after this peice of ifnoratmion - either  firewalld is a problem or your globals.yaml is misconfigured23:28
sdakebut we need this pecie of information first to see if we move on to the next peice of debug23:28
sdakeif there are failures in that last two lines thats a problem - and that needs fixing first23:28
*** tonanhngo has quit IRC23:28
v1k0d3nconfused.23:29
sdakeright, thtas why i debug 1 step at a time :)23:29
v1k0d3nit worked with exact same globals -- the TLS portion.23:29
sdakeyour next step is to produce the end output of the deploy operation :)23:30
Pavokolla-build, kolla-ansible bootsctrap server -i /usr/share/kolla/ansible/inventory/multinode, kolla-ansible certificates, kolla-ansible deploy -i /usr/share/kolla/ansible/inventory/multinode23:30
Pavothats all I do23:30
sdakepavo v1k0d3n is AIO atm23:30
Pavoah haven't done a deployment of aio yet23:31
PavoI would assume its easier though23:31
*** Jeffrey4l has joined #openstack-kolla23:31
sdakev1k0d3n are you in a vm?23:31
Pavokolla-ansible certificates has to be done before deploy though23:31
Pavofor tls23:31
v1k0d3nno bare metal host23:31
sdakev1k0d3n  you said it worked with same globals with tls, but then you changed (??X) and it stopped working23:32
v1k0d3nPavo: i did certificates23:32
sdakewhat is the ??X above?23:32
v1k0d3nsdake: ..... again....23:32
v1k0d3ni deployed without TLS (commented out...default).23:32
v1k0d3nworked fine.23:32
Pavov1k0d3n do you have a certifcates folder in /etc/kolla?23:32
v1k0d3ndestroyed.23:32
v1k0d3ncleared all images and containers23:32
v1k0d3ngen certs23:33
v1k0d3nredeploy23:33
v1k0d3nreally, really simple.23:33
v1k0d3nthat's it23:33
sdakeyou misse dthe step in there where you changed globals.yaml23:33
sdakewhat did you change it to?23:33
sdakedestroy has nothing to do with it :)23:33
sdakeespecially if you rebooted23:34
v1k0d3nsorry. of course i changed globals.23:34
sdakeright - can you apste that plz23:34
v1k0d3nkolla_enable_tls_external: "yes"23:34
v1k0d3nkolla_external_fqdn_cert: "{{ node_config_directory }}/certificates/haproxy.pem"23:34
v1k0d3nthat's it.23:34
v1k0d3ni wanted to change dns, but i didn't...i'm trying to step through.23:35
sdakewe really need to modify our docs to include tls setup23:35
v1k0d3nsdake: isn't that what this is?23:35
v1k0d3nhttp://docs.openstack.org/developer/kolla/advanced-configuration.html23:35
kollabot1Advanced Configuration — kolla 4.0.0.0b2.dev45 documentation23:35
Pavowell you also have to have kolla_external_fqdn: as a domain name23:36
v1k0d3nhttp://docs.openstack.org/developer/kolla/advanced-configuration.html#self-signed-certificates23:36
kollabot1Advanced Configuration — kolla 4.0.0.0b2.dev45 documentation23:36
sdakecool23:36
Pavoif no domain name then no certs gets made23:36
sdakedidn't know that hit the docs23:36
v1k0d3nPavo: to be fair....i actually changed the fqdn as well during one attempt23:36
v1k0d3nwell, certs get generated, but i think against the IP address and not a FQDN23:36
Pavowhat kolla deployment version are you deploying?23:37
sdake# This should be a VIP, an unused IP on your network that will float between23:37
sdake# the hosts running keepalived for high-availability. It defaults to the23:37
v1k0d3nnot quite sure on this.23:37
sdake# kolla_internal_vip_address, allowing internal and external communication to23:37
sdake# share the same address.  Specify a kolla_external_vip_address to separate23:37
sdake# internal and external requests between two VIPs.23:37
sdake#kolla_external_vip_address: "{{ kolla_internal_vip_address }}"23:37
PavoI use stable/mitaka23:37
v1k0d3nok...23:37
v1k0d3nso right....23:37
v1k0d3nand i am not using HA23:37
v1k0d3nsorry...haproxy23:37
sdaketls is enabled via haproxy...23:37
sdakewho on earth told you not to use haproxy?23:37
sdakei really wish people would stop giving out that advice23:38
sdakeits bad!23:38
v1k0d3nright...but up above...even when i try to deploy W/O TLS it doesn't work now23:38
v1k0d3nand now we're back to the start... :)23:38
Pavoalso v1k0d3n The kolla_internal_vip_address and kolla_external_vip_address must be different to enable TLS on the external network.23:38
sdakethe docs are incomplete and should record this information23:38
Pavothat is from the docs sdake23:39
Pavoin the Note23:39
sdakepavo cool :)23:39
v1k0d3nok, so i destroyed again.23:39
sdakehow about pasting your globals.yaml this time :)23:39
v1k0d3ni need external and internal FQDN's23:39
v1k0d3nsdake: be glad to...but at this point, i want to know what it should actually be.23:40
sdakeyou said you have haproxy disabled23:40
Pavobut I do think the docs need to be completely rewritten in more of a guide stand point not just about the different areas of different settings, its need to be able to walk someone through a complete setup and deploy23:40
sdakethat shouldn't be one of the things :)23:40
sdakeif you had pasted the full logs this would have been the second thing i looked at23:41
Pavono need for external and internal FQDNs, but you do need internal vip and external vip to be different23:41
v1k0d3nPavo: OMG YES TO THIS!!!23:41
v1k0d3nuser is guessing..."what is advanced?" "what is default"?23:41
Pavoexactly23:41
v1k0d3nhow should user know from the start what is advanced?23:41
Pavothey are better than they were but still need alot more work23:42
v1k0d3nthey will see..."enabled TLS" and think "oh, yeah...i wanted that".23:42
v1k0d3nthen deploy...no precheck...no error...boom.23:42
sdake0%23:42
sdakenot sure when my laptop drops off23:42
v1k0d3nwell, for AIO, it should just be....done.23:42
sdaketls has a 30-300% inpact on performance23:42
v1k0d3nno guessing. just defaults. TLS is security. include it.23:42
sdakeso we don't enable it by default23:42
v1k0d3nAIO sdake23:43
Pavothis is my multinode globals23:43
Pavohttp://pastebin.com/eZbDjtTE23:43
kollabot1################### # Kolla options ################### # Valid options are [ - Pastebin.com23:43
v1k0d3nnobody is looking for optimal performance for AIO23:43
*** sdake has quit IRC23:43
Pavocan someone try and get to https://ddi.hopto.org?23:44
v1k0d3nPavo: no love to that link....23:44
Pavodamnit23:44
PavoI hate my ISP23:44
v1k0d3nwho?23:45
Pavocan you try 1 more time23:45
v1k0d3nsame. no go.23:45
Pavowtf tcpdump shows you trying to get to it23:45
Pavoso port forwarding is working23:45
Pavowonder if my FQDN is messed up23:46
*** sdake has joined #openstack-kolla23:47
PavoI can get to it no problem from inside my network23:47
sdakev1k0d3n the problem with enabling TLS for AIO is it requires TWO vips not one23:47
sdakegetting epople to understand ONE VIP is a challenge ;)23:47
sdakeover 200 times i've debugged peoples envs that had TLS == host ip23:48
sdakei believe there is a precheck for that now ;-)23:48
sdake(note TLS==host ip doesn't work)23:48
Pavotrue statement23:48
sdakelearning kolla is incremental - not all at once :)23:49
sdakeotherwise its overwhelming23:49
sdakenot to say the docs don't suck23:49
sdakethey indeed are not optimal23:49
PavoI think the biggest issue with the docs right now is that both deployment methods are on the same page and is confusing23:50
Pavothey need to be seperated23:50
sdakeyou mean dev workflow and operator workflow pavo?23:50
Pavo1 page for production deploymnet and a seperate page for dev deployment23:50
sdakewriting docs is harder then writing implementations ;)23:50
v1k0d3nPavo: http://pastebin.com/LdrTdd8J23:50
kollabot1globals.yml - Pastebin.com23:50
Pavoand each one has a complete walk through on how to setup and deploy, then add advanced config examples23:51
v1k0d3nthat's a WIP.23:51
v1k0d3ni have two interfaces....23:51
sdakeyikes enable_haproxy: NO is wrong23:51
v1k0d3none, no ip address.23:51
v1k0d3nworking on it sdake23:51
v1k0d3nthere are examples out there without it.23:51
Pavov1k0d3n #kolla_external_vip_address: "{{ kolla_internal_vip_address }}" needs to be different23:51
v1k0d3nah...btw... sdake including cisco23:51
v1k0d3ncisco dev learning23:52
v1k0d3nwhich is where this came from orig23:52
sdakeline 41 needs needs a specific VIP23:52
sdakev1k0d3n cisco is a big place, not everyone talks to eveyrone else ;)23:52
Pavo#kolla_enable_tls_external: "yes" needs to be uncommented23:52
sdakev1k0d3n anyway when i work here i work upstream23:52
Pavo#kolla_external_fqdn_cert: "{{ node_config_directory }}/certificates/haproxy.pem" needs to be uncommented23:53
v1k0d3nso one sec...23:53
v1k0d3none at a time?23:53
v1k0d3ni will change haproxy now23:53
PavoI would also change #enable_central_logging: "no" to enable_central_logging: "yes"23:53
Pavoso you can TS easier23:53
v1k0d3nkolla external vip23:53
v1k0d3ni have two nics up23:53
v1k0d3none without an ip address23:53
v1k0d3nand the other is what i access horizon on.23:53
*** kristian2709 has quit IRC23:54
sdakeright the external vip needs to be an unused address on your network23:54
v1k0d3n192.168.70.25 is horizon23:54
v1k0d3nso that is external?23:54
sdakethe internal vip needs to be an unsued address on your network23:54
sdakeyou need both for tls to work correctly23:54
v1k0d3nso both need to be unused addresses?23:54
sdakeright - now you see why its not enabled by default ;)23:54
Pavointernal vip is network that only admins would use to access horizon, external vip would be an address that everyone else would use to access horizon23:54
v1k0d3nso i want to be able to access horizon on 70.25.23:55
Pavoand if you did a destroy you need to reboot because those vip address do not get unused until reboot23:55
v1k0d3ni have name resolution on that.23:55
v1k0d3nso i need to re-IP my host to .24 or something?23:55
sdakekolla_internal_vip_address: "192.168.70.25"23:56
sdakewhat is 70.25?23:56
sdakeyour host?23:56
Pavothe IP address on the nic that its connected doesn;t matter, you will use your internal and external vips to access horizon23:56
v1k0d3nyes, host is 192.168.70.25.23:56
sdakeok, that needs to be an unused address such as 192.168.70.20023:56
Pavo^23:56
sdake(if 200 is in your network and unused)23:56
v1k0d3nok, let me say this all in one string...maybe that is easier.23:56
sdakelets get aio with haproxy without tls working first ;)23:56
v1k0d3ni have DNS already set up to reach horizon...so the VIP needs to be 70.25....so i guess i need to re-IP address my host, is that correct?23:57
Pavoyou can do kolla-ansible reconfigure to get tls working after you actually get everything else up23:57
Pavov1k0d3nyes correct23:57
sdakeright, if you re-ip your host, then .25 will be free for your internal vip - but your internal vip will not be TLS-ified23:57
v1k0d3nok. let me readdress my host then.23:57
Pavobut make sure to reboot after you do23:58
Pavoso ARP can clear its cache23:58
v1k0d3nsdake: different issue. i am taking one step at a time.23:58
v1k0d3naddressing first.23:58
v1k0d3nPavo: rebooting now23:59
sdakepavo he had no haproxy prior23:59
sdakea reboot wasnt' necessary23:59
v1k0d3nhave ideas on how i can rewrite the docs.23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!