Sunday, 2016-03-20

*** vinkman has joined #kolla00:00
*** vinkman has left #kolla00:00
*** rstarmer has joined #kolla00:02
*** bmace has quit IRC00:07
*** bmace has joined #kolla00:07
*** SiRiuS_ has quit IRC00:08
openstackgerritSam Yaple proposed openstack/kolla: Add generate_passwords.py to generate passwords  https://review.openstack.org/29372800:11
openstackgerritSam Yaple proposed openstack/kolla: Move manila variable  https://review.openstack.org/29379700:11
openstackgerritSam Yaple proposed openstack/kolla: Change deprecated options throughout Kolla  https://review.openstack.org/29493100:11
*** mbound has quit IRC00:14
*** rstarmer has quit IRC00:15
SamYaplesdake: ping00:17
*** akwasnie1 has quit IRC00:22
*** rstarmer has joined #kolla00:27
*** mbound has joined #kolla00:31
sdakesup sam00:35
sdakeSamYaple ^00:35
*** vishwanathj has joined #kolla00:50
SamYaplehey sdake00:58
sdakewhat can i do for ya00:58
SamYaplesdake: I have gone throw most/all the options and fixed the deprecated ones this cycle00:58
sdakei saw that00:59
SamYapleive fixed alot of the smaller bugs throughout00:59
SamYapleive done a bunc hof deploys on all of this stuff now00:59
sdakemultinode?00:59
SamYapleits really _really_ solid00:59
SamYapleyup00:59
sdakenice00:59
SamYaplevm and baremetal00:59
sdakeexcept heat and magnum are busted ;)00:59
SamYapleheat i found the bug00:59
SamYaplehavent submitted patch yet00:59
SamYaplei also submitted a patch that builds _all_ images for ubuntu in 12m!01:00
sdakeoh nice i was going to start working on it tomorrow or monday01:00
sdakeon vacation atm01:00
sdakebeen in flagstaff since thursday01:00
sdakesummer break for the kids01:00
SamYaplewell the outlook is solid for mitaka (with all the patches in the queue)01:00
SamYapleits really really slick01:00
SamYapleyoull be happy when you get back from vaca01:01
sdakei agree mitaka is fantastically better hten iberty01:01
sdakei have deploeyd in thur and fri in vms single node01:01
sdakeand the for 3 weeks single node on real gear prior01:01
sdakegave a demo of reconfigure and upgrade and deploy - all in under 15 minutes01:01
SamYapleyup ive been doing that too01:02
sdakepeople in the call were like "Wtf how is it so fast"01:02
SamYaplei tested lights out, change passwords, the works01:02
sdakefunny thing during my demo i had a wireless dropout01:02
sdakenormally i use screen to deal with that01:02
sdakebut didn't this time01:02
sdakeso my deploy went south badly01:02
sdakeso i nuked it and was ready to go in 4 minutes01:02
sdakeits easier to nuke a deploy then fix a failure during deploy01:03
sdakeupgrade and reconfig is a totally different story01:03
SamYaplei agree, though ive been "fixing failues" mode01:03
sdakeyup i've got two weeks on my 3 node gear to beat it up and get it rolling01:03
sdakere heat fix, did you fix the lauching heat vms problem?01:03
SamYaplei believe so01:04
SamYaplebut i dont have the patch running in my env atm to test01:04
sdakenice01:04
SamYaplei will in a bit01:04
sdakei wont be able to test that until monday01:04
SamYapleit was an endpoint issue like swift01:04
sdakei can't actually launch vms in my alptop01:04
sdakei did run reconfigure in a loop 500 times without failure01:05
sdakeatually it was a file with reconfigure pasted 500times01:05
sdakebut you get the idea01:05
sdakeshould try same thing with update01:05
sdakeupgrade rather01:05
SamYaplei actually did a master->master upgrade01:06
SamYapleworked fantastic01:06
sdakei did a master to mater upgrade in my demo01:06
SamYaplelike a true one01:06
SamYaple2 week old master to trunk01:06
sdakenice01:06
SamYaplerunning for 2 week environment01:06
SamYapleit worked perfect01:06
SamYapleit was great01:06
SamYapleand so quick!01:07
sdakei have a two week old openstack at this point , i can do that upgrade test as well over and over01:07
SamYaplei included the docs in the password gen patch01:07
sdaketypically these lat copule of weeks i run things in loops 500 or 1000 times to make sure there are no wierd failure conditions01:07
sdakei saw looks good01:07
SamYaplehaving the rando passwords is fantastic too01:07
sdakeI think you should look into SystemRandom class01:07
sdakebut other hten that looks good01:08
*** rstarmer has quit IRC01:08
SamYapleno we cant use that (or maybe as an option)01:08
SamYaplerandom blocks01:08
sdakeyou mean /dev/urandom?01:08
SamYapleno01:08
SamYapleurandom is what my patch uses01:08
SamYaplei commented as such in the review01:08
sdakehave you seen it block?01:08
sdakethe python docs seem to indicate that to get /dev/urandom  rather then /dev/random you have to use systemrandom class01:09
sdakenm i'm a moron01:10
sdakei've got em backwards01:10
sdake /dev/random is the secure one01:10
SamYapleyes but random blocks01:12
SamYaplelike i said01:12
SamYapleit can block forever01:12
SamYapleif you dont have enough entropy01:12
SamYapleurandom is planty safe, but that can be an option if people want01:12
*** mbound has quit IRC01:13
sdakeAlmost all module functions depend on the basic function random(), which generates a random float uniformly in the semi-open range [0.0, 1.0). Python uses the Mersenne Twister as the core generator. It produces 53-bit precision floats and has a period of 2**19937-1. The underlying implementation in C is both fast and threadsafe. The Mersenne Twister is one of the most extensively tested random number generators in existence.01:13
sdakeHowever, being completely deterministic, it is not suitable for all purposes, and is completely unsuitable for cryptographic purposes.01:13
sdakeso basically random python api uses a deterministic random() c library call01:14
sdake /dev/urandom doesn't block /dev/random blocks01:14
SamYaplecorrect01:14
SamYaplethats waht ive been saying01:14
sdakeyes but the current code doens't use the os random01:14
sdakeits using one that the c library provides01:14
sdakewhat i'm getting at is /dev/urandom is not determinstic in most reaosnable cases except after a reboot01:15
SamYapleim pretty sure i use /dev/urandom01:15
sdakeand when entropy has run out01:15
SamYaplebut ill triplecheck01:15
sdakeit says right above in the python docs what random uses01:15
sdakeThe underlying implementation in C is both fast and threadsafe.01:15
sdakeit would be helpful is the kernel used interrupts for entropy pool01:17
sdakei dont know why it doesn't probaby a performance thing01:17
sdakewhen i gen pgp keys it always takes forever on my machines where i don't have a mosue and keyboard connected01:17
SamYapleremember im using random.choice01:18
SamYaplei believe that grabs urandomm, but like i said ill triple check01:18
sdakeya lets check the python lib code jsut to duble check01:18
sdakei could do but i dont hae a vm booted atm ;)01:19
sdakeso by let i mean you :)01:19
sdakelets01:19
sdakeyou can still use random.choice01:21
sdakeSystemRandom inherits from the random class01:21
sdakehttps://docs.python.org/3/library/random.html#random.SystemRandom01:22
*** weiyu_ has joined #kolla01:24
*** weiyu_ has quit IRC01:28
*** vhosakot has joined #kolla01:29
sdakethe source on my mac says random.choice uses random.random from _random.Random01:29
sdakelooks like _random may be c code, i don't see it on my mac01:30
*** iceyao has joined #kolla01:32
*** weiyu_ has joined #kolla01:33
sdakeSamYaple line 30 is how random() is implemented: http://nullege.com/codes/show/src%40m%40a%40main-HEAD%40External.LCA_RESTRICTED%40Languages%40CPython%4027%40Lib%40random.py/71/_random.Random/python01:35
sdakeSamYaple from the source code here is how python gets _random:01:42
sdakehttps://github.com/python/cpython/blob/master/Modules/_randommodule.c01:42
sdakedefined from here: https://github.com/python/cpython/blob/master/setup.py#L62501:43
sdakeclearly that c code does not use any operating system primitives ;)01:43
sdakeok off for a break01:43
*** vhosakot has quit IRC01:43
*** vhosakot has joined #kolla01:45
*** weiyu_ has quit IRC01:45
*** vhosakot has quit IRC01:46
*** vhosakot has joined #kolla01:48
SamYaplesdake: i have a patch up that speeds up the gate alot01:50
SamYaple12 min all images build for ubuntu01:50
sdakehow does it work01:51
SamYapletrims off 5-10m for centos01:51
sdakei'll look in queue01:51
SamYaplehttps://review.openstack.org/#/c/294854/501:51
SamYaplesame script i used last cycle at the end01:51
sdakeis this just source or binary as well?01:51
SamYaplebasically install packages that are installed >=10 times by other contaienrs01:52
sdakehavn't actuallly looked a your code :)01:52
sdakeya makes sense01:52
SamYaplesource, rhallisey said he wanted to do binary01:52
sdakewfm01:52
SamYaplebut it was a 12m gate run time!01:52
SamYaplethe deploy gate for ubuntu was sub 10m now!01:52
SamYaplealso shaves of about 5% of total build size01:52
sdakeought to get that script in the repo if you have one01:53
SamYaple600MB for centos01:53
SamYapleits a one liner with manual intervention01:53
sdakebummer01:53
SamYapleyea01:53
SamYaplei dont know its worth it to automate01:53
SamYapleit requires manual tweaks01:53
sdakemy only concern is if deps get dropped01:53
SamYaplewell they wont this cycle01:53
sdakebut as long as someone looks after it -w fm ;)01:53
sdakeright01:53
sdakei mean int he future01:53
SamYapleand i dug through them from last cycle01:53
SamYaplei diff'd01:54
SamYaplenothing dropped, only adds01:54
sdakei think pycrpto is going byebye01:54
SamYaplebut i agree, its a concern for the future01:54
sdakeor one of those crypto libs01:54
sdakei can't tkeep track which one is blessed ;-)01:54
SamYaplewhatevers on there is installed 10+ times by all packages01:54
sdakeya i just acked it01:55
sdake10 min delta between centos and ubuntu01:56
sdakemirrors ftw01:56
SamYapleisnt that crazy?01:56
SamYaple10m from build to vm01:56
sdakeya 35% delta01:56
sdakeno i mean 30 min centoso source build and 20 min ubuntu build from source01:57
*** vhosakot has quit IRC01:57
SamYapleyea i get that01:57
SamYaplebut ubuntu deploy was 10m gate!01:57
SamYaplebuild and deploy in 10mitues is crazy01:57
*** vhosakot has joined #kolla01:57
*** vhosakot has quit IRC02:03
*** vhosakot has joined #kolla02:03
openstackgerritHui Kang proposed openstack/kolla: Ceilometer needs pymongo to start  https://review.openstack.org/29472502:04
*** mbound has joined #kolla02:14
*** vhosakot has quit IRC02:14
*** weiyu_ has joined #kolla02:19
*** banix has joined #kolla02:19
*** mbound has quit IRC02:19
*** vhosakot has joined #kolla02:36
sdakefor folks that haven't watched the documntary "The Lottery of Life" I'd recommend it02:41
*** rstarmer has joined #kolla02:45
rstarmerwhat is this "recongifure" command you talk about. this sounds interesting.02:48
sdakerstarmer hey there02:48
sdakeit reconigures your openstack environment02:48
sdakeso if you ahve bridge A setup nad you instead want it to be bridge B02:49
sdakeit does that02:49
sdakeit merges /etc/kolla/config/* files into the main configuration files02:49
sdakeand overrides existing entries if present02:49
sdakeit allows for complete customization of openstack on deploy02:49
rstarmeroh.  wow, that is cool.02:49
sdakewhat we were lacking in liberty was a way to change those config options02:49
sdakereconfigure allows runtime changing with minimal downtime02:49
rstarmerVery very interesting.02:50
rstarmerI will need to investigate further.02:50
SamYaplerstarmer: it is a newer feature so it may have some bugs, but so far they are break-the-script bugs, not break-the-deploy bugs02:51
SamYaplethey way it works it would be hard/impossible to break the deploy anyway02:51
rstarmerMost excellent.  Will be playing with this as time permits this week!02:51
rstarmerI'm becoming more and more of a fan of this new fangled container based world... :)02:52
sdakeits about 2 years old ;)02:52
*** vhosakot has quit IRC02:53
sdakefinally we have delivered on upgrade02:53
sdakewhich is the whole reason we headed down the container road for openstack to begin02:53
*** klint has joined #kolla02:53
*** vhosakot has joined #kolla02:53
rstarmeryeah, I talk to so many people who are stuckc because upgrades, while "feasible" are just not something they actually want to touch.02:53
SamYaplerstarmer: im an ops guy and i say with no bias this is the easiest upgrade process for openstack02:54
SamYapleand ive used (and supported) most of them02:55
vhosakotrstarmer: here are the steps to reconfigure OpenStack services in kolla - https://etherpad.openstack.org/p/kolla-mitaka-testing-reconfigure02:55
SamYaplelove containers02:55
rstarmervhosakot: thanks, I needed that :D02:56
vhosakotrstarmer: cool, np :)02:56
rstarmerSamYaple: yeah, I help people with their Ops problems, often by comiserating with the mess they have...02:56
SamYaplekolla in the state it is in is actually alot of "man i hated doing xyz in <insert openstack tool>"02:57
SamYapleso hopefully ops do like this02:57
rstarmerI've got a project where I hope to be able to guide the customer into Kolla.  They're using puppet ATM, and that's where my previous experiences lie, and so I hope to be able to show them the light.03:01
SamYapleif they want 10minute multinode deploys, look no further!03:02
rstarmerI'l be demoing this to them on Tuesday if all goes well...03:03
openstackgerritHui Kang proposed openstack/kolla: Ceilometer needs pymongo to start  https://review.openstack.org/29472503:03
sdakekolla doesn't createa a mess the first place to clean up ;)03:04
SamYapleyes03:04
SamYaplei actually think thats a big reason kolla has come so far so quickly03:05
SamYaplebecause us devs can clean our environments so thorughly so quickly03:05
rstarmerI'm acutally using a simmilar approach (well, similar in that it containerizes the application middleware) for a Kubernetes project I'm supporting as well.  But there's no Kolla equivalent (at least that I've found) for Kubernetes, so that's something I might have to try as I get a better handle on the "system" of operation.03:06
*** sdake has quit IRC03:07
vhosakotrstarmer: kube.sh is pretty buggy ;)03:08
vhosakotrstarmer: you using vagrant+kubernetes ?03:09
rstarmervhosakot: no, the docker container based master/minion multi-node setup.03:09
vhosakotah, cool.. docker..  the vagrant setup + kube is not stable03:10
rstarmerit's all buggy. I only need it for some performance testing work, but I also need it to be re-produceable.03:10
*** sdake has joined #kolla03:10
*** sdake has quit IRC03:10
*** weiyu_ has quit IRC03:10
*** sdake has joined #kolla03:11
vhosakotah ok03:11
*** macsz has quit IRC03:36
*** vhosakot has quit IRC03:42
*** vhosakot has joined #kolla03:43
*** rstarmer has quit IRC03:44
*** weiyu_ has joined #kolla03:53
*** weiyu_ has quit IRC03:57
*** elmiko has quit IRC04:18
*** elmiko has joined #kolla04:18
*** dave-mcc_ has quit IRC04:24
*** jasonsb has quit IRC04:34
*** iceyao_ has joined #kolla04:39
*** iceyao has quit IRC04:40
*** iceyao has joined #kolla04:46
*** sdake has quit IRC04:47
*** iceyao_ has quit IRC04:50
*** vhosakot has quit IRC04:51
*** sdake has joined #kolla04:56
*** banix has quit IRC05:32
*** Windir has quit IRC05:45
*** sdake has quit IRC06:11
*** sdake has joined #kolla06:11
*** sdake has quit IRC06:20
*** achanda has joined #kolla06:33
*** Marga_ has quit IRC07:13
*** Marga_ has joined #kolla07:14
*** ssurana has joined #kolla07:23
*** stvnoyes has quit IRC08:03
*** stvnoyes has joined #kolla08:04
*** jmccarthy has quit IRC08:06
*** jmccarthy has joined #kolla08:06
*** achanda has quit IRC08:16
*** vishwanathj has quit IRC08:16
*** akwasnie has joined #kolla09:11
*** akwasnie has quit IRC09:18
*** akwasnie has joined #kolla09:19
*** akwasnie has quit IRC09:19
*** iceyao has quit IRC09:33
*** SiRiuS_ has joined #kolla09:41
*** akwasnie has joined #kolla09:44
*** achanda has joined #kolla09:46
akwasniesdake: ping09:48
akwasnieSamYaple: ping09:49
*** starmer has joined #kolla09:53
*** achanda has quit IRC09:53
*** starmer has quit IRC09:53
*** Serlex has joined #kolla09:54
*** ssurana has quit IRC10:09
*** pbourke has quit IRC10:13
*** pbourke has joined #kolla10:14
*** sdake has joined #kolla10:16
*** ccesario has joined #kolla10:17
*** iceyao has joined #kolla10:34
*** ccesario has quit IRC10:34
*** sdake has quit IRC10:48
*** sdake has joined #kolla10:50
*** achanda has joined #kolla10:50
*** achanda has quit IRC10:56
*** akwasnie has quit IRC10:58
*** Serlex has quit IRC11:38
*** dave-mccowan has joined #kolla12:00
*** Jeffrey4l_ has joined #kolla12:06
*** Jeffrey4l__ has quit IRC12:09
*** Liuqing has joined #kolla12:36
*** akwasnie has joined #kolla12:38
*** sdake has quit IRC12:47
*** achanda has joined #kolla12:54
*** akwasnie has quit IRC12:55
*** akwasnie has joined #kolla12:56
*** achanda has quit IRC13:00
*** klint has quit IRC13:01
*** dave-mcc_ has joined #kolla13:01
*** dave-mccowan has quit IRC13:02
*** dave-mcc_ has quit IRC13:06
*** rhallisey has quit IRC13:07
*** rhallisey has joined #kolla13:08
*** sdake has joined #kolla13:09
*** akwasnie has quit IRC13:11
*** akwasnie has joined #kolla13:16
*** Jeffrey4l_ has quit IRC13:18
*** sdake has quit IRC13:20
*** sdake has joined #kolla13:21
*** akwasnie has quit IRC13:32
*** akwasnie has joined #kolla13:33
*** Jeffrey4l_ has joined #kolla13:42
*** akwasnie has quit IRC13:46
*** akwasnie has joined #kolla13:46
*** achanda has joined #kolla13:56
SamYapleakwasnie: pong13:57
*** achanda has quit IRC14:01
*** akwasnie has quit IRC14:03
*** akwasnie has joined #kolla14:03
*** mbound has joined #kolla14:03
openstackgerritSam Yaple proposed openstack/kolla: Fix rax-ord gate  https://review.openstack.org/29500114:05
openstackgerritAndrei-Lucian Șerb proposed openstack/kolla: Attach external NIC to a NAT-Network if on Wi-Fi  https://review.openstack.org/29434014:17
*** mbound has quit IRC14:17
*** akwasnie has quit IRC14:26
*** akwasnie has joined #kolla14:27
*** diogogmt has quit IRC14:36
*** diogogmt has joined #kolla14:37
*** akwasnie1 has joined #kolla14:39
*** akwasnie has quit IRC14:39
*** dims has joined #kolla14:40
*** dwalsh has joined #kolla14:44
*** diogogmt has quit IRC14:47
akwasnie1sdake, SamYaple: it turns out that shield requires a licence - it starts with free 30 day trial, but after that additional licence is required. It is easy to install and configure (i tried and it works), but license stuff rather eliminates it from being used in kolla.14:53
akwasnie1so i think that we are back to nginx proxy14:53
SamYapleakwasnie1: agreed14:54
SamYaplenginx it is14:54
SamYaplehey akwasnie1 is the central logging stuff... functional?14:54
SamYaplei havent played with ti at all yet14:54
akwasnie1yes, it is :)14:55
SamYapleawesome! thats very cool14:55
SamYapleyouve all done great work on that14:55
SamYapleim going to build and install it now if thats the case14:55
SamYapleelasicsearch and kibana are the only containers right?14:55
akwasnie1thanks! :) should we create seperate container for nginx proxy (we might use this proxy for other services that does not have any authentication) or should we stick with configuring it inside kibana container?14:56
openstackgerritAndrei-Lucian Șerb proposed openstack/kolla: Attach external NIC to a NAT-Network if on Wi-Fi  https://review.openstack.org/29434014:56
akwasnie1elasticsearch and kibana are the only ones14:56
akwasnie1does -> do14:57
SamYapleakwasnie1: that depends, is nginx going to be a seperate process that needs to be watched? (i think it is)14:57
SamYapleif so new container14:57
SamYapleyou know akwasnie1 haproxy also does basic auth....14:58
SamYaplewe could just use that14:58
*** dims has quit IRC15:01
akwasnie1will try this tomorrow, then15:01
*** dims has joined #kolla15:01
SamYapleim wondering is there any downsides to it?15:02
*** akwasnie1 has quit IRC15:06
*** Liuqing has quit IRC15:07
sdakeonly requirement is password auth that is secure15:16
sdakehow its done, I don't care ;)15:16
sdakeif haproxy can do it that seems reasonable to me15:16
SamYaplehaproxy would do the same thing as nginx, Auth header15:17
SamYaplepbourke: you around?15:17
*** dims has quit IRC15:23
*** dims has joined #kolla15:27
*** iceyao has quit IRC15:32
*** ccesario has joined #kolla15:47
*** ccesario has quit IRC15:53
openstackgerritMerged openstack/kolla: Fix linuxbridge driver  https://review.openstack.org/29493615:57
*** vhosakot has joined #kolla15:57
*** achanda has joined #kolla15:59
*** vhosakot has quit IRC16:02
*** vhosakot has joined #kolla16:03
*** achanda has quit IRC16:04
SiRiuS_vhosakot, ping16:11
openstackgerritJeffrey Zhang proposed openstack/kolla: Fix the prechecks role idempotent  https://review.openstack.org/29501816:11
vhosakotSiRiuS_: hi16:11
SiRiuS_vhosakot, I answered the questions inside patch 516:12
SiRiuS_vhosakot, Basically, the globals.yml file is part of kolla itself, while the Vagrantfile is not part of kolla , but is used to provision kolla nodes16:13
vhosakotSiRiuS_: i don't see them. are they still in draft (red color). Could you hit "Reply" at the the top and click "post"16:13
*** akwasnie has joined #kolla16:14
*** akwasnie has left #kolla16:15
SiRiuS_vhosakot, is says Draft next to my answers16:15
SiRiuS_hmm16:15
vhosakotSiRiuS_: cool, I was just thinking if the default values used (network_interface: "eth0"  and  neutron_external_interface: "eth1") in globals.yml need to change, or the same values are good ?16:15
SiRiuS_vhosakot, how do I get the answers submitted, if they are in Draft. I'm new, as you can probably tell :)16:16
openstackgerritMerged openstack/kolla: Fix race condition in haproxy  https://review.openstack.org/29278216:17
vhosakotSiRiuS_: I found it unintuitive as well... so, select patch set 5 (where you see draft comments in red), click the green Up arrow at top right, click "Reply" at top, click "Post"16:18
vhosakotSiRiuS_: in the gerrit homepage, at top right, it shows the patch set number ("Patch Sets 5/7) at top right next to Download16:19
SiRiuS_vhosakot, I think I did it, thanks for the help16:21
vhosakotSiRiuS_: cool, np :)16:21
SiRiuS_vhosakot, if you have any followup questions, please ask :)16:24
vhosakotcool, thanks for the replies... so, bootstrap.sh sets network_interface and neutron_external_interface.. cool16:24
vhosakotSiRiuS_: https://github.com/openstack/kolla/blob/master/dev/vagrant/bootstrap.sh#L141-L142.. so, what is the value of network_interface here ? eth0 or eth1 ?16:24
SiRiuS_vhosakot, yup16:25
vhosakotSiRiuS_: eth0 or eth1 ?16:25
SiRiuS_vhosakot, it would be ethic16:25
SiRiuS_*eth216:25
rhalliseyhello everyone16:25
vhosakotSiRiuS_: neutron_external_interface is eth2, what about network_interface ?16:25
vhosakotrhallisey: o/16:25
SiRiuS_vhosakot, eth116:26
SiRiuS_vhosakot, just like the file16:26
vhosakotSiRiuS_: ah, cool... just wanted to make sure that they are not affected.. cool, thanks for the new patch set.. it looks good16:26
SiRiuS_vhosakot, sure, that's why i configured the other 2 interfaces (NIC 1 and NIC2), because if I didn't then it would change the order16:27
*** dwalsh has quit IRC16:28
SiRiuS_vhosakot, it's not documented in Vagrant, so I had to do it by trial and error16:28
vhosakotSiRiuS_: I see that in the order and the comment about the order in code... cool16:28
SiRiuS_vhosakot, first time I tested it, I just set up NIC 3, but when the guest OS booted, it would connect NIC3 to eth0, which was not good at all16:29
vhosakotah ok... I have seen the guest OS pick the configured  NIC and name it eth016:30
*** vhosakot has quit IRC16:32
openstackgerritMerged openstack/kolla: Ceilometer needs pymongo to start  https://review.openstack.org/29472516:38
SamYaplesdake: would you create teh stable/mitaka branch please?16:41
SamYapleto unblock master16:42
SamYaplealso we need to pin the versions of the projects were using, but need the stable branch for that16:43
sdakeSamYaple iirc we had planned to branch at rc216:52
sdakethe rationale is to keep people focused on fixing bugs in mitaka16:52
sdakeof which there are plenty in nthe bug tracker16:53
SamYaplewell the issue is everyone else branched already16:53
SamYapleand so now master is not mitaka16:53
SamYapleit is newton16:53
SamYapleso we need to pin versions16:53
SamYapleyou dont need to tag16:53
SamYapleyou need ot branch16:53
sdakei understand16:54
sdakedo you not agree if mitaka is branched people will do dev rather then fix bugs?16:54
SamYaplei dont think what i agree to matters, right now we arent gating mitaka16:54
SamYapleother projects are deving right now and we are consuming16:54
sdakei see16:55
SamYaplei think branching is the best for stability16:55
sdakeok that make sense16:55
SamYaplei have a bunch of bug fixes in the queue16:55
sdakesee when i send emials to the mailing list, all you ahve to do is respond :)16:55
SamYaplewhen did you send an email?16:55
sdakebasically dev is supposed to be done at mitaka-316:55
sdakewhen mitaka-3 was released16:55
sdakeneed subject?16:55
SamYapleah ok ill find it16:56
SamYaplebut i did not see it apaprently16:56
SamYaplemy email skills are not the best16:56
SamYaplei just need better filters16:56
*** jasonsb has joined #kolla16:56
sdakeok well i'll send note that master has branched16:57
SamYaplewell if we can stable/mitaka on monday after the queue has been merged that would be best i think16:57
sdakeok monday it is16:57
SamYapleyea there are some critical things in the queue needing merging16:57
sdakei think mitaka is in pretty good shape minus the fact that heat is broken16:57
SamYapleyea ill fix that16:57
sdakeif vms can be launched in heat, i'd be all for branching :)16:57
SamYaplei did test my fix16:57
sdakebackporting always fails on our end16:58
sdakethe idea with backports is you shouldn't do them unless absolutely necessary16:58
sdakehehnce the idea of waiting to branch16:58
sdakebut our project isa bit odd that it consumes other upstream projects for gating purposes16:58
SamYaplebackporting for us fails because of lack of focus. we dont manage our bugs well16:59
sdakecool so by test, did you llaunch a heat stack?16:59
SamYapleyea16:59
sdakei wholeheartedly agree16:59
SamYapleeven if focus wasnt there, good bug management could allow a single person to manage backports very easily16:59
*** vhosakot has joined #kolla17:00
sdakeyes but with all the trivailfix stuff happening which are really bugs17:00
sdakethat is hard17:00
sdakeneed to clamp down on that as a project, I think epople tend to abuse it17:00
SamYapleyea i think weve all done it17:00
SamYaplei agree17:00
sdakei file bugs if its not docs or a one liner17:01
SamYaplewhat we need instead of "trvialfix" is "AffectsMasterOnly"17:01
sdakeso people don't have to comb the git repo for fixes17:01
sdakewhat we need is for people to file bugs ;)17:01
sdakepeople use trivial fix for features, bugs, whatever suits their fancy ;)17:02
sdakethe big problem is it doesn't give us a true accounting of the bug work taking place17:03
SamYaplewell thats the thing. what do i do with something thats not a bug? its a feature, but not worthy of a blueprint?17:03
sdakeand no way to track = no way to backport17:03
sdakethose should be wishlist bugs17:03
sdakei dont know why docs get autoassigned wishlist either, since they arenot features17:03
sdakei prefer docs fixes get filed at high priority but i don't think there is control on that17:03
SamYaplethere is on the docs bot side17:04
SamYaplebut anyway yea we should just force bugs and not try to do a "affectsmasteronly" tag or anything17:04
*** dims_ has joined #kolla17:04
SamYapleand that means stop the trivalfix stuff17:04
SamYaplebecause "fix" means bug17:04
*** dims has quit IRC17:05
sdakewe implemented trivialfix because develoeprs were complaining about bug overhead17:05
SamYapleeh that doesnt realyl work either17:05
SamYapleyea17:05
sdakebecause filing a bug takes 5-10 minutes of someones life :)17:05
SamYaplehonestly the masteronly thing might work better17:05
SamYaplewe dont need a bug for masteronly fix17:05
SamYapleeven wishlist-type feature17:05
openstackgerritMerged openstack/kolla: Increase usefulness of openstack-base  https://review.openstack.org/29485417:05
SamYaplebut we need a bug for anythign that doesnt affect just master17:05
sdakethe problem is eveyrone applies tags differently using different semantics17:06
sdakethats why its better not to have any tags at all ;)17:06
sdakethis is the problem with process - different things to differnt people17:06
SamYapleforcing bugs is something no other projcet does17:06
SamYapleid be inclined to default to thier ways17:06
SamYapleit works for the rest of openstack better than what we do17:06
sdakenova allows features without a blueprint or bug tracker?17:06
SamYaplenova doesnt even allow bug fixes17:07
SamYaplebut lets not base anything we do on "nova does it"17:07
vhosakothow about bugs non non-one-liners and TrivialFix for _only_ one-liners ?17:07
SamYaplethey arent the pillar of working17:07
SamYaplevhosakot: what about a one-liner that needs to be backported?17:07
sdakewell they do have a big pile of stuff to deal with :)17:07
sdakevhosakot this issue that needs solving is hwo to manage backports17:07
sdakethe way I'd like to manage backports it just not do them by releasing sotwre that is good out of the gate ;)17:08
SamYapleI do like this MasterOnly thing, something that doesnt need a bug, isnt a blueprint, wont be backported17:08
vhosakotSamYaple: bug i that case.. one-liner in master but needed in older release in bug...17:08
sdakevhosakot how does the backporter know17:08
sdakecomb the commit to figure out if its a bug that needsa a backport or what?17:08
vhosakotsdake: right, the team needs to finalize/converge/freeze backports often17:09
sdakethe way other projects work is file a bug, then say affects which branches in launchpad17:09
sdakeor use a tag17:09
sdakewe are not using the bugs feature of launchpad very effectively as a team17:09
sdakei can't even get people to triage properly much less sort out how to do a backport17:10
sdakeit would be nice if ayone could asig themselves a bug, but not actually do the triaging17:10
vhosakotsdake: I think backport is a feaure-creep in older release and may un-stabilize the older release.. so, the team (at least the cores) need to know the status of backports..17:10
sdakethen we could reduce the size of the drivers team17:10
sdakebecause the drivers team's job is to do bug triage17:10
sdakevhosakot right backports turn into features17:11
SamYaplevhosakot: we commit to fixing bugs in backport17:11
sdakerather then "heat doesn't work"17:11
sdakeor whateer17:11
SamYaplevhosakot: not features17:11
sdakeour last release about 60 changes went in right after the branch which were not backports and not even slated for liberty17:11
vhosakotbugs in backport is different.. _backport_ itself is a new feature in the older release17:12
sdakeit almost gave me a heartattack17:12
sdakeSamYaple since i dont have my gear in flagstaff (back tonight) i'll test monday/tuesday and we can branch then17:12
sdakei want to test multinode on my own for centos17:12
sdakeI don't think kalot of folks are doing that17:12
SamYaplesdake: you dont recall do you? we agree to the 1.1.0 release right after liberty. thats why those chages are there17:13
sdakeyup but it wasn't staffed properly17:13
*** dwalsh has joined #kolla17:13
SamYapleno it was on track17:13
sdakei think agreeing to a 1.1.0 right after liberty was an error17:13
sdakebut we can't fix the past17:13
openstackgerritJeffrey Zhang proposed openstack/kolla: Install neutron in manila-share group only when enable_manila is true  https://review.openstack.org/29502417:13
sdakemorning Jeffrey4l_17:14
SamYaplenot following through was a bigger error17:14
*** vishwanathj has joined #kolla17:14
SiRiuS_vhosakot, can I get a +1 if you are satisfied with the patch :)17:14
SamYaplewe need to be better at following through17:14
sdakewe are good at followthrough on master17:14
vhosakotSiRiuS_: sure, 1 sec... was just testing your patch srt... looks great17:14
sdakeliberty was our first release where backporting was actually done17:15
Jeffrey4l_sdake, good night, it is 1 AM here :)17:15
SamYapleJeffrey4l_: go to sleep!17:15
SiRiuS_vhosakot, cool, I'll wait17:15
vhosakotJeffrey4l_: night night!17:15
Jeffrey4l_yea. I am going.17:15
vhosakotSiRiuS_: done, +1, nice work17:16
SiRiuS_vhosakot, thanks \m/ :)17:16
sdakei've been doing softwre for 20 years and have never seen backports handled properly17:17
vhosakotWhenevr I used Vagrant, I'm wired (bridge) so did not see the issue... this time, I used wireless NAT to test.. cool17:17
Jeffrey4l_sdake, about the magnum v3 issue. There are trustee_domain_id and trustee_domain_admin_id in trust section.17:18
sdakeits not like I havnet seen people try a million things17:18
Jeffrey4l_which domain/admin should we use?17:18
sdakeJeffrey4l_ recommend looking at how heat does it17:18
SamYaplesdake: did you file a bug on that heat issue?17:18
sdakeyes17:18
Jeffrey4l_default domain or create a new domain ( for example magnum_domain )17:18
Jeffrey4l_ok17:18
vhosakotI am not a fan of backports.. why touch older software when it if good.... backport something if it is _really_ needed for most users17:19
sdake"Heat doa in mitaka3"17:19
sdakeyup that is the only viable way to handle backports vhosakot17:19
sdakeas in don't do them unless absolutely necessary17:19
vhosakotsdake: cool17:19
Jeffrey4l_heat use the default domain. I will copy that.17:20
openstackgerritSam Yaple proposed openstack/kolla: Fix heat endpoint registration  https://review.openstack.org/29502517:20
SamYaplethere ya go sdake17:20
sdakenice thanks SamYaple17:20
Jeffrey4l_need go to sleep. bye guys.17:24
sdakelater lei17:24
sdakesay SamYaple17:25
sdakequestion relating to performance17:25
sdakewhy is mitaka so much faster then liberty?17:25
*** vhosakot has quit IRC17:25
*** vhosakot has joined #kolla17:26
SamYaplememcache all the things17:27
SamYaplebut if youre talking about horizon, thats thanks to jeffrey17:27
sdakei am talking about deplooy time17:27
SamYapleoh i optimized the crap outa it17:28
SamYaplereduced number of tasks drastically17:29
SamYapledid merge_configs17:29
SamYapleet17:29
SamYapleetc*17:29
sdakeso its just a reduction in number of tasks?17:29
SamYaplemostly17:29
SamYapleand sped up tasks17:29
sdakei seem to recall there were about 300 tasks in liberty17:29
sdakeand about 300 in mitaka17:29
SamYapleright but the number YOU use to deploy are less17:30
SamYaplewe added like 5 services17:30
SamYaplegate deplys in ~150 tasks17:30
SiRiuS_vhosakot, can you please click on "This bug affects you" , if you managed to reproduce it, here https://bugs.launchpad.net/kolla/+bug/155876617:34
openstackLaunchpad bug 1558766 in kolla "neutron not working inside virtualbox on wi-fi" [Undecided,In progress] - Assigned to Andrei-Lucian Șerb (lucian-serb)17:34
vhosakotSiRiuS_: as I said, I am always wired and _dont_ use vagrant for daily dev... I was not able to reproduce the bug, but I did not see the bug when I tested your patch set17:36
SiRiuS_vhosakot, I had a discussion with mandre about it, but he did not have VirtualBox, and was not able to reproduce it17:36
SiRiuS_vhosakot, oh, ok17:36
vhosakotSiRiuS_: I use the Vagrant libvirt plugin17:36
vhosakotSiRiuS_: I turned on wifi, shutdown my wired interface, tested your patch set, and did not see the bug..17:37
vhosakotSiRiuS_: are you using VirtualBox on Ubuntu or Mac/Windows ?17:38
SiRiuS_vhosakot, So practically it works on libvirt, without the patch, because libvirt was not patched17:38
SiRiuS_vhosakot, on Mac, but VirtualBox documentation did not specify Mac being the only one affected17:39
vhosakotSiRiuS_: yep, I have seen ubuntu+virtualbox not stable and I had to uninstall libvirt for it (which I dont want to, my libvirt is pretty stable)17:39
vhosakotSiRiuS_: ah ok17:39
SiRiuS_Bridging to a wireless interface is done differently from bridging to a wired interface, because most wireless adapters do not support promiscuous mode. All traffic has to use the MAC address of the host’s wireless adapter, and therefore VirtualBox needs to replace the source MAC address in the Ethernet header of an outgoing packet to make sure the reply will be sent to the host interface. When VirtualBox sees an incoming packet with a destination IP add17:39
SiRiuS_ress that belongs to one of the virtual machine adapters it replaces the destination MAC address in the Ethernet header with the VM adapter’s MAC address and passes it on. VirtualBox examines ARP and DHCP packets in order to learn the IP addresses of virtual machines.17:39
SiRiuS_vhosakot, that is what the docs say17:40
vhosakotSiRiuS_: ah ok.. let me read17:40
vhosakotSiRiuS_: isn;t this in the commit message17:40
SiRiuS_it is17:40
vhosakotcool17:40
vhosakotok, gotta step out a bit... SiRiuS_, I think I have a Ubuntu+Vagrant image minus libvirt.. will let you know if I find time to use and test it...17:42
SiRiuS_vhosakot, nice, that would be awesome17:43
vhosakotSiRiuS_: frankly, I don't spend much time debugging Vagrant issues as Vagrant is not used in production and no customer sees/worries about it mostly... a lot of devs use for daily dev/VM creation, teating and re-creating VMs easily17:44
SiRiuS_vhosakot, sure, it's mostly devs or people trying kolla for the first time17:44
vhosakotyep,17:45
SiRiuS_vhosakot, not everyone has a lab where they can do deployment :)17:45
vhosakotyep, use Ubuntu/Liux on laptop to begin with ;)17:45
SiRiuS_yeah17:46
vhosakotmost of the time, my host OS on laptop is Linux17:46
vhosakotI have seen libvirt/qemu in the host kernel a _lot_ stabler than virtualization software that _emulate_ like virtualBox/vmware fusion, etc17:47
openstackgerritMerged openstack/kolla: Fix the prechecks role idempotent  https://review.openstack.org/29501817:48
vhosakotplus, needless to say, the networking/nat magic these softwares do... but, yes, they are great if host OS on laptop is not Linux or without libvirt17:48
vhosakotok, gotta step out a bit... see you all17:49
sdakevirtualbox uses hardware virt if your mac is configured properly17:49
sdakeSiRiuS_ are yuo Anderi-Lucian?17:49
SiRiuS_sdake, yes17:50
sdakeSiRiuS_ could you explain how to manully setup virtualbox so that networking works properly?17:50
sdake(with wifi)17:50
sdaketo me - so i can document it17:50
vhosakotsdake: yes, agreed17:50
sdakeI use a mac for dev17:50
sdakeand i need to know :)17:50
SiRiuS_sdake, sure17:51
SiRiuS_I'll tell you the full story :D17:51
sdakelet me go get a monster drink ;)17:51
sdakej/k ;)17:51
SiRiuS_basically I downloaded kolla, and managed to reply fine17:52
SiRiuS_deploy*17:52
SiRiuS_I ran the init-runonce17:52
SiRiuS_so I had everything set up properly17:52
SiRiuS_but I could not get internet connectivity inside the nova VMs17:53
SiRiuS_neither could I connect to the nova VMs from my host17:53
sdakehow did you get into your vm from your mac?17:53
SiRiuS_NOVNC17:53
sdakeso not ssh17:53
SiRiuS_nope17:53
SiRiuS_ssh did not work17:54
sdakejust as a side do you know how  to access via ssh?17:54
SiRiuS_but I did not know that I had to set promiscuous mode to "allow-all" it's not in the kolla docs17:54
sdakenm we ca ntalk about that after ou finish :)17:54
sdakewhat is allow-all17:54
SiRiuS_I found out about promiscuous mode by reading through the OpenStack docs17:54
*** achanda has joined #kolla17:55
SiRiuS_VirtualBox filters L2 traffic if promiscuous mode is set to deny17:55
SiRiuS_and Neutron needs to pass L2 traffic to the external network gateway17:56
SiRiuS_but even with promiscuous mode set to allow-all, it still did not work fully17:56
SiRiuS_I managed to connect from my host to the nova VMs17:56
sdakewas this all in the context of vagrant or did you do it via the virtualbox ui first?17:57
SiRiuS_but the nova VMs could still not connect to the external network gateway, or to any other computer on the wireless network17:57
SiRiuS_first manually, without vagrant17:57
sdakewhere is the allow-all flag located?17:57
SiRiuS_Finally I read the VirtualBox docs17:58
SiRiuS_and the docs warned about Wi-Fi17:58
SiRiuS_where is it located in the docs?17:58
sdakethat would be rockin if they have already documented it - we could put that in our docs17:58
sdakelike how to get kolla running on virtualbox17:59
SiRiuS_I have virtualbox 5.0.1417:59
sdakeme too17:59
SiRiuS_if you go to help -> contents17:59
SiRiuS_it opens up the docs17:59
SiRiuS_on page 101 it says about Wi-Fi promiscuous mode and Wi-Fi18:00
SiRiuS_there is a Note18:00
sdakereading - continue on :)18:01
SiRiuS_practically it is impossible to send unmodified packages from neutron to the wifi router18:01
SiRiuS_so using the physical network as the neutron_external_network is not posible18:02
SiRiuS_luckily VirtualBox has the concept of a NAT-Network18:03
SiRiuS_which is practically a private network with its own virtual router18:03
sdakefamiliar with that18:03
sdakei want to be able to ssh into my vm :)18:04
sdakedo you know how to do that?18:04
sdakenot the nova vm, the vm i created originally18:04
SiRiuS_so we use the virtual router as the gateway for neutron18:04
SiRiuS_sdake, what VM exactly?18:05
sdakei created a centos7 vm to install koll ain18:05
sdakethe only way i can access this is via novnc which is annoying18:05
sdakei want to ssh into my vm18:05
sdakebut it is a nat network setup with wireless18:05
SiRiuS_what interfaces do you have connected to the vm ?18:05
sdakea nat interface18:06
SiRiuS_you should do it like Vagrant does it, it is a very good setup18:06
sdakeSiRiuS_ you arn't just teaching me, this is going in the docs18:06
sdakei dont understand vagrant too well unfortunately18:06
SiRiuS_you need a interface for internet connectivity18:06
sdakeso a host bridged network?18:07
SiRiuS_then another/multiple interfaces for internal stuff (management, are tunnel, storage)18:07
sdakeright18:07
SiRiuS_and another for the external network (which is usually the internet)18:07
sdakeso 4 nics then?18:08
SiRiuS_a minimum of 318:08
sdakelets say i'm not setting up kolla18:08
sdakei ust want  centos vm to ssh into18:08
SiRiuS_1 for internet connectivity inside the vm18:08
sdakewhat does the network type have to be?18:08
SiRiuS_2 for management, gre, storage18:08
SiRiuS_3 for external neutron network18:08
sdakeya got all that18:08
sdakejust want to focus on one thing at a time here :)18:09
SiRiuS_1 should be a simple NAT18:09
sdakehow do you ssh into it then?18:09
SiRiuS_set port forward18:09
sdakeis that a virtualbox option?18:09
SiRiuS_yup18:09
sdakesweet now we are getting somewhere18:09
sdakegot a pointer in the docs on that?18:09
SiRiuS_select "Attached to NAT"18:09
SiRiuS_then click "Advanced"18:09
SiRiuS_then click on "Port Forwarding18:10
SiRiuS_create a rule18:10
SiRiuS_Name: SSH18:10
SiRiuS_Protocol: TCP18:10
*** vhosakot has quit IRC18:10
SiRiuS_Host IP: 127.0.0.118:10
SiRiuS_Guest IP: leave blank18:11
SiRiuS_Guest Port: 2218:11
*** vhosakot has joined #kolla18:11
SiRiuS_now you can use ssh centos@localhost -p 222218:11
SiRiuS_I forgot Host Port: 222218:11
SiRiuS_you can use any port you like for the Host Port, just make sure is not used by other apps18:12
sdakeright18:12
SiRiuS_you should be able to ssh into centos with no problems18:13
SiRiuS_with just one interface attached to NAT18:13
SiRiuS_and port forwarding set up18:13
SiRiuS_let me know if it works18:14
sdakeis there a firewall to turn off?18:16
SiRiuS_also you should really give Vagrant a try, it's great, especially if you want to do multinode18:16
SiRiuS_I haven't touched any firewall settings18:16
sdakesweet it works!!18:17
sdakewow now i can do laptop dev of kolla18:17
sdakeSiRiuS_ you rock!18:17
SiRiuS_:D18:17
sdakeok on to the other network setups :)18:17
SiRiuS_glad to be of help18:17
sdakehow do i setup the other network interfaces?18:18
SiRiuS_so for NIC 218:18
SiRiuS_go  to virtualbox global preferences18:19
SiRiuS_there is a Network tab18:19
sdakeyes very fmailiar with all this stuff18:19
SiRiuS_go there, then click on Host-Only networks18:19
sdakejust could never get it to work18:19
SiRiuS_and create a default host-only network18:20
sdakevboxnet018:20
SiRiuS_with DHCP enabled18:20
SiRiuS_yes18:20
SiRiuS_while you are here18:20
SiRiuS_you can also set up a NAT Network18:21
SiRiuS_from the NAT Networks tab18:21
SiRiuS_give it a name and change the default CIDR18:21
SiRiuS_very important to change the default CIDR18:21
sdakewhat should it be set to?18:21
SiRiuS_because it conflicts with the NAT interface you created earlier18:21
sdake10.0.3?18:22
SiRiuS_the default NAT of the first interface is 10.0.2.0/2418:22
SiRiuS_you can change the CIDR to 192.168.x.x18:22
SiRiuS_like 192.168.2.0/2418:22
sdakeany port forwards?18:23
SiRiuS_and disable DHCP for the NAT Network,18:23
SiRiuS_when you want to access nova VMS18:23
sdakethis will probably require its own document18:23
sdakelots of steps - don't want to overload the QSG18:23
SiRiuS_to access nova VMs you need to set a port forward for each of the floating IPs18:24
SiRiuS_they will live inside the NAT Network18:24
SiRiuS_they will have IP like 192.168.2.12318:25
sdake192.168.1.150?18:25
sdakerather 2.150?18:25
SiRiuS_yes18:25
*** vishwanathj is now known as vishwanathj_away18:25
SiRiuS_depends on how you create them in neutron18:25
sdakehost and gueest port are both 22?18:25
sdakeinit-runocne starts floats at 15018:25
SiRiuS_nope18:25
SiRiuS_you have to use other ports18:25
SiRiuS_guys is 2218:26
SiRiuS_but host must be something other than 22 or 2222 set earlier18:26
sdakei see18:26
SiRiuS_i use 5502 for 192.168.2.10218:26
sdakeso host port should be 192.168.2.150 or guet ip?18:26
SiRiuS_5503 for 192.168.2.10318:26
SiRiuS_etc.18:26
SiRiuS_host port 5502..3..4..18:27
SiRiuS_host ip 127.0.0.118:27
SiRiuS_you need to set guest ip (unlike the NAT adapter earlier)18:27
SiRiuS_Guest IP: 192.168.2.15118:27
SiRiuS_(150 will be the neutron router if init-runonce is used)18:28
SiRiuS_Guest port: 2218:28
sdakeok i have two forwards setup18:28
sdakewhats next18:29
SiRiuS_click ok on everyhing18:29
SiRiuS_and run this18:29
SiRiuS_"VBoxManage list natnetworks"18:29
SiRiuS_inside the terminal18:29
SiRiuS_to see if the settings for the NAT Network are OK18:30
sdakehttp://paste.openstack.org/raw/491203/18:31
sdakedoesn't look quite right18:31
SiRiuS_Rule 1:tcp:[127.0.0.2]:5551:[192.168.2.151]:2218:31
SiRiuS_it should be 127.0.0.118:32
sdakeyup fixed18:32
sdakenow  whats next ;)18:32
* sdake likes learning :)18:32
* sdake likes teaching too18:32
SiRiuS_question: can we ssh into the neutron router?18:32
sdakeonly if you enter the namespace i think18:33
sdakevia ip netns exec18:33
sdakebut that is a WAG tbh18:33
SiRiuS_i saw you made a rule for 192.168.2.150, and that is the router in init-runonce18:33
SiRiuS_well next we need to create new interfaces18:34
SiRiuS_and connect them to the networks we created18:35
sdakehttp://paste.openstack.org/raw/491204/18:35
SiRiuS_ssh:tcp:[127.0.0.1]:55502:[192.168.2.152]:2218:35
SiRiuS_no good18:36
SiRiuS_that port is way to high18:36
sdakeoh typo18:36
sdakethanks18:36
SiRiuS_i recumbent 550x18:36
sdakeok fixed18:36
SiRiuS_if you are to have more floating IPs than 918:37
SiRiuS_55XX rather18:37
sdakewhat do you mean recumbent18:37
SiRiuS_*recommend18:38
sdakegot it18:39
sdakeok the final steps are which?18:39
sdake(btw this is all going in the docs as a separate doc - i'll add you as a coauthor since you figured all this stuff  out :)18:39
SiRiuS_next we need to create new interfaces18:39
SiRiuS_(thanks :) )18:39
sdakeadapter 2 should be setup how?18:40
SiRiuS_and connect them to the networks we created18:40
SiRiuS_for adapter 2 attach it to Host-only Adapter18:40
SiRiuS_with the name vboxnet018:40
sdakeand same deal for adapter 2 with nat network?18:41
SiRiuS_also I recommend setting the adapter type to virtio (for better performance)18:41
sdakerather adatper 318:41
SiRiuS_virtio for all the adapters18:41
sdakeall adapters paravirtualized18:41
SiRiuS_for adapter 3 we attach it to NAT Network18:42
SiRiuS_and also for the 3rd adapter set promiscuous mode to "Allow All"18:42
sdakeand fire up the vm?18:43
SiRiuS_yup18:43
SiRiuS_should be ok18:43
SiRiuS_you still need to change globals.yml with the new settings18:43
*** vhosakot has quit IRC18:43
SiRiuS_for deploying18:44
sdakeok I have an eth0 and eth1 an some other interfaces18:45
SiRiuS_you should have 3 interfaces inside CentOS18:45
SiRiuS_eth0 with an ip like 10.0.2.1518:45
SiRiuS_eth1 with an ip like 172.28.128.318:45
SiRiuS_and ethic with no ip18:46
SiRiuS_*eth218:46
sdakeeth2 didn't get a dhcp address it appears18:46
SiRiuS_yes18:46
sdakerather eth1 i mean18:46
SiRiuS_it's good that way18:46
SiRiuS_eth118:46
sdakeeth2 isn't configured it all18:46
SiRiuS_needs an ip18:46
SiRiuS_make sure DHCP is on in the Host-only Network18:47
SiRiuS_eth2 needs no IP because it will be use by neutron for the external interface18:47
sdakeright, i dont have an eth2, let me debug for a moment18:48
sdakewhat is this veth interfface18:48
sdakefound the problem , one of my devices was intel rather then virtio18:49
SiRiuS_weird18:49
SiRiuS_it should not cause any problems18:50
sdakehttp://paste.fedoraproject.org/343095/84998301/ -> http://paste.fedoraproject.org/343095/8499830118:50
sdakelooks right to me i think ;-)18:51
sdakeglobals.yml should look like what for the interfaces?18:51
SiRiuS_did you manually change the IPs for the Host-only network18:52
SiRiuS_because it is usually 172.28.128.0/2418:52
sdakei dont think i did, but i may have in  the past18:52
sdakei looked and it is 192.168.56.118:52
sdakefor th eadapter ip18:53
sdakedhcp server is 192.168.56.10018:53
SiRiuS_oh, it's ok, mine was 172.28.128.0/24 by default18:53
sdakedoes that part matter?18:54
sdakewhat is eth1 used for, storage network and friends?18:54
SiRiuS_as long as the networks don't cross, it's fine18:54
SiRiuS_management, storage, gre tunnel18:54
*** achanda has quit IRC18:58
sdakedeploying atm18:58
SiRiuS_kolla_internal_vip_address: "192.168.56.254"18:58
SiRiuS_network_interface: "eth1"18:59
SiRiuS_neutron_external_interface: "eth2"18:59
sdakebummer i deployed on eth0 :(18:59
SiRiuS_:D18:59
sdakeseems to be working tho ;)18:59
SiRiuS_well you will probably lose networking19:00
sdakeso SiRiuS_ how did you figure all that out?19:00
SiRiuS_because eth0 is used for ssh19:00
SiRiuS_and neutron will mess it up19:00
sdakenetwork_interface is the management network19:00
SiRiuS_yes19:01
sdakei dont think you really need 3 interfaces unless you want a storage network19:01
*** dwalsh has quit IRC19:01
SiRiuS_eth1 is used for managemen+gre tunnel+storage (all in one)19:02
SiRiuS_eth0 is used only to have internet access inside the VM and to ssh to it19:02
sdakegroan msg: The requested image does not exist: localhost:4000/kollaglue/centos-binary-neutron-metadata-agent:2.0.019:02
sdakeya lost networking :(19:03
SiRiuS_eth2 is used by neutron to provide access to the nova VMs and to give them internet access19:03
SiRiuS_told you so :)19:03
SiRiuS_just reboot the VM19:04
SiRiuS_change neutron_external_interface to ethic19:04
SiRiuS_*eth219:04
sdakeoh i lost networking because i was out of juice19:04
SiRiuS_eth0 should not be used for anything19:05
SiRiuS_it's just a utility network, not relevant to OpenStack19:05
sdakeok19:05
sdakerebuilding neutron - just a moment19:07
SiRiuS_sdake, On how did I figure all that out: trial and error :))19:09
sdakeSamYaple i'd like to see genpwd in prior to the branch if you can address the remaining comment19:09
sdakeSiRiuS_ what are your thoughts on pain points related to kolla19:10
SiRiuS_sdake, it took me a while to figure out how to get neutron working on virtualbox on wifi19:11
sdakeok well that will be fixed shortly19:11
SiRiuS_had no idea I need to set promiscuous mode to "allow-all"19:11
sdakebut thats a developer pain point - not really a real world use case ;)19:12
SiRiuS_for the neutron external interface19:12
sdakewhat about other pain points19:12
SiRiuS_well sdake, not just developer, because lots of people that will try kolla for the first time, will probably do it inside a VM19:12
SiRiuS_sdake, it is the easiest way to get started19:12
sdakeright19:12
sdakei think the easiest way to get started is bare metal ;)19:13
*** dims has joined #kolla19:13
sdakebut people may be afraid kolla makes a mess of their system19:13
sdakeused to devstack thinking19:13
SiRiuS_sdake, well bare metal is not always available, not everyone has a lab, or are willing to do i directly on their computer19:14
*** sbezverk has quit IRC19:14
sdakeok acknowledged19:14
sdakeanything else?19:14
*** dims_ has quit IRC19:14
SiRiuS_hmm, maybe ceph19:15
sdakecould you expand19:15
SiRiuS_I'm trying ceph as we speak19:15
sdakei hoep you read the docs first ;)19:15
sdakeI thought ceph was pretty straightforward the first time i deployed it19:15
SiRiuS_I had to delete my VM because it got messed up by ceph :)19:15
sdakeit edits /etc/fstab19:16
SiRiuS_well this is my first time, so maybe it will go smoothly the second time19:16
SiRiuS_ceph_erasure_profile is k=4 m=219:17
SiRiuS_so I need 6 discs right ?19:17
SiRiuS_and with caching enable i need 12 ?19:17
*** dims has quit IRC19:17
SiRiuS_6 with KOLLA_CEPH_OSD_BOOTSTRAP label19:17
SiRiuS_and 6 with KOLLA_CEPH_OSD_CACHE_BOOTSTRAP19:18
sdakehmm I haven't tried out erasure encoding since it was added19:18
sdakemy lab has been demolished for about 3 months unfortunately so i have only 1 macine availble to me bare metal atm19:18
sdakeSamYaple is your guy on ceph19:18
*** mbound has joined #kolla19:19
SiRiuS_I'm curious about setting it up on one VM (all-in-one deployment)19:21
sdakecurious as in the docs need improvement?19:21
SiRiuS_ceph osd pool set rbd size 1 (does 1 here signify the number of machines available for ceph ?)19:22
sdake1 = number of OSDs19:22
sdakeor disks available in essence19:22
sdakeiirc each disk gets an OSD19:22
SiRiuS_Oh, so i can set it to 6 if i have 6 disc in my VM19:23
SiRiuS_sdake, did you manage to deploy ?19:23
sdakei had to rebuild my neturon images19:23
sdakebut in process of deploying now19:23
sdakethen i'll ask for next steps ;)19:24
SiRiuS_ok19:24
SiRiuS_:D19:24
sdakeit takes about 5 minutes to deploy on my vm19:24
SamYaplehola19:26
SamYaplei got pinged19:26
SiRiuS_sdake, I got something like this from docker exec ceph_mon ceph -s -> http://paste.openstack.org/show/491210/19:26
sdakeyar - genpwd can you get that in prior to branch plz19:26
sdakeand SiRiuS_ is looking for ceph support on AIO with erasure encoding19:27
SamYaplesdake: just needs a rework to urandom, right?19:27
sdaketo SystemRandom I think19:27
*** achanda has joined #kolla19:27
sdakeshould be same code19:27
sdakejust different master class19:27
SamYapleos.urandom probably but yea19:27
SamYaplei can do that19:27
sdakeSystemRandom uses os.urandom19:27
sdakebut provides all the random() api calls19:27
sdakerather random class api calls19:27
SamYaplesdake: you seem to have a handle on this, do you want to take a crack at the patch?19:28
sdakeSystemrandom inherits from random but overrides random() api call19:28
sdakei could update yours if you liek19:28
sdakeits a 1 liner i think ;)19:28
SamYapleplease do19:28
SamYaplehavent looked at it at all today19:28
sdakeok - will be later tonight then19:28
sdakewe are going to be on the road shortly19:29
SamYapleSiRiuS_: if you have AIO earasure coding, youll likely have to rekick your deploy to make this work19:29
sdake2 hour drie19:29
SamYaplethe reason is you have to change the pool creation type19:29
SamYapleand you can't do that after the fact19:29
SamYapleSiRiuS_: just fyi though, you can't do erasure coding without a cache tier19:29
SamYapleSiRiuS_: ceph_erasure_profile: "k=4 m=2 ruleset-failure-domain=osd" <<< put that in your globals.yml19:30
sdakeok SiRiuS_ got a deploy19:30
sdakenow init-runonce needs modification I assume?19:30
SiRiuS_sdake, yes19:31
*** Jeffrey4l__ has joined #kolla19:32
SiRiuS_sdake, here is mine http://paste.openstack.org/show/491212/19:32
SiRiuS_the networking part19:32
SiRiuS_what is important is the part: start=192.168.2.101,end=192.168.2.200 public-net 192.168.2.0/24 --gateway 192.168.2.119:33
SiRiuS_the gateway is the NAT-Network gateway19:33
*** Jeffrey4l_ has quit IRC19:33
SiRiuS_SamYaple, the default is ceph_erasure_profile: "k=4 m=2 ruleset-failure-domain=host"19:35
SiRiuS_SamYaple, what does osd do if I change it to that?19:35
SamYapleSiRiuS_: you have a single host, youll never be able to get proper object placement without a MINIMUM of 3 hosts with that ruleset19:36
SamYapleor actually, 6 hosts19:36
SiRiuS_not even with 6 discs on my host?19:36
SamYaplechanging the failure domain to osd means you only need 6 osds19:36
SamYapleno because the failure domain is hosts, not osds19:36
SiRiuS_OOOOh19:36
SiRiuS_got it19:37
SiRiuS_:)19:37
SamYaplebut you have t orekick19:37
SamYapleyou cant change that live19:37
SiRiuS_I'm starting fresh19:37
SiRiuS_SamYaple, sdake, that should be in the docs19:38
SamYapleSS19:40
SamYapleSiRiuS_: maybe. Its a ceph thing and its in the ceph docs19:40
SamYapleerasure coding with cache tiers is some advanced ceph stuff19:40
SamYapleive made it easy in kolla, but its still advanced ceph stuff19:41
SamYaplewe cant document everything19:41
*** mbound has quit IRC19:41
SiRiuS_SamYaple, there is already some stuff about all-in-one, that why i suggested it19:41
SamYapleI rarely ever block docs patches S, you are free to write it19:42
SiRiuS_SamYaple, I looked over the ceph docs, and yes, it is quite advanced, thanks for making it easy :)19:42
SiRiuS_sdake, is everything working ?19:44
sdakeSiRiuS_ seems to be19:45
sdakelaunching a vm now19:45
SiRiuS_tell me if you have internet access inside the VM19:45
SiRiuS_and if you can ssh into it19:45
SiRiuS_from your host19:45
sdakegroan no valid host found19:46
SiRiuS_hmmm19:47
sdakewhy is that the only error that comes out of nova and is competely useless19:47
sdakeoh i'm ina vm19:47
sdakelet me reconfigure19:47
sdakehrm that didn't seem to work19:52
sdakeitworked when i demoed it however19:52
sdakemaybe the patch hasn't hit the repo19:52
SamYapleno valid host is likely a networking issue, or other resource issue19:53
SamYaplecheck scheduler and compute logs19:53
SamYapleit _will_ have the reason why19:53
sdakether ei no hypervisor in nova hypervisor-list19:53
SamYaplecheck rabbitmq19:53
sdakethere is no19:53
sdakewell i just blew away my deploy19:53
SiRiuS_sdake, I suggest you start fresh with a blank VM19:53
sdakedont need a blank vm19:53
sdakejust need to cleanup19:54
sdakethats the beuty of iolla - it makes no mess :)19:54
SamYapleit can make an uncleanup-able mess if you dont properly removed openvswitch between runs19:54
SamYaplejust fyi19:54
SamYaplefor bridge in $(docker exec -it openvswitch_vswitchd ovs-vsctl list-br); do for port in $(docker exec -it openvswitch_vswitchd ovs-vsctl list-ports ${bridge}); do docker exec -it openvswitch_vswitchd ovs-vsctl del-port ${bridge} ${port}; done; docker exec -it openvswitch_vswitchd ovs-vsctl del-br ${bridge}; done; ip a | awk -F'\''[: ]'\'' '\''/^[0-9]/ {print $3}'\'' | awk '\''/^q|^tap/'\'' | xargs -n1 ip l d dev; rmmod vport_vxlan openvswitch19:55
SamYapleyoure welcome19:55
SamYapleor even;19:56
SamYaplefor bridge in $(docker exec -it openvswitch_vswitchd ovs-vsctl list-br); do for port in $(docker exec -it openvswitch_vswitchd ovs-vsctl list-ports ${bridge}); do docker exec -it openvswitch_vswitchd ovs-vsctl del-port ${bridge} ${port}; done; docker exec -it openvswitch_vswitchd ovs-vsctl del-br ${bridge}; done; ip a | awk -F'[: ]' '/^[0-9]/ {print $3}' | awk '/^q|^tap/' | xargs -n1 ip l d dev; rmmod vport_vxlan openvswitch19:56
sdakei rebooted19:56
sdakei'd think a reboot would get the job done19:56
SamYaplesdake: if you want to cleanup inbteween runs, use that command above _BEFORE_ you tear down contaienrs19:56
SamYapleit will19:57
SamYaplebut you ahve to reboot19:57
sdakealready deploying :)19:57
SiRiuS_sdake, I recommend enabling shared folders in VirtualBox19:57
sdakethis time with virt driver of qemu19:57
SiRiuS_and put your docker registry there19:57
sdakeSiRiuS_ are ou usin qemu driver?19:57
SiRiuS_yes, that is the only way19:58
SiRiuS_I forgot about that :)19:58
SiRiuS_you HAVE to use qemu19:58
sdake /etc/kolla/config/nova/nova-compute.conf19:58
SiRiuS_in there, yes19:58
sdakeas already documented19:58
SiRiuS_anyway, try to use shared folders in VIrtualBox, and put your docker images in there19:59
SiRiuS_that way, you can completely destroy a vm, without having to rebuild images20:00
SiRiuS_all of this stuff is done automatically by Vagrant, by the way :)20:01
sdake2016-03-20 20:01:55.177 1 ERROR nova.virt.libvirt.host [req-9a4357b7-037e-4ac2-9e8c-8505d06f606c - - - - -] Connection to libvirt failed: unable to connect to server at '192.168.56.101:16509': Connection refused20:05
sdakefantastic libvirt is in a restart loop20:06
SiRiuS_sdake, can i see the globals.yml and init-runonce ?20:07
sdakelet me debug for  a bit20:07
sdake[libvirt]20:09
sdakeconnection_uri = "qemu+tcp://192.168.56.101/system"20:09
sdakevirt_type = qemu20:09
SamYaplesdake: dont set connection_uri20:09
sdakeSiRiuS_ what does your file /etc/kolla/nova/nova.conf contain20:09
SamYapleit should get set properly20:09
sdakei am not setting it20:09
SamYapleoh ok20:09
sdakethat is what is in /etc/kolla/nova/nova.conf20:10
sdakeSamYaple shouldn't that be set to the haproxy ip?20:10
*** vhosakot has joined #kolla20:10
SamYaplesdake: no20:11
sdakeSiRiuS_ http://paste.fedoraproject.org/343126/85047251/ -> http://paste.fedoraproject.org/343126/8504725120:12
sdakeSamYaple so its a SPOF?20:12
sdakewell ya i guess libvirt would be20:12
sdakenm20:12
SamYaplesdake: is a compute node a SPOF?20:12
SiRiuS_[libvirt]20:12
SamYapleyea20:12
SiRiuS_connection_uri = "qemu+tcp://172.28.128.3/system"20:12
SiRiuS_virt_type = qemu20:12
SiRiuS_looks good sdake20:13
sdakeya well libvirt is in a restart loop - no idea why20:13
sdakebecause we have no libvirt logging20:13
* sdake groans20:13
sdakeis .3 yoru eth1?20:14
SiRiuS_yup20:14
sdakei wonder why my system is choosing 10120:14
SiRiuS_you had Host-Networking set up with DHCP giving address from 101 up20:15
SiRiuS_*Host-only Networking20:15
SiRiuS_mine is the default20:15
SamYaplesdake: we have libvirt logging20:15
sdakemy ip address is not 10120:15
sdakehttp://paste.fedoraproject.org/343129/58504960/ -> http://paste.fedoraproject.org/343129/5850496020:16
SiRiuS_ inet 192.168.56.101  netmask 255.255.255.0  broadcast 192.168.56.25520:16
SiRiuS_it says 10120:16
sdakeoh right20:17
sdakebrain not woring well today it appears ;)20:17
SamYaplebtw sdake if you have a domain and want a cert for free https://hub.docker.com/r/samyaple/letsencrypt/20:17
sdakeSamYaple where are the libvirt logs located20:18
sdakeI dont' see anything in the heka container20:18
SamYaplein libvirt container20:18
SamYaplewe havent moved them yet20:18
sdakeright which is in a restart loop20:18
SiRiuS_sdake, by the way, because eth1 is a Host-only adapter, you can connect directly to IPs in the network 192.168.56.0/2420:18
SiRiuS_try and ping 192.168.56.101 from your host20:18
SamYapleget to them through /var/lib/docker sdake20:18
sdakegood idea sam20:19
sdake2016-03-20 19:57:51.454+0000: 10137: error : virPidFileAcquirePath:422 : Failed to acquire pid file '/var/run/libvirtd.pid': Resource temporarily unavailable20:19
sdakewell I wont even metnion what just happened20:20
*** achanda has quit IRC20:21
SiRiuS_sdake, did you run out of free space ? :)20:22
SamYaplesdake: is libvirtd runing in the vm already20:22
sdakeno i had libvirtd runnnig in my vm20:22
SamYaplelol20:22
sdakeyah funny huh :)20:22
SamYapleprechecks!20:22
sdakei typically turn that off first step ;)20:23
sdakebut i mad ehtis vm several weeks ago20:23
SiRiuS_sdake, :D told you to start fresh :)20:23
*** achanda has joined #kolla20:24
sdake7 minutes until deploy :)20:24
sdakei'm so excited tho that going to be able to develop on a laptop20:25
SamYaplei got an accout with dreamhost today20:25
sdakewho the hell knows when my lab will be back in order20:25
SamYapleim goign to start doing the blog thing20:25
sdakei use wordpress.com20:25
sdakebut they are all likely teh same20:25
SamYaplethis is using wordpress too20:25
SamYaplebut its more than that20:25
SamYaplei got it because ayoung recommended it20:26
sdakeya blogging helpful to share your thoughts with people and create a record20:26
ayoungSamYaple, you on Twitter?20:26
SamYapleayoung: no20:26
SamYapleits more "i dont rememebr how i fixed that a year ago"20:26
ayoungSamYaple, Ok, was going to point people at it, to include the Dreamhost folks20:26
sdaketwitter is powerful if you know how to use it20:26
ayoungsdake, I think he meant the blog...20:27
SamYapleayoung: i dont know how to twitter or facebook or social media20:27
ayoungsdake, I was watching some of your old Heat presentations...20:27
sdakemy wife is a  twitter pro20:27
sdakeayoung learn anything old ? :)20:27
ayoungsdake, heh...well, one question for you20:27
sdakesure20:27
ayounghow do I update a stack after deploy?20:27
ayounglike, say I want to ship a new policy file out to all the keystone servers?20:28
ayoungdoes Heat support that?20:28
sdakeno idea - update came after I left the project20:28
sdakeor a working implementation atlaets20:28
sdakeshardy is your go to on that one20:28
ayoungYeah, I'll bug him...but I have the same question on Kolla20:28
SamYapleayoung: i saw that ML thread20:28
sdakeayoung you use reconfigure20:29
ayoungWhat happens if we want to publish a newe Policy file?  Does that mean we need new containers?20:29
SamYapleayoung: so basically we dont have a way to tweak policy file right now, but the infrastructure is there to do so20:29
sdakepolicy files are not yet configurable20:29
SamYapleayoung: no, just container restart WHEN we configure them20:29
ayoungCool.  And how about the notification when something changes in Keystone?20:29
SamYapleayoung: come newton release all of the paste/policy type files will be configurable, and then its just a container restart to do what you want20:29
SamYaplethat is a bit trickier20:30
sdakeayoung i don't understand your last question20:30
SamYaplebecause we dont have a deploy service always running it would be up to the deployer to run a reconfigure task20:30
ayoungsdake, ok, flow is somethning like this20:30
ayoungI upload a new policy file to Keystone...no change yet20:30
SamYapleso technically keystone could trigger this task, but thats on the deploy to setup20:30
ayoungthen, I associate that new policy file with glance.  Now I want to update all the glance servers with the new policy file20:31
ayoungI can have Keystone send out a notification "policy for glance has changed"20:31
ayoungbut who would listen for it (besides ceilometer that listens to everythigng) and make it happen?20:31
sdakeayoung I see - we dont have a daemon listening for oslo notifications at this time20:32
sdakebut i'm sure its pretty easy to write - the issue is having a daemon running20:32
ayoungdake, so, I think that we are going to end up with Kolla as the Overcloud layer for Tripleo. I might push for it being the responsibility of the undercloud Heat server to do this20:32
sdakewe haven't quite figured out  the use cases for all that yet20:32
ayoungsdake, Kolla has no daemon at all, right?20:33
sdakeayoung no daemon20:33
sdakepeople have tried to add one in the past20:33
ayoungsdake, So we could write a simple Keystone listener that does that.  There have been requests for something like that for autoprovisioning as well, but it sounds like Mistral is supposed to be the workflow kickoff thing...20:33
sdakebut i think they didn't have a clear set of use cases of what the rest api should look like and how it should be secured20:33
*** achanda has quit IRC20:34
sdakethat creates a bit of a circular dep on keystone triggering kolla-ansible20:34
*** dwalsh has joined #kolla20:34
sdakethe right place for it is in kolla-ansible via a rest api20:35
sdakebut to date, no rest api exists20:35
ayoungsdake, Well, Kolla could drive the initial setup, and then respond to notifications later20:35
ayoungI don;t think is should20:35
sdakerather not kolla-ansible20:35
sdakebut kolla i mean20:35
ayoungWe should push RH to Open up Tower and let that be the Ansible daemon20:35
ayoungI am sure that is coming20:35
sdakeI think its wierd to have keystone run kolla commands :)20:35
ayoungHeh20:35
sdakeayoung i've pushed red hat around enough already :)20:36
ayoungmore like Kolla respond to Keystone notifications, but yeah20:36
sdakeya that would be good, but then we are back at the kolla needs a a rest api discussion20:36
sdakehere is the thing about a rest api20:36
ayoungsdake, I would be really surprised if an Open version of Tower was not announced at the RH summit20:36
sdakewe dont want to make one unless there is a legitimate need20:36
sdakei think there is a need20:37
sdakeothers dont20:37
sdakeits up in the air atm20:37
ayoungAgreed, and I don;t think we need one. But something listening to Keystone notifications that can then make ansible calls ... sounds like Heat business to me20:37
sdakeyou can't make ansible calls from python code20:37
ayoungOf couirse I can.  I do it all the time20:37
sdakeit viralizes the code base with gplv320:37
sdakeit has to be run as a subprocess20:37
ayoungNah,  popen is just fine20:38
ayoungagreed20:38
sdakeright popen is accceptable20:38
sdakei thought you meant import ansible20:38
ayoungI personally have no problem with gplv3. This would be a stand alone listener.20:38
ayoungEIther way would work20:38
ayoungOK, so the piece I need Kolla to tell me is: if I trigger a policy update, here is what you do...20:39
ayoungand then I get Heat to kick that off in the Tripleo-Kolla future I envision20:40
sdakei suppose we would send a notification of either policy updates were successful or failed20:42
sdakea reconfigure can take about 1-2 minutes on bare metal20:42
sdakereal1m6.037s20:42
sdakereconfigure in a vm20:42
sdakeso its not immediate - have to wait for a response20:42
sdakei wrote the oslo port for heat20:43
SamYaplesdake: yaple.net20:43
sdakeso i understand how notifications work20:43
SamYaplefirst post ^20:43
*** vhosakot has quit IRC20:44
sdakeSamYaple nice recommend deleting your hello world post ;)20:44
SamYaplei was thinking of leaving it20:44
SamYaplei specifcally talk about the way of ceph bootstraping with external journals fyi20:45
sdakethe post before that one20:46
sdakeyou have two posts in your blog20:46
SamYaplei know20:46
SamYaplei was thinking of leaving the default one20:46
sdakei see20:46
sdakewell whatever works :)20:46
SiRiuS_sdake, any update on the status of the deployment ?20:47
SamYapleeven after typing up the steps, its crazy how few there are20:47
*** TheBall has joined #kolla20:48
ayoungNah, drop the Hello world post SamYaple it makes you look like a Muggle20:48
SiRiuS_SamYaple, I think I fount a bug in globals.yml related to ceph20:49
SiRiuS_SamYaple, ceph_use_cache: "yes" instead of ceph_enable_cache: "yes"20:50
SamYapleno post is going to tell me what to do ayoung!20:50
SamYapleSiRiuS_: thats patched already20:50
SiRiuS_oh20:50
SiRiuS_ok, I haven't updated20:50
SamYaplefine ayoung youve twisted my arm20:50
*** The_Ball has quit IRC20:51
ayoungSamYaple, So I have been using a plugin for code that I like20:53
ayoungyou are going to want soemthing for code samples20:53
SamYaplethat is a plugin20:53
SamYaplesyntaxhighlighter20:53
SamYapleis hte name of it20:53
ayoungSyntax Highlighter and Code Prettifier Plugin for WordPress20:53
ayoungyep20:53
SamYapleyea im using that there20:54
* SamYaple is not a web guy at all.....20:54
ayoungSam so, if you are using that one, the way I do it is (in text view)  <pre class="brush:bash">  stuff </pre>20:55
ayoungYour might be better integrated...20:55
SamYaplei did something. i did <pre>[bash]bash code[/bash]</pre>20:56
SamYapleit has other languages too20:56
ayoungNice20:56
SamYaplethe [bash] block i wrote out and it just translates it it appears20:56
*** sdake has quit IRC20:56
ayoungSamYaple, the one gotcha I have found is that I often need to replace < and >  inside the code with &lt; and &gt;20:57
SamYapleayoung: let me test that20:57
SamYaplewell no ayoung i have => in the ansible blocks20:58
SamYapleit rendered fine20:58
ayoungSamYaple, Yeah, it is not an all the time thing.  It is more the <  that messes up the rendering, but you will catch it in preview20:58
SamYaplegotcha20:58
SamYaplewill watch out for it20:59
*** sdake has joined #kolla21:12
sdakeSiRiuS_ my instance build timed out21:12
sdakeayoung maybe I don't get it, but why bother having an under and overcloud21:14
sdakewhy not just one cloud - two layers seems more complex and prone to failure21:14
ayoungsdake, I was not part of that decision21:14
ayoungsdake, however, I think the idea is that you need something to manage the cluster of hardware, separate from the services running on it. And Ironic was that.21:15
SamYapleayoung: excuses excuses21:15
SiRiuS_sdake, what do you mean by instance build? nova VMs ?21:15
sdakei think ironic can be made to run in one cloud21:15
ayoungSince the services running the undercloud are not exposed to end users, there is less pressure to keep them up-to-date security wise...you can fix them at a different cycle21:15
sdakeSiRiuS_ yes21:16
ayoungsdake, what I would love to see is Tripleo supporting multiple overclouds on a single undercloud. Then the whole thing would make a lot more sense to me21:16
SiRiuS_sdake, weird, I never had trouble with nova21:16
SamYapleayoung: agree there21:17
SiRiuS_can you please upload globals.yml and init-runonce21:17
ayoungAnd I would love the OverCloud to be Kolla.  So, I'm going to be pushing for that21:17
sdakei like ironic, and I think it can be made to work in a hybrid mode where it does bare metal deploy for cloud operators21:18
SiRiuS_sdake, did you deploy binary ?21:19
sdakeyup21:19
SiRiuS_sdake, does horizon work?21:20
SiRiuS_on 192.168.56.25421:21
sdakenot from my mac21:21
SiRiuS_well, it should work21:21
SiRiuS_can you ping 192.168.56.101 ?21:22
sdakeSiRiuS_ it works within the vm21:22
sdakebut not outside the vm21:22
SiRiuS_it should work from the mac as well21:22
SiRiuS_that's the idea behind Host-only networking21:22
sdakeconnectoin refused21:23
SiRiuS_there is something wrong with eth121:23
sdakepossibly a firwall issue21:24
sdaketurned off firewall - no go21:25
sdakei see the vboxnet0 on the host21:26
SiRiuS_i can ping both the VM ip on eth1 and the VIP21:26
sdakevboxnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 150021:27
sdake        ether 0a:00:27:00:00:0021:27
sdake        inet 192.168.56.1 netmask 0xffffff00 broadcast 192.168.56.25521:27
sdakewell with that i can make a good start at the docs for setting up virtualbox21:28
sdakesans vagrant21:28
SiRiuS_it looks good21:28
SiRiuS_can you ping 192.168.56.121:29
SiRiuS_?21:29
sdakeyes and also 10121:29
SiRiuS_can you ping 192.168.56.25421:29
SiRiuS_?21:29
SiRiuS_from the mac21:30
sdakei can also ssh into 10121:30
sdakeso I dont think the first interface is necessary at all21:30
sdakeonly need the two interfaces21:30
sdakeeth1 and eth221:30
SiRiuS_sdake, you would be wrong :)21:30
sdakepossible could you expand21:30
SiRiuS_without the first one, you would not have internet inside the VM21:30
SiRiuS_Host-only network is just that21:31
sdakeno route out huh21:31
sdakegot it21:31
SiRiuS_"host-only"21:31
SiRiuS_can you ping 192.168.56.254 ?21:32
sdake.254 responds to ping21:32
sdake253 does not respond to pign21:32
SiRiuS_well should it? I don't think you should have anything on 25321:33
*** dwalsh has quit IRC21:33
sdakenope it shouldn't21:33
sdakei didn't expect 254 to respond to ping either21:33
sdakesince 254 is a vip to haproxy21:33
SiRiuS_254 attaches to eth1: inet 172.28.128.254/32 scope global eth121:35
SiRiuS_run ip addr21:35
SiRiuS_inside the vm21:35
SiRiuS_try to connect to horizon21:35
SiRiuS_192.168.56.254  in your web browser21:36
SiRiuS_it should work21:36
sdakeyes i did try that21:39
sdakeit works inside vm but not on mac host21:39
sdakebut i turnedoff the firwall, i am going to reboot the vm21:40
*** rstarmer has joined #kolla21:40
sdakeand try a fresh deploy21:40
SiRiuS_so you can ping 192.168.56.254 from the mac but can't connect to horizon on 192.168.56.254 from the mac21:40
SiRiuS_that is not normal21:40
sdakeright21:42
sdakewhich webbrowser do you use?21:42
SiRiuS_safari21:42
sdakei am thinking its the firewall - mcffree endpoint protection21:42
SiRiuS_disable all firewall on hot & VM21:43
SiRiuS_*host21:43
sdakeya was the vm firewall21:44
sdakedoing fresh deploy21:44
sdakepossibly reason my vm was stuck not starting as well21:44
SiRiuS_probalby21:45
sdakewhen i worked on corosync firewall was problem #1, #2, and #3 in most cases ;)21:46
SiRiuS_remember to disable it not just stop it, because you will probably forget to stop it again after a reboot21:47
SiRiuS_:)21:47
openstackgerritRyan Hallisey proposed openstack/kolla: Improve openstack-base image for centos binary  https://review.openstack.org/29504821:47
sdakeya i alread ydid that21:47
sdakeSiRiuS_ hae ou tried other deployment tools21:50
SiRiuS_sdake, nope, I looked at the options, and basically kolla was the only one for me, because of docker21:51
sdakecould you expand on that21:52
SiRiuS_the idea of deploying and then cleaning up sound like the best thing for me21:52
SiRiuS_not dirtying the OS is what I like21:53
sdakeyup horizon is working from the mac now21:53
sdake\o/21:53
SiRiuS_see if you can crete a nova instance21:53
sdake2016-03-20 21:39:13.749 1 ERROR nova.compute.manager [req-77362aa1-788a-489d-b44d-1d703e6c12d4 6ad2a7a93a664144a43f297262012ac6 33cec6588a1145bb9bd2b66fb8dbf522 - - -] Instance failed network setup after 1 attempt(s)21:57
sdakewhich nova boot operation are you using?21:57
SiRiuS_directly from horizon21:57
sdakenever done that21:58
sdakei dont see a create instance operation21:59
sdakenm got it21:59
*** rstarmer has quit IRC22:01
sdakenope doesn't work from horizon22:02
*** asalkeld has joined #kolla22:05
SiRiuS_with the same error ?22:05
sdakeno it doens't get that far22:06
sdakehorizon errors out immediately22:06
sdakei am going to try booting with heat22:06
sdakesince sam fixed that and tested it boots there22:06
SiRiuS_what does it say ?22:06
sdakenot sure i hae to redeploy22:06
sdakesince its a bootstrap chane22:07
*** achanda has joined #kolla22:07
sdakebbin10mins22:08
SamYaplerhallisey: i commented on your centos-binary openstack-base patch with my oneliner (changed for centos)22:11
*** vhosakot has joined #kolla22:11
sdakeback22:17
sdakeSamYaple i just tested your patch and stack create fails22:19
sdake| steak         | 0c26741b-2eaa-4eef-98ff-4faef523fb17 | Authorization failed.  | CREATE_FAILED   | 2016-03-20T22:02:00 |22:20
sdakehttp://paste.fedoraproject.org/343157/51253414/ -> http://paste.fedoraproject.org/343157/5125341422:22
sdakeSamYaple endpoints ^^22:22
sdakenova boot with a --nic net-id= also fails to complete22:26
*** akwasnie has joined #kolla22:26
*** akwasnie has left #kolla22:27
SamYaplesdake: you should probably look into that second one22:28
SamYaplebut the auth failed seems liek a trust thing22:28
sdake[sdake@localhost heat]$ nova boot --image cirros --flavor m1.small --nic net-id=c01aea0d-9c5a-4d0e-8220-2c6ae35fa29e steak22:28
sdakeERROR (ConnectFailure): Unable to establish connection to http://192.168.56.254:8774/v2/4d6e71ec5325427ebc9e939bf8959106/servers22:28
SamYapleim not using trusts22:28
sdakei should look into which second one22:28
SamYaple?22:29
sdakethe nova boot fails or heat fails22:29
SamYaplenova boot22:29
sdakeya no idea why that happens22:29
sdakei can telnet to the server just fine22:29
*** Marga_ has quit IRC22:30
sdakenova boot without --nic net-id ends up in failed state as well22:31
SamYaplewait without?22:32
SamYaplenet-id is requied22:32
SamYapleoh nvm22:32
SamYaplereread22:32
sdakenova will boot without --nic netid=22:33
sdakebut it fails to boot22:33
sdakecan't allocate networks22:33
SamYapleright22:33
sdakealthough you have to dig through the logs to find the one line that says that22:34
sdakei dont understand why nova can't fix these basic things22:34
sdakeyou think if tht is what the problem was it woud be reported by nova show22:35
sdakebut nova boot --nic net-id= completely bombs as well22:35
sdakehow do you boot a vm SamYaple ?22:35
SamYaplehere is the command i normally use22:36
SamYapleopenstack server create --image $(openstack image list | awk '/trusty/ {print $2;exit}') --nic net-id=$(openstack network list | awk '/internal/ {print $2;exit}') --flavor 4 --key-name sam sy-test0122:37
sdakehttp://paste.fedoraproject.org/343161/14585134/ -> http://paste.fedoraproject.org/343161/1458513422:37
sdake[sdake@localhost heat]$ openstack server create --image cirros --nic net-id=c01aea0d-9c5a-4d0e-8220-2c6ae35fa29e --flavor m1.small steak22:39
sdakelooks like same story22:39
sdakecan you paste your openrc?22:39
SamYaplesdake im not sure your keystone is working22:40
SamYaplecan you `openstack token issue`22:40
sdakethat returns something22:40
sdakehttp://paste.fedoraproject.org/343163/85136821/ -> http://paste.fedoraproject.org/343163/8513682122:41
SamYapledo you have a compute endpoint?22:41
*** rhallisey has quit IRC22:41
sdakehttp://paste.fedoraproject.org/343164/13716145/ -> http://paste.fedoraproject.org/343164/1371614522:42
sdakeyes admin public and internal22:42
sdakei can also telnet to port 877422:42
sdakeand get a reponse back in if i type nonsense22:42
SamYaplenova service-list is all good i assume?22:43
SamYapleyou did the normal trobuleshooting22:43
SamYaple8774 telnet would be from haproxy22:43
sdakeservice list looks good22:43
sdakehttp://paste.fedoraproject.org/343165/51384214/ -> http://paste.fedoraproject.org/343165/5138421422:44
sdakethis worked for me last week22:44
sdakebut i was on bare metal22:44
sdakei am in a vm atm22:44
*** rhallisey has joined #kolla22:44
SamYapleidk man22:44
*** Marga_ has joined #kolla22:46
sdakehttp://paste.fedoraproject.org/343166/14585140/ -> http://paste.fedoraproject.org/343166/1458514022:47
sdakecheck out the end of that log22:47
SamYapleyup seems like a rabbitmq issue22:50
SamYapleremember the ip has to resolve to the hostname22:50
SamYaplenot localhost22:51
SamYapleif you want to merge this patch it would have caught it..... https://review.openstack.org/#/c/287969/22:51
*** Marga_ has quit IRC22:51
sdakeso /etc/hosts should resolve localhost to which ip then 254?22:51
SamYaplei dont know your schema22:52
sdake254 is my floating ip22:52
sdake56.101 is my management network22:52
SamYapleno, your hostname should never resolve to your vip22:52
sdakebbi10min22:57
sdakewith a properly setup /etc/hosts file22:57
sdakeand hostname22:58
openstackgerritAndrei-Lucian Șerb proposed openstack/kolla: Attach external NIC to a NAT-Network if on Wi-Fi  https://review.openstack.org/29434023:03
*** sdake has quit IRC23:09
openstackgerritSam Yaple proposed openstack/kolla: Update to latest stable versions  https://review.openstack.org/29505023:09
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505123:09
openstackgerritAndrei-Lucian Șerb proposed openstack/kolla: Attach external NIC to a NAT-Network if on Wi-Fi  https://review.openstack.org/29434023:10
*** sdake has joined #kolla23:11
SiRiuS_sdake, I did the modification to the patch23:13
sdakeSiRiuS_ cool thanks :)23:13
*** Marga_ has joined #kolla23:13
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505223:17
*** vhosakot has quit IRC23:17
*** Marga_ has quit IRC23:20
*** Marga_ has joined #kolla23:21
sdakeSamYaple this is with my hostname set23:22
sdakehttp://paste.fedoraproject.org/343174/45851615/ -> http://paste.fedoraproject.org/343174/4585161523:22
sdakerabbitmq doesn't have much interesting in the way of logs23:22
openstackgerritSam Yaple proposed openstack/kolla: Update version pins for mitaka  https://review.openstack.org/29505223:23
openstackgerritSam Yaple proposed openstack/kolla: Split versions onto new line  https://review.openstack.org/29505123:23
*** alisonh has quit IRC23:38
sdakeSiRiuS_ which distro are you on23:41
*** dave-mcc_ has joined #kolla23:41
SiRiuS_centos 723:41
sdakesource or binary ?23:41
SiRiuS_source23:41
*** alisonh has joined #kolla23:42
sdakewhat is in your /etc/hosts file23:43
*** dave-mccowan has joined #kolla23:43
SiRiuS_sdake, http://paste.openstack.org/show/491216/23:44
sdakewhat about when you did a manual setup?23:45
sdakedo you recall?23:45
*** vhosakot has joined #kolla23:45
sdakecan you paste our ifconfig?23:45
sdakefpaste /etc/ifconfig23:45
sdakerather23:45
sdakeifconfig | fpaste23:45
*** dave-mcc_ has quit IRC23:46
SiRiuS_sdake, http://paste.fedoraproject.org/343179/14585180/ -> http://paste.fedoraproject.org/343179/1458518023:53
sdakeSiRiuS_ can you paste your openrc plz23:58
SiRiuS_sdake, I didn't touch /etc/hosts when I did manual setup, iirc23:58
SiRiuS_http://paste.fedoraproject.org/343181/58518357/ -> http://paste.fedoraproject.org/343181/5851835723:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!