Wednesday, 2015-12-02

*** gtt116_ has joined #openstack-ansible00:05
*** gtt116__ has quit IRC00:08
bgmccollumi see in Kilo the lxc package isn't getting pinned, but it is in Juno. Should this pin be removed as part of the upgrade?00:24
*** mss_ has quit IRC00:24
*** markvoelker has quit IRC00:45
*** sdake has quit IRC00:53
*** ctina_ has joined #openstack-ansible00:55
*** eil397 has quit IRC00:59
*** ctina_ has quit IRC01:00
cloudnullbgmccollum: yes/no I believe the pin is in group_vars for Juno which will be removed when you change the playbooks01:00
cloudnullif the pin is in user_vars then its up to the deployer if it should be removed01:01
cloudnullbgmccollum: RE: yes to adding openrc_ in front of the various variables.01:06
openstackgerritKevin Carter proposed openstack/openstack-ansible: Ability to define the mtu globally (for all the container networks)
openstackgerritKevin Carter proposed openstack/openstack-ansible: Use Ansible AIO playbook for
openstackgerritKevin Carter proposed openstack/openstack-ansible: Added to ability to set mtu for lxcbr0
openstackgerritKevin Carter proposed openstack/openstack-ansible: Added in keystone reserved port
jamielennoxcloudnull: so is there any intention to publish some of these modules individually?01:16
openstackgerritKevin Carter proposed openstack/openstack-ansible: Update Master SHA's - 01.12.2015
jamielennoxi was trying to use os_keystone for some development stuff and isolate it from the whole openstack-ansible deployment01:17
cloudnulljamielennox: absolutly.01:17
cloudnullwe're in the process of doing that right now01:17
cloudnullso far I've done the host only roles.01:17
jamielennoxbut things are very intertwined, turned out to be impossible without also including rabbit, memcache and something failed in there that i gave up on01:17
cloudnullthats the spec for it01:18
cloudnullso we're breaking it all out as independent roles and then creating gate tests for the role as a stand alone role01:18
jamielennoxcloudnull: are you expecting to keep the role dependencies?01:19
jamielennox(hoping for no)01:19
cloudnullwhich ones? in the meta?01:20
cloudnullor all of the galera, rabbit bits and such ?01:20
*** mss has joined #openstack-ansible01:20
jamielennoxcloudnull: let me find some examples01:21
cloudnullRE: these deps ?01:21
jamielennoxcloudnull: some of those01:23
cloudnullyes and no. things like pip_lock_down, no -- things like memcahed when the token provider is a memcached driver, probably.01:24
jamielennoxthere's a few things like groups['rabbitmq_all'] which just fail if you don't set things up specifically01:24
cloudnullbut if there's things that you'd like to see in that role as we break it out im game to work that in01:24
cloudnullah yes those will be out.01:25
*** alkari has quit IRC01:25
cloudnullthe plan so far is to break out the "crud" ops from the os_* roles01:25
cloudnulllikely in its own role or in an optional running section.01:26
jamielennoxideally even for the memcache drivers i'd kind of like to just provide keystone_memcache as a va01:26
jamielennoxi'm not a good enough ansibler but i would expect you could integrate them via defaults if available01:26
cloudnullso theres just a connection string and assume memcached is simply installed at that local ?01:27
cloudnullas an example01:27
jamielennoxwell some way that i can orchestate memcache and just feed the appropriate vars to keystone01:28
jamielennoxsame with galera_client i guess01:28
jamielennoxi want to deal with setting up mysql/galera/postgres whatever as a seperate role01:29
cloudnullfor sure.01:29
cloudnullthat will be done01:29
bgmccollumcloudnull: re upgrade pins...10.1.17 has lxc pinned to 1.0.7. trying to upgrade to 11.0.4 barfs because lxc 1.0.8 is required, but nothing cleared out the lxc pin (its appears unbound in kilo)...juno head has updated lxc pin to
bgmccollumactually, i think the problem is the upgrade scripts runs lxc-hosts-setup.yml before re-templating the pins...01:30
kyssehmh. I should install ssd disks to those infra nodes, I hate slow disks )-;01:30
jamielennoxcloudnull: ok, sweet - just looks like i was trying to use individually roles a bit prematurely01:30
cloudnullyea, sorry about that01:30
cloudnullitll be there soon-ish01:30
* cloudnull goes to find more hours in the day :)01:31
jamielennoxnah, i had to do a bunch of hacking and forking just to get the setup to run01:31
cloudnulljamielennox: i hope to have it done for most of the OS_* roles by the end of the month. my next batch will be all of the galera, rabbit, memcached, etc .01:32
jamielennoxwill simplify some of my test deployments a lot to have it01:32
cloudnullbut those should go quic,k01:32
cloudnullbgmccollum: is that the new or the old upgrade script ?01:33
jamielennoxcan't believe there aren't standard modules already for things like rabbit, memcache, galera01:33
cloudnullthere are "some" but from what we found they're a bit suspect01:33
bgmccollumcloudnull: old :/01:34
cloudnulldebopts has some nice roles but they wont work cross platfor which is something that we're going toward soonish also01:34
cloudnullbgmccollum: that too01:34
bgmccollumcloudnull: i think i need to just set apt_pinned_packages: [] in the user vars...01:34
bgmccollumso its "defined", but empty...should clean out the pins01:34
cloudnullbgmccollum: is the apt pinning should run. is the pin in the user_vars ?01:35
cloudnullkysse:  great success yet ?01:35
kyssenot yet, I have slow disks.01:35
cloudnullah. now i get it01:35
cloudnullload in another hampster :p01:36
cloudnulljamielennox:  when i get to the keystone role ill ping you, if you dont mind, so that we get it working for your usecase01:36
kyssenot everyone has 10 node cluster01:36
bgmccollumcloudnull: the pin was laid down in 10.1.17...i dont see any pins defined in 11.0.4, so when it gets to here to actually templating the pinning, its skipped --
cloudnullkysse: this is true01:36
cloudnullah, yea that would do it bgmccollum01:37
cloudnullits not removing the pins because nothing is defined01:37
bgmccollumso i just need to ensure apt_pinned_packages: [] is in my user vars01:37
jamielennoxcloudnull: sounds good, be glad to try it out, there's a lot of stuff in os_keystone that i'm keen to have an automated way to do01:37
cloudnullmakes sense now01:37
bgmccollumyeah... :/01:37
cloudnulljamielennox: sweet, then will be now soon. ill let you know.01:38
cloudnullbgmccollum: i guess you could re-pin it to a new version ?01:39
cloudnulli dont think it would be a good idea for us to remove pins by default because there may be other things in that list of pinned packages which some deployer may need .01:40
cloudnullbut then again i dont think we should be pinning packages ever. but thats a conversation for the ages.01:41
kysseI ran into weird problem with haproxy,  'AnsibleError: host not found:. hngh.01:41
cloudnullkinda like maintianing a private fork, never updating, and then complaining the deployment lacks functionality or is insecure. :\01:42
kysseoh. I'm running it too early..01:42
cloudnullok i've got ot go , have a good one all. ttyl01:49
*** networkninja has quit IRC01:53
*** alkari has joined #openstack-ansible02:48
*** iceyao has quit IRC03:21
*** cemmason has joined #openstack-ansible03:36
*** iceyao has joined #openstack-ansible03:38
*** sacharya has joined #openstack-ansible03:47
*** alkari has quit IRC03:58
*** tlian has quit IRC04:47
*** fawadkhaliq has joined #openstack-ansible04:48
*** sacharya has quit IRC04:50
*** sacharya has joined #openstack-ansible04:55
*** darrenc is now known as darrenc_afk05:10
*** alkari has joined #openstack-ansible05:22
*** markvoelker has joined #openstack-ansible05:24
*** darrenc_afk is now known as darrenc05:32
*** markvoelker has quit IRC05:41
*** sacharya has quit IRC05:48
*** sacharya has joined #openstack-ansible05:49
kyssecloudnull: it worked05:55
*** alkari has quit IRC06:08
*** targon has joined #openstack-ansible06:13
*** harlowja has quit IRC06:16
*** javeriak has joined #openstack-ansible06:22
*** mss has quit IRC06:25
*** sacharya has quit IRC06:32
*** phiche has joined #openstack-ansible06:45
*** phiche has quit IRC06:50
*** phiche has joined #openstack-ansible07:00
*** mss has joined #openstack-ansible07:21
*** sacharya has joined #openstack-ansible07:33
*** hassaan has joined #openstack-ansible07:35
hassaanI have a clean juno Openstack setup. I am trying to upgrade juno to kilo using the run-upgrade scripts. I ran into an error on the task: lxc_hosts | Install apt packages. The error is as follows: The following packages have unmet dependencies: lxc-dev : Depends: lxc (= 1.0.8-0ubuntu0.3) but 1.0.7-0ubuntu0.7 is to be installed07:36
*** sacharya has quit IRC07:38
*** mpavone has joined #openstack-ansible07:45
mattthassaan: i heard chatter of this one yesterday08:17
mattthassaan: mancdaz can probably fill you in w/ more info08:17
hassaanmattt: Thank you for the lead08:19
mattthassaan: from what i gather openstack-ansible pins LXC version, and that causes some breakage w/ a recent Ubuntu 14.04 lxc package update08:19
mattthassaan: i'm sure we'll have a fix for this one out shortly08:20
hassaanmattt: I understand. Thanks!08:22
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Make instance info output to stdout on failure
*** egonzalez has joined #openstack-ansible08:29
*** fawadkhaliq has quit IRC08:30
odyssey4mehassan mattt we just merged a patch to update the pin yesterday08:31
odyssey4mewe're waiting for a few other patches to land before we tag the next juno release08:31
odyssey4mehassaan ^08:31
*** javeriak has quit IRC08:32
hassaanodyssey4me: Can I get a link to the patch so I can apply it on my local setup08:33
*** sdake has joined #openstack-ansible08:34
odyssey4mehassaan you can either apply, or simply add the updated apt_pinned_packages dict to your user_variables08:34
*** sdake has quit IRC08:37
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: [WIP] Make instance info output to stdout on failure
*** sdake has joined #openstack-ansible08:40
*** fawadkhaliq has joined #openstack-ansible08:49
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Convert AIO bootstrap from bash to Ansible
*** gus has quit IRC09:07
*** gus has joined #openstack-ansible09:09
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Gate: Make instance info output to stdout on failure
*** egonzalez has quit IRC09:14
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Gate: Make instance info output to stdout on failure
*** raginbajin has joined #openstack-ansible09:17
hassaanodyssey4me: Thankyou09:22
odyssey4mehassaan if you're setting up a new environment, I recommend that you rather opt to setup a Kilo or Liberty environment... Juno is EOL09:25
hassaanodyssey4me: Noted. Thanks alot09:25
*** javeriak has joined #openstack-ansible09:45
evrardjpodyssey4me: was there a decision about the changes done on the roles that will are already moved in the IRR? cf. conversation here:
*** javeriak has quit IRC09:59
*** hassaan has quit IRC10:03
*** hassaan_ has joined #openstack-ansible10:03
*** hassaan_ has quit IRC10:05
*** gparaskevas has joined #openstack-ansible10:12
*** javeriak has joined #openstack-ansible10:14
*** alkari has joined #openstack-ansible11:04
*** javeriak has quit IRC11:40
*** fawadkhaliq has quit IRC11:40
*** alkari has quit IRC11:47
*** hassaan has joined #openstack-ansible11:47
*** javeriak has joined #openstack-ansible11:48
*** javeriak has quit IRC11:53
hassaanPlease note, I was able to continue with the upgrade from juno to kilo only after I updated the  liblxc1 package even with the fix applied. Without that the upgrade keeps failing since lxc package cannot be updated  for the lib-dev to be installed. Hence the script fails.11:54
*** javeriak has joined #openstack-ansible11:56
*** hassaan has quit IRC11:59
*** mpavone has quit IRC12:00
*** javeriak has quit IRC12:01
*** egonzalez has joined #openstack-ansible12:13
*** hassaan has joined #openstack-ansible12:42
odyssey4meevrardjp sorry, was out - what decision are you looking for? I'm a little confused. :)12:55
odyssey4mehassaan was this with the patch applied (and the playbooks re-run), or was this with the application of updated pins in user_variables?12:56
*** sdake has quit IRC13:00
mgariepygood morning everyone13:02
*** hassaan has quit IRC13:04
odyssey4meo/ mgariepy13:06
mgariepyhow are you doing ?13:06
odyssey4memorning mhayden - so we're beyond the flavor differences, and now into services not behaving well for
odyssey4meso I guess that something isn't being done quite right in the config somewhere... I'd appreciate your help trying to figure out where13:15
* mhayden is looking13:16
*** javeriak has joined #openstack-ansible13:16
mhaydenu'message': u'No valid host was found. There are not enough hosts available.'13:17
mhaydeni wonder if we're hitting a ram limit13:17
odyssey4memhayden unlikely, that's be from one of the schedulers13:19
odyssey4meso we need to figure out which one, then why it thinks the host is not available13:19
mhaydenaren't we only running one scheduler?13:20
mhaydeni'm going to build this on a rax 8gb right quick so i can mess with it13:21
kmARCQuick question guys: what's the preferred method to restart a service (let's say, neutron)?13:23
kmARCCurrently I'm doing a setup-everything. Maybe there is --limit neutron_all or something?13:24
kmARCI see the os_neutron/handlers/main.yml which has a hint (Restart neutron services). Can I invoke this via openstack-ansible or do I need some pure ansible machinery?13:28
gparaskevasdo you need to restart all neutron serviceS?13:28
kmARCyes, for example13:29
kmARCI guess it should be with some delays13:29
gparaskevasjust a restart no config change?13:29
kmARCI'm interested in both13:30
mhaydenodyssey4me: that build came up with three rabbitmq containers13:32
mhaydenbut it should have had one13:33
* mhayden is looking for where that logic went13:33
odyssey4memhayden don't worry about that, I reverted that logic back to the original intent13:33
mhaydenoh, so we want to have 3?13:33
odyssey4meit was cut down because we thought that there was a resource issue, but it turned out to be a disk space issue13:33
odyssey4mebut that's easy enough to cut them down as a tester13:34
gparaskevaskmARC: if you need to deploy a new change then as you mentioned you do the change to the desired ansible template and then openstack-ansible --limit neutron_all --tags "the tag(s) name you want to call"13:35
mhaydenthere seems to be enough RAM... the scheduler claims > 5GB RAM available13:35
kmARCgparaskevas: thanks. what limit options are there I can choose from, and also what entities are denoted by `tags`?13:36
gparaskevaskmARC: every task in openstack ansible is taged for that specific reason so you can call explicit set of tasks.13:36
mhaydenbut the scheduler claimed that something else fialed13:36
kmARCah, ok13:36
*** markvoelker has joined #openstack-ansible13:36
mhaydenodyssey4me: libvirtError: internal error: no supported architecture for os type 'hvm'13:38
mhaydeni can't find where we're setting libvirt_type anywhere13:40
mhaydenit's been a while since i looked at nova's defaults13:40
*** markvoelker_ has joined #openstack-ansible13:41
odyssey4meaha, so this is the issue that was being discussed yesterday13:41
odyssey4meI deliberately left out the hard setting of the virt_type as there is code that's support to detect it. Apparently that code doesn't work. :/13:42
odyssey4melet me track it down.13:42
*** markvoelker has quit IRC13:42
odyssey4methanks for finding that!13:42
gparaskevaslimit option are ragard to the inventory so you can check /etc/openstack_deploy/env.d/neutron.yml13:43
gparaskevaskmARC:  limit option are ragard to the inventory so you can check /etc/openstack_deploy/env.d/neutron.yml13:43
mhaydenodyssey4me: no problem! :)13:48
kmARCgparaskevas: ah, I see, it makes sense now13:49
gparaskevaskmARC: yeah , always pleasure!13:50
kmARCgparaskevas: so and what about tags?13:50
kmARCI saw some in the playbooks13:50
*** adac has joined #openstack-ansible13:51
adacGuys, i did a fresh installation of openstack-ansible today but still the galera cluster, executing the command ansible galera_container -m shell -a "cat /var/lib/mysql/grastate.dat" right after the installation just shows just the following:
*** javeriak has quit IRC14:00
mhaydenso those flags can be passed through without nested virt being enabled14:00
mhaydenperhaps we should add a nested virt check?14:00
odyssey4mewell, I just tested on a vm i cloud14:02
odyssey4meI get a zero value, so my virt_type should be qemu - which should work as before14:02
odyssey4meah, but the value is being set incorrectly in nova.conf14:02
kmARCOK, I have a networking related question now. gparaskevas maybe you can help me with this too :)14:06
odyssey4memhayden and I see why - it's because nova_virt_type is defined as a default... I'll patch this up14:07
kmARCSo on the config file the examples enumerate some networks. What I need, is flat networking, however, in the example it maps to eth12, which is good in the containers, not correct for the compute node though (it doesn't have containers)14:07
kmARCwhat's kind of suspicous is the host_bind_override: "eth12" line. However I have no idea what this host_bind_override does14:07
mhaydenodyssey4me: woot14:08
mhaydenlet me know when the review is up and i'll gander14:08
mhaydeni will apply a +1 with gusto14:08
gparaskevasmhayden: ok, dont bother if its not on the compute nodes because thereis an option that overides that for the compute nodes "host_bind_override:" and there you define the specific interface of the compute node14:10
gparaskevasmhayden: whrong usename :P14:11
mhaydengparaskevas: i still like ya14:11
kmARC(Just for the record, I might have found it: - it seems this configuration sets the corresponding mapped interface)14:11
gparaskevaskmARC: ok, dont bother if its not on the compute nodes because thereis an option that overides that for the compute nodes "host_bind_override:" and there you define the specific interface of the compute node14:11
kmARCYepp, found it14:11
gparaskevaskmARC: the option can be found in the openstack_user_config.yml14:11
gparaskevaskmARC: great14:12
*** hassaan has joined #openstack-ansible14:12
kmARCok now it seems the config changed, let me check14:13
hassaanodyssey4me, sorry for late reply, was away. Yes this was with the application of updated pins in user_variables and playbook rerun14:13
*** fawadkhaliq has joined #openstack-ansible14:15
*** hassaan has quit IRC14:18
openstackgerritDarren Birkett proposed openstack/openstack-ansible: turn neutron notifications off by default
gparaskevaskmARC: what brancj are you on?14:24
kmARCgparakevas: so I'm trying to debug something now while looking at rsyslog messages, however I'm still getting OLD messages but don't know why. Have a look here:
kmARCgparaskevas: I'm on Liberty14:25
gparaskevaskmARC: this is from compute node?14:28
*** tlian has joined #openstack-ansible14:29
*** hassaan has joined #openstack-ansible14:31
kmARCno, it's from the rsyslog container14:36
kmARCgparaskevas: anyways, I think I have multiple issues here :) It'd be great if I'd have the logs in cronological order, and then I would able to fix my 'no compute host found' issue.14:39
gparaskevaskmARC: good luck :)14:42
gparaskevaskmARC: you are going to need  it!14:42
gparaskevaskmARC:  make small changes so you dont get lost always14:43
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Fix nova_virt_type auto-detection
odyssey4memhayden ^14:43
* mhayden ganders14:44
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Convert AIO bootstrap from bash to Ansible
*** fawadkhaliq has quit IRC14:44
mhaydenodyssey4me: you get my strongest +114:45
mhaydenthat's a clean fix14:45
mhaydeni like it14:45
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Fix nova_virt_type auto-detection
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Convert AIO bootstrap from bash to Ansible
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Gate: Make instance info output to stdout on failure
odyssey4memhayden spelling error in the commit msg, fixed :)14:48
*** KLevenstein has joined #openstack-ansible14:53
*** spotz_zzz is now known as spotz14:56
*** mss has quit IRC14:58
*** leifmadsen has quit IRC15:01
matttthis kvm auto detect thing is on interest15:01
*** leifmadsen has joined #openstack-ansible15:03
openstackgerritMajor Hayden proposed openstack/openstack-ansible: Use Ansible AIO playbook for
*** baker has joined #openstack-ansible15:06
*** sigmavirus24_awa is now known as sigmavirus2415:09
adacalso additonally run this command does not solve it "openstack-ansible -e galera_ignore_cluster_state=true galera-install.yml" Not sure what I can try else. If it is not even working wehn istalling everything from scratch with a clean ubuntu installation :/15:16
*** targon has quit IRC15:16
*** raginbajin has quit IRC15:16
*** raginbajin has joined #openstack-ansible15:17
*** linggao has joined #openstack-ansible15:21
*** hassaan has quit IRC15:26
*** fawadkhaliq has joined #openstack-ansible15:28
*** markvoelker_ has quit IRC15:28
adacThere is also a log node, can I check the galera/mariadb log probably there?15:29
*** ChanServ has quit IRC15:34
*** raginbajin has quit IRC15:34
*** gparaskevas has quit IRC15:34
*** marekd has quit IRC15:34
*** grumpycatt has quit IRC15:34
*** pabelanger has quit IRC15:34
*** egonzalez has quit IRC15:34
*** gtt116_ has quit IRC15:34
*** mcarden has quit IRC15:34
*** jbweber has quit IRC15:34
*** erikmwilson has quit IRC15:34
*** jwagner has quit IRC15:34
*** misc has quit IRC15:34
*** xek has quit IRC15:34
*** metral has quit IRC15:34
*** stevelle has quit IRC15:34
*** dalees has quit IRC15:34
*** finchd has quit IRC15:34
*** mgagne has quit IRC15:34
*** b3rnard0 has quit IRC15:34
*** fawadkhaliq has quit IRC15:34
*** adac has quit IRC15:34
*** cemmason has quit IRC15:34
*** alextricity has quit IRC15:35
*** jlvillal has quit IRC15:35
*** thrawn01 has quit IRC15:35
*** e-vad has quit IRC15:35
*** mattoliverau has quit IRC15:35
*** SamYaple has quit IRC15:35
*** mrda has quit IRC15:35
*** phschwartz has quit IRC15:35
*** charz has quit IRC15:35
*** mgariepy has quit IRC15:35
*** rev_dr has quit IRC15:35
*** Sam-I-Am has quit IRC15:35
*** rcarrillocruz has quit IRC15:35
*** jhesketh has quit IRC15:35
*** dweaver has quit IRC15:35
*** dmsimard has quit IRC15:35
*** sirushti has quit IRC15:35
*** wabu has quit IRC15:35
*** arbrandes has quit IRC15:35
*** evrardjp has quit IRC15:35
*** ggillies has quit IRC15:35
*** pellaeon_ has quit IRC15:35
*** sshen_ has quit IRC15:35
*** dstanek has quit IRC15:35
*** persia has quit IRC15:35
*** lkoranda has quit IRC15:35
*** andymccr has quit IRC15:35
*** coolj has quit IRC15:35
*** dolpher has quit IRC15:35
*** Ti-mo has quit IRC15:35
*** bapalm has quit IRC15:35
*** richoid has quit IRC15:35
*** galstrom_zzz has quit IRC15:35
*** d9k has quit IRC15:35
*** toddnni has quit IRC15:35
*** _hanhart has quit IRC15:35
*** rackertom has quit IRC15:35
*** etoews has quit IRC15:35
*** maximov_ has quit IRC15:35
*** meteorfox has quit IRC15:35
*** mhayden has quit IRC15:35
*** errr has quit IRC15:35
*** baker has quit IRC15:35
*** leifmadsen has quit IRC15:35
*** KLevenstein has quit IRC15:35
*** mancdaz has quit IRC15:35
*** jaypipes has quit IRC15:35
*** tiagogomes has quit IRC15:35
*** mrodden has quit IRC15:35
*** kore has quit IRC15:35
*** elo has quit IRC15:35
*** Tridde has quit IRC15:35
*** darrenc has quit IRC15:35
*** niknakpaddywak has quit IRC15:35
*** toan has quit IRC15:35
*** McMurlock has quit IRC15:35
*** mattt has quit IRC15:35
*** Guest4784 has quit IRC15:35
*** kysse has quit IRC15:35
*** tlbr has quit IRC15:35
*** antonym has quit IRC15:35
*** logan- has quit IRC15:35
*** admiralboom has quit IRC15:35
*** fignew- has quit IRC15:35
*** miguelgrinberg has quit IRC15:35
*** rromans has quit IRC15:35
*** jmccrory has quit IRC15:35
*** prometheanfire has quit IRC15:35
*** jcannava has quit IRC15:35
*** larsks has quit IRC15:35
*** andyhky has quit IRC15:35
*** git-harry has quit IRC15:35
*** bryan_att has quit IRC15:35
*** spotz has quit IRC15:35
*** serverascode has quit IRC15:35
*** lbragstad has quit IRC15:35
*** jroll has quit IRC15:35
*** Apsu has quit IRC15:35
*** jamielennox has quit IRC15:35
*** rETROpunK has quit IRC15:35
*** mattronix has quit IRC15:35
*** cloudnull has quit IRC15:35
*** bogeyon18 has quit IRC15:35
*** d34dh0r53 has quit IRC15:35
*** sigmavirus24 has quit IRC15:35
*** dolphm has quit IRC15:35
*** eglute has quit IRC15:35
linggaoWhy did everyone quit? What did I do?15:36
*** palendae has quit IRC15:37
*** sacharya has joined #openstack-ansible15:37
*** dweaver has joined #openstack-ansible15:37
*** dmsimard has joined #openstack-ansible15:37
*** sirushti has joined #openstack-ansible15:37
*** wabu has joined #openstack-ansible15:37
*** arbrandes has joined #openstack-ansible15:37
*** evrardjp has joined #openstack-ansible15:37
*** ggillies has joined #openstack-ansible15:37
*** pellaeon_ has joined #openstack-ansible15:37
*** sshen_ has joined #openstack-ansible15:37
*** palendae has joined #openstack-ansible15:38
*** mhayden has joined #openstack-ansible15:38
evrardjpodyssey4me: sorry for the late answer, but you answered yourself with a -1 workflow on the commit :)15:38
mhaydenhooray for netsplits15:38
linggaoMorning ansible.15:39
*** egonzalez has joined #openstack-ansible15:39
*** gtt116_ has joined #openstack-ansible15:39
*** mcarden has joined #openstack-ansible15:39
*** jbweber has joined #openstack-ansible15:39
*** erikmwilson has joined #openstack-ansible15:39
*** jwagner has joined #openstack-ansible15:39
*** misc has joined #openstack-ansible15:39
*** xek has joined #openstack-ansible15:39
*** metral has joined #openstack-ansible15:39
*** stevelle has joined #openstack-ansible15:39
*** dalees has joined #openstack-ansible15:39
*** finchd has joined #openstack-ansible15:39
*** mgagne has joined #openstack-ansible15:39
*** b3rnard0 has joined #openstack-ansible15:39
*** d9k has joined #openstack-ansible15:39
*** galstrom_zzz has joined #openstack-ansible15:39
*** toddnni has joined #openstack-ansible15:39
*** _hanhart has joined #openstack-ansible15:39
*** rackertom has joined #openstack-ansible15:39
*** etoews has joined #openstack-ansible15:39
*** maximov_ has joined #openstack-ansible15:39
evrardjpmorning everyone15:39
*** marekd has joined #openstack-ansible15:39
*** grumpycatt has joined #openstack-ansible15:39
*** palendae has quit IRC15:39
*** palendae has joined #openstack-ansible15:39
*** mgariepy has joined #openstack-ansible15:40
evrardjpmhayden: so I'm not the only one to have these issues? It's been like 1 week of issues, maybe it's time to move to a more recent protocol, like xmpp?15:40
evrardjpsorry mhayden, the complete sentence was not for you.15:40
*** raginbajin has joined #openstack-ansible15:40
mhaydenlots of DDoS going on this week :(15:40
*** antonym has joined #openstack-ansible15:40
*** logan- has joined #openstack-ansible15:40
*** admiralboom has joined #openstack-ansible15:40
*** fignew- has joined #openstack-ansible15:40
*** miguelgrinberg has joined #openstack-ansible15:40
*** rromans has joined #openstack-ansible15:40
*** jmccrory has joined #openstack-ansible15:40
*** prometheanfire has joined #openstack-ansible15:40
*** jcannava has joined #openstack-ansible15:40
*** larsks has joined #openstack-ansible15:40
*** andyhky has joined #openstack-ansible15:40
mhaydenlooks like people are coming back online now15:40
*** jhesketh has joined #openstack-ansible15:41
mhaydeni was on which seemed to avoid some of the shenanigans until today15:41
evrardjpmhayden: DDoS are the origin of the disconnections?15:41
kmARCanyone see anything like this: I can list, show the image, my compute node cannot15:41
mhaydenevrardjp: yeah, freenode is getting DDoS'ed in a few places15:41
mhaydennot sure if this message came through earlier, but some eyes on would be appreciated15:41
evrardjppff. ain't nobody got time for that?15:42
*** sacharya has quit IRC15:42
*** gparaskevas has joined #openstack-ansible15:42
*** dstanek has joined #openstack-ansible15:42
*** persia has joined #openstack-ansible15:42
*** coolj has joined #openstack-ansible15:42
*** lkoranda has joined #openstack-ansible15:42
*** andymccr has joined #openstack-ansible15:42
*** dolpher has joined #openstack-ansible15:42
*** Ti-mo has joined #openstack-ansible15:42
*** bapalm has joined #openstack-ansible15:42
*** richoid has joined #openstack-ansible15:42
*** meteorfox has joined #openstack-ansible15:42
*** errr has joined #openstack-ansible15:42
*** pabelanger has joined #openstack-ansible15:42
linggaoHi all. What is the status for ironic support in ansible?  Can I get draft to try?15:43
*** git-harry has joined #openstack-ansible15:44
*** bryan_att has joined #openstack-ansible15:44
*** spotz has joined #openstack-ansible15:44
*** serverascode has joined #openstack-ansible15:44
*** lbragstad has joined #openstack-ansible15:44
*** jroll has joined #openstack-ansible15:44
*** Apsu has joined #openstack-ansible15:44
*** jamielennox has joined #openstack-ansible15:44
*** rETROpunK has joined #openstack-ansible15:44
*** mattronix has joined #openstack-ansible15:44
*** cloudnull has joined #openstack-ansible15:44
*** bogeyon18 has joined #openstack-ansible15:44
*** d34dh0r53 has joined #openstack-ansible15:44
*** sigmavirus24 has joined #openstack-ansible15:44
*** dolphm has joined #openstack-ansible15:44
*** eglute has joined #openstack-ansible15:44
*** phschwartz_ has joined #openstack-ansible15:44
*** phschwartz_ has quit IRC15:45
*** phschwartz_ has joined #openstack-ansible15:48
evrardjpopenstack-ansible doesn't currently have any ironic support, or at least none that I'm aware of15:48
evrardjpbut I'd happy to see some of it15:48
linggaoevrardjp, I heard that someone was working on it.  I forgot the name of the person.15:49
evrardjpthere is a work done on ironic standalone and ansible, it's called bifrost15:49
evrardjpmorning cloudnull15:50
evrardjpcloud: I'll adapt my mtu thingies inside your new roles (IRR) if you want15:51
linggaoevrardjp, yes, I tried that.  I am using openstack-ansible to install nodes. Just hope it support ironic.15:51
*** ctina has joined #openstack-ansible15:51
evrardjplinggao: at the moment, the deployer chooses its way to deploy the bare metal nodes, and openstack-ansible takes the relay further15:51
evrardjphowever ironic as nova backend (I don't know how you call that), seems very interesting IMO15:52
evrardjpI'm not sure I understood your question to be honest :p15:54
stevelleI thought I heard that someone may have been building a role for ironic but I don't know status of it and don't recall a name of the author either linggao15:54
*** markvoelker has joined #openstack-ansible15:55
linggaoevrardjp, like I am using openstack-ansible to install nova, neutron, glance etc. I hope it can do ironic.15:55
linggaoserverascode, I guess 'a role for ironic' is the stuff I should ask. thanks :-)15:56
linggaoI saw,  is it related to this project?15:57
stevelleI don't believe that is15:58
odyssey4melinggao is the current effort which is ready for some eyes, but not yet necessarily ready for integration15:58
*** adac has joined #openstack-ansible15:59
*** fawadkhaliq has joined #openstack-ansible15:59
*** alextricity has joined #openstack-ansible15:59
*** Sam-I-Am has joined #openstack-ansible15:59
*** jlvillal has joined #openstack-ansible15:59
*** thrawn01 has joined #openstack-ansible15:59
*** e-vad has joined #openstack-ansible15:59
*** mattoliverau has joined #openstack-ansible15:59
*** SamYaple has joined #openstack-ansible15:59
*** mrda has joined #openstack-ansible15:59
*** charz has joined #openstack-ansible15:59
*** rev_dr has joined #openstack-ansible15:59
*** rcarrillocruz has joined #openstack-ansible15:59
odyssey4mestevelle evrardjp ^15:59
linggaoodyssey4me, thanks, will it eventually be checked in to openstack-ansible tree?16:00
odyssey4meevrardjp those roles aren't cloudnull's - they are now the community's - they're being used for the gate tests :)16:00
stevellelinggao: file:///Users/hairaldo/dev/
odyssey4melinggao they will become part of the umbrella, yes - I'll be discussing the status with mrda later this evening16:01
*** k_stev has joined #openstack-ansible16:01
stevelleerr sorry linggao but in the docs we have an entry for adding galaxy roles16:01
odyssey4mehaha stevelle I think you meant:
evrardjpodyssey4me: indeed, but cloudnull started the work on it, that's why I say that, sorry for my wrong language :p16:01
*** adaccada has joined #openstack-ansible16:04
stevelleI did, odyssey4me. forgot the link I had open was from a local build16:04
cloudnullmattt hughsaunders if you have a moment can you review again16:06
cloudnullim blocked on any more of the IRR work until that goes in16:06
*** ctina has quit IRC16:07
adaccadaThis is the output of mysld.log: and here is the output of mysql_safe.log: all looks fine and the recovery seems to have taken place and the mysqld server is running, however a cat on /var/lib/mysql/grastate.dat still only shows
*** ctina has joined #openstack-ansible16:07
adaccadanot sure why this is happening16:08
adaccadaIt is a fresh installation, I did not modify anything yet16:08
*** adac has quit IRC16:09
*** fawadkhaliq has quit IRC16:09
*** alextricity has quit IRC16:09
*** jlvillal has quit IRC16:09
*** thrawn01 has quit IRC16:09
*** e-vad has quit IRC16:09
*** mattoliverau has quit IRC16:09
*** SamYaple has quit IRC16:09
*** mrda has quit IRC16:09
*** charz has quit IRC16:09
*** rev_dr has quit IRC16:09
*** Sam-I-Am has quit IRC16:09
*** rcarrillocruz has quit IRC16:09
cloudnulladaccada: looks like its running16:11
cloudnullare you not able to connect to the galera cluster ?16:11
adaccadacloudnull, not sure how to test that?16:12
*** javeriak has joined #openstack-ansible16:12
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_hosts: Added to ability to set mtu for lxcbr0
*** jaypipes has joined #openstack-ansible16:14
*** palendae has quit IRC16:14
adaccadacloudnull, I thought that this command would give me the overview over the cluster: ansible galera_container -m shell -a "cat /var/lib/mysql/grastate.dat" but when I start it (before I did all the steps from here:, then I only get this output:
*** palendae has joined #openstack-ansible16:16
adaccadaSo as odyssey4me told me this does mean the cluster is not really ready16:16
prometheanfireodyssey4me: do those ldap review need testing with the memberof plugin? or just basic ldap?16:17
*** javeriak has quit IRC16:19
*** javeriak has joined #openstack-ansible16:22
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-lxc_container_create: Ability to define the mtu globally (for all the container networks)
cloudnulladaccada: its showing the cluster state, however to know if the cluster is "ready" you should query: show status like '%wsrep%';16:22
cloudnullthat will look at the wsrep state16:23
adaccada"wsrep_local_state_comment : Synced",  wsrep_cluster_size   2, wsrep_connected  ON16:23
adaccadacloudnull, yeah just did that, seems to be online16:23
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Add documentation for HA ceilometer
adaccadajay!! :) Not sure why I have such a brutal output then with ansible galera_container -m shell -a "cat /var/lib/mysql/grastate.dat"16:24
*** javeriak_ has joined #openstack-ansible16:25
gparaskevascloudnull: Hello cloudnull, just an irelevant question regarding openstack(i hoped you maybe know) does it support boot from iso? I have seen an post about it but i havent doen it with success16:26
*** javeriak has quit IRC16:26
gparaskevascloudnull: just in case you know if it is suported or not :)16:27
evrardjpgparaskevas: not sure but I think you should use cinder for that16:27
gparaskevasevrardjp: i have read in post that when you specify an iso image then nova mounts a cinder volume as well16:27
evrardjpdid you check if glance has something to do with iso?16:28
cloudnullgparaskevas:  boot from ISO in creating a VM ?16:28
* cloudnull looking16:28
gparaskevasevrardjp: yes16:28
gparaskevascloudnull: yes, launch instance with image type ISO16:28
cloudnullthat should work16:28
gparaskevascloudnull: let me try once more16:29
gparaskevasthanks both16:29
cloudnullyou might need to change the console type for your install, if i remember correct.16:30
cloudnulli think it worked well with novnc16:30
*** alkari has joined #openstack-ansible16:30
cloudnullbut ive not tried with spice16:30
gparaskevasmmm let me check16:30
cloudnullsorry , recalling memories w/out enough coffee16:31
evrardjpyeah that's really far for me too16:31
evrardjplike 2 years ago16:31
*** javeriak has joined #openstack-ansible16:31
gparaskevascloudnull: lol I am on redbull today...16:32
evrardjpmaybe less I can't remember, which doesn't help. Anyway, cloudnull you I've see there was a swapiness in your change here
evrardjpI didn't set it in the follow-up commit (because it's not related to mtu), so if you need it, feel free to create another commit :)16:33
*** tlbr has joined #openstack-ansible16:34
*** Guest4784 has joined #openstack-ansible16:34
*** mattt has joined #openstack-ansible16:34
*** McMurlock has joined #openstack-ansible16:34
*** toan has joined #openstack-ansible16:34
*** niknakpaddywak has joined #openstack-ansible16:34
*** darrenc has joined #openstack-ansible16:34
*** Tridde has joined #openstack-ansible16:34
*** elo has joined #openstack-ansible16:34
*** kore has joined #openstack-ansible16:34
*** kysse has joined #openstack-ansible16:34
*** mrodden has joined #openstack-ansible16:34
*** tiagogomes has joined #openstack-ansible16:34
*** mancdaz has joined #openstack-ansible16:34
*** Klev-meeting has joined #openstack-ansible16:34
*** leifmadsen has joined #openstack-ansible16:34
*** baker has joined #openstack-ansible16:34
*** mss has joined #openstack-ansible16:34
*** fawadkhaliq has joined #openstack-ansible16:34
*** alextricity has joined #openstack-ansible16:34
*** Sam-I-Am has joined #openstack-ansible16:34
*** jlvillal has joined #openstack-ansible16:34
*** thrawn01 has joined #openstack-ansible16:34
*** e-vad has joined #openstack-ansible16:34
*** mattoliverau has joined #openstack-ansible16:34
*** SamYaple has joined #openstack-ansible16:34
*** mrda has joined #openstack-ansible16:34
*** charz has joined #openstack-ansible16:34
*** rev_dr has joined #openstack-ansible16:34
*** rcarrillocruz has joined #openstack-ansible16:34
*** ysm_ has joined #openstack-ansible16:34
*** javeriak_ has quit IRC16:35
*** eil397 has joined #openstack-ansible16:36
cloudnullevrardjp: no need from me, good change though. :)16:36
evrardjpswappiness is damn important, but I'm not sure in which role it would be best to put it... a common one? ;)16:37
evrardjpor just let the deployer choose.16:37
gparaskevasswappiness regarding every host?16:38
*** _hanhart has quit IRC16:39
linggaoHi mrda, I saw you are working on ironc now and also read your design.  How far is it from being useful?16:42
evrardjpgparaskevas: yes16:44
evrardjpdefine just something that defines the swapiness16:44
evrardjpmaybe with a variable depending on the group16:44
evrardjpdefaulting to something really low16:44
evrardjpbut that's not really a priority16:44
*** mss has quit IRC16:45
*** fawadkhaliq has quit IRC16:45
*** alextricity has quit IRC16:45
*** jlvillal has quit IRC16:45
*** thrawn01 has quit IRC16:45
*** e-vad has quit IRC16:45
*** mattoliverau has quit IRC16:45
*** SamYaple has quit IRC16:45
*** mrda has quit IRC16:45
*** charz has quit IRC16:45
*** rev_dr has quit IRC16:45
*** Sam-I-Am has quit IRC16:45
*** rcarrillocruz has quit IRC16:45
*** ysm_ has quit IRC16:45
*** baker has quit IRC16:45
*** leifmadsen has quit IRC16:45
*** Klev-meeting has quit IRC16:45
*** mancdaz has quit IRC16:45
*** tiagogomes has quit IRC16:45
*** mrodden has quit IRC16:45
*** kore has quit IRC16:45
*** elo has quit IRC16:45
*** Tridde has quit IRC16:45
*** darrenc has quit IRC16:45
*** niknakpaddywak has quit IRC16:45
*** toan has quit IRC16:45
*** McMurlock has quit IRC16:45
*** mattt has quit IRC16:45
*** Guest4784 has quit IRC16:45
*** kysse has quit IRC16:45
*** tlbr has quit IRC16:45
evrardjpanother round of netsplits!16:45
*** gparaskevas has quit IRC16:45
*** ysm_ has joined #openstack-ansible16:46
*** rcarrillocruz has joined #openstack-ansible16:46
*** rev_dr has joined #openstack-ansible16:46
*** charz has joined #openstack-ansible16:46
*** mrda has joined #openstack-ansible16:46
*** SamYaple has joined #openstack-ansible16:46
*** mattoliverau has joined #openstack-ansible16:46
*** e-vad has joined #openstack-ansible16:46
*** thrawn01 has joined #openstack-ansible16:46
*** jlvillal has joined #openstack-ansible16:46
*** Sam-I-Am has joined #openstack-ansible16:46
*** alextricity has joined #openstack-ansible16:46
*** fawadkhaliq has joined #openstack-ansible16:46
*** mss has joined #openstack-ansible16:46
*** baker has joined #openstack-ansible16:46
*** leifmadsen has joined #openstack-ansible16:46
*** Klev-meeting has joined #openstack-ansible16:46
*** mancdaz has joined #openstack-ansible16:46
*** tiagogomes has joined #openstack-ansible16:46
*** mrodden has joined #openstack-ansible16:46
*** kysse has joined #openstack-ansible16:46
*** kore has joined #openstack-ansible16:46
*** elo has joined #openstack-ansible16:46
*** Tridde has joined #openstack-ansible16:46
*** darrenc has joined #openstack-ansible16:46
*** niknakpaddywak has joined #openstack-ansible16:46
*** toan has joined #openstack-ansible16:46
*** McMurlock has joined #openstack-ansible16:46
*** mattt has joined #openstack-ansible16:46
*** Guest4784 has joined #openstack-ansible16:46
*** tlbr has joined #openstack-ansible16:46
-cloudnull- Freenode is having issues today: -- if you have a message missed or otherwise ignored please repost it, its likely the message was never seen16:48
*** sdake has joined #openstack-ansible16:51
cloudnullevrardjp: sorry was in another meeting . the swappieness is in openstack_hosts16:52
odyssey4meprometheanfire the ldap plugins - honestly I have no idea right now - I have yet to look at any of it16:53
evrardjpok it could make sense to add it on lxc hosts too then16:53
cloudnulli guess we could16:54
prometheanfireodyssey4me: just not sure why that was one of the instructions16:54
cloudnulli'd keep the value between the openstack_hosts and lxc_hosts roles the same so it doesnt flip flop16:55
evrardjpcloudnull: roger that. Will try do it tomorrow.16:55
cloudnullidk if we should have the value be 5 or 1016:55
cloudnullbut ill leave that up to you16:55
evrardjpyeah like this is gonna make a huge difference16:55
odyssey4melinggao mrda is in Australia, and therefore likely asleep at this time. :)16:56
*** baker has quit IRC17:01
*** baker has joined #openstack-ansible17:02
*** ChanServ has joined #openstack-ansible17:05
*** sets mode: +o ChanServ17:05
*** Mudpuppy has joined #openstack-ansible17:05
*** mgariepy has quit IRC17:06
*** mgariepy has joined #openstack-ansible17:06
*** Mudpuppy is now known as Mudpuppy_awa17:06
*** baker has quit IRC17:06
*** adaccada has quit IRC17:08
*** Mudpuppy_awa is now known as Mudpuppy17:11
linggaoodyssey4me, thanks let me know. :-).17:14
*** sacharya has joined #openstack-ansible17:18
*** phiche has quit IRC17:19
*** eil397 has quit IRC17:23
*** woodard has joined #openstack-ansible17:24
*** hassaan has joined #openstack-ansible17:24
*** phschwartz_ is now known as phschwartz17:25
*** sacharya_ has joined #openstack-ansible17:26
*** woodard has quit IRC17:28
*** k_stev has quit IRC17:28
*** k_stev1 has joined #openstack-ansible17:28
*** k_stev1 has quit IRC17:28
*** sacharya has quit IRC17:29
*** egonzalez has quit IRC17:29
*** k_stev has joined #openstack-ansible17:29
*** eil397 has joined #openstack-ansible17:30
*** baker has joined #openstack-ansible17:31
*** k_stev has quit IRC17:32
*** alkari has quit IRC17:32
*** k_stev has joined #openstack-ansible17:32
*** hassaan has quit IRC17:33
*** targon has joined #openstack-ansible17:33
*** targon has quit IRC17:36
*** Klev-meeting is now known as KLevenstein17:39
*** shausy has joined #openstack-ansible17:40
*** hassaan has joined #openstack-ansible17:45
*** b3rnard0 is now known as b3rnard0_away17:45
*** hassaan has quit IRC17:51
*** javeriak_ has joined #openstack-ansible17:52
*** Bjoern_ has joined #openstack-ansible17:52
*** hassaan has joined #openstack-ansible17:54
*** shausy has quit IRC17:54
*** javeriak has quit IRC17:56
*** shausy has joined #openstack-ansible17:56
*** baker has quit IRC17:57
*** sigmavirus24 is now known as sigmavirus24_awa17:57
*** tiagogomes has quit IRC17:57
*** alkari has joined #openstack-ansible18:00
*** shausy has quit IRC18:00
*** shausy has joined #openstack-ansible18:00
*** hassaan has quit IRC18:01
*** galstrom_zzz is now known as galstrom18:04
*** sdake has quit IRC18:06
*** sacharya_ has quit IRC18:10
*** harlowja has joined #openstack-ansible18:12
*** sacharya has joined #openstack-ansible18:13
*** hassaan has joined #openstack-ansible18:13
*** Bjoern_ is now known as BjoernT18:14
*** rebase_ has joined #openstack-ansible18:20
*** b3rnard0_away is now known as b3rnard018:23
-cloudnull- Freenode is having issues today: -- if you have a message missed or otherwise ignored please repost it, its likely the message was never seen18:24
*** alkari has quit IRC18:34
*** hassaan_ has joined #openstack-ansible18:36
cloudnullkysse: so liberty is all happy now ?18:38
cloudnullwas going to ask earlier however IRC was not happy18:38
cloudnullany core arond to do a review on ?18:40
*** hassaan has quit IRC18:40
cloudnullodyssey4me mattt hughsaunders andymccr d34dh0r53 ? ^18:40
cloudnullstevelle: ty for the review already18:40
*** bryan_att has quit IRC18:41
cloudnullcores we need that to go in too ^18:44
cloudnulllarge deployments of liberty are effectively broken unless neutron_l2_population is enabled.18:45
*** ysm_ has quit IRC18:45
*** ysm_ has joined #openstack-ansible18:46
*** adaccada has joined #openstack-ansible18:48
*** fawadkhaliq has quit IRC18:49
*** fawadkhaliq has joined #openstack-ansible18:49
*** ysm_ has quit IRC18:50
*** k_stev has quit IRC18:55
odyssey4mecloudnull let me understand this properly18:56
odyssey4meif l2pop is enabled, l3ha is unusable18:56
odyssey4mehowever if l3ha is enabled, l2pop must be disabled - rendering neutron unusable18:56
*** javeriak_ has quit IRC18:56
*** hassaan_ has quit IRC18:57
Sam-I-Ami thought l3ha and l2pop got along in liberty18:57
Sam-I-Amits l2pop and dvr that do not18:57
odyssey4meis this disaster a special case for when we only have one agents container?18:57
cloudnullSam-I-Am:  nope18:57
cloudnullthe issue comes up when there are multiple agents , a single l3 agent everything is ok18:57
Sam-I-Amwell, l3ha with only one agent is not ha18:58
odyssey4meso if l2pop is disabled, and l3ha enabled, and there are multiple agents - then everything is fine?18:58
odyssey4meand conversely if l2pop is enabled, l3ha disabled and there is one agent - then everything is fine?18:58
cloudnullodyssey4me:  and there is something eles on the backend that is doing l2 population for the deployment yes18:58
cloudnullto the first one18:59
cloudnulll2 pop enabled, l3ha disabled works fine for multiple network nodes18:59
Sam-I-Ami'm looking for the bug about this18:59
odyssey4meoh, so l3ha is thus pretty much broken then.... unless we implement something to replace l2pop18:59
*** bryan_att has joined #openstack-ansible18:59
*** BjoernT has quit IRC19:00
cloudnullthis would be the thrid release where l3ha is functionally unusable.19:00
cloudnullwhich is why we still have the att neutron tool19:00
odyssey4mecloudnull in that case we should disable the automagic stuff and let the deployer decide - let's leave l3ha disabled by default and l2pop enabled by default19:01
cloudnullSam-I-Am:  if it works, then theres something else going on, because kysse beat on this for several days and its busted.19:01
Sam-I-Amwe enabled support for l3ha in liberty, right?19:02
Sam-I-Ami seem to recall this patch19:02
odyssey4mebefore we do that, though, I'd like us to engage with some neutron devs to figure this one out19:02
odyssey4meSam-I-Am yes19:02
*** baker has joined #openstack-ansible19:02
Sam-I-Amodyssey4me cloudnull -
openstackLaunchpad bug 1365476 in neutron "HA routers interact badly with l2pop" [High,Fix released] - Assigned to Mike Kolesnik (mkolesni)19:03
*** baker has quit IRC19:03
*** Bjoern_ has joined #openstack-ansible19:03
odyssey4meso this seems to imply that l2pop and l3ha can work togetheR?19:04
*** phiche has joined #openstack-ansible19:04
Sam-I-Amthat is the idea19:05
odyssey4mespecifically part of the commit message: This patch also changes the L3 HA failover test to use l2pop. Note that the test does not pass when using l2pop without this patch.19:05
Sam-I-Ambut my l3ha guy is afk right now19:05
*** baker has joined #openstack-ansible19:05
odyssey4memhayden ping19:06
*** Bjoern_ is now known as BjoernT19:06
Sam-I-Amodyssey4me: that commit message makes sense19:06
Sam-I-Amat least to me19:06
Sam-I-Amalthough does it mean that l3ha does not work without l2pop19:07
* Sam-I-Am sticks head in sand19:07
Sam-I-Amaios only have 1 agents container, right?19:08
cloudnullSam-I-Am: correct19:08
cloudnullwhich is why we didnt see the issue in the gate19:08
cloudnullkysse:  had 2 network nodes which caused the issue19:08
mhaydenodyssey4me: echo reply19:09
*** sigmavirus24_awa is now known as sigmavirus2419:09
Sam-I-Amthere's a lot of patches that the gate doesnt check19:09
cloudnullwe can do an affinity group for agent containers and test to make sure l2 pop is in fact working when l3ha is enabled19:09
Sam-I-Amnot specifically ours, but neutrons19:09
Sam-I-Amthat was my next thought19:09
cloudnulland that l3ha is working/the stack can pass tempest19:09
Sam-I-Amalthough i also have a non-OA lab that might work19:10
mhaydenodyssey4me: last AIO ansible failure is peculiar -- it seems we have a new nodepool in bluebox's cloud?!19:11
odyssey4memhayden yep, another new provider is ovh19:12
Sam-I-Amhopefully they suck less19:13
mhaydenseems like bluebox might not have a lot of base disk space nor extra disks from what i see19:13
mhayden(initial look)19:13
*** javeriak has joined #openstack-ansible19:14
*** BjoernT is now known as Bjoern_zZzZzZzZ19:15
mhaydenit looks like bare-trusty will ensure we get on hp/rax19:17
*** Bjoern_zZzZzZzZ is now known as BjoernT19:18
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Gate: Make instance info output to stdout on failure
odyssey4memhayden ^ will help us find that out19:19
*** ysm has joined #openstack-ansible19:20
*** mancdaz has quit IRC19:25
odyssey4mecloudnull it seems to me that either we need to enable both l3ha and l2pop if the number of agents>1, and we need to disable l3ha if # agents == 119:25
Sam-I-Amwhy is adding a distro package to a bug in launchpad such a bitch19:25
Sam-I-Amodyssey4me: or test this before we do things?19:25
odyssey4meSam-I-Am they see you ridin', they hatin'19:25
Sam-I-Amlike disabling arp spoof protection?19:26
odyssey4meSam-I-Am eh?19:26
*** mancdaz has joined #openstack-ansible19:26
*** adaccada has quit IRC19:27
Sam-I-Ami thought we disable arp spoof protection by default in liberty because reasons19:27
Sam-I-Amsomething i think i was supposed to look into, but time and unicorns19:27
*** baker has quit IRC19:28
*** fawadkhaliq has quit IRC19:28
odyssey4meSam-I-Am it got disabled by default because it resulted in severe brokenness19:28
*** fawadkhaliq has joined #openstack-ansible19:28
odyssey4mewhile we could build, we could never pass a full tempest test19:28
Sam-I-Amwhich is odd, because i havent seen that here19:28
odyssey4meSam-I-Am you perhaps forget that I worked with you to cover every single avenue that time permitted19:29
odyssey4meI am happy to revisit that, but I can confidently tell you that it broke networking.19:29
odyssey4meNote also that there is no linuxbridge voted testing in the gate upstream.19:30
*** BjoernT is now known as Bjoern_zZzZzZzZ19:30
*** Bjoern_zZzZzZzZ is now known as BjoernT19:31
odyssey4meAlso it seemed to me at the time that I looked into it that upstream does not do floating address testing in the gate for integrated tests... which we do.19:31
odyssey4meThe blocker for upstream linuxbridge gate testing merged yesterday, finally. The next step will be to revisit the patch to make it voting.19:32
Sam-I-Amdo you recall which tests failed? i have fips with arp spoof in all of my labs and nothing obvious breaks19:32
*** shausy has quit IRC19:32
*** sc68cal has joined #openstack-ansible19:32
odyssey4methe tempest test for floating ip access failed19:32
*** niknakpaddywak has quit IRC19:32
Sam-I-Amodyssey4me: meet sc68cal19:32
Sam-I-Amwho is working on the lb gate job19:33
sc68calbear with me - let me read the eavesdrop log for this channel to catch up19:33
*** maximov_ has quit IRC19:33
*** etoews has quit IRC19:33
*** serverascode has quit IRC19:33
*** meteorfox has quit IRC19:33
odyssey4mehaha, sc68cal is the owner of :)19:33
odyssey4meit's time to revise the patch, as the dependent patch has merged - finally19:34
odyssey4memuch bike shedding was had19:34
Sam-I-Amsc68cal: tl;dr - we had to disable arp spoof protection (even with port_security ext) in liberty due to issues of some sort19:34
Sam-I-Amthis sort of came out of a 'does l3ha work with l2pop yet' discussion19:34
sc68calok I've caught up on today's scrollback19:35
sc68calodyssey4me: oh did it????19:35
sc68calPRAISE THE SUN19:35
odyssey4meyep, it is done - time for some rebase magic to prep for the next bike shed19:36
sc68calodyssey4me: ack. Let's let it cook for a bit and see what our pass rate gets to with that fix in19:37
sc68calsince reason I set it as a depends-on in the voting patch was so I don't get hit with a wall of hate19:37
sc68calwhen LB job starts failing at the gate19:37
odyssey4mesc68cal sure, happy to let it sit for a bit - that patch set should probably also be backported to liberty at least19:38
sc68calodyssey4me: good point19:38
*** baker has joined #openstack-ansible19:39
odyssey4mesc68cal so we found that arp spoofing broke our integrated gate check consistently19:40
odyssey4meour build uses linuxbridge19:40
sc68calodyssey4me: what's the job name?19:40
odyssey4mesc68cal which job - one that has arp spoofing enabled?19:41
*** niknakpaddywak has joined #openstack-ansible19:41
odyssey4meI can whip a review up quickly to re-test all that. I haven't tried it since just before Liberty release.19:42
sc68calodyssey4me: ok - but you said "our integrated gate check" - who is our19:42
odyssey4meour = OpenStack-Ansible19:42
odyssey4mesc68cal ie
*** adaccada has joined #openstack-ansible19:43
* sc68cal looks in project-config19:43
odyssey4mesc68cal lemme whip up a test review19:44
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: [WIP] Enable Neutron prevent_arp_spoofing
*** sdake has joined #openstack-ansible19:46
* sc68cal looks19:46
*** luckyinva has joined #openstack-ansible19:48
sc68calodyssey4me: ok I think I'm starting to follow now :)19:48
sc68calodyssey4me: is there a bug on the neutron side already for this issue? if not, and your patch comes back and explodes let's go ahead and make a bug on the neutron side to help track19:49
odyssey4mesc68cal I would be absolutely happy to orientate you as necessary. Ping me any time.19:49
sc68calSam-I-Am: thanks for the introduction19:49
*** sacharya has quit IRC19:49
odyssey4mesc68cal sure, can do that - and I'm happy to facilitate discussions between whichever parties to resolve some of the issues we pick up19:50
sc68calodyssey4me: cool. Once CI reports back on that patch we can use that as an artifact if we open a bug on the neutron side19:51
Sam-I-Amsc68cal: sure19:51
odyssey4mesc68cal grab a coffee, it'll take around an hour once it kicks off19:51
Sam-I-Am(or beer)19:52
Sam-I-Amodyssey4me: heard from the l3ha guy. l2pop and l3ha should work together in kilo and liberty, but use of l2pop and l3ha is not recommended until further performance refinements19:53
Sam-I-Amsc68cal: o-a is probably one of the larger deployment tools using lb19:55
Sam-I-Ammultinode-ish testing is already possible on a single node with affinity groups19:55
Sam-I-Amnow if only i can get my ovs testing done on it :)19:55
*** markvoelker has quit IRC19:55
odyssey4meSam-I-Am (cc cloudnull) so what I'm hearing is that we should enable l2pop and disable l3ha by default, and add some documentation around the known issue - is there a bug/blueprint for the improvements to fix performance?19:56
odyssey4meit'd be good to know at what scale this is an issue19:56
odyssey4meSam-I-Am if you can find a sponsor who cares about OVS, the patch is quite simple :) we just need someone who cares enough and understands OVS enough... because we don't19:57
Sam-I-Ami think its deployment-specific19:57
Sam-I-Amvlan-type networks can't use l2pop, so there's much less of an issue with l3ha there19:57
Sam-I-Amalso only applies to installs with project networks in the first place19:58
*** rackertom has quit IRC19:59
*** markvoelker has joined #openstack-ansible19:59
alextricityI literally just got done turning on l2_population on my mutli-neutron-server deployment20:01
alextricityI should of grabbed the error..20:01
odyssey4mealextricity your finding?20:02
alextricityWhen l2_pop was disabled..I was getting a port binding error20:02
alextricityI didn't really look into though20:02
* alextricity regrets it20:02
alextricityI just turned on l2_pop and it seemed to fix it20:02
odyssey4meyep, that's the same experience that was being discussed over the last few days20:02
*** sacharya has joined #openstack-ansible20:02
odyssey4mealextricity we need to definitively determine whether l2pop must be enabled when l3ha is enabled - it all relates to (see the review comments)20:05
*** BjoernT has quit IRC20:05
*** luckyinva has quit IRC20:05
alextricityI was looking at that earlier today. I was about to +1 one it until I saw your comments20:05
*** sdake has quit IRC20:05
alextricityThe changed seemed to fix my port binding issues20:06
alextricityI can look back to see what error I was getting exactly20:06
*** etoews has joined #openstack-ansible20:07
odyssey4meperhaps l3ha should only be enabled if #agents>1, but l2pop should always be enabled20:08
*** spotz is now known as spotz_zzz20:08
odyssey4mebut perhaps the decision should lie with the deployer - and those are just our defaults20:08
*** sdake has joined #openstack-ansible20:08
alextricity That'll definitely do it. However, l3ha still wouldn't be working in it's currently state, right? At least that's what i'm getting at after reading the patch set20:10
alextricitymodprobe: ERROR: could not insert 'ip_vs': Operation not permitted20:11
alextricityneutron-ha-tool: ERROR <html><body><h1>503 Service Unavailable</h1>#012No server is available to handle this request.#012</body></html>20:11
alextricityERROR neutron.plugins.ml2.managers [req-32a1ea1e-aedb-4528-a9c5-3dc17f751a65 - - - - -] Failed to bind port 20cc04e0-fdc3-40a8-a8e3-8023866427ff on host osa-infra3_neutron_agents_container-40a619db20:11
alextricityUnfortunately, the only thing that doesn't go over my head is the neutron-ha-tool 503 error20:14
*** sdake has quit IRC20:14
odyssey4mealextricity the ha tool only handles the non-ha routers, as I understand it20:15
*** serverascode has joined #openstack-ansible20:16
*** revdr has joined #openstack-ansible20:16
*** sigmavirus24 is now known as sigmavirus24_awa20:20
*** sdake has joined #openstack-ansible20:21
*** njohnston has joined #openstack-ansible20:21
*** meteorfox has joined #openstack-ansible20:29
*** javeriak has quit IRC20:32
*** k_stev has joined #openstack-ansible20:33
*** rackertom has joined #openstack-ansible20:34
Sam-I-Amodyssey4me: does it break if it sees an ha router?20:38
odyssey4meSam-I-Am it's been patched ot ignore ha routers20:38
*** hybridpollo has joined #openstack-ansible20:42
*** spotz_zzz is now known as spotz20:43
*** maximov_ has joined #openstack-ansible20:44
*** k_stev has quit IRC20:44
*** spotz is now known as spotz_zzz20:47
*** git-harry has quit IRC20:49
*** fawadkhaliq has quit IRC20:49
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Convert AIO bootstrap from bash to Ansible
*** git-harry has joined #openstack-ansible20:50
openstackgerritMajor Hayden proposed openstack/openstack-ansible-security: Check mode compatibility for security role
openstackgerritMajor Hayden proposed openstack/openstack-ansible-security: Updating tests for openstack-ansible-security
openstackgerritMajor Hayden proposed openstack/openstack-ansible-security: Adjusting commonly failing tasks
odyssey4memhayden I'm dropping the absolute AIO minimum space on / to 50GB, as that's what seems to be required for gate checks20:57
mhaydengood stuff20:57
*** baker has quit IRC21:00
*** phiche has quit IRC21:02
*** phiche has joined #openstack-ansible21:02
*** sigmavirus24_awa is now known as sigmavirus2421:03
*** Bjoern_ has joined #openstack-ansible21:06
*** adaccada has quit IRC21:19
*** spotz_zzz is now known as spotz21:30
*** Bjoern_ is now known as Bjoern_zZzZzZzZ21:30
*** Bjoern_zZzZzZzZ is now known as Bjoern_21:33
openstackgerritMerged openstack/openstack-ansible: Add documentation for HA ceilometer
*** galstrom is now known as galstrom_zzz21:36
*** dolphm is now known as dolphm_41821:38
*** ctina_ has joined #openstack-ansible21:45
*** Bjoern_ is now known as Bjoern_zZzZzZzZ21:49
*** ctina has quit IRC21:49
*** Bjoern_zZzZzZzZ is now known as Bjoern_21:49
*** ctina_ has quit IRC21:50
*** loquacities has joined #openstack-ansible21:52
*** Bjoern_ has quit IRC21:54
*** KLevenstein has quit IRC21:54
*** Bjoern_ has joined #openstack-ansible21:56
odyssey4mealextricity please backport to liberty21:58
odyssey4meSam-I-Am sc68cal and would you look at that, passed the gate check :)22:01
alextricityodyssey4me: Sure22:01
odyssey4mealextricity good patch :)22:02
openstackgerritMiguel Alex Cantu proposed openstack/openstack-ansible: Add documentation for HA ceilometer
odyssey4mestevelle FYI, not sure if you're aware of this but alextricity has contributed most of the Ceilometer related stuff into OSA so far. If you haven't met before, consider this an intro. :)22:03
odyssey4methanks alextricity for yet another good patch :)22:03
odyssey4memcarden welcome :)22:05
odyssey4meiceyao ping?22:05
*** Mudpuppy has quit IRC22:06
*** phiche has quit IRC22:07
mcardenodyssey4me: Thanks22:11
cloudnullodyssey4me: sorry was in meetings for the most of the day22:14
*** baker has joined #openstack-ansible22:24
openstackgerritJesse Pretorius proposed openstack/openstack-ansible: Use PyPi packages for ceph python bindings
*** mancdaz has quit IRC22:30
openstackgerritKevin Carter proposed openstack/openstack-ansible: FIX: provider_networks module for  multiple vlans
*** mancdaz has joined #openstack-ansible22:31
*** eil397 has quit IRC22:38
*** Bjoern_ is now known as BjoernT22:40
stevellewe've talked a couple of times, I believe alextricity22:43
*** mss_ has joined #openstack-ansible22:45
*** mss has quit IRC22:48
*** sacharya_ has joined #openstack-ansible22:49
*** eil397 has joined #openstack-ansible22:49
*** rromans has quit IRC22:52
*** sacharya has quit IRC22:52
*** njohnston has quit IRC22:53
*** hybridpollo has quit IRC22:55
odyssey4memhayden so it appears that we have some flavours in openstack-ci with 75G disks now:
*** rromans has joined #openstack-ansible22:56
odyssey4meso that's one more issue down - now to try and figure out the next issue22:57
linggaoHi mrda22:57
odyssey4memhayden the last check failed with a build timeout - waiting for 15mins for the swift proxy container to be alive :/22:58
*** hybridpollo has joined #openstack-ansible22:59
linggaoodyssey4me, do you when is best time to talk to mrda?23:00
odyssey4melinggao I expect that he's alive - not sure whether he's stuck in meetings or something23:00
linggaook. I will try later. thanks.23:01
odyssey4melinggao do you have a running openstack-ansible setup?23:01
linggaoodyssey4me, yes a all-in-one23:01
odyssey4melinggao from what I understand, ironic needs ipmi access to host bmc devices... this is usually the blocker for testing23:02
odyssey4meand this is an issue we're trying to work out23:02
odyssey4mewe want to be able to gate test it, and this is not easy23:03
odyssey4meideally we need a sponsor for some sort of external CI I think23:03
kysseOur galera gluster went borken, cluster size is now 1. We booted galera nodes, first node 1 and after it came up we booted node 2.23:04
kyssewhat to do? Only node 2 is up and running now, when I launch node 1 it says like 10 secs that cluster size is 2, and then it fails.23:05
linggaoodyssey4me, I can help with the initial test if needed. I have been working on ironic. But I do not think I have time to CI, setting up a CI takes time. We did it for ironic before.23:06
odyssey4melinggao ah, so you know ironic then - that helps from a review standpoint23:09
linggaoodyssey4me, yes.23:11
sc68calodyssey4me: it's a christmas miracle23:11
sc68calodyssey4me: I know that there was some work done recently on arp spoofing - I just don't remember if it was OVS only or LB only23:12
*** sigmavirus24 is now known as sigmavirus24_awa23:12
odyssey4mesc68cal this is why we can't have nice things :p23:12
odyssey4mesc68cal with a voting lb gate, we could have nice things :)23:12
sc68calodyssey4me: haha - ok ok I'll rebase the patch tomorrow. promise23:13
odyssey4mesc68cal I'll buy you a C.H.I.P. if you get it voted through by the end of the week. :)23:15
* Sam-I-Am appear23:16
sc68calodyssey4me: no idea what a CHIP is - is it beer?23:17
Sam-I-Amsc68cal: its a thing that goes well with beer23:17
*** baker has quit IRC23:18
odyssey4mesc68cal - with beer, it runs openstack23:18
* sc68cal googles23:18
sc68caloh. I already have 2 RasPis23:18
sc68calI'll take a $3 beer at the summit - that saves you 5 DOLLARS!23:18
odyssey4mesc68cal and now you could have diversity :)23:19
Sam-I-Amwith beer anything runs openstack23:19
odyssey4mesc68cal I'll buy you a scotch, or a beer as a fall back.23:19
sc68calodyssey4me: deal. :)23:19
*** ysm has quit IRC23:20
sc68calSam-I-Am: we need to make that a success thingy that the IRC bot picks up23:20
*** baker has joined #openstack-ansible23:21
Sam-I-Amsc68cal: ha, didnt know that existed23:21
Sam-I-Ami know cloudnull wrote a successerator function23:21
*** baker has quit IRC23:21
stevellejust saw it used over in #openstack-glance23:22
stevellepretty neat23:22
odyssey4mehow does it work? #magic ?23:22
odyssey4me#success I just worked out how to make successbot work! :)23:24
openstackstatusodyssey4me: Added success to Success page23:24
Sam-I-Amsuccess - open source didnt kill me today23:25
Sam-I-Amthen again no one could agree on a patch to accomplish that task :)23:25
odyssey4mealright, on that note - I'm out for the night23:25
Sam-I-Amodyssey4me: scotch23:26
odyssey4meSam-I-Am that'd be PS4-ish :)23:27
*** jaypipes has quit IRC23:38
*** eil397 has quit IRC23:40
*** eil397 has joined #openstack-ansible23:45
*** sacharya_ has quit IRC23:49
*** sacharya has joined #openstack-ansible23:50
*** spotz is now known as spotz_zzz23:51
*** dolphm_418 is now known as dolphm23:57

Generated by 2.14.0 by Marius Gedminas - find it at!