Thursday, 2015-09-10

*** markvoelker has quit IRC00:01
*** CheKoLyN has joined #openstack-ansible00:14
*** Mudpuppy has joined #openstack-ansible00:14
*** shoutm has quit IRC00:17
*** shoutm has joined #openstack-ansible00:20
*** Mudpuppy has quit IRC00:25
*** CheKoLyN has quit IRC00:29
*** Mudpuppy has joined #openstack-ansible00:32
*** darrenc_afk is now known as darrenc00:32
*** CheKoLyN has joined #openstack-ansible00:42
*** Mudpuppy has quit IRC00:42
openstackgerritMerged stackforge/os-ansible-deployment: Set the Ubuntu mirror used based on the environment  https://review.openstack.org/22166600:43
*** Mudpuppy has joined #openstack-ansible00:49
openstackgerritMerged stackforge/os-ansible-deployment: Updated the configuration to current dynamic inventory standards  https://review.openstack.org/22185400:52
*** markvoelker has joined #openstack-ansible00:58
*** alop has quit IRC01:00
*** Mudpuppy has quit IRC01:01
*** markvoelker has quit IRC01:02
*** abitha has quit IRC01:04
*** tlian2 has joined #openstack-ansible01:09
*** Mudpuppy has joined #openstack-ansible01:12
openstackgerritKevin Carter proposed stackforge/os-ansible-deployment: Set affinity to 1 if running on hpcloud  https://review.openstack.org/22195701:12
*** tlian has quit IRC01:13
openstackgerritKevin Carter proposed stackforge/os-ansible-deployment: Set affinity to 1 if running on hpcloud  https://review.openstack.org/22195701:15
*** mrstanwell has quit IRC01:16
*** Mudpuppy has quit IRC01:28
*** Mudpuppy has joined #openstack-ansible01:31
*** Mudpuppy has quit IRC01:49
*** Mudpuppy has joined #openstack-ansible01:56
openstackgerritMerged stackforge/os-ansible-deployment: Fix keystone shibboleth template to detect ssl usage correctly  https://review.openstack.org/22134702:04
openstackgerritMerged stackforge/os-ansible-deployment: Be more clear about used_ips, mostly in the example file  https://review.openstack.org/22171302:11
*** CheKoLyN has quit IRC02:24
*** Mudpuppy has quit IRC02:25
*** tlian2 has quit IRC02:26
*** alejandrito has joined #openstack-ansible02:28
*** Mudpuppy has joined #openstack-ansible02:37
*** arbrandes has quit IRC02:43
*** Mudpuppy has quit IRC02:47
*** arbrandes has joined #openstack-ansible02:52
*** markvoelker has joined #openstack-ansible02:55
*** sdake_ has joined #openstack-ansible02:56
*** Mudpuppy has joined #openstack-ansible02:57
*** sdake__ has joined #openstack-ansible02:57
*** sdake has quit IRC02:58
*** sdake_ has quit IRC03:01
*** Mudpuppy has quit IRC03:07
*** alejandrito has quit IRC03:10
*** alextricity_h has quit IRC03:14
*** dolphm has quit IRC03:27
*** sigmavirus24 has quit IRC03:27
*** eglute has quit IRC03:27
*** d34dh0r53 has quit IRC03:27
*** palendae has quit IRC03:27
*** d34dh0r53 has joined #openstack-ansible03:28
*** Mudpuppy has joined #openstack-ansible03:28
*** eglute has joined #openstack-ansible03:28
*** dolphm has joined #openstack-ansible03:28
*** palendae has joined #openstack-ansible03:29
*** sdake__ is now known as sdake03:30
openstackgerritMerged stackforge/os-ansible-deployment: Adds the ability to provide user certificates to HAProxy  https://review.openstack.org/22166703:30
openstackgerritMerged stackforge/os-ansible-deployment: Fixes jinja typo in keystone-httpd.conf.j2  https://review.openstack.org/22197703:30
*** sigmavirus24 has joined #openstack-ansible03:32
*** palendae has quit IRC03:36
*** palendae has joined #openstack-ansible03:37
*** Mudpuppy has quit IRC03:38
*** sigmavirus24 is now known as sigmavirus24_awa03:39
openstackgerritMerged stackforge/os-ansible-deployment: Allow testing of cinder multi backends w/ tempest  https://review.openstack.org/22185103:51
*** markvoelker has quit IRC04:55
*** sdake_ has joined #openstack-ansible05:12
*** sdake has quit IRC05:15
*** sdake has joined #openstack-ansible05:15
*** sdake_ has quit IRC05:16
*** sdake__ has joined #openstack-ansible05:16
*** sdake__ has quit IRC05:17
*** sdake_ has joined #openstack-ansible05:18
*** sdake has quit IRC05:20
*** sdake_ is now known as sdake05:21
*** shausy has joined #openstack-ansible05:28
*** sdake_ has joined #openstack-ansible05:29
*** sdake has quit IRC05:30
*** shausy has quit IRC05:32
*** markvoelker has joined #openstack-ansible05:56
*** markvoelker has quit IRC06:01
*** sdake has joined #openstack-ansible06:08
*** sdake_ has quit IRC06:11
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Add galera_address to all playbooks installing galera_client  https://review.openstack.org/22130106:27
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Set client versions deployed to use global requirements  https://review.openstack.org/22005006:27
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Enabling log compression by default  https://review.openstack.org/22192406:27
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Install nfs-common with nova-compute  https://review.openstack.org/22184406:28
*** sdake has quit IRC06:28
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Updated MariaDB to the new release version (10.0)  https://review.openstack.org/17825906:28
*** sdake has joined #openstack-ansible06:28
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Changed certificate order for pem generation with CA files  https://review.openstack.org/22168206:31
*** mcarden has quit IRC06:40
*** javeriak has joined #openstack-ansible07:30
*** javeriak has quit IRC07:37
*** javeriak has joined #openstack-ansible07:38
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Shift irqbalance package from lxc_hosts to openstack_hosts  https://review.openstack.org/21835407:41
*** javeriak has quit IRC07:53
*** markvoelker has joined #openstack-ansible07:57
*** bapalm has quit IRC07:58
*** jmccrory has quit IRC07:59
*** shoutm_ has joined #openstack-ansible08:00
*** bapalm has joined #openstack-ansible08:00
evrardjpgood morning everyone08:01
*** javeriak has joined #openstack-ansible08:01
*** shoutm has quit IRC08:01
*** markvoelker has quit IRC08:02
*** jmccrory has joined #openstack-ansible08:03
*** javeriak has quit IRC08:09
*** javeriak has joined #openstack-ansible08:10
*** sdake has quit IRC08:17
*** javeriak_ has joined #openstack-ansible08:33
*** javeriak has quit IRC08:35
odyssey4meo/ evrardjp08:35
*** javeriak has joined #openstack-ansible08:38
*** javeriak_ has quit IRC08:39
evrardjpquick question: I'd like to discuss about the inventory approach for keepalived component, what's the best way to do it? Have a point in open topics in the meeting today?08:39
odyssey4meevrardjp yes, that would be a good way to do it - perhaps add it to the reviews agenda item08:42
*** javeriak has quit IRC08:42
odyssey4memattt hughsaunders git-harry there was quite a bit of discussion in-channel regarding https://review.openstack.org/202822 yesterday, but none of those views are recorded in the review - please can you review the patch and add your thoughts accordingly08:43
*** javeriak_ has joined #openstack-ansible08:46
*** gparaskevas has joined #openstack-ansible08:48
evrardjpodyssey4me: done08:52
*** markvoelker has joined #openstack-ansible08:58
*** markvoelker has quit IRC09:02
*** jmccrory has quit IRC09:04
*** shoutm has joined #openstack-ansible09:05
*** jmccrory has joined #openstack-ansible09:06
*** javeriak has joined #openstack-ansible09:07
*** shoutm_ has quit IRC09:07
*** javeriak_ has quit IRC09:09
hughsaundersgit-harry: you should comment on https://review.openstack.org/20282209:12
*** jmccrory has quit IRC09:12
*** jmccrory has joined #openstack-ansible09:15
*** javeriak_ has joined #openstack-ansible09:24
*** javeriak has quit IRC09:24
*** javeriak_ has quit IRC09:32
odyssey4mehughsaunders I agree with what you've said, but I think that perhaps a revisit of what's being done here should be a proposal for a new role in a subsequent patch/09:36
odyssey4meactually, this is a perfect opportunity for a new blueprint/spec to create something like a rabbitmq-admin role which the service playbooks can consume09:37
*** javeriak has joined #openstack-ansible09:39
hughsaundersI'll see what cloudnull says first09:43
*** javeriak has quit IRC09:49
*** javeriak has joined #openstack-ansible09:50
openstackgerritMerged stackforge/os-ansible-deployment: Shift irqbalance package from lxc_hosts to openstack_hosts  https://review.openstack.org/21835409:52
odyssey4meevrardjp can you please backport https://review.openstack.org/221713 to kilo?10:01
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Fixes jinja typo in keystone-httpd.conf.j2  https://review.openstack.org/22197410:03
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Fixes jinja typo in keystone-httpd.conf.j2  https://review.openstack.org/22197410:04
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Fixes jinja typo in keystone-httpd.conf.j2  https://review.openstack.org/22197410:05
odyssey4memattt can you backport https://review.openstack.org/221851 to kilo please10:06
odyssey4meunless that's not an intention for the patch?10:06
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Shift irqbalance package from lxc_hosts to openstack_hosts  https://review.openstack.org/22210410:07
matttodyssey4me: i can, i wasn't sure if that was necessary tho since it's effectively introducing a new feature10:15
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Set affinity to 1 if running on hpcloud  https://review.openstack.org/22195710:17
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Set affinity to 1 if running on hpcloud  https://review.openstack.org/22195710:19
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Set affinity to 1 if running on hpcloud  https://review.openstack.org/22195710:19
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: More changes to neutron db migrations  https://review.openstack.org/21975910:20
odyssey4memattt I think it should be backported - it's a new testing feature and ideally I'd like master and kilo to have the same testing capabilities10:22
odyssey4meit's not a customer facing feature10:22
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: Allow testing of cinder multi backends w/ tempest  https://review.openstack.org/22211910:27
openstackgerritJean-Philippe Evrard proposed stackforge/os-ansible-deployment: Be more clear about used_ips, mostly in the example file  https://review.openstack.org/22212110:29
evrardjpwhat's the total scale of the solution? I have the feeling it's starting to be a pain to deploy when you start to have more than 4 controllers10:31
evrardjpsorry, not really intended to be here10:32
*** javeriak has quit IRC10:43
odyssey4meevrardjp so the primary design it was built around was with 3 controllers10:52
odyssey4meas I recall, Rabbit is the biggest bottleneck due to the way the clustering works and the way we're using it10:52
odyssey4meevrardjp https://review.openstack.org/202822 is set to improve that10:53
odyssey4mehowever, it'd be good to know what your experience is and how we could possibly improve things10:53
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: More changes to neutron db migrations  https://review.openstack.org/21975910:53
*** markvoelker has joined #openstack-ansible10:59
*** markvoelker has quit IRC11:04
*** shoutm has quit IRC11:24
openstackgerritHugh Saunders proposed stackforge/os-ansible-deployment: Install logrotate with ryslog  https://review.openstack.org/22214311:25
*** shoutm has joined #openstack-ansible11:26
evrardjpodyssey4me: I agree, but if I want to go in production soon, I'd avoid messing up with these "not tested" ways ;)11:37
evrardjpI'll downscale then11:37
evrardjpI'll test the 4 controllers later11:39
cloudnullmorning11:54
*** markvoelker has joined #openstack-ansible11:55
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: More changes to neutron db migrations  https://review.openstack.org/21975912:00
mgariepygood morning everyone12:02
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: More changes to neutron db migrations  https://review.openstack.org/21975912:07
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: More changes to neutron db migrations  https://review.openstack.org/21975912:14
mhaydenmorning12:31
mhaydenand good afternoon mattt and odyssey4me12:31
matttmhayden: morning2u :)12:37
*** woodard has joined #openstack-ansible12:48
*** woodard has quit IRC12:49
*** woodard has joined #openstack-ansible12:50
evrardjpgood morning cloudnull and mhayden12:50
cloudnullo/ evrardjp12:52
cloudnullodyssey4me: https://review.openstack.org/#/q/status:open+project:stackforge/os-ansible-deployment+branch:kilo,n,z looks like kilo is having a wheel build issue. im testing now will see whats what in a min12:53
*** javeriak has joined #openstack-ansible12:57
cloudnullgit-harry hughsaunders odyssey4me - RE: https://review.openstack.org/#/c/202822/  - My stance remains unchanged. However, If the "community" wants to seperate these tasks into other roles, or just tasks within a given play, I'm fine with that. You guys let me know and I'll update the review accordingly.12:59
hughsaundersmorning cloudnull13:01
odyssey4meo/ mgariepy cloudnull mhayden13:03
odyssey4mecloudnull yeah, I was about to figure out the wheel build issue, but happy that you're on it13:04
hughsaundersI think it would be useful to keep the ability to use an external rabbit. As you and git-harry have said, duplication is a wider issue and can't be addressed in a review discussion.13:04
odyssey4mehughsaunders cloudnull I'd like us to table a discussion at the summit around opportunities for managing technical debt which should include figuring out a 'standard' or 'policy' regarding where another role should or shouldn't be used13:05
odyssey4mepersonally I think the change is fine for now as it keeps with the current conventions, but I think we do need to discuss and agree on whether the existing conventions are good13:06
matttagreed, this change is in line w/ how we're doing things now, but i do think it begs some discussion as to how we do this going forward13:09
odyssey4mecloudnull fyi https://review.openstack.org/221957 will not change the hpcloud-b4 build failures... those are not resource related, they appear to be networking related as the issue is that the apt get upgrade/installs can't successfully complete13:09
cloudnullodyssey4me:  on https://review.openstack.org/221957 last night I was seeing lots of build failures in hpb.* so i made that, its not quite right but I wanted to see resource usage w/ affinity=1 vs not13:13
odyssey4memattt hughsaunders I've added my feedback in the review and encourage you to do the same.13:13
odyssey4mecloudnull yeah - I adjusted it this morning, but figured that it might be a tester :) I've done this before as well, but didn't find it to make a significant difference at the time.13:14
openstackgerritgit-harry proposed stackforge/os-ansible-deployment: Remove unused Juno-specific groups from inventory  https://review.openstack.org/22218613:15
cloudnullon https://review.openstack.org/#/c/202822 would it be better for this discussion to move the tasks into the play indstead of the role ? would that garner more agreement in implementation?13:16
odyssey4mecloudnull where I am seeing it make a difference is in the success of the tempest tests13:16
odyssey4mecloudnull I think that'd be a bad idea, personally13:16
evrardjpgood morning mgariepy! I've changes for you, and I guess a possible discussion to bring room for improvements :)13:17
odyssey4mepersonally I think that there is room to have a role which handles the rabbitmq administration bits separately from the rabbitmq setup13:17
odyssey4mecloudnull ie the openstack service roles should be built with the assumption that mysql and rabbitmq databases/vhosts/users/etc are already setup when they run13:18
odyssey4methe playbooks, however, should orchestrate the creation of the vhosts, users, databases, etc through the use of a role that sets them up after setting up the database13:19
odyssey4medoes that make sense?13:19
*** pradk has joined #openstack-ansible13:19
evrardjpideally it's not even about that... it's about letting your customers to work the way they want...13:19
odyssey4mein other words, it's not a role dependancy - it's a playbook orchestration13:19
evrardjpbecause they should know better what they need13:19
odyssey4meevrardjp exactly13:19
odyssey4methis way we are more loosely coupled, and the playbooks do the coupling13:20
mgariepyevrardjp, I agree with not stopping haproxy13:20
odyssey4methis would also mean that the rabbitmq cluster could be externally setup and managed13:20
odyssey4mesame with galera13:20
evrardjpmgariepy: ok. stopping backends or not?13:20
odyssey4meand I'm sure there are other opportunities for this13:20
evrardjpI don't think that would be smart13:20
mgariepywe just need to make sure it's running, if not mark keepalive as faulted.13:21
evrardjpit's kinda error prone, and easy to miss for the lambda administrator13:21
evrardjpmgariepy: I agree13:21
evrardjpI've done my change on the galaxy role13:21
mgariepyfor the first iteration i guess kiss is better.13:21
evrardjpmgariepy: I agree13:21
odyssey4mecloudnull does that make sense?13:22
odyssey4mecloudnull note, though, that I don't think this method refactor needs to be done in that patch - it's probably better suited to a follow on patch13:22
mgariepywould be nice to have email sent when state change also haha13:23
evrardjpthat's something you could do13:23
evrardjpwithout even changing the notify script13:23
mgariepyyeah it'13:24
mgariepys keepalived config.13:24
evrardjpoh btw, I removed the logging part13:24
evrardjpfor the next iteration13:24
evrardjpbecause most of the important is already logged in syslog13:24
evrardjpmost of the important events*13:25
* cloudnull was breaking the fast13:26
cloudnullodyssey4me:  makes sense. just wnated to give everyone a chance to weigh in with the hope that more  folks would also comment on the review .13:27
cloudnullalso on https://review.openstack.org/#/c/221957/ resource reduction seems to make a difference - HPB2. SUCCESS in 1h 30m 07s which is just under the build timeout wire :)13:28
cloudnullbut I do agree with mattt kinda in that we may not catch some things especially in cases where there are code paths that execute on N+113:29
*** tlian has joined #openstack-ansible13:32
*** vdo has joined #openstack-ansible13:35
matttcloudnull: i'm +2, except had a question before I actually +2 :)13:38
*** shoutm has quit IRC13:55
*** Mudpuppy has joined #openstack-ansible14:01
openstackgerritChristopher H. Laco proposed stackforge/os-ansible-deployment: Enable log_slave_updates  https://review.openstack.org/22219814:02
*** cloudtrainme has joined #openstack-ansible14:03
*** sigmavirus24_awa is now known as sigmavirus2414:03
*** KLevenstein has joined #openstack-ansible14:04
*** shoutm has joined #openstack-ansible14:07
evrardjphughsaunders: I tried to answer your questions on https://review.openstack.org/#/c/214089/14:08
evrardjpif it's still not clear, I'll rephrase14:08
hughsaundersevrardjp: thanks, will have a look14:08
evrardjpI'm not sure if I'm clear ;)14:08
evrardjpunderstandable*14:09
*** javeriak has quit IRC14:10
*** spotz_zzz is now known as spotz14:12
evrardjpcloudnull: if you don't mind I added a point to the agenda for one review, but it's more a discussion about already discussed topic, so I guess you can push me later if required... this item shouldn't last more than 5 minutes14:12
cloudnulli dont mind at all. thanks evrardjp :)14:13
*** javeriak has joined #openstack-ansible14:17
javeriakhello folks, need a little help trying to deploy from master, so I can't figure out where the repo-hosts are set up. The configure-hostlist.rst doc refers to a '/etc/openstack_deploy/user_group_vars.yml' which is actually 'playbooks/inventory/group_vars/hosts.yml', am I missing something here?14:23
javeriakand the 'openstack_repo_url' parameter in that file points to my deploy node, which isn't hosting anything, which brings me back to my original question, where is the repo hosting installation done in the plays?14:25
*** Mudpuppy has quit IRC14:25
matttjaveriak: repo-install.yml gets called in setup-infrastructure.yml14:27
*** Mudpuppy has joined #openstack-ansible14:29
*** agireud has joined #openstack-ansible14:29
*** Mudpuppy has quit IRC14:30
matttjaveriak: does that answer your question or did it side-step entirely?  :)14:30
*** Mudpuppy has joined #openstack-ansible14:30
javeriakmattt yes and no :). problem is the pip_install task fails during setup-host.yml because the repo its looking for doesn't exist,14:31
palendaejaveriak: Ah, yeah...I don't know that the repo servers are set up by then.14:33
evrardjpthey aren't14:33
evrardjpoh wait14:33
palendaeThe openstack_repo_url option can be left out if you specify the repo servers14:34
*** alextricity has joined #openstack-ansible14:34
javeriakWill it work if I point it to "https://rpc-repo.rackspace.com/" ?14:34
evrardjpthe repo doesn't exist or some packages are not yet up to date?14:34
javeriakevrardjp, it doesnt exist at all, i mean nothings being hosted14:35
palendaeif you specify repo-infra_hosts, the repos will be made, populated from rpc-repe.rackspace.com14:35
palendaeI don't remember exactly, but I think it needs to be the os-releases diir14:36
palendaehttps://rpc-repo.rackspace.com/os-releases/14:36
evrardjpjaveriak: I'm not sure why you need that at that point14:36
javeriakpalendae, yes but if that happens in the setup-infrastructure.yml, how are the previous installations supposed to work since nothing is up yet14:36
evrardjpduring setup hosts you setup lxc IIRC14:36
evrardjpand other stuff on your hosts14:36
evrardjpyeah, I'm not sure you need that... what's the part of the play that fails?14:37
palendaejaveriak: That's a great question...I'm not sure I have an answer. Maybe cloudnull does, but I think you need a repo server already up if you're specifying openstack_repo_url14:37
palendaeIf you want repo servers to be built, define the repo-infra_hosts in user_config14:38
evrardjpI'm not sure, but I don't think it should read it during the setup-hosts14:38
palendaeAnd leave openstack_repo_url undefined14:38
evrardjpI'll check14:38
javeriakevrardjp, it tries to install pip during setup hosts, this one fails [pip_install | Install PIP (fall back mode)]14:38
matttjaveriak: now that you mention it, i thought i did see a failure when i ran a deploy last, but it didn't fail the run14:39
javeriakmatt: it wont fail if it finds pip already there from a previous install, but as it happens one of my machines doesnt have it, so it tries to contact the repo14:40
matttjaveriak: ah i see14:40
palendaejaveriak: Are you trying to avoid building local repos?14:40
evrardjplxc_hosts dependencies pip install ><14:40
javeriakpalendae, nope I defined the repo-infra_hosts, same as my infra_hosts14:41
palendaeOk14:41
cloudnullpalendae:  whats the issue ?14:41
palendaeMake sure you also undefine openstack_repo_url14:41
javeriakwas i supposed to remove "openstack_repo_url" too?14:41
cloudnullo/ javeriak :)14:41
palendaejaveriak: Yeah, that takes precedence14:41
* cloudnull reading14:41
*** shoutm has quit IRC14:42
cloudnullpip should attempt to install using the repo infra for a locked down version of pip however if its not there the options are removed and the pip install simply installs the upstream version.14:44
cloudnullwhich on a greenfield deployment should be ok as what is used from the upstream install is also the thing used within the initial repo build process.14:45
palendaecloudnull: But if you define openstack_repo_url to a place that doesn't exist or doesn't contain the repo dir structure, it'll fail, no?14:45
evrardjpI think it should work fine14:45
cloudnullit should be fine14:46
cloudnullthat variable is used in the following two places14:46
cloudnullhttp://cdn.pasteraw.com/hekgwt1uge4csanckyrktgnzx38vnkf14:46
javeriakcloudnull palendae, yes its trying to install through fall back mode, no options, but the error I get is still: RetryError: HTTPConnectionPool(host='10.22.37.149', port=8181): Max retries exceeded with url: /os-releases/11.2.2/ (Caused by ResponseError('too many 503 error responses',))14:47
javeriaknothings being hosted on 10.22.37.149 (my deploy)14:47
javeriakits picking that up from pip.conf  I believe14:48
cloudnulldo your hosts have the file ~/.pip/pip.conf that is locking pip down to that adress.14:48
palendaeAh, your pip.conf probably didn't get updated after removing the openstack_repo_url var14:49
cloudnullso maybe the Install PIP (fall back mode) task needs to run with the --isolated flag to ignore pip.conf ?14:50
javeriakcloudnull yep its there, i guessing it doesnt get overwritten on a new install, I see an entry to https://mirror.rackspace.com/rackspaceprivatecloud/python_packages/10.1.11/ from my prev install14:50
evrardjpquestion, not related at all... Your rackspace maas is updated for kilo? Is it upstreamed somewhere?14:50
cloudnulljaveriak: previous meaning Juno ?14:50
javeriakcloudnull, yes juno14:50
cloudnullevrardjp: https://github.com/rcbops/rpc-openstack14:51
javeriak10.1.11 to be precise14:51
evrardjpthanks14:51
cloudnullthe plugins are here: https://github.com/rcbops/rpc-openstack/tree/master/maas14:51
cloudnullmaas roles are here https://github.com/rcbops/rpc-openstack/tree/master/rpcd/playbooks/roles/rpc_maas14:51
cloudnulljaveriak:  this is an upgrade issue sadly.14:52
cloudnullwe have an upgrade script that does the translation from juno (rax technical debt) to kilo (community osad)14:52
cloudnullhttps://github.com/stackforge/os-ansible-deployment/blob/master/scripts/run-upgrade.sh#L600-L60414:52
cloudnullthats the line where thats resolved.14:52
cloudnullstructurally there are quite a few changes between juno and kilo (master as well) and so we've taken on the job of translating all of those changes for upgrade.14:54
javeriakcloudnull, ah okay, yes so it works by specifying --isolated or taking out the local repo entry in pip.conf, so we do need to un-specify the openstack_repo_url while specifying repo-hosts ? shouldnt that be put in the doc somewhere?14:54
cloudnullyou shouldnt need to set openstack_repo_url14:55
cloudnulli think it may be a good idea to add the --isolated flag to the fall back mode for pip install.14:55
javeriakalso this: https://github.com/stackforge/os-ansible-deployment/blob/master/doc/source/install-guide/configure-hostlist.rst needs to be updated with the correct path for this variable14:55
cloudnullthat too14:56
javeriakno the variable is already there by default14:56
javeriakwould you like me to take care of it? I can push a patch14:56
cloudnullthatd be awesome !14:56
cloudnullif you have the cycles to do it , I would appreciate it :)14:56
javeriaksure, we should comment out that variable by default then as well, or ask the deployer to do it explicitly in the example user config14:58
palendaeIs it not commented?14:58
javeriaknope: https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/inventory/group_vars/hosts.yml14:59
palendaeI guess stepping even further back15:00
palendaeWhat exactly are you trying to do? Upgrade Juno to Kilo?15:00
javeriakpalendae, not exactly, but I didnt remove all of the juno config before starting, only the previous containers and some main directories15:04
cloudnullso its a rekick but there was an existing deployment15:08
cloudnullwhich was torn down ?15:08
cloudnulli assume ?15:09
javeriakyes15:10
*** jwagner_away is now known as jwagner15:10
openstackgerritMerged stackforge/os-ansible-deployment-specs: Move specs to appropriate cycle folders  https://review.openstack.org/22092915:10
javeriakso I should've run the upgrade script then I think? What exactly is the use case for it, if you have an up and running juno setup, and you want to move to kilo? You run the script and then run all new playbooks15:11
cloudnullthe upgrade script will take you from juno to kilo wholesale15:24
cloudnullwhich also does the clean up15:24
cloudnullbut in your case it seems that you were doing a greenfield deployment on hosts that were used in the past.15:25
cloudnullwhich just needed a bit of clean up before running the new deployment.15:25
javeriakokay cool, I'll just push a fix for the doc paths, and comment out the openstack_repo_url in another,15:31
javeriakshould we add the isolated flag to the fall back pip install as well?15:31
*** jwagner is now known as jwagner_away15:33
cloudnulli think that might be best.15:33
odyssey4mejaveriak you may ba better off archiving your old config and starting fresh15:33
odyssey4meseveral of your old configs will still work, but you'll need to consider whether each of the items you had should be there now, and whether they have been renamed or not15:34
odyssey4methe upgrade script took care of all that for you, but you blew your environment away :p15:34
palendaeHm15:36
palendaeActually, I don't think openstack_repo_url should be commented out by default15:36
palendaeI'm probably misremembering a bunch of this15:36
javeriakodyssey4me: true :),  I should probably clean it up properly. I don't require any of the old configs, I'm interested in a fresh install anyway. Shouldnt have taken the quicker way15:36
palendaeBut now that I see it's the internal LB by default, I'm pretty sure that's necessary15:36
palendaejaveriak: Yeah, I thought you were doing a 100% clean install15:37
palendaeI think I was wrong; if you completely remove openstack_repo_url, nothing will be able to find the repo servers you define15:38
*** alop has joined #openstack-ansible15:38
javeriakpalendae, yes but why is it the internal LB, where are we setting that up as a repo? I havent looked through the infrastructure setup plays yet, but I assume they overwrite this to the ones you define as repo hosts15:38
palendaeSo should probably clarify that pointing to the internal LB VIP is needed for the repo-shared_infra hosts15:38
javeriakoh okay I see, it points to the LB which redirects it to the actual repo hosts..15:39
palendaeRight15:39
palendaeWe don't overwrite the variable15:39
palendaeThe repo hosts themselves ignore tha var15:39
palendaeAnd pull from upstream15:40
palendaeEverything else points to the LB15:40
javeriakits not really being used anywhere except for the pip-lock down file15:40
palendaeRight, which basically locks down everything to the internal repo15:40
javeriakokay I'll leave the variable to what it is, just add --isolated to the pip install so that initial installs dont fail15:41
palendaeI've not seen this behavior in pure greenfield deploys15:41
*** mrstanwell has joined #openstack-ansible15:41
javeriakwell it failing makes sense, the repo isnt there yet, so it cant work. But dunno why it hasnt been seen before. Let me check what happens on a new install and let you guys know if anything happens15:43
*** arbrandes has quit IRC15:44
cloudnullthanks javeriak15:46
odyssey4mejaveriak if you're doing a fresh install on hosts that have been used for previous builds then you'll need to do some clean-up15:49
odyssey4merm -rf the /root/.pip directory, for one15:49
odyssey4meI'm not sure if the teardown script might be useful for you?15:50
javeriakodyssey4me, i had a local cleanup playbook thats probably missing some items, now im just going to do a fresh ubuntu15:51
javeriakyes i used the teardown script as source :)15:51
odyssey4mejaveriak it takes longer, but is probably the best action to save some of your hair :p15:51
palendaeThe teardown script has missed stuff too15:52
javeriakyes hair is important :p, i need to go configure my cluster with maas now first15:52
palendaeThings get added to the project, but not to that15:52
*** gparaskevas has quit IRC15:53
*** jwagner_away is now known as jwagner15:53
cloudnullmeeting in 7 min in #openstack-meeting-4 -- cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, mancdaz, dolphm, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden15:54
*** vdo has quit IRC16:04
openstackgerritJaveria Khan proposed stackforge/os-ansible-deployment: Fix doc paths for openstack_repo_url  https://review.openstack.org/22226216:07
*** Bjoern_ has joined #openstack-ansible16:11
*** jwagner is now known as jwagner_away16:11
*** Bjoern_ is now known as BjoernT16:11
*** jwagner_away is now known as jwagner16:15
*** tlian has quit IRC16:19
*** tlian has joined #openstack-ansible16:26
openstackgerritMerged stackforge/os-ansible-deployment: Fix of haproxy ssl misconfiguration with nova_console  https://review.openstack.org/22185316:29
openstackgerritMerged stackforge/os-ansible-deployment: Update Ansible to v1.9.3-1  https://review.openstack.org/22189916:30
openstackgerritMerged stackforge/os-ansible-deployment: Create config_template plugin  https://review.openstack.org/22021216:36
openstackgerritJaveria Khan proposed stackforge/os-ansible-deployment: Add isolated flag to pip fall back installation  https://review.openstack.org/22228516:38
prometheanfiremhayden: what return do you get if that log is overloaded?16:41
mhaydenLXC says everything's cool16:42
mhaydenno errors, even with debug16:42
mhaydenin its mind, it told the kernel to do something and it hopes the kernel actually does it16:42
prometheanfireI mean from the kernel16:42
mhaydenbut when it floods the netlink message queue with deletions, bad things happen16:42
mhaydennothing from the kernel16:42
prometheanfirehuh, sounds like a kernel bug16:42
prometheanfirekernel should say that the queue is full16:43
prometheanfireso anything using it can try again16:43
mhaydenprometheanfire: i poked rlove and got https://twitter.com/rlove/status/64128654828611584016:46
mhaydenhaven't gone about working with that yet16:46
Apsuprometheanfire: Netlink is not a reliable transport with retransmissions and such like TCP16:48
prometheanfiremhayden: I'm saying that we should get a return code if overloaded, we probably shouldn't care about the size16:48
ApsuNetlink is more like UDP, and the kernel doesn't tell you jack over the netlink socket.16:48
mhaydenthe size was mainly a curiosity ;)16:48
prometheanfireApsu: ah, better option?16:48
ApsuYou are responsible for making sure what you asked happened.16:48
mhaydenfedora kernel devs said "don't use netlink, it's old"16:49
prometheanfiredon't blame them16:49
mhaydenbut that sounds like a big fat patch for lxc :P16:49
palendaeprometheanfire: gus (I think) mentioned you basically have to keep pinging with back offs to communicate over netlink16:49
mhaydeni'll toss what i found into a big fat LXC issue later today on the githubs16:49
palendaemhayden: LXc needs some big fat patches from what I hear16:49
palendaeSome just being style consistency16:49
mhaydenyeah16:50
mhaydenas soon as i mention lxc, the kernel devs seem to hide under their desks16:50
mhaydenthen again, i do the same16:50
palendaelol16:52
palendaeWhat about when you mention docker?16:52
prometheanfiredocker?16:52
prometheanfiredockerdocker?16:52
mhaydendockerxc?16:55
*** jmckind has joined #openstack-ansible16:56
alextricityHey. I'm probably late to the party, but does anyone know what's going on with the InsecurePlatform warning while running the repo plays?16:57
evrardjpcurl https?17:01
prometheanfirepaste?17:01
evrardjpI guess an update from curl that now mentions certificate warnings17:01
odyssey4mein case y'all are wondering why the kilo gate is broken, you can thank a library: https://bugs.launchpad.net/openstack-gate/+bug/149434717:01
openstackLaunchpad bug 1494347 in OpenStack-Gate "PyECLib 1.0.9 released on 9/10 blowing up grenade jobs due to "src/c/pyeclib_c/pyeclib_c.c:32:25: fatal error: erasurecode.h: No such file or directory"" [Undecided,New]17:01
prometheanfirelol, it's because they aren't doing it right17:02
evrardjp:)17:02
evrardjpI'm off for today17:02
alextricityodyssey4me: thanks17:03
prometheanfireodyssey4me: https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/PyECLib/files/1.0.7-erasurecode_locations.patch17:03
odyssey4mewe'll have to bump the sha for juno / kilo to accommodate the global-requirements updates to work around the issue while upstream sorts their stuff out17:04
odyssey4melol prometheanfire is that patch sent upstream?17:04
odyssey4meupstream bug: https://bitbucket.org/kmgreen2/pyeclib/issues/76/pyeclib-109-fails-to-install-if-not17:05
prometheanfireI could17:06
prometheanfiresuppose I should too17:06
cloudnullo/ evrardjp have a good one.17:06
prometheanfirebut I'm busy buying a domain17:06
odyssey4meprometheanfire there's mention of what you've done in the log17:07
prometheanfireneat17:07
odyssey4menot what you've done, but the paths you've changed are mentioned17:07
prometheanfireya, this might be a packaging problem on their end17:09
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Replaced the copy_update module  https://review.openstack.org/21679017:14
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Allow testing of cinder multi backends w/ tempest  https://review.openstack.org/22211917:15
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Fixes jinja typo in keystone-httpd.conf.j2  https://review.openstack.org/22197417:17
openstackgerritKevin Carter proposed stackforge/os-ansible-deployment: Used named veth pairs that match container  https://review.openstack.org/22229517:23
cloudnullodyssey4me: hughsaunders: other: on https://review.openstack.org/#/c/217030/ < do we want to have the config template be applied in one patch or many patches to the various openstack services ?17:27
odyssey4mecloudnull I would suggest one patch per role17:27
odyssey4meit makes it quicker and simpler to review17:27
cloudnullit also makes it a backport mess but i agree its simpler to review .17:28
odyssey4mea backport mess? once the first patch is in - all the others dep only on the module being there, surely?17:29
cloudnullyes its just lots of patches is all .17:29
cloudnullbut if thats better for folks im game .17:29
odyssey4meyeah, it's a bit of a pain - but big patches are scary :p17:30
* cloudnull shakes :)17:30
* cloudnull lunching afk a bit17:31
*** jwagner is now known as jwagner_away17:55
*** phschwartz has joined #openstack-ansible17:59
*** arbrandes has joined #openstack-ansible18:10
openstackgerritMerged stackforge/os-ansible-deployment: Set client versions deployed to use global requirements  https://review.openstack.org/22005018:11
openstackgerritMerged stackforge/os-ansible-deployment: Add galera_address to all playbooks installing galera_client  https://review.openstack.org/22130118:11
openstackgerritMerged stackforge/os-ansible-deployment: Changed certificate order for pem generation with CA files  https://review.openstack.org/22168218:12
*** arbrandes has quit IRC18:25
*** ChanServ sets mode: +o palendae18:29
cloudnulllook at that, three merges in a row :)18:29
palendaeTopic: Launchpad: https://launchpad.net/openstack-ansible Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible  || Repo rename from stackforge/os-ansible-deployment to openstack/openstack-ansible happens Sept 11 2015 23:00 to 23:3018:30
*** palendae changes topic to "Topic: Launchpad: https://launchpad.net/openstack-ansible Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Repo rename from stackforge/os-ansible-deployment to openstack/openstack-ansible happens Sept 11 2015 23:00 to 23:30"18:30
svgThere goes 'osad'. What's the new short name for the project?18:30
palendaeOA?18:31
svgOSA?18:31
cloudnullOSAD 4 life!18:31
mrstanwellohappy?18:31
*** palendae changes topic to "Topic: Launchpad: https://launchpad.net/openstack-ansible Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Repo rename from stackforge/os-ansible-deployment to openstack/openstack-ansible happens Sept 11 2015 23:00 to 23:30. See https://review.openstack.org/#/c/200730/"18:31
mrstanwell(because you can actually install ostack with it...)18:31
*** ChanServ sets mode: +o palendae18:32
palendaeUn-op!18:32
*** ChanServ sets mode: -o palendae18:33
cloudnullmrstanwell: related https://twitter.com/cloudnull/status/64204351897891225618:34
mrstanwellcloudnull:  Excellent!18:35
*** jwagner_away is now known as jwagner18:36
*** arbrandes has joined #openstack-ansible18:42
*** phalmos has joined #openstack-ansible18:52
*** cloudtrainme has quit IRC19:14
*** cloudtrainme has joined #openstack-ansible19:25
openstackgerritMerged stackforge/os-ansible-deployment: Add profiling for Ansible tasks  https://review.openstack.org/21684919:34
*** javeriak has quit IRC19:37
*** javeriak has joined #openstack-ansible19:48
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment-specs: Add spec for Independent Role Repositories  https://review.openstack.org/21377919:53
odyssey4memhayden can you backport https://review.openstack.org/216849 please ?19:54
mhaydenodyssey4me: of course -- now how do i do that? ;)19:54
*** sdake has joined #openstack-ansible19:54
odyssey4memhayden seeing as it's already merged and doesn't rely on anything else, it's easy19:55
odyssey4meeven a windows engineer could do it ;)19:55
odyssey4meclickety on the link19:55
odyssey4meclickety on the cherry-pick button ont he left19:55
odyssey4metypey 'kilo' as the branch to cherry-pick to19:56
odyssey4meclickety 'cherry pick change'19:56
odyssey4meet voila!19:56
* mhayden prepares for clickeyu19:56
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment: Add profiling for Ansible tasks  https://review.openstack.org/22234819:57
odyssey4meet voila!19:57
mhaydenodyssey4me: damn, never have done that before -- thanks!19:57
odyssey4memhayden thanks :) we'll do the more complex backports another time :)19:58
*** sdake_ has joined #openstack-ansible20:02
javeriak odyssey4me should I rebase https://review.openstack.org/#/c/222262/ ? kinda hard to make out why its failing from the logs20:03
odyssey4mejaveriak looking20:03
odyssey4meah, that's a build timeout which is exactly 90 minutes20:04
odyssey4meyou can either rebase, or recheck20:04
javeriakoh i see it now20:04
javeriakrecheck?20:04
*** sdake has quit IRC20:05
odyssey4mejaveriak yeah, just comment on it with 'recheck build timeout'20:06
odyssey4mewhich is of the format 'recheck <reason why>'20:06
odyssey4me:)20:06
javeriakcool thanks, I assume this will have retriggered the dsvm pipeline20:08
*** cloudtrainme has quit IRC20:09
*** KLevenstein has quit IRC20:09
odyssey4mejaveriak yup - you can see progress at http://status.openstack.org/zuul/20:09
odyssey4mefilter by 'stackforge/os-ansible-deployment' and expand by default20:10
*** kukacz has joined #openstack-ansible20:10
*** cloudtrainme has joined #openstack-ansible20:10
odyssey4meit's busy waiting for nodes from nodepool, and I think there's some trouble with nodepool which -infra is working out20:10
odyssey4meso the nodes are trickling through20:11
javeriakyep i see it now :)20:11
javeriakbtw what happens when the repo changes to openstack/ do we need to reclone our local copies?20:11
javeriakwhat about pending commits20:11
odyssey4mejaveriak the pending commits should be fine20:12
odyssey4meyou will need to update your remotes, or reclone, yes20:12
javeriakah okay, will you guys be sending out an official statement or something?20:13
odyssey4mejaveriak we'll send a note out to the ML20:13
*** KLevenstein has joined #openstack-ansible20:14
javeriakgreat, I'll have something share with my folks20:15
*** cloudtra_ has joined #openstack-ansible20:24
*** cloudtrainme has quit IRC20:27
*** javeriak has quit IRC20:30
openstackgerritBjoern Teipel proposed stackforge/os-ansible-deployment: Implement Neutron LBAAS using haproxy  https://review.openstack.org/22036520:59
*** sdake_ has quit IRC21:13
*** jbweber_ has quit IRC21:18
*** sdake has joined #openstack-ansible21:26
*** jmckind has quit IRC21:33
*** jwagner is now known as jwagner_away21:36
*** kukacz has quit IRC21:36
openstackgerritBjoern Teipel proposed stackforge/os-ansible-deployment: Implement Neutron LBAAS using haproxy  https://review.openstack.org/22036521:40
*** phalmos has quit IRC21:43
*** mcarden has joined #openstack-ansible21:52
* cloudnull out21:55
cloudnullhave a good one21:55
BjoernTby bye21:56
*** KLevenstein has quit IRC21:56
*** Mudpuppy_ has joined #openstack-ansible21:58
*** agireud has quit IRC21:58
*** Mudpuppy has quit IRC22:02
*** Mudpuppy_ has quit IRC22:02
*** sigmavirus24 is now known as sigmavirus24_awa22:18
*** BjoernT has quit IRC22:37
*** alop has quit IRC22:45
*** woodard_ has joined #openstack-ansible23:22
*** woodard has quit IRC23:26
*** cloudtra_ has quit IRC23:28
*** woodard_ has quit IRC23:33
*** spotz is now known as spotz_zzz23:36
*** arbrandes has quit IRC23:40
openstackgerritBjoern Teipel proposed stackforge/os-ansible-deployment: Implement Neutron LBAAS using haproxy  https://review.openstack.org/22036523:46
openstackgerritBjoern Teipel proposed stackforge/os-ansible-deployment: Implement Neutron LBAAS using haproxy  https://review.openstack.org/22036523:47
*** shoutm has joined #openstack-ansible23:51
*** arbrandes has joined #openstack-ansible23:56
*** rromans has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!