*** arbrandes has joined #openstack-ansible | 00:09 | |
*** sdake_ has joined #openstack-ansible | 00:13 | |
*** sdake has quit IRC | 00:16 | |
*** BjoernT has quit IRC | 00:17 | |
*** openstack has joined #openstack-ansible | 00:35 | |
*** woodard has quit IRC | 00:41 | |
*** cloudtrainme has joined #openstack-ansible | 00:46 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 00:46 | |
*** shoutm has quit IRC | 00:47 | |
*** shoutm has joined #openstack-ansible | 00:49 | |
*** sdake has joined #openstack-ansible | 00:56 | |
*** sdake_ has quit IRC | 00:59 | |
*** shoutm has quit IRC | 01:00 | |
*** shoutm has joined #openstack-ansible | 01:00 | |
*** shoutm has quit IRC | 01:07 | |
*** shoutm has joined #openstack-ansible | 01:09 | |
*** shoutm has quit IRC | 01:27 | |
*** cloudtrainme has quit IRC | 01:33 | |
*** shoutm has joined #openstack-ansible | 01:40 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Read affinity from environment https://review.openstack.org/215903 | 01:57 |
---|---|---|
*** javeriak has joined #openstack-ansible | 02:57 | |
*** javeriak has quit IRC | 03:15 | |
*** fawadkhaliq has joined #openstack-ansible | 03:58 | |
*** tlian has quit IRC | 04:21 | |
*** shoutm_ has joined #openstack-ansible | 04:32 | |
*** shoutm has quit IRC | 04:34 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Removed default lxc profile on container create https://review.openstack.org/216301 | 04:52 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Fixes loops for bashate https://review.openstack.org/215904 | 05:38 |
*** shausy has joined #openstack-ansible | 05:41 | |
*** shausy has quit IRC | 05:53 | |
*** shausy has joined #openstack-ansible | 05:54 | |
*** javeriak has joined #openstack-ansible | 06:43 | |
evrardjp | xar-: znc is indeed nice :) | 06:46 |
evrardjp | good morning everyone | 06:46 |
mattt | evrardjp: morning2u | 06:52 |
*** javeriak has quit IRC | 07:14 | |
*** fawadkhaliq has quit IRC | 07:15 | |
*** cristicalin has joined #openstack-ansible | 07:22 | |
cristicalin | anybody else seeing kernel traces about corrupt packets and bad offloading when setting up containers for OSAD with ubuntu 14.04 ? | 07:25 |
cristicalin | I managed to repro this with all 3 official kernel generations for 14.04 on different hardware | 07:25 |
cristicalin | as far as I can tell the trace is generated my the internal interfaces (veths or linux bridges) not the external interface | 07:26 |
cristicalin | http://pastebin.com/5jgXVCRd here's a kern.log for anybody interested and knowledgable to look into the issue | 07:27 |
*** gparaskevas has joined #openstack-ansible | 07:30 | |
*** fawadkhaliq has joined #openstack-ansible | 07:34 | |
*** fawadkhaliq has quit IRC | 07:38 | |
odyssey4me | cristicalin is this for an AIO or for a multinode deployment? also, is it on real hardware/vm's/? | 07:40 |
odyssey4me | this is something we've seen, and it seems that it relates to container veths which shouldn't be there any more - Apsu put together a script to clean up after containers when you restart them: https://gist.github.com/Apsu/7947a3347fcc86bb45a7 | 07:42 |
odyssey4me | that said, what you're seeing may not be the same thing - would you mind registering a bug for this so that we can look into it properly and perhaps also report it upstream if necessary | 07:42 |
cristicalin | odyssey4me, it's an AIO running in an openstack instance (so VM) | 07:43 |
cristicalin | sure, grad to register a bug, but where ? | 07:43 |
odyssey4me | cristicalin ok, so it may be the same thing that I've often seeing - but it has no impact on general functionality | 07:43 |
cristicalin | on github ? | 07:43 |
odyssey4me | cristicalin https://bugs.launchpad.net/openstack-ansible | 07:44 |
cristicalin | odyssey4me, yes, looks like it completes even with the traces so I guess it's harmless | 07:44 |
odyssey4me | cristicalin I'd like us to follow through with it though, just in case it becomes a problem later - so a bug registered would be greatly appreciated :) | 07:44 |
*** gparaskevas_ has joined #openstack-ansible | 07:48 | |
*** gparaskevas has quit IRC | 07:49 | |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Removed default lxc profile on container create https://review.openstack.org/217014 | 07:54 |
*** fawadkhaliq has joined #openstack-ansible | 07:54 | |
odyssey4me | mattt andymccr hughsaunders please review the horde of backports and key patches in flight - only two days to go before we release 11.2.0: https://review.openstack.org/#/q/starredby:%22Jesse+Pretorius%22+project:stackforge/os-ansible-deployment,n,z | 07:55 |
odyssey4me | hughsaunders please check if your query has been adequately covered in https://review.openstack.org/207939 | 07:56 |
odyssey4me | cloudnull it appears that https://review.openstack.org/215905 is causing some trouble in the container setup process - please look into it | 07:59 |
*** vdo has joined #openstack-ansible | 08:01 | |
*** benwh4 has joined #openstack-ansible | 08:02 | |
benwh4 | hello | 08:02 |
cristicalin | odyssey4me, https://bugs.launchpad.net/openstack-ansible/+bug/1488815 done | 08:02 |
openstack | Launchpad bug 1488815 in openstack-ansible "Kernel traces with skb_warn_bad_offload showing up during an AIO deployment on Ubuntu 14.04" [Undecided,New] | 08:02 |
odyssey4me | o/ benwh4 | 08:03 |
odyssey4me | thanks cristicalin :) | 08:03 |
cristicalin | odyssey4me, I'm still waiting for that deployment to finish so I'm not 100% sure about that not impacted part | 08:04 |
*** fawadkhaliq has quit IRC | 08:04 | |
benwh4 | in the bug I think you mean dist-upgrade not diet-upgrade | 08:04 |
cristicalin | :) true | 08:05 |
cristicalin | it was not exactly a copy & paste | 08:05 |
cristicalin | but it would be nice as a feature to apt-get, wouldnt it? | 08:06 |
benwh4 | a diet-upgrade ! sure it would be dope | 08:07 |
*** javeriak has joined #openstack-ansible | 08:07 | |
odyssey4me | hahaha | 08:07 |
odyssey4me | I've added the note about the leftover veth's and a link to the clean-up script - once Apsu's online later perhaps he can put a little more time into figuring out the root cause. It may have to wait though as there is more focus on ensuring that upgrades are better right now. | 08:09 |
cristicalin | so upgrade will be supported from kilo to liberty ? | 08:09 |
cristicalin | I mean fully ? | 08:09 |
cristicalin | hmm, I think I just borked my deployment with that fix script | 08:10 |
odyssey4me | cristicalin the current work is to improve the juno to kilo upgrades as we've found that it causes some down-time for neutron routing, which isn't great | 08:10 |
cristicalin | I tried to run it during the deployment ... | 08:10 |
odyssey4me | but yes, once liberty goes into code freeze we'll be finalising changes to release liberty on the same day as the upstream projects and part of the testing will be to ensure that upgrades work | 08:11 |
odyssey4me | cristicalin oops - I haven't tried the script myself, so I can't vouch for it, but that wasn't perhaps the greatest of ideas :p | 08:12 |
cristicalin | yeah, figured that one out the hard way | 08:12 |
cristicalin | oh well , terminate and relaunch | 08:12 |
odyssey4me | cristicalin you may wish to use the process outlined here instead of your method: https://github.com/stackforge/os-ansible-deployment/blob/master/development-stack.rst | 08:12 |
cristicalin | ok, I'll use that, might as well learn the right way to do it | 08:14 |
odyssey4me | cristicalin there isn't really anything wrong with the method you used, but it's a once-off run method - whereas the method outlined there breaks it down a little more to understand things better | 08:15 |
*** fawadkhaliq has joined #openstack-ansible | 08:17 | |
benwh4 | does this error break the deployment or is it just impacting ssh ? and why the message return 127.0.0.1 and no other IP address ? | 08:19 |
benwh4 | TASK: [Wait for ssh to be available] ****************************************** | 08:19 |
benwh4 | failed: [infra1_utility_container-1aa5c074 -> 127.0.0.1] => {"elapsed": 302, "failed": true} | 08:19 |
benwh4 | msg: Timeout when waiting for search string OpenSSH in 10.200.239.61:22 | 08:19 |
benwh4 | ?? | 08:19 |
odyssey4me | benwh4 I thought we'd covered this the last time you asked? | 08:22 |
mattt | odyssey4me: WELL COVER IT AGAIN | 08:22 |
mattt | sheesh | 08:22 |
mattt | :P | 08:22 |
benwh4 | yes but I changed the value in my user_config but still the message appear | 08:23 |
odyssey4me | benwh4 can you post your updated config to pastebin please :) | 08:24 |
odyssey4me | last I recall you have both a container and management network, and I suggested that you consolidate them | 08:24 |
odyssey4me | I also noticed that your LB addresses weren't in the list of used addresses, so your containers may conflict with them | 08:25 |
cloudnull | mornings | 08:25 |
odyssey4me | you may wish to refer to the logs near the top of http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2015-08-24.log.html :) | 08:25 |
odyssey4me | gparaskevas_ when you're around it'd be great if you could backport https://review.openstack.org/215019 to kilo - just cherry-pick it to kilo from the gerrit interface :) | 08:27 |
benwh4 | odyssey4me hter is the config file : http://pastebin.com/eaAB8UXH | 08:28 |
gparaskevas_ | odyssey4me: hey there! sure no problem | 08:28 |
odyssey4me | benwh4 thanks - it looks better | 08:29 |
odyssey4me | the indent on line 7 is wrong - it needs to line up with the others in the same section | 08:29 |
cloudnull | odyssey4me: on https://review.openstack.org/#/c/215905/ seems odd that the change would be causing issues in kilo as that change has been in master for some time now. | 08:29 |
odyssey4me | cloudnull yep - it might just be a flaky gate, but kilo's gate has been pretty stable and that patch is failing every time (although in different places) - it may just warrant a test build to see if there's anything going on to be concerned about | 08:30 |
cloudnull | i have that going presently | 08:30 |
benwh4 | odyssey4me the line 7 is a paste bug it is ok in the config file though | 08:31 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: [WIP] Adds the crud_template to ceilometer https://review.openstack.org/217030 | 08:31 |
odyssey4me | benwh4 the affinity settings can also be removed - the default is 1, so you're implementing an unnecessary override | 08:32 |
*** javeriak has quit IRC | 08:32 | |
cloudnull | also https://review.openstack.org/#/c/216790/ replaces the copy_update module and https://review.openstack.org/#/c/217030/ shows how it can be used to allow configurations files that are json, yaml, or in ini format to be dyanmically updated. | 08:32 |
cloudnull | i did ceilometer first as it uses all three for config | 08:33 |
odyssey4me | cloudnull awesome - looks good, I think it's time to restore https://review.openstack.org/168976 and update it with this method :) | 08:37 |
cloudnull | yea ill try and bang on that today | 08:37 |
odyssey4me | benwh4 ok, so I don't see anything else in the openstack_user_config that is funky | 08:40 |
*** shoutm_ has quit IRC | 08:40 | |
odyssey4me | benwh4 once you'd fixed up the environment, did you tear it down and restart - or have you tried to re-use the existing environment? | 08:42 |
*** javeriak has joined #openstack-ansible | 08:48 | |
gparaskevas_ | odyssey4me: I get an error : code review error cherry pick failed | 09:00 |
odyssey4me | gparaskevas_ ah, there must be a dependent patch that hasn't merged into kilo | 09:01 |
gparaskevas_ | odyssey4me: ok then! | 09:02 |
*** ChanServ changes topic to "Launchpad: https://launchpad.net/openstack-ansible Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible" | 09:02 | |
odyssey4me | gparaskevas_ hmm, are you sure - it seems to cherry-pick cleanly for me | 09:04 |
gparaskevas_ | odyssey4me: let me see gain | 09:04 |
odyssey4me | gparaskevas_ note that you must select the right branch to cherry pick into - and it must all be lower case | 09:05 |
gparaskevas_ | odyssey4me: i select from the latest patch(7) and press cherry pick, then on the modal window i type-select kilo and i leave the commit message as is, I can see that i has the cherry pick from id. | 09:06 |
odyssey4me | gparaskevas_ odd, i see that it fails via the browser | 09:06 |
odyssey4me | ok well, do you have an appropriate connection to be able to do it via cli? | 09:07 |
gparaskevas_ | yes let me try from chrome as well maybe firefox is the issue | 09:07 |
gparaskevas_ | then cli | 09:07 |
benwh4 | odyssey4me I teardown and rebuild | 09:10 |
gparaskevas_ | git fetch https://gparask@review.openstack.org/stackforge/os-ansible-deployment refs/changes/19/215019/7 && git cherry-pick FETCH_HEAD | 09:10 |
odyssey4me | git fetch https://review.openstack.org/stackforge/os-ansible-deployment refs/changes/19/215019/7 && git cherry-pick -x FETCH_HEAD | 09:11 |
benwh4 | odyssey4me the heck is that the setup-hosts is ok for the target hosts but not for the container inside and the ssh error abort the playbook | 09:11 |
odyssey4me | ie add the '-x' just before the end | 09:11 |
gparaskevas_ | ok | 09:11 |
odyssey4me | then git review | 09:11 |
odyssey4me | benwh4 please remind me - are you executing ansible from a workstation on another network, or from one of the hosts? | 09:12 |
gparaskevas_ | do i need to be on the desired repo folder? or it will clone it? | 09:12 |
*** shoutm has joined #openstack-ansible | 09:12 | |
odyssey4me | gparaskevas_ you need to get into the repo folder, git fetch to update the refs, then git checkout origin/kilo | 09:12 |
odyssey4me | then execute your cherry-pick | 09:13 |
benwh4 | odyssey4me I execute ansible from a sevrer in the same network as my targets 10.200.0.0 for the management | 09:14 |
benwh4 | odyssey4me it was my pxe server for the targets as well | 09:15 |
odyssey4me | benwh4 ok, so let's step through where you're at | 09:16 |
*** gparaskevas_ has quit IRC | 09:16 | |
odyssey4me | you've pxe installed your hosts, bootstrapped ansible (which version are you using?) on your pxe host, cloned the repo, implemented /etc/openstack_deploy/{openstack_user_config.yml,user_variables.yml,user_secrets.yml} ? | 09:17 |
odyssey4me | and the /etc/openstack_deploy/{conf.d,env.d} directories with all the contents of env.d from the repo clone | 09:18 |
odyssey4me | benwh4 does that sound right so far? | 09:19 |
benwh4 | odyssey4me I didn't touched the env.d,conv.d directory but yes it sound familiar | 09:25 |
benwh4 | odyssey4me I use ansible 1.9.2 | 09:27 |
*** misc_ has joined #openstack-ansible | 09:27 | |
*** mhayden has quit IRC | 09:28 | |
*** jmccrory has quit IRC | 09:28 | |
*** misc has quit IRC | 09:28 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment-specs: Tunable OpenStack Configuration Specification https://review.openstack.org/168976 | 09:28 |
*** jmccrory has joined #openstack-ansible | 09:28 | |
cloudnull | odyssey4me: ^ | 09:30 |
benwh4 | do you think I can go on even if I have this after the play recap ... | 09:38 |
benwh4 | log1 : ok=46 changed=7 unreachable=0 failed=0 | 09:38 |
benwh4 | log1_rsyslog_container-c5d3f7e7 : ok=16 changed=10 unreachable=0 failed=1 | 09:39 |
benwh4 | all of my containers have encouter error during the task wait for ssh available task | 09:39 |
*** mhayden has joined #openstack-ansible | 09:40 | |
*** gparaskevas has joined #openstack-ansible | 09:50 | |
*** shoutm has quit IRC | 09:50 | |
gparaskevas | odyssey4me: hey sorry for late reply we are having connection issues at the office. i did git fetch to update my refs, i did git checkout origin/kilo and now i have detatched head(head shows kilo) should i git review? | 09:51 |
*** cristicalin has quit IRC | 10:02 | |
*** shausy has quit IRC | 10:10 | |
*** gparaskevas has quit IRC | 10:12 | |
*** gparaskevas has joined #openstack-ansible | 10:15 | |
*** benwh4 has quit IRC | 10:32 | |
cloudnull | this is ithe error that is consistently happening within kilo per the recent patches http://paste.openstack.org/show/428169 | 10:35 |
*** fawadkhaliq has quit IRC | 10:40 | |
*** javeriak has quit IRC | 10:46 | |
odyssey4me | benwh4 can you please pastebin a copy of your sshd_config from one of the hosts? | 10:48 |
odyssey4me | it sounds to me like the sshd config isn't right | 10:48 |
odyssey4me | gparaskevas after git checkout origin/kilo, do the cherry pick, then git review | 10:48 |
*** fawadkhaliq has joined #openstack-ansible | 10:50 | |
openstackgerrit | George Paraskevas proposed stackforge/os-ansible-deployment: Enable HAProxy Stats Web UI https://review.openstack.org/217094 | 11:08 |
*** misc_ is now known as misc | 11:14 | |
odyssey4me | gparaskevas +1 :) | 11:16 |
gparaskevas | odyssey4me: thanks! :) always pleasure | 11:18 |
*** javeriak has joined #openstack-ansible | 11:23 | |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Updated juno to include fix for CVE-2015-3241 - 26 Aug 2015 https://review.openstack.org/217098 | 11:27 |
*** fawadkhaliq has quit IRC | 11:30 | |
cloudnull | odyssey4me: fixes openstack problems https://gist.github.com/cloudnull/9361461 | 11:32 |
*** fawadkhaliq has joined #openstack-ansible | 11:33 | |
cloudnull | if youre curious how the tool works i have a tool for that too https://pypi.python.org/pypi/AdvancedSearchDiscovery | 11:35 |
*** javeriak has quit IRC | 11:35 | |
*** javeriak has joined #openstack-ansible | 11:35 | |
*** javeriak has quit IRC | 11:36 | |
*** javeriak has joined #openstack-ansible | 11:37 | |
*** javeriak has quit IRC | 11:41 | |
*** javeriak has joined #openstack-ansible | 11:42 | |
odyssey4me | hughsaunders cloudnull mattt andymccr please work through these reviews asap: https://review.openstack.org/#/q/starredby:%22Jesse+Pretorius%22+project:stackforge/os-ansible-deployment,n,z | 11:45 |
odyssey4me | hughsaunders please check if your question has been appropriately addressed? https://review.openstack.org/207939 | 11:45 |
openstackgerrit | Hugh Saunders proposed stackforge/os-ansible-deployment: Add ebtables to neutron agent configuration https://review.openstack.org/217103 | 11:51 |
odyssey4me | cloudnull hughsaunders https://bugs.launchpad.net/neutron/+bug/1274034 | 11:56 |
openstack | Launchpad bug 1274034 in neutron "Neutron firewall anti-spoofing does not prevent ARP poisoning" [High,Fix released] - Assigned to Kevin Benton (kevinbenton) | 11:56 |
*** benwh4 has joined #openstack-ansible | 12:00 | |
evrardjp | ouch | 12:03 |
evrardjp | this is a bad one | 12:03 |
evrardjp | I've seen we have another CVE today | 12:04 |
evrardjp | https://bugs.launchpad.net/nova/+bug/1387543 | 12:04 |
openstack | Launchpad bug 1387543 in OpenStack Compute (nova) "[OSSA 2015-015] Resize/delete combo allows to overload nova-compute (CVE-2015-3241)" [High,Fix committed] - Assigned to Abhishek Kekane (abhishek-kekane) | 12:04 |
odyssey4me | evrardjp it's an upstream CVE for juno | 12:04 |
evrardjp | odyssey4me: which one? | 12:05 |
evrardjp | Arp poisoning? | 12:05 |
odyssey4me | ah, no that one isn't a CVE - but it is s security issue across all branches | 12:05 |
odyssey4me | the reviews are still in progress upstream though | 12:05 |
odyssey4me | ah, I see the one you linked needs a patch update for us - let me do that quickly | 12:06 |
evrardjp | I didn't got the chance to do it myself, sorry | 12:07 |
*** rward has quit IRC | 12:08 | |
evrardjp | still the arp poisoning is a bad one, I'll check which it hasn't been merged | 12:08 |
evrardjp | why* | 12:08 |
evrardjp | mmm a lot of work | 12:09 |
evrardjp | I guess | 12:09 |
evrardjp | https://review.openstack.org/#/c/209705/ | 12:10 |
evrardjp | any of your customers has asked this? | 12:10 |
odyssey4me | evrardjp we hit that when the changes merged to master and stuff didn't work any more - it turns out that the implementation right now is incomplete | 12:12 |
odyssey4me | we have therefore registered https://bugs.launchpad.net/neutron/+bug/1483315 | 12:12 |
openstack | Launchpad bug 1483315 in neutron "ebtables ARP rules don't account for floating IPs on LinuxBridge" [Undecided,In progress] - Assigned to Kevin Benton (kevinbenton) | 12:12 |
odyssey4me | once that's fixed, we can turn it on in master - for now we've disabled it in https://review.openstack.org/210593 | 12:13 |
evrardjp | ok | 12:15 |
evrardjp | linux bridge really deserves more love. | 12:17 |
evrardjp | cloudnull: nice spec | 12:18 |
evrardjp | https://review.openstack.org/#/c/168976/2/specs/kilo/tunable-openstack-configuration.rst | 12:18 |
*** woodard has joined #openstack-ansible | 12:21 | |
*** shoutm has joined #openstack-ansible | 12:23 | |
*** javeriak_ has joined #openstack-ansible | 12:23 | |
odyssey4me | evrardjp linuxbridge is no voting in the gate, so it's an equal to ovs from a voting standpoint | 12:23 |
odyssey4me | the issue with the arp filtering things is that they added the feature, but explicity turned off the tests for both ovs and linuxbridge | 12:24 |
odyssey4me | so whoever added that, and whoever approved that, needs their heads checked :p | 12:24 |
*** javeriak has quit IRC | 12:25 | |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Updated juno to include fix for CVE-2015-3241 - 26 Aug 2015 https://review.openstack.org/217114 | 12:35 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Updated kilo to include fix for CVE-2015-3241 - 26 Aug 2015 https://review.openstack.org/217114 | 12:35 |
*** pradk has joined #openstack-ansible | 12:37 | |
mhayden | odyssey4me / palendae: my bouncer blew up last night and i wasn't sure where the APLv2 vs MIT licensing discussion went | 12:57 |
mhayden | i just tossed some notes into https://review.openstack.org/#/c/216849/ | 12:58 |
openstackgerrit | Jean-Philippe Evrard proposed stackforge/os-ansible-deployment: Adds the ability to provide user certificates to HAProxy https://review.openstack.org/215525 | 13:04 |
*** scarlisle has joined #openstack-ansible | 13:12 | |
*** tlian has joined #openstack-ansible | 13:16 | |
Apsu | Morning. | 13:47 |
evrardjp | good morning Apsu | 13:53 |
*** fawadkhaliq has quit IRC | 13:56 | |
*** shoutm has quit IRC | 13:56 | |
*** jmckind has joined #openstack-ansible | 14:00 | |
*** fawadkhaliq has joined #openstack-ansible | 14:02 | |
*** shoutm has joined #openstack-ansible | 14:03 | |
*** KLevenstein has joined #openstack-ansible | 14:04 | |
*** sigmavirus24_awa is now known as sigmavirus24 | 14:05 | |
*** spotz_zzz is now known as spotz | 14:08 | |
odyssey4me | mhaydenI saw the notes, thanks - will read them more thoroughly in a bit once this CVE is dealt with | 14:09 |
*** cloudtrainme has joined #openstack-ansible | 14:10 | |
*** Mudpuppy has joined #openstack-ansible | 14:15 | |
evrardjp | is gerrit slow today or it's just me? | 14:17 |
cloudnull | its slow today... | 14:18 |
cloudnull | :( | 14:18 |
evrardjp | I'll leave earlier then | 14:20 |
evrardjp | have a nice day! | 14:20 |
*** shoutm has quit IRC | 14:22 | |
benwh4 | how do I used the new patch impacting regex ssh ? does I need to re-clone the git repo ? | 14:24 |
*** cristicalin has joined #openstack-ansible | 14:25 | |
cristicalin | anybody have a idea how I can set horizon to use the keystone v3 api instead of the default v2 ? | 14:26 |
cristicalin | I see the template for local_settings.py contains some logic to do it based on the API endpoint URL | 14:27 |
cristicalin | but the catalog looks weird the publicURL for identity is v3 but the internal and admin are v2 and horizon is using the internalURL | 14:27 |
odyssey4me | cristicalin yes, it adapts based on what you enter as the keystone api | 14:27 |
odyssey4me | and you'll need to make sure that you have a catalog entry to match it | 14:28 |
cristicalin | and where is the catalog configured ? | 14:28 |
odyssey4me | cristicalin so it all depends on what you really want to achieve | 14:30 |
odyssey4me | some warnings - keystone v3 endpoints are not yet fully supported for all service to service comms | 14:30 |
cristicalin | odyssey4me, my end goal is to get Horizon to show me keystone domains and be able to manage them in Horizon | 14:30 |
odyssey4me | so start with just setting https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/roles/os_horizon/defaults/main.yml#L86 | 14:30 |
cristicalin | ok, that could work | 14:31 |
cristicalin | can I override that in user_variables.yml ? | 14:31 |
odyssey4me | ie in user_variables, set - horizon_keystone_endpoint: "{{ keystone_service_internalurl_v3 }}" | 14:32 |
odyssey4me | or you can simply set it to the appropriate url instead of using a variable | 14:32 |
*** gparaskevas has quit IRC | 14:32 | |
odyssey4me | cristicalin this will then result in the template adding the right bits when it's dropped on the horizon hosts: https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/roles/os_horizon/templates/horizon_local_settings.py.j2#L45-L54 | 14:34 |
cristicalin | https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/roles/os_horizon/templates/horizon_local_settings.py.j2#L56-L58 | 14:35 |
cristicalin | this should also be changed to True | 14:35 |
cristicalin | so I have to change the template to support multiple domains | 14:35 |
*** fawadkhaliq has quit IRC | 14:35 | |
cristicalin | would be nice to have that into a variable | 14:35 |
odyssey4me | ah, so yes - can you register a bug to request the bits you need for horizon multi-domain support | 14:36 |
odyssey4me | we'll add it as a wishlist and I can prep a patch quickly, unless you'd like to prep a patch for it? | 14:36 |
cristicalin | I can do the patch if you accept it | 14:37 |
cristicalin | will try to do it tomorrow | 14:37 |
odyssey4me | cristicalin sure - we may need to do some to and fro initially around style, etc - but if you're comfortable doing so, we love patches :) | 14:37 |
odyssey4me | we're doing a release tomorrow, then the next will be in two weeks (unless we hit a CVE) - bear that in mind if this is essential for your requirements :) | 14:38 |
cristicalin | odyssey4me, I'm just learning the tools at the moment so I'm in no hurry | 14:39 |
odyssey4me | cristicalin if you can register bugs as you find them, then someone may pick them up along the way - if you find things that are missing, register bugs for those too and we'll assess them as wishlist items | 14:40 |
odyssey4me | we discuss new bugs in the tuesday meetings, then longer term plans in the thu meetings | 14:41 |
* odyssey4me points at the channel topic :) | 14:41 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment-specs: Tunable OpenStack Configuration Specification https://review.openstack.org/168976 | 14:42 |
*** gparaskevas has joined #openstack-ansible | 14:42 | |
*** gparaskevas_ has joined #openstack-ansible | 14:42 | |
cristicalin | ok, will keep that in mind | 14:43 |
*** javeriak_ has quit IRC | 14:47 | |
*** yaya has joined #openstack-ansible | 14:51 | |
*** gparaskevas has quit IRC | 14:52 | |
*** gparaskevas_ has quit IRC | 14:52 | |
cristicalin | odyssey4me, one more question if it's not too much pestering | 14:54 |
cristicalin | is changing haproxy_ssl from no to yes supported ? | 14:54 |
odyssey4me | cristicalin you may pester as much as you like :) if we're so busy that we can't answer, we won't :p | 14:55 |
cristicalin | do I need to re-run all the playbooks or just the haproxy one and the os-keystone to update the catalog | 14:55 |
odyssey4me | (or we may not be here) | 14:55 |
*** javeriak has joined #openstack-ansible | 14:55 | |
odyssey4me | the answer is yes :) | 14:55 |
cristicalin | ok, dumb question, too late in the evening for me | 14:55 |
cristicalin | so which one ? run all or just the haproxy and keystones ? | 14:56 |
odyssey4me | cristicalin more info in the commit message: https://github.com/stackforge/os-ansible-deployment/commit/36640a8f436fae8d0957d92f033dd4baf9e8af3f | 14:56 |
odyssey4me | there is also a review in flight to add user provided certificate support: https://review.openstack.org/215525 | 14:57 |
cristicalin | self signed works for me atm | 14:58 |
*** sdake has quit IRC | 14:58 | |
odyssey4me | cristicalin so you can use haproxy to do ssl offloading for keystone, or you can do ssl on keystone itself (see https://github.com/stackforge/os-ansible-deployment/commit/4b35b3e929cbc728b903bf19d8d169e376920832 ) and you can theoretically also do ssl on both if you want | 14:58 |
*** CheKoLyN has joined #openstack-ansible | 14:59 | |
odyssey4me | be warned though, haproxy is primarily a dev/test tool for the project - it hasn't been fine tuned for production use | 14:59 |
cristicalin | I prefer the haproxy approach at the moment it's cleaner | 14:59 |
odyssey4me | evrardjp has been submitting patches to improve it for production use and will continue to improve it over time | 15:00 |
cristicalin | also I can separate and tune it outside the main node | 15:00 |
*** javeriak has quit IRC | 15:01 | |
odyssey4me | most production deployments using the project are using F5's for load balancing and SSL offloading - they scale better for heavy loads | 15:01 |
*** daneyon has joined #openstack-ansible | 15:01 | |
*** fawadkhaliq has joined #openstack-ansible | 15:02 | |
cristicalin | mine is chugging along just fine with haproxy but it's a small one <20 nodes | 15:03 |
*** yaya has quit IRC | 15:03 | |
*** gparaskevas has joined #openstack-ansible | 15:04 | |
cristicalin | odyssey4me, is that patch backported to the kilo branch ? | 15:08 |
*** cristicalin has quit IRC | 15:16 | |
*** yaya has joined #openstack-ansible | 15:19 | |
xar- | evrardjp: yes it is ;) | 15:20 |
xar- | morning everyone | 15:20 |
*** mhayden has left #openstack-ansible | 15:26 | |
odyssey4me | andymccr mattt cloudnull hughsaunders sigmavirus24 please review: https://review.openstack.org/#/q/starredby:%22Jesse+Pretorius%22+project:stackforge/os-ansible-deployment,n,z | 15:26 |
*** mhayden has joined #openstack-ansible | 15:26 | |
*** gparaskevas has quit IRC | 15:27 | |
*** jwagner is now known as jwagner_away | 15:31 | |
cloudnull | anyone else getting a 503 from os infra ? | 15:43 |
*** fawadkhaliq has quit IRC | 15:43 | |
-openstackstatus- NOTICE: restarting gerrit due to a slow memory leak | 15:43 | |
sigmavirus24 | cloudnull: ^? | 15:44 |
odyssey4me | o/ cloudnull | 15:44 |
cloudnull | yup thats a thing | 15:45 |
odyssey4me | xar- an early morning to you | 15:45 |
cloudnull | xor not an early morning any longer | 15:45 |
*** benwh4 has quit IRC | 15:55 | |
*** Bjoern_ has joined #openstack-ansible | 15:56 | |
*** yaya has quit IRC | 15:56 | |
*** Bjoern_ is now known as BjoernT | 15:57 | |
*** alop has joined #openstack-ansible | 15:57 | |
*** vdo has quit IRC | 16:02 | |
*** k_stev has joined #openstack-ansible | 16:03 | |
*** k_stev has quit IRC | 16:03 | |
*** k_stev has joined #openstack-ansible | 16:04 | |
*** jwagner_away is now known as jwagner | 16:11 | |
*** yaya has joined #openstack-ansible | 16:18 | |
*** sdake has joined #openstack-ansible | 16:20 | |
*** fawadkhaliq has joined #openstack-ansible | 16:23 | |
*** Mudpuppy_ has joined #openstack-ansible | 16:27 | |
*** Mudpupp__ has joined #openstack-ansible | 16:29 | |
*** Mudpuppy has quit IRC | 16:29 | |
*** jmckind has quit IRC | 16:29 | |
*** Mudpuppy has joined #openstack-ansible | 16:31 | |
*** Mudpuppy_ has quit IRC | 16:32 | |
*** Mudpupp__ has quit IRC | 16:33 | |
*** cloudtrainme has quit IRC | 16:33 | |
xar- | :) | 16:37 |
*** jwagner is now known as jwagner_away | 16:39 | |
*** jwagner_away is now known as jwagner | 16:41 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Use dict args for ceph_config slurp https://review.openstack.org/216608 | 16:51 |
openstackgerrit | Hugh Saunders proposed stackforge/os-ansible-deployment: Add variable for cirros url https://review.openstack.org/217310 | 17:08 |
*** k_stev has quit IRC | 17:21 | |
*** daneyon has quit IRC | 17:27 | |
*** k_stev has joined #openstack-ansible | 17:30 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Set iptables-persistent install execution to append to log https://review.openstack.org/215495 | 17:39 |
*** cloudtrainme has joined #openstack-ansible | 17:56 | |
openstackgerrit | Steve Lewis proposed stackforge/os-ansible-deployment: Add sorting_method to swift proxy config as needed https://review.openstack.org/208817 | 18:13 |
*** openstackgerrit has quit IRC | 18:17 | |
*** openstackgerrit has joined #openstack-ansible | 18:17 | |
*** yaya has quit IRC | 18:39 | |
*** javeriak has joined #openstack-ansible | 18:42 | |
openstackgerrit | Steve Lewis proposed stackforge/os-ansible-deployment: Ensure rsync restarts fully during swift setup https://review.openstack.org/217341 | 18:45 |
*** javeriak has quit IRC | 18:50 | |
*** javeriak has joined #openstack-ansible | 18:53 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Trigger restart after adding user to cephkeys https://review.openstack.org/216320 | 18:57 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Enable admin level on the haproxy stats socket https://review.openstack.org/215899 | 18:58 |
*** jmckind has joined #openstack-ansible | 19:27 | |
*** daneyon has joined #openstack-ansible | 19:29 | |
*** javeriak has quit IRC | 19:33 | |
*** daneyon has quit IRC | 19:33 | |
*** javeriak has joined #openstack-ansible | 19:34 | |
openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Set default container apparmor profile to uncontained https://review.openstack.org/217367 | 19:46 |
*** daneyon has joined #openstack-ansible | 19:48 | |
openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Set default container apparmor profile to unconfined https://review.openstack.org/217367 | 19:49 |
*** klindgren has joined #openstack-ansible | 19:53 | |
*** daneyon has quit IRC | 19:54 | |
klindgren | Sam-I-Am, who was it that you said had a branch to run OSAD on top of redhat/cent? | 19:55 |
*** daneyon has joined #openstack-ansible | 19:55 | |
*** KLevenstein has quit IRC | 19:58 | |
*** BjoernT has quit IRC | 19:58 | |
Sam-I-Am | klindgren: cloudnull | 20:04 |
Sam-I-Am | klindgren: let me find it... | 20:04 |
Sam-I-Am | klindgren: https://github.com/cloudnull/os-ansible-deployment/tree/master-rhel | 20:04 |
*** KLevenstein has joined #openstack-ansible | 20:06 | |
klindgren | Sam-I-Am, thanks | 20:08 |
*** daneyon has quit IRC | 20:09 | |
Sam-I-Am | klindgren: you didnt hear it from me :) | 20:10 |
klindgren | Sam who? I found it on github ;-) | 20:10 |
*** cloudtrainme has quit IRC | 20:12 | |
*** k_stev has quit IRC | 20:12 | |
*** cloudtrainme has joined #openstack-ansible | 20:13 | |
*** k_stev has joined #openstack-ansible | 20:17 | |
*** k_stev1 has joined #openstack-ansible | 20:18 | |
Sam-I-Am | klindgren: lol | 20:18 |
*** javeriak has quit IRC | 20:19 | |
*** k_stev has quit IRC | 20:19 | |
*** javeriak has joined #openstack-ansible | 20:19 | |
*** daneyon has joined #openstack-ansible | 20:24 | |
*** fawadkhaliq has quit IRC | 20:25 | |
*** k_stev1 has quit IRC | 20:29 | |
*** yaya has joined #openstack-ansible | 20:41 | |
*** javeriak has quit IRC | 20:42 | |
*** openstackgerrit has quit IRC | 21:01 | |
*** k_stev has joined #openstack-ansible | 21:02 | |
*** openstackgerrit has joined #openstack-ansible | 21:02 | |
*** woodard has quit IRC | 21:11 | |
*** daneyon_ has joined #openstack-ansible | 21:16 | |
*** k_stev1 has joined #openstack-ansible | 21:18 | |
*** daneyon has quit IRC | 21:19 | |
*** k_stev1 has quit IRC | 21:19 | |
*** k_stev1 has joined #openstack-ansible | 21:19 | |
*** k_stev has quit IRC | 21:19 | |
odyssey4me | klindgren that work will be submitted into osad at some point soon - if you're interested in maintaining it then perhaps you could become that person | 21:21 |
*** k_stev1 has quit IRC | 21:35 | |
*** d34dh0r53 has quit IRC | 21:38 | |
*** d34dh0r53 has joined #openstack-ansible | 21:39 | |
*** alejandrito has joined #openstack-ansible | 21:39 | |
stevelle | miguelgrinberg: any idea why I might be getting issues with the heat config when turning on rsync on the host in an aio? | 21:40 |
*** KLevenstein has quit IRC | 21:40 | |
miguelgrinberg | stevelle: can you be more specific? what issues are you seeing? | 21:40 |
stevelle | http://paste.openstack.org/show/9OR5JgmodVC3yY1kUgrt/ | 21:40 |
miguelgrinberg | stevelle: doesn't seem related to rsync. You have an undefined var. | 21:41 |
stevelle | miguelgrinberg: I don't. I didn't change anything related to heat in that diff | 21:42 |
*** Mudpuppy has quit IRC | 21:42 | |
stevelle | my builds work fine, this just misbehaving in gate. gate has an undefined var :) | 21:42 |
stevelle | my hunch was maybe heat also uses rsync or something | 21:42 |
stevelle | not that this makes any sense | 21:43 |
coolj | palendae: odyssey4me does 1488315 make sense? need more info? seems to be effecting all 10.1.11+ deployments | 21:43 |
miguelgrinberg | stevelle: so this var is not a regular var | 21:43 |
miguelgrinberg | stevelle: it is registered internally: https://github.com/stackforge/os-ansible-deployment/blob/f665c58d38c58ee395229c566d3aab77d94e1a6e/playbooks/roles/os_heat/tasks/heat_domain_setup.yml#L118 | 21:43 |
miguelgrinberg | or actually, on line 126 of that file, a few lines below the one I highlighted | 21:44 |
miguelgrinberg | stevelle: try running the openstack cli command manually and see what you get | 21:44 |
stevelle | "openstack: 'keystone_service_adminurl_v3' is not an openstack command. See 'openstack --help'." | 21:47 |
stevelle | stupid substitution | 21:47 |
odyssey4me | coolj it makes sense, but the workaround is specified in the bug comments | 21:47 |
odyssey4me | those commands should ideally be implemented from the utility container until a suitable fix can be found | 21:48 |
miguelgrinberg | stevelle: yeah, that should be "host_ip:5000/v3" | 21:48 |
stevelle | miguelgrinberg: I have another 6 flags to resolve or something like that | 21:49 |
miguelgrinberg | are you doing this on the utility container? | 21:49 |
stevelle | yes | 21:50 |
stevelle | pruned all the noise, command resolves just fine | 21:51 |
miguelgrinberg | stevelle: you can just list all domains and find the one for heat | 21:51 |
stevelle | but this is in my testing aio | 21:51 |
miguelgrinberg | stevelle: source the openrc, then "openstack domain list" | 21:51 |
stevelle | openstack domain show heat | 21:51 |
stevelle | works fine | 21:51 |
*** alejandrito has quit IRC | 21:51 | |
miguelgrinberg | do you have a domain with name "heat"? | 21:51 |
stevelle | yes | 21:52 |
stevelle | id bdf9e15718e84a8bb78c1679692c9a9b | 21:52 |
miguelgrinberg | if you do "openstack domain show heat" and then pipe it to the grep command that we use in the playbook do you get the id captured? | 21:52 |
miguelgrinberg | stevelle: this thing: grep -oE -m 1 "[0-9a-f]{32}" | 21:53 |
stevelle | miguelgrinberg: I do | 21:53 |
miguelgrinberg | then I guess we have to assume that task was skipped for some reason? Do you have the ansible output to check if this task executed? | 21:54 |
stevelle | url for the review is in the paste | 21:55 |
stevelle | says it skipped the task | 21:55 |
miguelgrinberg | stevelle: it was skipped. I wonder why. | 21:56 |
stevelle | miguelgrinberg: would seem to be because "inventory_hostname == groups['heat_all'][0]" was false | 21:58 |
miguelgrinberg | stevelle: was this a first run? | 21:58 |
stevelle | https://review.openstack.org/#/c/217341/ miguelgrinberg | 21:59 |
stevelle | it was the gate check there | 21:59 |
miguelgrinberg | stevelle: the post_install task runs for all heat containers, but as you noticed, the domain setup task runs for the first | 22:00 |
miguelgrinberg | I'm guessing it is an unrelated bug | 22:00 |
miguelgrinberg | stevelle: the other heat container is bad it seems | 22:01 |
miguelgrinberg | search for "fatal: [aio1_heat_apis_container-d847bf57]" in your log | 22:01 |
stevelle | SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh | 22:02 |
miguelgrinberg | right, so that's your first heat container, the one that should have set that domain id thing | 22:03 |
stevelle | thanks, I got caught on the wrong error | 22:03 |
miguelgrinberg | so go take a look at that container | 22:03 |
stevelle | does anything else with the review present an issue? seems like your initial comments are addressed | 22:04 |
miguelgrinberg | stevelle: my only problem with it is that the restart rsync just stops rsync | 22:04 |
stevelle | you do understand that currently that play just stops rsync? | 22:05 |
miguelgrinberg | I understand what you are saying, but before this change, restart was a restart, and now it is a stop | 22:05 |
stevelle | before this change "restart" didn't work. it never triggered the start handler | 22:06 |
miguelgrinberg | stevelle: yes, I do. So maybe this was broken before, I get that. | 22:06 |
miguelgrinberg | But now that we understand the problem, shouldn't we rename it to stop, if that is all it does? | 22:06 |
stevelle | Frankly I think it was named wrong before, under the pretense that it worked. | 22:06 |
miguelgrinberg | stevelle: well, I think it tried to do the right thing, right? It stopped, then notified the other handler that does the start | 22:07 |
stevelle | In my eyes it tried to pawn part of it's job off on another handler. I can change the name though. | 22:10 |
miguelgrinberg | stevelle: we have many other restart handlers that do actually a restart | 22:13 |
stevelle | agreed | 22:13 |
miguelgrinberg | if we can't do a clean restart with rsync, then to make it clear that we have this limitation the handlers should be called stop and start, or something like that | 22:13 |
openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Remove temporary upgrade task that removes profile https://review.openstack.org/217367 | 22:13 |
stevelle | miguelgrinberg: I'll rename it in another revision, just digging a little further into this gate failure I see there are no logs for that container, but the container was created and given a backing store etc. | 22:15 |
*** shoutm has joined #openstack-ansible | 22:15 | |
*** sdake_ has joined #openstack-ansible | 22:15 | |
stevelle | the missing container logs is a bit troubling | 22:15 |
miguelgrinberg | stevelle: before you invest a lot of time, I would do a recheck to see if it is repeatable | 22:17 |
miguelgrinberg | stevelle: and regarding the handlers, the other thing to double check is that ansible invokes multiple handlers one after the other, and not in parallel. That could also break your rsync restarts. | 22:18 |
*** sdake has quit IRC | 22:18 | |
stevelle | miguelgrinberg: I did test it, n=9, to ensure consistent execution order and service always up at the end | 22:19 |
*** jmckind has quit IRC | 22:19 | |
miguelgrinberg | stevelle: ok, sounds good | 22:23 |
*** cloudtrainme has quit IRC | 22:31 | |
*** shoutm has quit IRC | 22:32 | |
openstackgerrit | Steve Lewis proposed stackforge/os-ansible-deployment: Ensure rsync restarts fully during swift setup https://review.openstack.org/217341 | 22:32 |
stevelle | that is another way to recheck ^ | 22:32 |
*** spotz is now known as spotz_zzz | 22:33 | |
*** shoutm has joined #openstack-ansible | 22:36 | |
*** scarlisle has quit IRC | 22:37 | |
coolj | odyssey4me: sorry i had to step away for a meeting. the issue is effecting nova-compute on all the hosts--when it tries to get image info from glance it gets back an empty Image object and raises an AttributeError | 22:38 |
*** sdake_ is now known as sdake | 22:41 | |
*** yaya has quit IRC | 22:42 | |
*** markvoelker has quit IRC | 22:46 | |
*** jwagner is now known as jwagner_away | 22:53 | |
*** markvoelker has joined #openstack-ansible | 22:54 | |
coolj | another new 10.1.11 deployment hit | 23:07 |
coolj | https://bugs.launchpad.net/openstack-ansible/+bug/1488315 | 23:07 |
openstack | Launchpad bug 1488315 in openstack-ansible trunk "The python-requests package is pulled in by apt via dependency" [Medium,Confirmed] | 23:07 |
miguelgrinberg | stevelle: regarding order of handler execution. Looks like ansible executes handlers not in the order they are given in the notify line, they run in the order in which they are defined in the handlers file. Something to keep in mind. | 23:14 |
stevelle | miguelgrinberg: good to know | 23:15 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Enable tempest testing of ceilometer https://review.openstack.org/209568 | 23:16 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Add configurable ssh_delay https://review.openstack.org/216429 | 23:16 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Remove read only disks from lvm candidates https://review.openstack.org/216327 | 23:16 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Updated kilo to include fix for CVE-2015-3241 - 26 Aug 2015 https://review.openstack.org/217114 | 23:16 |
*** CheKoLyN has quit IRC | 23:23 | |
*** alop has quit IRC | 23:54 | |
*** shoutm_ has joined #openstack-ansible | 23:54 | |
*** klindgren has quit IRC | 23:57 | |
*** shoutm has quit IRC | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!