*** sdake has quit IRC | 00:03 | |
*** stevemar has joined #openstack-ansible | 00:06 | |
*** galstrom_zzz is now known as galstrom | 00:19 | |
*** BjoernT has quit IRC | 00:31 | |
*** stevemar has quit IRC | 00:32 | |
*** daneyon_ has quit IRC | 00:55 | |
*** darrenc is now known as darrenc_afk | 01:40 | |
*** darrenc_afk is now known as darrenc | 01:52 | |
*** galstrom is now known as galstrom_zzz | 01:58 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 02:33 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated for the 11.0.1 tag https://review.openstack.org/179906 | 02:38 |
---|---|---|
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Allow configuration of Nova SQLAlchemy options https://review.openstack.org/178870 | 02:41 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated RabbitMQ to the new release version https://review.openstack.org/178230 | 02:42 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated RabbitMQ to the new release version https://review.openstack.org/178230 | 02:43 |
*** galstrom_zzz is now known as galstrom | 02:49 | |
*** JRobinson__ is now known as JRobinson__afk | 03:22 | |
*** JRobinson__afk is now known as JRobinson__ | 03:33 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Set the neutron default workers As stated in neutron's default conf file: # This feature is experimental until issues are addressed and testing # has been # enabled for various plugins for compatibility. https://review.openstack.org/180020 | 03:37 |
*** stevemar has joined #openstack-ansible | 03:43 | |
*** galstrom is now known as galstrom_zzz | 03:56 | |
*** galstrom_zzz is now known as galstrom | 04:17 | |
*** sdake_ has quit IRC | 04:38 | |
*** galstrom is now known as galstrom_zzz | 04:49 | |
*** sdake has joined #openstack-ansible | 04:57 | |
*** JRobinson__ has quit IRC | 05:49 | |
*** openstackgerrit has quit IRC | 06:23 | |
*** openstackgerrit has joined #openstack-ansible | 06:23 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 07:06 |
*** sandywalsh has quit IRC | 07:11 | |
*** sandywalsh has joined #openstack-ansible | 07:12 | |
*** stevemar has quit IRC | 07:22 | |
*** JRobinson__ has joined #openstack-ansible | 08:14 | |
openstackgerrit | Serge van Ginderachter proposed stackforge/os-ansible-deployment: [WIP] first shot at implementing Ceph/RBD support https://review.openstack.org/173229 | 09:09 |
*** JRobinson__ has quit IRC | 09:44 | |
*** markvoelker has quit IRC | 09:48 | |
openstackgerrit | Matt Thompson proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 10:26 |
openstackgerrit | Matt Thompson proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 10:28 |
openstackgerrit | Matt Thompson proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 10:37 |
*** sandywalsh has quit IRC | 11:12 | |
*** sandywalsh has joined #openstack-ansible | 11:14 | |
*** markvoelker has joined #openstack-ansible | 11:51 | |
openstackgerrit | Matt Thompson proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 11:54 |
*** galstrom_zzz is now known as galstrom | 11:59 | |
*** galstrom is now known as galstrom_zzz | 12:53 | |
*** KLevenstein has joined #openstack-ansible | 13:00 | |
cloudnull | mornings | 13:00 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 13:51 |
cloudnull | mattt ^ same changes you made, however on a local AIO test i found that if the reqs are moved to the build server everything works while also not adding additional installed pkgs on the deployment host. | 13:52 |
cloudnull | and i figured while all of gating seems jammed up this was a good time to make the change. | 13:53 |
mattt | cloudnull: dang, didn't realise that was necessary, thanks ! | 13:53 |
mattt | (s/necessary/possible/) | 13:54 |
cloudnull | its not really necessary as in it didn't work before . what you had was 100% functional, It just added deps to the deployment host. i couldve left it alone (sorry for being OCD) :\ . | 13:55 |
mattt | cloudnull: now that i think about it, do we need to explicitly install those in the heat_engine container ? | 13:55 |
*** Mudpuppy has joined #openstack-ansible | 13:55 | |
mattt | cloudnull: no that makes perfect sense, i didn't like having to install those on teh deployment host | 13:55 |
cloudnull | no. theyll be build as part of the heat deps / wheels | 13:55 |
mattt | k | 13:55 |
cloudnull | pbr makes me sad , in a lot of ways. | 13:56 |
mattt | i'm trying to understand what we need to do w/ https://review.openstack.org/#/c/179671/ | 13:58 |
mattt | seems the heat devs are reluctant to specify versions | 13:58 |
mattt | which results in a wheel that isn't functional | 13:58 |
*** fawadkhaliq has joined #openstack-ansible | 13:59 | |
cloudnull | well we'd have to change our setup to make that happy. | 14:00 |
cloudnull | we build the wheel using the setup.py | 14:00 |
cloudnull | however when the contrib plugin is deployed we clone and install using pip. | 14:00 |
cloudnull | so we could remove the clone part and simply install the built wheel. | 14:01 |
mattt | oh derp | 14:01 |
mattt | i thought that's what we were doing | 14:01 |
cloudnull | but thats a change on our part. | 14:01 |
* cloudnull double checking. | 14:01 | |
cloudnull | https://github.com/stackforge/os-ansible-deployment/blob/master/playbooks/roles/os_heat/tasks/heat_install_plugins.yml | 14:01 |
mattt | whatever we do isn't working | 14:01 |
mattt | but if i remove and manually install with pip it looks OK | 14:02 |
*** sigmavirus24_awa is now known as sigmavirus24 | 14:03 | |
cloudnull | yea, i think if we just specifiy the package name without using the cloned source , we should be ok. | 14:03 |
cloudnull | which i think is "heat_contrib_extraroute" | 14:03 |
* cloudnull looking | 14:03 | |
mattt | cloudnull: yep | 14:03 |
mattt | because it's getting built as a wheel so that whole task file is not needed | 14:04 |
cloudnull | yup. | 14:04 |
cloudnull | just the entry item in the pip list | 14:04 |
cloudnull | we could make the change now and have it depend on https://review.openstack.org/180044 because we know its coming in some form or another. | 14:05 |
mattt | i dislike this version tho: heat-contrib-extraroute==2015.1.0rc1.post224 | 14:07 |
mattt | that's really misleading | 14:07 |
sigmavirus24 | mattt: it is | 14:07 |
cloudnull | it is indeed. | 14:07 |
mattt | cloudnull: i'll put together the PR tho and we can have it there incase we want to put it through | 14:08 |
cloudnull | +1 | 14:08 |
sigmavirus24 | Anyone else still having problems with testrepository: http://logs.openstack.org/29/178429/9/check/os-ansible-deployment-dsvm-check-commit/cf9aeba/logs/ansible-logging/ansible.log ? I rebased onto cloudnull's review from yesterday (which merged already) and I'm still seeing failures | 14:22 |
sigmavirus24 | Oh testscenarios too now | 14:23 |
mattt | and fixtures | 14:25 |
cloudnull | sigmavirus24 the trail of fail is strong these last few days | 14:29 |
sigmavirus24 | Yep | 14:30 |
cloudnull | mattt found the extra pins that we needed and added them to https://review.openstack.org/180044 | 14:30 |
cloudnull | all because pbr > 0.11.0 | 14:30 |
cloudnull | and the fact that we are using a heat contrib plugin | 14:31 |
cloudnull | which turns out is not really anything that people in the heat community want to maintain... | 14:31 |
cloudnull | :\ | 14:31 |
openstackgerrit | Matt Thompson proposed stackforge/os-ansible-deployment: Install heat-contrib-extraroute wheel https://review.openstack.org/180170 | 14:33 |
sigmavirus24 | cloudnull: yeah, that stuff is just disappointing | 14:43 |
sigmavirus24 | If you don't want to maintain it move it to the -archive section of gerrit | 14:43 |
sigmavirus24 | release a last packaged version of it and then mark it as deprecated | 14:43 |
sigmavirus24 | If people want to maintain it, they'll step up to do so | 14:43 |
*** sdake_ has joined #openstack-ansible | 14:43 | |
*** yaya has joined #openstack-ansible | 14:47 | |
*** sdake has quit IRC | 14:47 | |
cloudnull | +1 | 14:52 |
cloudnull | i think that those contrib packages should be moved into stackforge and be pulled into mainline heat due to some method of promotion | 14:52 |
sigmavirus24 | cloudnull: so you're volunteering to write a heat spec for it? Cool =P | 14:53 |
cloudnull | sure. | 14:53 |
cloudnull | in a week or two | 14:53 |
cloudnull | im all over it | 14:53 |
sigmavirus24 | :D | 14:57 |
sigmavirus24 | b3rnard0: ^ | 14:57 |
* b3rnard0 adds it as an action item | 15:01 | |
*** sdake_ is now known as sdake | 15:01 | |
*** erikmwilson is now known as Guest32970 | 15:04 | |
*** Guest32970 has quit IRC | 15:04 | |
*** erikmwilson_ has joined #openstack-ansible | 15:04 | |
*** galstrom_zzz is now known as galstrom | 15:04 | |
*** jaypipes has quit IRC | 15:13 | |
sigmavirus24 | b3rnard0++ | 15:18 |
*** jwagner_away is now known as jwagner | 15:36 | |
*** sdake has quit IRC | 15:37 | |
*** sdake has joined #openstack-ansible | 15:40 | |
cloudnull | meeting times | 15:59 |
cloudnull | ^ cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, mancdaz, dolphm, _shaps_, BjoernT, claco, echiu, dstanek, jwagner | 16:00 |
*** yaya has quit IRC | 16:00 | |
dstanek | o/ | 16:00 |
stevelle | o/ | 16:00 |
Apsu | o/ | 16:01 |
d34dh0r53 | % | 16:01 |
*** daneyon has joined #openstack-ansible | 16:02 | |
b3rnard0 | oh oh | 16:02 |
jwagner | present | 16:02 |
d34dh0r53 | bah-lak-e? | 16:02 |
Sam-I-Am | helloooo | 16:02 |
rromans | hi | 16:02 |
*** daneyon_ has joined #openstack-ansible | 16:03 | |
cloudnull | so to start we only have two new issues that need to be brought up | 16:03 |
cloudnull | https://bugs.launchpad.net/openstack-ansible/+bug/1451217 | 16:03 |
openstack | Launchpad bug 1451217 in openstack-ansible "net.netfilter.nf_conntrack_max not changed on swift server" [Undecided,New] | 16:03 |
palendae | Hi | 16:04 |
cloudnull | according to the issue bjorne wants to up the net.netfilter.nf_conntrack_max to > 256K | 16:05 |
cloudnull | when it pertains to swift storage nodes. | 16:05 |
cloudnull | in master /kilo we set # playbooks/roles/openstack_hosts/defaults/main.yml:82: - { key: 'net.netfilter.nf_conntrack_max', value: 262144 } | 16:05 |
cloudnull | same in juno | 16:05 |
*** daneyon has quit IRC | 16:06 | |
cloudnull | check that we set it on neutron and compute nodes in juno | 16:06 |
cloudnull | so it looks like we could add another var to swift container nodes for juno and call it good? Apsu thoughts ? | 16:07 |
Apsu | cloudnull: looking | 16:07 |
cloudnull | in juno # rpc_deployment/vars/config_vars/container_config_nova_compute.yml:61: - { key: 'net.netfilter.nf_conntrack_max', value: 262144 } | 16:07 |
Apsu | cloudnull: Yep. Agreed. Just apply to the swift container config vars too | 16:08 |
*** yaya has joined #openstack-ansible | 16:08 | |
cloudnull | next https://bugs.launchpad.net/openstack-ansible/+bug/1450580 | 16:09 |
openstack | Launchpad bug 1450580 in openstack-ansible "run-upgrade/plays should remove existing rpc_release.link" [Undecided,New] | 16:09 |
cloudnull | this issue is a matter of cleaning up the rpc_release llnk file | 16:10 |
cloudnull | while it was left for redundancy in an environment that may be running "other" links besides what we use upstream, it seems that its not something that we need. | 16:11 |
cloudnull | and removing the rpc_release.link file shouldn't impact any other links that may be present. | 16:11 |
cloudnull | i think we can do something like # ansible hosts:all_containers -m shell -a "rm /root/.pip/links.d/rpc_release.link" somewhere around https://github.com/stackforge/os-ansible-deployment/blob/master/scripts/run-upgrade.sh#L291 and it should be all good. | 16:14 |
Apsu | Sounds reasonable to me. | 16:14 |
*** erikmwilson has joined #openstack-ansible | 16:15 | |
cloudnull | done. | 16:15 |
cloudnull | any other issues that we want to talk about ? | 16:15 |
cloudnull | or any other things we want to talk about ? | 16:15 |
cloudnull | we're waiting on zuul http://status.openstack.org/zuul/ | 16:16 |
Apsu | I like turtles | 16:16 |
cloudnull | which seems like its very much stuck | 16:16 |
Apsu | Zuul will run on Zuul's own time | 16:16 |
*** Bjoern__ has joined #openstack-ansible | 16:17 | |
stevelle | adjust your clocks, Zuul is the reference for all time | 16:17 |
Apsu | #WWZD | 16:17 |
palendae | Probably crash | 16:18 |
Apsu | More like WWZR? When Will Zuul Run? | 16:18 |
palendae | Never | 16:18 |
palendae | Or always | 16:18 |
cloudnull | or always never | 16:18 |
palendae | When Will Zuul Finish | 16:18 |
d34dh0r53 | almost always never | 16:18 |
Bjoern__ | Hey | 16:18 |
*** Bjoern__ is now known as BjoernT | 16:18 | |
cloudnull | palendae: tick question. | 16:19 |
cloudnull | *trick | 16:19 |
cloudnull | hi BjoernT | 16:20 |
cloudnull | did you have anything that you wanted to highlight ? | 16:20 |
BjoernT | let me look | 16:20 |
BjoernT | I think we covered the cinder backup job | 16:20 |
*** daneyon_ has quit IRC | 16:21 | |
*** daneyon has joined #openstack-ansible | 16:21 | |
BjoernT | #1440784 Spice console is freezing | 16:22 |
BjoernT | I believe we were not able to reproduce it | 16:22 |
BjoernT | did we look in the code what might cause the situation ? | 16:23 |
*** yaya has quit IRC | 16:23 | |
cloudnull | if the spice console freezes in production and nobody's able to reproduce it, did it really freeze? :) | 16:23 |
Apsu | NOTABUG | 16:24 |
cloudnull | BjoernT i have not. | 16:24 |
d34dh0r53 | OPINION | 16:24 |
BjoernT | what happened was that port 6082 is open but not giving back the normal directory listing, if you just do a curl http://localhost:6082 | 16:24 |
BjoernT | not even using the spice_auto.html, it did not serve any files basically | 16:25 |
BjoernT | so my hope was that we look at the code to get an idea what to check next time | 16:26 |
cloudnull | so nope nothing new on that front. | 16:27 |
BjoernT | ok apsu : #1436999 LVM/DM/UDEV out of sync inside cinder-volumes container | 16:27 |
* cloudnull not it | 16:28 | |
Apsu | BjoernT: Ah yes, our old friend. | 16:28 |
BjoernT | yeah, won't just die | 16:29 |
BjoernT | I like to get udev working right in containers, at least for cinder volumes | 16:29 |
Apsu | I can provide slightly more info, but I'm not sure what the solution is just yet. I assume apparmor profiles combined with lvm.conf using udev. | 16:29 |
Apsu | But... I was helping you guys troubleshoot an issue like this and we noticed that there was a mismatch between what dmsetup though the device major/minor was and the actual device file in /dev | 16:30 |
Apsu | Which pretty much has to mean udev not triggering to rewrite the device file | 16:30 |
*** yaya has joined #openstack-ansible | 16:31 | |
BjoernT | right, I think it is a simple as /dev is not mounted over udev | 16:31 |
Apsu | Well udev is enabled in the containers already. The lvm.conf has it disabled for LVM currently, but... | 16:32 |
BjoernT | other than that, I don't have anything else | 16:32 |
Apsu | Supposedly Ubuntu's apparmor profiles for LXC already deal with udev cascading issues | 16:32 |
*** yaya has quit IRC | 16:32 | |
Apsu | But we'll have to test the current state of the union there, with udev enabled for lvm | 16:32 |
BjoernT | ok | 16:33 |
openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Harden our copy of Glance's policy https://review.openstack.org/178429 | 16:33 |
sigmavirus24 | (Sorry for disrupting the meeting) | 16:33 |
cloudnull | so https://bugs.launchpad.net/openstack-ansible/+bug/1436999 | 16:36 |
openstack | Launchpad bug 1436999 in openstack-ansible "LVM/DM/UDEV out of sync inside cinder-volumes container" [Medium,Confirmed] - Assigned to Evan Callicoat (apsu-2) | 16:36 |
cloudnull | is a wip | 16:36 |
Apsu | yep | 16:36 |
cloudnull | b3rnard0 ^ action item | 16:36 |
b3rnard0 | k | 16:36 |
cloudnull | anything else ? | 16:36 |
Apsu | Three cheers for device file shenanigans | 16:37 |
cloudnull | pi pi ta! | 16:37 |
*** jaypipes has joined #openstack-ansible | 16:38 | |
cloudnull | ok so we're done here ? | 16:38 |
Apsu | I sure am :D | 16:38 |
cloudnull | thanks everyone | 16:41 |
b3rnard0 | thank you | 16:41 |
b3rnard0 | notes here: https://etherpad.openstack.org/p/openstack_ansible_bug_triage.2015-05-05-16.00 | 16:42 |
cloudnull | tyvm b3rnard0 | 16:43 |
*** sdake_ has joined #openstack-ansible | 16:45 | |
*** sdake has quit IRC | 16:49 | |
*** erikmwilson has left #openstack-ansible | 17:05 | |
*** jwagner is now known as jwagner_away | 17:11 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 17:12 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Harden our copy of Glance's policy https://review.openstack.org/178429 | 17:13 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Install heat-contrib-extraroute wheel https://review.openstack.org/180170 | 17:13 |
*** sdake_ has quit IRC | 17:14 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated RabbitMQ to the new release version https://review.openstack.org/178230 | 17:19 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Update the tempest install environment https://review.openstack.org/180044 | 17:26 |
cloudnull | ^ mattt i squashed your heat-contrib commit into that one because its still blocked due to the clone . | 17:28 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated RabbitMQ to the new release version https://review.openstack.org/178230 | 17:29 |
cloudnull | sigmavirus24/miguelgrinberg: i hate your heat and your pbr. | 17:30 |
cloudnull | :) | 17:30 |
miguelgrinberg | cloudnull: :) | 17:31 |
sigmavirus24 | hey hey hey | 17:31 |
sigmavirus24 | Don't lump me in with either heat or pbr | 17:31 |
miguelgrinberg | I sort of got in a beef with a core dev over this | 17:31 |
sigmavirus24 | I don't work on either | 17:31 |
cloudnull | sigmavirus24 your fault via proxy | 17:31 |
cloudnull | :D | 17:31 |
* sigmavirus24 throws arms up | 17:32 | |
cloudnull | hahahah | 17:32 |
miguelgrinberg | cloudnull: are we okay staying with the old pbr, or do we need to change how we install the plugin? | 17:32 |
miguelgrinberg | I assume they are going to remove the versions from the plugins | 17:32 |
cloudnull | i built the plugin and added it to our index. mattt made a commit that simply installs the plugin instead of attempting to build it. | 17:33 |
cloudnull | and i recently squashed that into the above commit . | 17:33 |
miguelgrinberg | ah okay, I thought I saw you guys pinned pbr to the previous release | 17:33 |
cloudnull | we did. but heat has a run time dependency on pbr which requires lover versions of an ever increasing set of newer packages. | 17:34 |
cloudnull | *lower | 17:34 |
miguelgrinberg | fun | 17:34 |
cloudnull | so attempting to pin pbr is next to impossible over any given period of time. | 17:34 |
cloudnull | it worked yesterday, but not today. | 17:35 |
Sam-I-Am | cloudnull: you're adding too much hipster | 17:36 |
cloudnull | this is true . | 17:36 |
cloudnull | so maybe its not all sigmavirus24 's fault. :D | 17:36 |
miguelgrinberg | there is a summit design session to discuss bringing all those plugins into the main codebase, some of the cores feel like they don't want to officially support them, so not sure if it'll happen | 17:37 |
cloudnull | add in extra route move the rest to stackforge | 17:37 |
cloudnull | and then we can forget that they even exist. | 17:37 |
miguelgrinberg | that'd be nice | 17:38 |
cloudnull | it would, but that'd be too easy . | 17:38 |
cloudnull | so im sure theres a way that we can make this all more complicated. | 17:38 |
cloudnull | and drag out the migration for a few more releases | 17:39 |
*** sdake has joined #openstack-ansible | 17:41 | |
*** yaya has joined #openstack-ansible | 17:57 | |
*** logan2 has quit IRC | 18:28 | |
*** jwagner_away is now known as jwagner | 18:29 | |
*** fawadkhaliq has quit IRC | 18:30 | |
*** logan2 has joined #openstack-ansible | 18:31 | |
openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Harden our copy of Glance's policy https://review.openstack.org/178429 | 18:37 |
*** sdake_ has joined #openstack-ansible | 18:43 | |
*** fawadkhaliq has joined #openstack-ansible | 18:43 | |
*** sdake has quit IRC | 18:47 | |
*** BjoernT has quit IRC | 18:53 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Set the neutron default workers As stated in neutron's default conf file: # This feature is experimental until issues are addressed and testing # has been # enabled for various plugins for compatibility. https://review.openstack.org/179973 | 18:58 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Set the neutron default workers As stated in neutron's default conf file: # This feature is experimental until issues are addressed and testing # has been # enabled for various plugins for compatibility. https://review.openstack.org/180020 | 18:58 |
*** yaya has quit IRC | 19:01 | |
d34dh0r53 | cloudnull: think we may have problems with rsyslog logging, do you have a 'trusted' install running somewhere? | 19:04 |
cloudnull | Trusted ? | 19:05 |
d34dh0r53 | :) | 19:09 |
d34dh0r53 | been running for a while? | 19:09 |
*** KLevenstein__ has joined #openstack-ansible | 19:12 | |
*** KLevenstein has quit IRC | 19:12 | |
*** KLevenstein__ is now known as KLevenstein | 19:12 | |
cloudnull | Would the iad3 lab work? | 19:13 |
cloudnull | Its not running current but its close enough I'd say? | 19:13 |
d34dh0r53 | cloudnull: yeah, maybe, I'll look there | 19:13 |
cloudnull | OK. | 19:13 |
*** subscope_ has joined #openstack-ansible | 19:27 | |
*** Bjoern__ has joined #openstack-ansible | 19:32 | |
*** sdake has joined #openstack-ansible | 19:42 | |
*** sdake has quit IRC | 19:42 | |
*** sdake_ has quit IRC | 19:45 | |
*** sdake has joined #openstack-ansible | 19:46 | |
d34dh0r53 | cloudnull: yep, no mysql logs in the /var/log/log-storage/*galera direcories, spool files are in the containers and everything looks to be setup correctly but no logs | 19:47 |
d34dh0r53 | IAD3 btw | 19:47 |
*** sdake_ has joined #openstack-ansible | 19:47 | |
*** sdake has quit IRC | 19:51 | |
sigmavirus24 | d34dh0r53: wrong channel? | 19:52 |
d34dh0r53 | it was mentioned earlier so I thought I'd put it here, this is referring to os-a-d logging not rpc-extras work | 19:53 |
sigmavirus24 | okay | 19:53 |
openstackgerrit | Matthew Kassawara proposed stackforge/os-ansible-deployment: Add documentation to user config file https://review.openstack.org/180296 | 19:57 |
*** sdake has joined #openstack-ansible | 20:01 | |
*** sdake_ has quit IRC | 20:05 | |
*** sdake has quit IRC | 20:10 | |
*** sdake has joined #openstack-ansible | 20:10 | |
cloudnull | d34dh0r53 we're you able to figure out the issue? | 20:18 |
d34dh0r53 | cloudnull: not yet, still looking into it | 20:21 |
cloudnull | rerunning the client role didn | 20:22 |
cloudnull | *didn't make it happy | 20:22 |
cloudnull | ? | 20:22 |
cloudnull | also what was spooling ? | 20:22 |
d34dh0r53 | cloudnull: there are spool files, but no log files on the logging server | 20:23 |
d34dh0r53 | cloudnull: the only files on the logging server are cron.log,dhclient.log,kernel.log etc... | 20:23 |
cloudnull | do we see this behavior on an AIO ? | 20:27 |
d34dh0r53 | yep | 20:27 |
*** KLevenstein has quit IRC | 20:31 | |
cloudnull | d34dh0r53 i had an env up running kilo this morning where i had all the logs on the log server. | 20:35 |
cloudnull | im kicking kilo & master right now | 20:35 |
cloudnull | will know more in a min | 20:35 |
*** persia has quit IRC | 20:40 | |
*** d34dh0r53 has quit IRC | 20:41 | |
*** persia has joined #openstack-ansible | 20:41 | |
*** persia has quit IRC | 20:41 | |
*** persia has joined #openstack-ansible | 20:41 | |
*** d34dh0r53 has joined #openstack-ansible | 20:45 | |
*** fawadkhaliq has quit IRC | 20:51 | |
*** KLevenstein has joined #openstack-ansible | 20:52 | |
*** jwagner is now known as jwagner_away | 21:05 | |
cloudnull | d34dh0r53 on my two AIO's i have a whole mess of logs in [ /openstack/aio1_rsyslog_container-*/log-storage/**/* ] | 21:07 |
d34dh0r53 | cloudnull: look in log-storage/<one of your galera containers> and see if you see galara_server* | 21:08 |
cloudnull | i put keys on 162.209.124.25 | 21:08 |
d34dh0r53 | cloudnull: http://paste.openstack.org/show/215069/ | 21:10 |
d34dh0r53 | cloudnull: and then http://paste.openstack.org/show/215071/ | 21:11 |
d34dh0r53 | I've tried changing all kinds of stuff in the rsyslog_client config and nothing so far is making it go | 21:11 |
cloudnull | i see the entries in /etc/rsyslog.d/99-galera-rsyslog-client.conf | 21:11 |
d34dh0r53 | yep | 21:12 |
d34dh0r53 | that file looks fine, I've tried moving the ip down to the bottom, removing the RFC tag thing, changing the tag to be unique, cant figure it out, rsyslog is definitely shipping logs, just not those logs | 21:12 |
cloudnull | -rw-r----- 1 mysql root 39638 May 5 21:12 galera_server_error.log | 21:13 |
cloudnull | -rw-rw---- 1 mysql mysql 1599759 May 5 21:13 mysql-slow.log | 21:14 |
cloudnull | maybe a perms issue as to why its not being shipped ? | 21:14 |
d34dh0r53 | possibly, although I just got it to ship on IAD3 | 21:15 |
d34dh0r53 | one sed, gonna muck around with your aio | 21:16 |
d34dh0r53 | s/sed/sec | 21:16 |
cloudnull | d34dh0r53 changing to the perms to 644 made the logs ship | 21:20 |
cloudnull | look on my server under /openstack/aio1_rsyslog_container-cd7d7179/log-storage/aio1_galera_container-6f9a76ba/galera_server_error.log | 21:20 |
d34dh0r53 | yep, appears so | 21:21 |
cloudnull | so d34dh0r53 maybe the correct action here is to, within the rsyslog_client role, ensure that permissions are 0644 ? | 21:25 |
cloudnull | that way all logs regestered via the client are ensured to log correctly. | 21:25 |
cloudnull | or we can move the logs into /var/log from within the galera_server role instead of having them in the /var/log/mysql directory. | 21:26 |
d34dh0r53 | hmm | 21:30 |
d34dh0r53 | any security implications to making them 644? | 21:31 |
cloudnull | yes. | 21:32 |
palendae | Would those security implications be the same if you put them in /var/log? | 21:32 |
d34dh0r53 | :) that's what I thought | 21:32 |
*** KLevenstein__ has joined #openstack-ansible | 21:33 | |
cloudnull | d34dh0r53 if the logs were in /var/log/ instead they'd be craeted with owner "mysql:adm" which syslog is a member of adm so that should work | 21:34 |
d34dh0r53 | ok, that works | 21:34 |
*** sdake_ has joined #openstack-ansible | 21:35 | |
*** KLevenstein has quit IRC | 21:35 | |
*** KLevenstein__ is now known as KLevenstein | 21:35 | |
d34dh0r53 | is syslog set in [mysqld_safe] in my.cnf? | 21:35 |
cloudnull | we have nothing there. | 21:36 |
cloudnull | except [mysqld_safe] | 21:36 |
cloudnull | socket = /var/run/mysqld/mysqld.sock | 21:36 |
cloudnull | nice = 0 | 21:36 |
d34dh0r53 | if we set syslog that may work as well | 21:36 |
*** JRobinson__ has joined #openstack-ansible | 21:37 | |
d34dh0r53 | that will work for the error log, not sure about the slow-query-log | 21:37 |
cloudnull | if we simply put syslog under the [mysqld_safe] section error logging should go there by defualt. | 21:38 |
d34dh0r53 | no, log-error takes precedence | 21:38 |
d34dh0r53 | just read that | 21:38 |
*** sdake has quit IRC | 21:39 | |
d34dh0r53 | I think /var/log is the best option | 21:39 |
d34dh0r53 | going to try that on my aio | 21:41 |
cloudnull | im am too :) | 21:42 |
*** stevemar has joined #openstack-ansible | 21:47 | |
cloudnull | so that didnt work | 21:47 |
d34dh0r53 | really? | 21:49 |
d34dh0r53 | same perms? | 21:49 |
cloudnull | yup. mysql:root | 21:49 |
d34dh0r53 | we could chgrp to adm later in the play | 21:49 |
*** sdake has joined #openstack-ansible | 21:50 | |
d34dh0r53 | bbl, going to gym | 21:51 |
cloudnull | d34dh0r53 https://gist.github.com/cloudnull/c50f14f2b82f6850fe48 | 21:52 |
cloudnull | that made it go for me | 21:52 |
cloudnull | i moved the logs to a directory with a stick adm group. | 21:53 |
*** sdake__ has joined #openstack-ansible | 21:54 | |
*** sdake_ has quit IRC | 21:54 | |
*** Bjoern__ has left #openstack-ansible | 21:55 | |
*** sdake has quit IRC | 21:58 | |
*** galstrom is now known as galstrom_zzz | 22:22 | |
*** stevemar has quit IRC | 22:24 | |
*** KLevenstein has quit IRC | 22:31 | |
*** subscope_ has quit IRC | 22:34 | |
*** erikmwilson_ is now known as erikmwilson | 22:35 | |
*** Mudpuppy has quit IRC | 22:45 | |
*** jbweber has quit IRC | 23:18 | |
*** jbweber has joined #openstack-ansible | 23:18 | |
d34dh0r53 | cloudnull: nice, want me to PR that? | 23:20 |
cloudnull | sure. | 23:20 |
cloudnull | unless you have a better solution | 23:20 |
cloudnull | its up to you brother. | 23:21 |
cloudnull | :) | 23:21 |
d34dh0r53 | not at the moment, I'll PR that in the morning, maybe I'll come up with something between now and then | 23:21 |
cloudnull | kk | 23:21 |
d34dh0r53 | have a good one sir, I'm getting out of here | 23:21 |
cloudnull | have a good night. | 23:22 |
*** sdake__ has quit IRC | 23:42 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!