| *** alop has quit IRC | 00:08 | |
| *** sdake_ has quit IRC | 00:18 | |
| *** Guest42648 has quit IRC | 00:20 | |
| *** sdake has joined #openstack-ansible | 00:33 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Block requests 2.8.0, oslo.messaging 2.6.0 & cap WebOb<1.5.0 https://review.openstack.org/233756 | 00:33 |
|---|---|---|
| openstackgerrit | Merged openstack/openstack-ansible: Fix run-aio-build.sh for curl one-liner https://review.openstack.org/232964 | 00:34 |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible: Implement Neutron LBAAS using haproxy https://review.openstack.org/220365 | 00:41 |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible: Implement Neutron LBAAS using haproxy https://review.openstack.org/220365 | 00:48 |
| *** BjoernT has quit IRC | 00:56 | |
| -cloudnull- the master gate is blocked due to several oslo related changes | 00:59 | |
| -cloudnull- the master gate is blocked due to several oslo related changes current package differences which were released within the last 24 hours https://gist.github.com/cloudnull/a551628cc136a5036cfb | 01:00 | |
| openstackgerrit | Merged openstack/openstack-ansible: Make bootstrap-ansible script compatible with RHEL https://review.openstack.org/233330 | 01:04 |
| *** opal has joined #openstack-ansible | 01:11 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38498: Audit log file permissions https://review.openstack.org/232056 | 01:21 |
| *** opal has left #openstack-ansible | 01:21 | |
| mhayden | prometheanfire: todo improved ^^ | 01:21 |
| cloudnull | lol | 01:21 |
| prometheanfire | mhayden: done | 01:22 |
| * mhayden high fives prometheanfire | 01:23 | |
| mhayden | # TODO: bring beer for cloudnull and prometheanfire | 01:23 |
| prometheanfire | scotch | 01:23 |
| mhayden | # and wine cooler for d34dh0r53 | 01:23 |
| prometheanfire | lol | 01:23 |
| * cloudnull will work for beer | 01:23 | |
| cloudnull | :) | 01:23 |
| prometheanfire | ofc | 01:23 |
| prometheanfire | any alcohol is appreciated | 01:23 |
| *** elo has quit IRC | 01:32 | |
| *** tlian has quit IRC | 01:45 | |
| *** Mudpuppy has joined #openstack-ansible | 02:23 | |
| *** ggillies has quit IRC | 02:25 | |
| *** sdake has quit IRC | 02:29 | |
| *** kerwin_bai has joined #openstack-ansible | 02:52 | |
| *** CBR09 has joined #openstack-ansible | 02:53 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Block requests 2.8.0, oslo.messaging 2.6.0 & cap WebOb<1.5.0 https://review.openstack.org/233756 | 02:56 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updates the lint check to ignore templates https://review.openstack.org/231101 | 02:58 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement nova venv support https://review.openstack.org/230727 | 02:59 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Seperated out Telemetry Alarming (Aodh) https://review.openstack.org/232224 | 03:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement swift venv support https://review.openstack.org/230733 | 03:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement neutron venv support https://review.openstack.org/230726 | 03:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement keystone venv support https://review.openstack.org/229513 | 03:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement horizon venv support https://review.openstack.org/229226 | 03:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement heat venv support https://review.openstack.org/229225 | 03:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement glance venv support https://review.openstack.org/229221 | 03:01 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement ceilometer venv support https://review.openstack.org/229212 | 03:01 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement cinder venv support https://review.openstack.org/225463 | 03:01 |
| *** jhesketh has quit IRC | 03:08 | |
| *** jhesketh has joined #openstack-ansible | 03:13 | |
| *** elo has joined #openstack-ansible | 03:25 | |
| *** sdake has joined #openstack-ansible | 03:33 | |
| *** sdake has quit IRC | 03:37 | |
| *** elo has quit IRC | 03:37 | |
| *** sdake has joined #openstack-ansible | 03:38 | |
| *** sdake has quit IRC | 04:30 | |
| *** sdake has joined #openstack-ansible | 04:31 | |
| *** kerwin_bai has quit IRC | 04:33 | |
| *** kerwin_bai has joined #openstack-ansible | 04:56 | |
| *** opal has joined #openstack-ansible | 05:19 | |
| *** opal has left #openstack-ansible | 05:32 | |
| *** javeriak has joined #openstack-ansible | 05:37 | |
| *** elo has joined #openstack-ansible | 05:39 | |
| *** elo has quit IRC | 05:39 | |
| *** daneyon has joined #openstack-ansible | 05:40 | |
| *** daneyon has quit IRC | 05:41 | |
| *** javeriak has quit IRC | 06:11 | |
| *** Mudpuppy has quit IRC | 06:14 | |
| *** Mudpuppy_ has joined #openstack-ansible | 06:14 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Break apart and document the upgrade process https://review.openstack.org/224137 | 06:20 |
| *** javeriak has joined #openstack-ansible | 06:21 | |
| *** openstack has joined #openstack-ansible | 06:30 | |
| *** daneyon has joined #openstack-ansible | 06:34 | |
| *** javeriak has joined #openstack-ansible | 06:39 | |
| *** daneyon has quit IRC | 06:39 | |
| *** javeriak_ has joined #openstack-ansible | 06:43 | |
| *** javeriak has quit IRC | 06:44 | |
| *** javeriak_ has quit IRC | 06:59 | |
| *** gparaskevas has joined #openstack-ansible | 07:13 | |
| *** jhesketh has quit IRC | 07:22 | |
| *** jhesketh has joined #openstack-ansible | 07:23 | |
| *** neilus has joined #openstack-ansible | 07:24 | |
| *** javeriak has joined #openstack-ansible | 07:25 | |
| *** javeriak has quit IRC | 07:26 | |
| *** javeriak has joined #openstack-ansible | 07:26 | |
| *** javeriak has quit IRC | 07:28 | |
| *** daneyon has joined #openstack-ansible | 07:29 | |
| *** javeriak has joined #openstack-ansible | 07:29 | |
| *** daneyon has quit IRC | 07:34 | |
| *** ggillies has joined #openstack-ansible | 07:37 | |
| *** persia has quit IRC | 07:50 | |
| *** persia has joined #openstack-ansible | 07:51 | |
| *** subscope has joined #openstack-ansible | 07:52 | |
| *** javeriak has quit IRC | 07:59 | |
| *** Mudpuppy_ has quit IRC | 08:03 | |
| *** CBR09 has left #openstack-ansible | 08:11 | |
| odyssey4me | mattt did you see my update in https://review.openstack.org/233172 ? | 08:17 |
| mattt | odyssey4me: i have now :) | 08:18 |
| mattt | odyssey4me: why does devstack still use the other library then? | 08:19 |
| odyssey4me | mattt according to sdague it doesn't | 08:20 |
| odyssey4me | but bear this in mind - galera still requires the other library | 08:20 |
| odyssey4me | there's a hard dependency there - that's why we don't remove the library altogether, all we do is remove it from the openstack services | 08:20 |
| *** javeriak has joined #openstack-ansible | 08:20 | |
| mattt | odyssey4me: https://github.com/openstack-dev/devstack/blob/35814a7b6e4248f3c890019a0eddee4b4b76c564/files/debs/keystone | 08:21 |
| mattt | odyssey4me: sorry https://github.com/openstack-dev/devstack/blob/master/files/debs/keystone | 08:21 |
| mattt | odyssey4me: https://github.com/openstack-dev/devstack/blob/master/files/debs/neutron | 08:21 |
| mattt | etc. etc. | 08:21 |
| odyssey4me | mattt that's likely for mysql | 08:21 |
| mattt | wut | 08:22 |
| *** daneyon has joined #openstack-ansible | 08:23 | |
| *** subscope has quit IRC | 08:23 | |
| mattt | odyssey4me: i'm happy to push that commit through, i'd just hate for us to have some issues surface in production as a result | 08:24 |
| *** subscope has joined #openstack-ansible | 08:24 | |
| *** mgoddard has joined #openstack-ansible | 08:25 | |
| *** javeriak has quit IRC | 08:26 | |
| *** sdake has quit IRC | 08:27 | |
| *** daneyon has quit IRC | 08:28 | |
| *** openstack has joined #openstack-ansible | 08:49 | |
| *** subscope has quit IRC | 09:03 | |
| *** harvy has quit IRC | 09:06 | |
| *** harvy has joined #openstack-ansible | 09:08 | |
| *** elo has joined #openstack-ansible | 09:11 | |
| *** elo has quit IRC | 09:11 | |
| *** subscope has joined #openstack-ansible | 09:15 | |
| *** daneyon has joined #openstack-ansible | 09:17 | |
| *** daneyon has quit IRC | 09:22 | |
| evrardjp | hello everyone | 09:31 |
| gparaskevas | yellow! | 09:31 |
| robak | any reason why I'd be getting error like this, when running ansible/nova.py for openstack vm launching? "fatal: [localhost] => One or more undefined variables: list object has no element 0" | 09:34 |
| *** openstackstatus has joined #openstack-ansible | 09:37 | |
| *** ChanServ sets mode: +v openstackstatus | 09:37 | |
| *** ashisjain has joined #openstack-ansible | 09:38 | |
| -openstackstatus- NOTICE: gerrit is undergoing an emergency restart to investigate load issues | 09:40 | |
| *** ChanServ changes topic to "gerrit is undergoing an emergency restart to investigate load issues" | 09:40 | |
| *** kerwin_bai has quit IRC | 09:41 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible: Implementation of keepalived for haproxy https://review.openstack.org/234063 | 09:41 |
| *** kerwin_bai has joined #openstack-ansible | 09:41 | |
| evrardjp | I'd like to help with reviews, but it seems it's a bad day for gerrit | 09:44 |
| ashisjain | hello | 09:50 |
| ashisjain | Need some help :) | 09:50 |
| ashisjain | I was finally able to install osad on 5 nodes | 09:50 |
| ashisjain | Need some help to stabilise it | 09:51 |
| ashisjain | My neutron services are continuosly oscillating in on/of mode | 09:51 |
| ashisjain | I have also seen some errors in galera as well as rmq | 09:51 |
| ashisjain | I have tried restarting rabbitmq-server and than neutron-server, it seems to be solving the problem for few minutes but than all the neutron services running on agents go down | 09:52 |
| ashisjain | when I run neutron agent-list I see variable number of ':-) " and 'xxx' all the time | 09:53 |
| ashisjain | and at times I see all 'xxx' | 09:53 |
| ashisjain | Here is one of the error which I am seeing in neutron-server log : oslo_messaging.rpc.dispatcher TimeoutError: QueuePool limit of size 30 overflow 10 reached, connection timed out, timeout 120 | 09:54 |
| ashisjain | In rabbitmq I see lot of errors like this "no exchange 'reply_7037f84350cd48088c3f1088542ce1b0' in vhost '/'"" | 09:55 |
| ashisjain | and when I run rabbitmq list_exchanges I am unable to find any exchange with above name | 09:56 |
| odyssey4me | fyi mancdaz http://docs.openstack.org/developer/openstack-ansible/install-guide/app-minorupgrade.html | 09:56 |
| odyssey4me | o/ evrardjp | 09:57 |
| ashisjain | I have tried restarting rabbitmq, neutron-server and mysql till now, but nothing seems to be helping | 09:57 |
| odyssey4me | ashisjain do you have proper network time consistency? | 09:57 |
| odyssey4me | ie do you have ntp setup on your hosts to a reliable source? | 09:57 |
| odyssey4me | robak uh, I'm guessing that you're referring to the ansible modules for openstack, rather than https://github.com/openstack/openstack-ansible which is the focus of this channel | 09:59 |
| ashisjain | odyssey4me: Yes all the hosts are in sync with respect to ntp | 09:59 |
| odyssey4me | robak if you're looking for assistance with using the openstack modules for ansible, you'll likely have better luck in #ansible - we can try to help, but most of us are not necessary familiar with the old modules | 09:59 |
| odyssey4me | (we have our own) | 09:59 |
| ashisjain | here are the ntp servers address | 09:59 |
| ashisjain | server 1.in.pool.ntp.org server 1.asia.pool.ntp.org server 2.asia.pool.ntp.org | 09:59 |
| odyssey4me | ashisjain and have you confirmed that they're all in sync from a time standpoint? | 10:00 |
| *** openstackgerrit has quit IRC | 10:01 | |
| *** openstackgerrit has joined #openstack-ansible | 10:02 | |
| ashisjain | odyssey4me: yes all the 5 nodes are in sync from time standpoint. | 10:02 |
| ashisjain | all have the same time and date | 10:03 |
| ashisjain | I have also tried increasing the max connection setting in my.cnf of galera on one of the lxc host but that also has not helped | 10:09 |
| *** daneyon has joined #openstack-ansible | 10:11 | |
| odyssey4me | ashisjain if you monitor your neutron server log, what do you see? | 10:12 |
| ashisjain | odyssey4me: there are continuous error messages of "RROR oslo_messaging.rpc.dispatcher [req-caebfc5b-d195-4e98-87b9-a43a697b85c0 ] Exception during message handling: QueuePool limit of size 30 overflow 10 reached, connection timed out, timeout 120" | 10:14 |
| ashisjain | followed by stack trace | 10:14 |
| ashisjain | odyssey4me: Just one question how is this whole osad suppose to work which mysql it is going to use or which neutron-server it is going to use as I have got 3 infra hosts. | 10:15 |
| *** daneyon has quit IRC | 10:16 | |
| odyssey4me | ashisjain try setting 'neutron_rpc_conn_pool_size' to a number higher than 30 in user_variables.yml (30 is the default), then re-run os-neutron-install.yml | 10:19 |
| ashisjain | odyssey4me: the aove error was from one of the neutron server node | 10:19 |
| odyssey4me | ashisjain you can inspect the haproxy configuration to see how the load balancing is setup | 10:19 |
| ashisjain | I another node I see the following message "Oct 13 15:53:43 openstack007_neutron_server_container-28aea5e2 neutron-server: 2015-10-12 16:35:22.910 15319 CRITICAL neutron [-] OperationalError: (OperationalError) (1045, "Access denied for user 'neutron'@'openstack006' (using password: YES)") None None" | 10:19 |
| odyssey4me | ashisjain ah, so you have a problem there - it would seem that you may not have all the neutron servers setup the same way? | 10:20 |
| odyssey4me | did you fully populate user_secrets properly? | 10:20 |
| odyssey4me | have you compared the default user_secrets with yours to ensure that you have the full set of vars needed? | 10:20 |
| odyssey4me | have you edited anything in the code tree? | 10:20 |
| ashisjain | odyssey4me: I have used the script which comes with osad for user secret generation | 10:21 |
| ashisjain | No I have not modified the code | 10:23 |
| ashisjain | odyssey4me: What is this "have you compared the default user_secrets with yours to ensure that you have the full set of vars needed" ? | 10:23 |
| ashisjain | I have run the os-neutron-install.yml for neutron | 10:24 |
| ashisjain | and I have run it many times | 10:24 |
| ashisjain | because of failures here and there | 10:24 |
| ashisjain | shall I re-run the neutron playbook with the settings as you have suggested? | 10:26 |
| odyssey4me | yup, you'll need to figure out why all your neutron servers aren't able to read the DB | 10:27 |
| odyssey4me | but increasing the number of RPC pools for rabbit access is not a bad thing | 10:27 |
| ashisjain | neutron_rpc_conn_pool_size: 40 | 10:27 |
| *** kerwin_bai has quit IRC | 10:27 | |
| ashisjain | Is this value fine? | 10:27 |
| odyssey4me | I'd say perhaps 100 would be better | 10:27 |
| ashisjain | okay | 10:28 |
| ashisjain | what shall i do about that password errror? | 10:28 |
| ashisjain | access error? | 10:28 |
| ashisjain | Running neutron playbook will not fix that if at all their is a setup issue? | 10:28 |
| odyssey4me | check neutron.conf on all the neutron servers and validate that they all have the right user & password values | 10:28 |
| odyssey4me | then check each Galera DB to validate that they're all happy and in sync | 10:29 |
| ashisjain | neutron server or agent conf? | 10:29 |
| odyssey4me | you need to validate the health of your environment, which is not something I can step you through in detail | 10:29 |
| ashisjain | okay I will check out all these details | 10:29 |
| odyssey4me | whatever is giving you the db errors, which I expect would be the neutron server | 10:29 |
| ashisjain | and let u know | 10:29 |
| tiagogomes | cloudnull, do you know whether it is possible to create HA routers by default? | 10:32 |
| evrardjp | mhayden: what's funny is that most of the changes can be done thanks to open source software, like OSSEC for example | 10:33 |
| evrardjp | mhayden: (about security changes and followups, in the openstack-ansible-security role) | 10:33 |
| *** harvy has quit IRC | 10:38 | |
| ashisjain | odyssey4me: I have checked the username/password for all the 3 neutron server containers and all of them are in sync with each other and also user_secrets.yml | 10:43 |
| ashisjain | odyssey4me: Another step as you have suggested is to look into the db, can you please which tables in which db I need to check? | 10:44 |
| odyssey4me | ashisjain not the tables - check whether the cluster's synchronisation is healthy | 10:44 |
| odyssey4me | it's a mariadb cluster, so check whether they're all up to date | 10:44 |
| odyssey4me | ashisjain this has some clues: http://docs.openstack.org/developer/openstack-ansible/install-guide/ops-galera-recoverymulti.html | 10:45 |
| odyssey4me | otherwsie check the mariadb docs | 10:45 |
| ashisjain | odyssye4me: hatop -s /var/run/haproxy.stat suggest all the nodes are up | 10:46 |
| ashisjain | including galera | 10:47 |
| odyssey4me | ashisjain that tells you the ports are up, it doesn't tell you whether the mariadb is healthy | 10:47 |
| ashisjain | odyssey4me: yup I will check the health of clusters | 10:47 |
| odyssey4me | ashisjain you should probably do the same for rabbitmq | 10:48 |
| ashisjain | okay | 10:48 |
| ashisjain | when shall I rerun the neutron playbook | 10:48 |
| ashisjain | after I suppose fixing all this issue? | 10:48 |
| odyssey4me | I thought you had already? | 10:48 |
| *** kerwin_bai has joined #openstack-ansible | 10:48 | |
| odyssey4me | but yes - verify that your infrastructure is healthy from the bottom-up, as would be standard for any troubleshooting situation | 10:49 |
| ashisjain | No not yet I thought I should verify the passwords first | 10:49 |
| openstackgerrit | Merged openstack/openstack-ansible: Break apart and document the upgrade process https://review.openstack.org/224137 | 10:49 |
| ashisjain | Okay I will run the neutron playbook with higher value for rpc.... | 10:50 |
| *** javeriak has joined #openstack-ansible | 10:52 | |
| ashisjain | odyssey4me: galera cluster is healthy..here is the paste"http://paste.openstack.org/show/476102/" | 10:54 |
| *** javeriak has quit IRC | 10:55 | |
| ashisjain | odyssey4me:Is their a similar command to check the rabbitmq cluster health? | 10:55 |
| *** subscope has quit IRC | 10:56 | |
| mattt | ashisjain: rabbitmqctl cluster_status | 10:57 |
| *** jaypipes has joined #openstack-ansible | 10:59 | |
| ashisjain | rabbitmq cluster status seems okay http://paste.openstack.org/show/476107/ | 11:02 |
| ashisjain | mattt: thanks for the command | 11:03 |
| odyssey4me | ashisjain ok, if the agent status is still bouncing all the time - tail each neutron server's log to see whether you see anything | 11:03 |
| *** daneyon has joined #openstack-ansible | 11:05 | |
| *** daneyon has quit IRC | 11:10 | |
| *** ChanServ changes topic to "Topic: Launchpad: https://launchpad.net/openstack-ansible Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Repo rename from stackforge/os-ansible-deployment to openstack/openstack-ansible happens Sept 11 2015 23:00 to 23:30. See https://review.openstack.org/#/c/200730/" | 11:15 | |
| -openstackstatus- NOTICE: Gerrit has been restarted and is responding to normal load again. | 11:15 | |
| *** subscope has joined #openstack-ansible | 11:38 | |
| pellaeon | Hi, I have glance_nfs_client settings, but the glance containers don't mount NFS on boot | 11:54 |
| pellaeon | `df` doesn't show the mountpoint, while `mount` shows: | 11:55 |
| pellaeon | /dev/mapper/ansible--vg-root on /var/lib/glance/images type ext4 (rw,intr,soft,_netdev) | 11:55 |
| pellaeon | I can attach to the container and `mount -a` and it will mount successfully | 11:56 |
| pellaeon | I'm also seeing dmesg: | 11:57 |
| pellaeon | type=1400 audit(1444735720.105:160): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxc-openstack" name="/run/rpc_pipefs/" pid=3230 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs" flags="rw" | 11:57 |
| pellaeon | this was solved by adding `mount fstype=rpc_pipefs` to apparmor rules, reference http://bridge.grumpy-troll.org/2014/03/lxc-routed-on-ubuntu/ | 11:58 |
| pellaeon | but after solving this it still doesn't mount automatically | 11:59 |
| *** daneyon has joined #openstack-ansible | 11:59 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: [WIP] Test reduced constraints https://review.openstack.org/234169 | 12:01 |
| *** tlian has joined #openstack-ansible | 12:04 | |
| *** daneyon has quit IRC | 12:04 | |
| mhayden | evrardjp: you might be right, but OSSEC certainly isn't trivial to configure ;) | 12:25 |
| mhayden | happy tuesday, folks | 12:25 |
| ashisjain | odyssey4me: hello. | 12:27 |
| ashisjain | odyssey4me: My playbook run finished and here is what I am seeing in errors as I tail the log | 12:28 |
| ashisjain | http://paste.openstack.org/show/476122/ | 12:29 |
| ashisjain | http://paste.openstack.org/show/476121/ | 12:29 |
| ashisjain | http://paste.openstack.org/show/476123/ | 12:29 |
| ashisjain | These are the logs from 3 different neutron server host, 1st is from the host where haproxy is also running | 12:30 |
| ashisjain | This looks more of a rabbitmq issue now | 12:30 |
| ashisjain | Here is the run of rabbitmqctl list_exchanges "http://paste.openstack.org/show/476124/" | 12:32 |
| ashisjain | Here is a paste of log from rabbitmq host which also seems to be logging the error of channel not found | 12:34 |
| ashisjain | http://paste.openstack.org/show/476125/ | 12:34 |
| odyssey4me | ashisjain last time I dug into logs and saw those, it was kinda normal behaviour - however that was over a year ago so things may have changed | 12:35 |
| odyssey4me | perhaps someone else can comment on whether this is normal | 12:35 |
| ashisjain | odyssey4me: all the neutron agents are down | 12:37 |
| ashisjain | Where is the exchanges information defined? | 12:41 |
| odyssey4me | ashisjain they're created by each service | 12:43 |
| ashisjain | L3 agent on host is giving this warning 015-10-13 18:20:32.122 30825 WARNING neutron.agent.l3.agent [req-4e88b1f1-d54b-4ee0-a401-daf2077783b1 ] l3-agent cannot check service plugins enabled on the neutron server. Retrying. Detail message: Timed out waiting for a reply to message ID cef380d0ced14256a45f95bee7c7abb3 | 12:43 |
| odyssey4me | most of them are dynamic | 12:43 |
| odyssey4me | many of them also get deleted automatically | 12:43 |
| odyssey4me | that's why I'm not sure whether that's expected behavior or not | 12:44 |
| ashisjain | odyssey4me: okay will wait for someone to help verify this | 12:44 |
| odyssey4me | ashisjain I'd advise you to keep digging meanwhile | 12:45 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38500: No UID 0 accounts except root https://review.openstack.org/232070 | 12:45 |
| ashisjain | yeah i will keep doing it | 12:45 |
| odyssey4me | use google too - you're now in openstack territory, not OSA territory | 12:45 |
| *** woodard has joined #openstack-ansible | 12:46 | |
| ashisjain | odyssey4me: Sure | 12:47 |
| ashisjain | odyssey4me: thanks for your help and time | 12:47 |
| ashisjain | and patience too :) | 12:48 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38501, V-38573: Disable accounts after failed logins https://review.openstack.org/232074 | 12:50 |
| *** daneyon has joined #openstack-ansible | 12:53 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-3851{1,2,3}, V-38686: IPv4 security controls https://review.openstack.org/232088 | 12:58 |
| *** daneyon has quit IRC | 12:58 | |
| *** scarlisle has joined #openstack-ansible | 13:07 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Block/cap incompatible libraries https://review.openstack.org/233756 | 13:07 |
| *** KLevenstein has joined #openstack-ansible | 13:09 | |
| *** alejandrito has joined #openstack-ansible | 13:09 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Glance Configuration for Liberty https://review.openstack.org/229967 | 13:13 |
| odyssey4me | :( mattt I rebased https://review.openstack.org/229967 and lost your vote | 13:13 |
| mhayden | a swift re-review is required | 13:14 |
| * mhayden giggles | 13:14 | |
| mhayden | oh, i need more caffeine | 13:14 |
| mattt | odyssey4me: no worries | 13:15 |
| odyssey4me | thanks mattt | 13:16 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38622: Restricted mail relaying https://review.openstack.org/234204 | 13:20 |
| *** maximov has joined #openstack-ansible | 13:27 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38683: Check for non-unique usernames https://review.openstack.org/234209 | 13:28 |
| *** harvy has joined #openstack-ansible | 13:28 | |
| *** mgoddard_ has joined #openstack-ansible | 13:29 | |
| ashisjain | How to use rabbitmq in a standalone mode in osad? | 13:30 |
| ashisjain | I have already setup rabbitmq in osad in a cluster mode .... is it possible to change it now? | 13:31 |
| *** mgoddard has quit IRC | 13:32 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38681: GID's in /etc/passwd & /etc/group https://review.openstack.org/234215 | 13:35 |
| *** Mudpuppy has joined #openstack-ansible | 13:43 | |
| *** daneyon has joined #openstack-ansible | 13:48 | |
| *** daneyon has quit IRC | 13:53 | |
| *** phalmos has joined #openstack-ansible | 13:55 | |
| *** woodard has quit IRC | 13:57 | |
| *** woodard has joined #openstack-ansible | 14:00 | |
| *** k_stev has joined #openstack-ansible | 14:00 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:01 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-51739: LSM device labeling exception https://review.openstack.org/234227 | 14:02 |
| odyssey4me | ashisjain sure, remove two rabbitmq servers from the cluster - remove them from your openstack_user_config - remove the containers from the inventory - destroy the containers on the hosts - then re-run setup-openstack to reconfigure all the openstack bits | 14:03 |
| ashisjain | odyssey4me: How can I remove rabbitmq from openstack_user_config, there is no separate section on user_config | 14:06 |
| ashisjain | on rabbitmq | 14:06 |
| ashisjain | If I just stop the 2 rabbitmq containers will that not help? | 14:06 |
| odyssey4me | ashisjain oh, I think you may need ot set the affinity to 0 - I'm not sure exactly, but I know it's possible | 14:06 |
| ashisjain | where is this affinity set? | 14:07 |
| *** Bjoern_ has joined #openstack-ansible | 14:08 | |
| Bjoern_ | Do we know when we're going to fix the requests issue in master ? | 14:11 |
| *** Bjoern_ is now known as BjoernT | 14:11 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38699: Public directories exception https://review.openstack.org/234235 | 14:11 |
| odyssey4me | BjoernT it's an upstream issue - they're releasing another RC today or tomorrow. We do have a workaround fix in though to unblock us: https://review.openstack.org/233756 | 14:13 |
| odyssey4me | That took around 16 hours of my life to figure out that I will never get back. | 14:13 |
| BjoernT | i see, still in review | 14:14 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38685: Temporary accounts (exception) https://review.openstack.org/234237 | 14:14 |
| odyssey4me | BjoernT yes, it was just finalised - waiting for it to pass again before we push it through. | 14:14 |
| BjoernT | Yepp, requirements.txt sucks from upstream | 14:14 |
| odyssey4me | BjoernT not really, dependent libraries do things outside of the openstack community's control | 14:15 |
| *** neilus has quit IRC | 14:15 | |
| odyssey4me | they decide to more strictly enforce things suddenly | 14:15 |
| BjoernT | lol, it still sucks | 14:15 |
| odyssey4me | so it seems likely that better gating will be enforced on the libraries to find these issues early on | 14:15 |
| odyssey4me | and of course the same holds true for us | 14:15 |
| odyssey4me | note that you only know about this issue because of our gating - it has not affected an actual tagged release | 14:16 |
| BjoernT | yes I know | 14:16 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-58901: sudo requires auth https://review.openstack.org/234239 | 14:20 |
| *** jmckind has joined #openstack-ansible | 14:21 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated the repo-build process https://review.openstack.org/230716 | 14:26 |
| *** phalmos has quit IRC | 14:28 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38697: Sticky bit (exception) https://review.openstack.org/234249 | 14:30 |
| cloudnull | tiagogomes: if you pull this change in https://review.openstack.org/#/c/233389/ you should be able to create ha routers by default the main issue is that we have the l3ha config in the l3 config file and neutron expects it in the neutron.conf file | 14:30 |
| tiagogomes | cloudnull ah! | 14:31 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-386{85,90}: Temporary/emergency accounts (exception) https://review.openstack.org/234237 | 14:33 |
| *** ganderson has joined #openstack-ansible | 14:38 | |
| *** Mudpuppy has quit IRC | 14:38 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-386{67,70,95,96}, V-38700: Run AIDE via cron https://review.openstack.org/233231 | 14:38 |
| *** Mudpuppy has joined #openstack-ansible | 14:38 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-386{67,70,95,96,98}, V-38700: Run AIDE via cron https://review.openstack.org/233231 | 14:39 |
| *** ganderson has quit IRC | 14:39 | |
| *** daneyon has joined #openstack-ansible | 14:42 | |
| *** ashisjain has quit IRC | 14:43 | |
| tiagogomes | is every python packages installed from the container repo? Or are there some exceptions | 14:44 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-51391: Initialize AIDE https://review.openstack.org/234264 | 14:44 |
| tiagogomes | I am asking because I am seeing allow_all_external in pip.conf. But I am not sure of what this setting really entails | 14:44 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Block/cap incompatible libraries https://review.openstack.org/233756 | 14:46 |
| *** ganderson has joined #openstack-ansible | 14:47 | |
| *** daneyon has quit IRC | 14:47 | |
| cloudnull | tiagogomes: all pip package installs are done from within the env unless an --isolated flag is set | 14:47 |
| tiagogomes | cloudnull ta | 14:48 |
| *** mgoddard_ has quit IRC | 14:50 | |
| *** mgoddard has joined #openstack-ansible | 14:50 | |
| *** BjoernT has quit IRC | 15:01 | |
| *** woodard_ has joined #openstack-ansible | 15:15 | |
| *** woodard_ has quit IRC | 15:16 | |
| *** woodard_ has joined #openstack-ansible | 15:16 | |
| *** woodard has quit IRC | 15:18 | |
| *** jmckind has quit IRC | 15:18 | |
| *** woodard has joined #openstack-ansible | 15:23 | |
| *** woodard_ has quit IRC | 15:26 | |
| *** phalmos has joined #openstack-ansible | 15:30 | |
| *** daneyon has joined #openstack-ansible | 15:36 | |
| *** daneyon has quit IRC | 15:41 | |
| *** jwagner is now known as jwagner_away | 15:45 | |
| *** daneyon has joined #openstack-ansible | 15:47 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement cinder venv support https://review.openstack.org/225463 | 15:49 |
| *** daneyon_ has joined #openstack-ansible | 15:49 | |
| *** jwagner_away is now known as jwagner | 15:49 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement neutron venv support https://review.openstack.org/230726 | 15:50 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement nova venv support https://review.openstack.org/230727 | 15:50 |
| *** daneyon has quit IRC | 15:53 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement aodh venv support https://review.openstack.org/233401 | 15:53 |
| *** mgoddard has quit IRC | 15:53 | |
| *** mgoddard has joined #openstack-ansible | 15:53 | |
| *** alop has joined #openstack-ansible | 15:54 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement aodh venv support https://review.openstack.org/233401 | 15:54 |
| *** fawadkhaliq has joined #openstack-ansible | 15:57 | |
| cloudnull | mattt: that those changes should address the rootwrap issue you were seeing | 16:00 |
| prometheanfire | time? | 16:00 |
| odyssey4me | bug triage cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, mancdaz, dolphm, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle | 16:03 |
| d34dh0r53 | o/ | 16:03 |
| evrardjp | o/ | 16:03 |
| cloudnull | o/ | 16:03 |
| jwagner | o/ | 16:03 |
| prometheanfire | \o | 16:03 |
| scarlisle | \o | 16:03 |
| odyssey4me | first up https://bugs.launchpad.net/openstack-ansible/+bug/1503411 | 16:03 |
| openstack | Launchpad bug 1503411 in openstack-ansible "Plays fail to mount NFS glance store inside glance containers if local_path is set to /var/lib/glance/images" [Undecided,New] | 16:03 |
| jwagner | i filed this one | 16:05 |
| jwagner | it only fails if the nfs mount is set to that path | 16:05 |
| jwagner | if u change it it works fine | 16:05 |
| jwagner | this is because that path is getting bind mounted into the container before the nfs plays run so it is already set up | 16:05 |
| cloudnull | jwagner: is it this path /var/lib/glance/nfs_images | 16:07 |
| odyssey4me | so it sounds to me like we're assuming that a file store is on the host when we shouldn't | 16:07 |
| odyssey4me | ie if the file store is an nfs store, we shouldn't bind mount it? | 16:07 |
| *** Bjoern_ has joined #openstack-ansible | 16:07 | |
| jwagner | cloudnull no that is the path that i set that worked | 16:07 |
| jwagner | if you set it to just /images it fails | 16:07 |
| jwagner | and the default / doc example is /images | 16:07 |
| cloudnull | ah. | 16:08 |
| jwagner | also if you overwrite that path, cinder.conf still uses /images | 16:08 |
| jwagner | it doesnt use the custom path u set up | 16:08 |
| *** harvy has quit IRC | 16:08 | |
| jwagner | so you have to then go overwrite the conf manually | 16:08 |
| *** phalmos_ has joined #openstack-ansible | 16:10 | |
| jwagner | not sure the correct way to fix it, but the mount point gets laid down in the container run, so it gets set pretty early | 16:10 |
| jwagner | and it always gets laid down as /images | 16:11 |
| *** woodard_ has joined #openstack-ansible | 16:11 | |
| jwagner | no matter if you change it in your config or not | 16:11 |
| *** woodard_ has quit IRC | 16:11 | |
| *** woodard_ has joined #openstack-ansible | 16:12 | |
| *** phalmos has quit IRC | 16:13 | |
| jwagner | https://etherpad.openstack.org/p/rpc_vnx_integration | 16:13 |
| jwagner | if you go to the GLANCE section in that etherpad you can see the manual steps you hvae to run to get it working | 16:13 |
| jwagner | line 55 | 16:13 |
| andymccr | imo we dont bind mount images dir | 16:13 |
| andymccr | you can already set the size of the glance container | 16:13 |
| andymccr | so increase that | 16:13 |
| stevelle | that makes it container-ephemeral? | 16:14 |
| odyssey4me | it's only used for the glance cache | 16:14 |
| odyssey4me | oh no, it's for the glance store in this case | 16:14 |
| *** woodard has quit IRC | 16:14 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated the neutron l3HA tool to use v3 https://review.openstack.org/229053 | 16:14 |
| stevelle | kinda want the option of a mount for that | 16:15 |
| scarlisle | what about for customers who want to utilize nfs for glance? | 16:15 |
| scarlisle | stevelle yea | 16:15 |
| andymccr | i dont like the idea of "well we're running out of space on the container so lets just push that issue to the host" that assumes infinite storage on the host which is not true | 16:15 |
| andymccr | so you are just pushing an issue to somewhere where you won't see it potentially, but when you do its more impactful | 16:15 |
| evrardjp | so we need to improve glance documentation/setup with nfs... | 16:17 |
| Sam-I-Am | lots of docs improvements are needed | 16:17 |
| andymccr | if we are going to bind mount that dir, then its more of a bug - since we shouldn't bind mount that dir when using nfs or swift or any non-local storage options | 16:17 |
| odyssey4me | so I think the bind-mount to the host should be an opt-in, rather than an opt-out | 16:18 |
| evrardjp | andymccr: +1 | 16:18 |
| jwagner | well even if you fix the doc the glance-api and registry configs get /images no matter if you set a custom mount or not | 16:18 |
| jwagner | i had to lineinfile replace them manually | 16:18 |
| jwagner | which wont survive a playbook run | 16:18 |
| evrardjp | it needs a complete study: how to make work glance better with NFS, that was my opinion | 16:20 |
| odyssey4me | evrardjp we have gate split work planned, and part of that work will be to implement an integration test and documentation for glance/nfs | 16:21 |
| andymccr | so we'd have to change teh cinder path based on glance's custom path? | 16:21 |
| andymccr | how about you just allow that cinder option to be a var and you can override it if you change it in glance | 16:21 |
| andymccr | then it becomes a docs change | 16:21 |
| odyssey4me | the issue here is that the bind mount is happening when the container should be mounting remotely instead | 16:21 |
| evrardjp | it's a large impact: if it's done on master, you'll use the new template module but cannot be straightforwardly backported in kilo + lxc containers setup + multi component change | 16:23 |
| evrardjp | impact is not the good term | 16:23 |
| evrardjp | sorry | 16:23 |
| evrardjp | odyssey4me: good to hear :) | 16:24 |
| odyssey4me | evrardjp a deployer can use the template module to override any settings | 16:24 |
| odyssey4me | in master and kilo | 16:24 |
| *** Bjoern_ has quit IRC | 16:24 | |
| stevelle | tagging this upgrade-impact now | 16:25 |
| odyssey4me | however as part of the glue that pulls this all together, we're making 'smart' decisions around where things should happen in a 'standard' environment | 16:25 |
| evrardjp | ok I was not sure config_template changes on each component was already merged in kilo | 16:26 |
| *** woodard has joined #openstack-ansible | 16:27 | |
| odyssey4me | evrardjp it'll be included in the next tag | 16:27 |
| evrardjp | ok | 16:27 |
| *** elo has joined #openstack-ansible | 16:27 | |
| odyssey4me | ok, so it seems we have two issues here | 16:29 |
| odyssey4me | one is that the location to mount should be customisable, but it's not | 16:29 |
| odyssey4me | and the other is that there's a bind mount intefering with mounts which are for remote resources | 16:30 |
| *** phalmos_ has quit IRC | 16:30 | |
| *** Bjoern_ has joined #openstack-ansible | 16:30 | |
| odyssey4me | any volunteers to pick this up? | 16:30 |
| *** woodard_ has quit IRC | 16:30 | |
| andymccr | i'll take it | 16:30 |
| andymccr | assigny | 16:30 |
| andymccr | do the typey typey | 16:30 |
| odyssey4me | importance? | 16:31 |
| andymccr | i think its pretty random, like we havnt run into this because nobody is really doing that so i'd put low-med? | 16:31 |
| stevelle | seems right to me | 16:31 |
| odyssey4me | I'm thinking medium. | 16:31 |
| odyssey4me | While it's not a common use-case, it seems, it has a high impact when you hit it. | 16:32 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated the neutron l3HA tool to use v3 https://review.openstack.org/229053 | 16:32 |
| odyssey4me | objections? | 16:32 |
| palendae | None here | 16:33 |
| evrardjp | none | 16:33 |
| cloudnull | nope | 16:33 |
| odyssey4me | next https://bugs.launchpad.net/openstack-ansible/+bug/1504226 | 16:33 |
| openstack | Launchpad bug 1504226 in openstack-ansible "nova management network should be dynamic" [Undecided,New] - Assigned to Rahul U Nair (rahulunair) | 16:33 |
| scarlisle | I think we're ok from support side. We have a couple of customers using netapp for glance images, but I think we have a workaround | 16:33 |
| odyssey4me | cloudnull ^ | 16:33 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38623: rsyslog file permissions https://review.openstack.org/234331 | 16:33 |
| *** gparaskevas has quit IRC | 16:33 | |
| odyssey4me | I think cloudnull has a review up for this already? | 16:34 |
| cloudnull | yup | 16:34 |
| *** subscope has quit IRC | 16:34 | |
| cloudnull | https://review.openstack.org/#/c/232666/ | 16:34 |
| cloudnull | related issue https://bugs.launchpad.net/openstack-ansible/+bug/1504208 | 16:34 |
| openstack | Launchpad bug 1504208 in openstack-ansible "cinder storage network should be dynamic" [Medium,In progress] - Assigned to Kevin Carter (kevin-carter) | 16:34 |
| cloudnull | and pr https://review.openstack.org/#/c/232637/ | 16:34 |
| cloudnull | both correct the same thing for the different services | 16:35 |
| odyssey4me | the impact here is that the address ends up being inappropriate | 16:36 |
| cloudnull | yes, however that causes traffic to go over the wrong network | 16:36 |
| odyssey4me | so for instance, cinder-volume publishes the management address, and therefore all the cinder traffic goes over the wrong network | 16:36 |
| odyssey4me | the same for the spice/vnc traffic | 16:37 |
| *** woodard_ has joined #openstack-ansible | 16:37 | |
| odyssey4me | marked as medium and assigned to cloudnull - any objections? | 16:37 |
| *** Bjoern_ has quit IRC | 16:37 | |
| stevelle | none | 16:37 |
| *** woodard_ has quit IRC | 16:37 | |
| *** woodard_ has joined #openstack-ansible | 16:38 | |
| cloudnull | tiagogomes: cc- on those issues, he help work through them | 16:38 |
| odyssey4me | ok, that's it for new bugs - are there any other bugs that need triage/discussion? | 16:38 |
| cloudnull | https://review.openstack.org/#/c/229053 -- https://bugs.launchpad.net/openstack-ansible/+bug/1499708 - | 16:40 |
| openstack | Launchpad bug 1499708 in openstack-ansible trunk "Migrate neutron-ha-tool.py to use Keystone API v3" [High,In progress] - Assigned to Kevin Carter (kevin-carter) | 16:40 |
| *** woodard has quit IRC | 16:40 | |
| cloudnull | tiagogomes, palendae, prometheanfire, d34dh0r53 i updated that review to make sure that the log error doesnt happen and it should now support routers that were created with ha=True | 16:40 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38546: Disable IPv6 system-wide https://review.openstack.org/234333 | 16:40 |
| palendae | cloudnull:Thanks | 16:40 |
| odyssey4me | ok, be warned that it will not pass the gate yet as liberty is broken right now thanks to library updates | 16:42 |
| odyssey4me | the fix is imminent | 16:42 |
| odyssey4me | so go ahead and review regardless - the gate doesn't currently test this anyway | 16:42 |
| odyssey4me | any other bugs/reviews? | 16:44 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-51391: Initialize AIDE https://review.openstack.org/234264 | 16:44 |
| *** Bjoern_ has joined #openstack-ansible | 16:45 | |
| *** mgoddard_ has joined #openstack-ansible | 16:45 | |
| palendae | Appears mhayden has a lot of security reviews up ;) | 16:45 |
| mhayden | pfft | 16:46 |
| mhayden | i just pushed in the last STIG control a few moments ago | 16:46 |
| evrardjp | disable ipv6 system-wide? Hopefully it's an opt-in :p | 16:47 |
| mhayden | definitely is | 16:47 |
| mhayden | that one gives me the sads | 16:47 |
| evrardjp | me too | 16:47 |
| palendae | mhayden: Complete noob question - where do the V-XXXXX designations come from? | 16:47 |
| evrardjp | it's documented | 16:48 |
| mhayden | https://www.stigviewer.com/stig/red_hat_enterprise_linux_6/ | 16:48 |
| palendae | Is it like CVEs, where there's a group publishing them? | 16:48 |
| mhayden | yeah, those are the ID's applied by the US Gov't | 16:48 |
| palendae | Gotcha | 16:48 |
| mhayden | the folks at UCF make a handy graphical viewer there | 16:48 |
| mhayden | the STIG downloads from the Govt are one big ol' fat zip file with everything under the sun in it | 16:48 |
| *** mgoddard has quit IRC | 16:49 | |
| palendae | So these are the things you follow to ensure the NSA can get in? :) | 16:49 |
| evrardjp | and friends | 16:49 |
| mhayden | SRSLY. U GUYS. | 16:49 |
| mhayden | :P | 16:49 |
| palendae | evrardjp: Well, I think they share it out | 16:49 |
| mhayden | palendae: it's best practices ;) | 16:49 |
| palendae | Though there are 4 other Eyes | 16:49 |
| evrardjp | depending on the country ;) | 16:49 |
| palendae | mhayden: Best for whom, HMMMMMM? | 16:49 |
| palendae | evrardjp: Right | 16:49 |
| cloudnull | setenforce 0 FTW! | 16:49 |
| mhayden | oh i was waiting for that | 16:49 |
| evrardjp | :D | 16:49 |
| *** mgoddard_ has quit IRC | 16:49 | |
| stevelle | I haven't found the one that says "Salt all passwords with THIS super-secure key and DSA-64" | 16:50 |
| * mhayden deducts one bottle of beer from cloudnull's tally | 16:50 | |
| palendae | Open all ports, disable firewalls, set banner to "HELLO NSA" | 16:50 |
| Bjoern_ | The nsa firiends already rewrote code so they can get in everywhere no need for the STIG stuff | 16:50 |
| cloudnull | palendae: lololol | 16:50 |
| *** Bjoern_ is now known as BjoernY | 16:50 | |
| evrardjp | stevelle: or set this wheel user password to "NSAforever" | 16:50 |
| palendae | BjoernY: But open source! | 16:50 |
| cloudnull | mhayden: :( | 16:51 |
| palendae | There *can't* be backdoors or bugs in stuff where people can read the code! | 16:51 |
| stevelle | needs more open | 16:51 |
| stevelle | drop the iptables rules | 16:51 |
| evrardjp | palendae: they call this "Frontdoor" | 16:51 |
| BjoernY | Yeah that worked when they hacked the ipsec stack in xxBSD | 16:51 |
| palendae | BjoernY: I'm being sarcastic :) | 16:51 |
| evrardjp | BjoernY: it's not proven, IIRC :) | 16:51 |
| BjoernY | Yes I know | 16:51 |
| palendae | -1 Day bugs! | 16:51 |
| *** BjoernY is now known as BjoernT | 16:52 | |
| stevelle | I've used an OS I would call a -1 day bug, before | 16:52 |
| evrardjp | sadly we all use NIC that have closed firmware... | 16:52 |
| mhayden | evrardjp: i'll go test the module change and see how it affects sysctl | 16:52 |
| palendae | stevelle: Windows? | 16:52 |
| stevelle | ofc | 16:53 |
| palendae | evrardjp: Wireless especially | 16:53 |
| evrardjp | palendae: on servers? >< | 16:53 |
| odyssey4me | no-one needs to write back doors, openssl is a very wide open door | 16:53 |
| evrardjp | but yeah | 16:53 |
| evrardjp | odyssey4me: true | 16:53 |
| evrardjp | ;) | 16:53 |
| evrardjp | mhayden: ok | 16:53 |
| palendae | evrardjp: Ok, well maybe not there | 16:53 |
| palendae | But wireless APs sure | 16:53 |
| evrardjp | mhayden: it's just I have the feeling that using modprobe to remove ipv6 wasn't the recent way of doing it | 16:54 |
| palendae | That firmware's more tightly controlled than standard ethernet NICs | 16:54 |
| mhayden | you could be right | 16:54 |
| palendae | odyssey4me: Yep, and it's really re-assuring to see the OpenBSD people tear it apart but not send patches upstream :( | 16:54 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Glance Configuration for Liberty https://review.openstack.org/229967 | 16:54 |
| evrardjp | palendae: there are other libs | 16:54 |
| evrardjp | nobody uses them or they are not as freely available | 16:55 |
| palendae | evrardjp: Right, more concerned about all the problems being laid bare and not being fixed | 16:55 |
| evrardjp | (licensing issues ... thanks lawyers!) | 16:55 |
| evrardjp | I guess we could all redo the world at some point, by setting another way to verify certificates, etc ... but I guess we should keep this conversation with a beer at the summit, right? ;) | 16:56 |
| odyssey4me | palendae have you read their reasons why? | 16:56 |
| odyssey4me | the reasons are that they did, then got sick of the upstream custodian ignoring them | 16:56 |
| evrardjp | they first blamed, which didn't help for good relations, IIRC | 16:57 |
| odyssey4me | the upstream custodian, they say, is more interested in making consulting money based on their crappy software's bugs | 16:57 |
| palendae | odyssey4me: Ah, a shame | 16:57 |
| palendae | Also not surprising | 16:57 |
| odyssey4me | yeah, that's why the bsd crowd decided to make their own new stuff | 16:57 |
| stevelle | which bsd crowd ;) | 16:58 |
| odyssey4me | it's quite a fascinating analysis for mere mortals like myself | 16:58 |
| evrardjp | http://it.slashdot.org/story/14/04/10/1343236/theo-de-raadts-small-rant-on-openssl | 16:58 |
| palendae | stevelle: I think Open | 16:58 |
| palendae | I mean, I get it | 16:58 |
| stevelle | BSD was like a fractal community, you see a complex thing, zoom, see another thing just as complex, zoom, repeat | 16:58 |
| palendae | stevelle: was? | 16:59 |
| stevelle | I stopped looking... | 16:59 |
| odyssey4me | https://www.google.co.uk/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0CCAQFjAAahUKEwik_Lfr9b_IAhVIWhQKHSf3C4w&url=http%3A%2F%2Fwww.libressl.org%2F&usg=AFQjCNE0mrEqSJRL6JQLratpfEt-hXJCOQ&sig2=gMKdTz7Qi_YXW8aY2IJaCQ&bvm=bv.104819420,d.bGg | 16:59 |
| stevelle | walked away before 2000 | 16:59 |
| odyssey4me | bah | 16:59 |
| odyssey4me | http://www.libressl.org/ | 16:59 |
| palendae | odyssey4me: Yeah, that's what I was referring to | 16:59 |
| mhayden | evrardjp: weird, you might be right on the v6 stuff | 16:59 |
| evrardjp | walked away later, but still they lay down interesting stuff/possible future problems | 16:59 |
| mhayden | it came up with v6 still enabled | 16:59 |
| evrardjp | mhayden: weird that I'm right? ;) | 17:00 |
| *** mgoddard has joined #openstack-ansible | 17:00 | |
| evrardjp | mhayden: also, pay attention to where you edit sysctl, sometimes it's a pain :p | 17:00 |
| mhayden | right | 17:00 |
| mhayden | so using sysctl killed v6 immediately -- no reboot req'd | 17:00 |
| mhayden | i'll use that instead | 17:00 |
| evrardjp | you may want to test it | 17:01 |
| evrardjp | if it stays upon reboots or ifup/ifdown s | 17:01 |
| mhayden | well it has to go into sysctl.conf to persist | 17:01 |
| mhayden | but it looks like ifup/ifdown makes stuff come up with v4-only | 17:01 |
| mhayden | i'll try it with .default.disable_ipv6 too | 17:02 |
| evrardjp | odyssey4me: did you look at the favicon of libressl? | 17:02 |
| evrardjp | I recall of having set default and all to sysctl using ansible sysctl module | 17:02 |
| odyssey4me | evrardjp heartbleed | 17:02 |
| evrardjp | touché! | 17:02 |
| odyssey4me | that's what got the BSD nuts on the rampage | 17:03 |
| palendae | Heartbleed was kind of genius, in that giving it a name and a logo raised awareness in a way a CVE number hasn't | 17:03 |
| odyssey4me | the flame war was awesome | 17:03 |
| evrardjp | yeah | 17:03 |
| evrardjp | the bug too | 17:03 |
| palendae | odyssey4me: What one aren't? | 17:03 |
| evrardjp | no log and having fun gathering passwords | 17:04 |
| evrardjp | it was world's biggest honeypot | 17:04 |
| palendae | Should still go back and read Tenanbaum (sp) and Torvalds on minix vs linux | 17:04 |
| evrardjp | mhayden: what's weird it's the handling of NICs for disable_ipv6... if it's a sysctl to disable ipv6 on a specific nic, it needs to be set in post-up in /etc/network/interfaces, IIRC, instead of sysctl | 17:05 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-38546: Disable IPv6 system-wide https://review.openstack.org/234333 | 17:06 |
| evrardjp | I'm off! | 17:06 |
| *** k_stev1 has joined #openstack-ansible | 17:07 | |
| *** k_stev has quit IRC | 17:07 | |
| *** woodard has joined #openstack-ansible | 17:09 | |
| cloudnull | have a good one evrardjp | 17:10 |
| *** kerwin_bai has quit IRC | 17:11 | |
| *** woodard_ has quit IRC | 17:12 | |
| *** b3rnard0 is now known as b3rnard0_away | 17:18 | |
| *** sdake has joined #openstack-ansible | 17:25 | |
| stevelle | sigmavirus24: any reason this doesn't have workflow? https://review.openstack.org/#/c/231870 | 17:28 |
| *** phalmos has joined #openstack-ansible | 17:29 | |
| sigmavirus24 | no clue stevelle | 17:29 |
| odyssey4me | stevelle nope, simply an oversight | 17:30 |
| stevelle | all good now | 17:30 |
| sigmavirus24 | probably thanks to gertty | 17:30 |
| *** sdake has quit IRC | 17:31 | |
| *** sdake has joined #openstack-ansible | 17:36 | |
| *** gparaskevas has joined #openstack-ansible | 17:44 | |
| *** sdake_ has joined #openstack-ansible | 17:52 | |
| *** sdake has quit IRC | 17:52 | |
| *** tiagogomes has quit IRC | 17:55 | |
| *** abitha has joined #openstack-ansible | 17:55 | |
| *** fawadkhaliq has quit IRC | 17:58 | |
| *** fawadkhaliq has joined #openstack-ansible | 18:07 | |
| *** gparaskevas has quit IRC | 18:09 | |
| *** b3rnard0_away is now known as b3rnard0 | 18:11 | |
| *** elo has quit IRC | 18:19 | |
| *** fawadkhaliq has quit IRC | 18:21 | |
| *** fawadkhaliq has joined #openstack-ansible | 18:21 | |
| *** fawadkhaliq has quit IRC | 18:24 | |
| *** fawadkhaliq has joined #openstack-ansible | 18:24 | |
| *** fawadkhaliq has quit IRC | 18:24 | |
| *** KLevenstein_ has joined #openstack-ansible | 18:25 | |
| *** phalmos has quit IRC | 18:25 | |
| *** subscope has joined #openstack-ansible | 18:26 | |
| *** ashishjain has joined #openstack-ansible | 18:26 | |
| *** KLevenstein has quit IRC | 18:28 | |
| *** KLevenstein_ is now known as KLevenstein | 18:28 | |
| *** elo has joined #openstack-ansible | 18:28 | |
| *** elo has quit IRC | 18:29 | |
| *** elo has joined #openstack-ansible | 18:30 | |
| *** phalmos has joined #openstack-ansible | 18:53 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Removed unnecessary comment in the user_secrets for ceph variable https://review.openstack.org/233152 | 18:56 |
| *** daneyon has joined #openstack-ansible | 18:56 | |
| ashishjain | hello | 18:58 |
| *** daneyon_ has quit IRC | 19:00 | |
| *** daneyon has quit IRC | 19:00 | |
| *** mgoddard has quit IRC | 19:01 | |
| *** alejandrito has quit IRC | 19:20 | |
| *** alejandrito has joined #openstack-ansible | 19:22 | |
| ashishjain | Hello | 19:22 |
| ashishjain | My neutron agent services are bouncing all the time in on and off state | 19:23 |
| ashishjain | I have tried restarting rabbitmq as well as re-running neutron playbook | 19:24 |
| ashishjain | Neutron l3 agent seems to be showing the warning "WARNING neutron.agent.l3.agent [req-8707164c-36e4-46d2-b1e9-eeb208110488 ] l3-agent cannot check service plugins enabled on the neutron server. Retrying. Detail message: Timed out waiting for a reply to message ID c32e4a4e1d774c2c909a59fdc0a60489" | 19:24 |
| ashishjain | I have shutdown 2 out of 3 rabbitmq servers considering that mirroring is probably not working as expected but I still the issue | 19:25 |
| ashishjain | In the rabbitmq log I see the following "Oct 14 00:44:55 openstack006_rabbit_mq_container-7df79f87 rabbit@openstack006_rabbit_mq_container-7df79f87: "no exchange 'reply_9443c66abbfb4cf3ad55183576dff90a' in vhost '/'"," | 19:26 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Use inventory instead of hostfile parameter https://review.openstack.org/231870 | 19:27 |
| ashishjain | any advice on how shall i go about debugging this up | 19:27 |
| ashishjain | In one of the neutron server I am seeing the following ct 14 01:08:05 openstack007_neutron_server_container-28aea5e2 neutron-server: 2015-10-12 17:26:40.242 18468 TRACE neutron OperationalError: (OperationalError) (1045, "Access denied for user 'neutron'@'openstack006' (using password: YES)") None None | 19:29 |
| *** alejandrito has quit IRC | 19:32 | |
| *** ganderson has quit IRC | 19:35 | |
| *** cloudtrainme has joined #openstack-ansible | 19:46 | |
| openstackgerrit | Merged openstack/openstack-ansible: Block/cap incompatible libraries https://review.openstack.org/233756 | 19:46 |
| openstackgerrit | Merged openstack/openstack-ansible: Add minor upgrade documentation to the install guide https://review.openstack.org/232522 | 19:46 |
| *** sdake has joined #openstack-ansible | 19:52 | |
| *** sdake_ has quit IRC | 19:52 | |
| *** ashishjain has quit IRC | 19:57 | |
| openstackgerrit | Merged openstack/openstack-ansible: Fix the nodepool file check https://review.openstack.org/233090 | 19:57 |
| *** mgoddard has joined #openstack-ansible | 19:59 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Cinder Configuration for Liberty https://review.openstack.org/227205 | 20:02 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Nova Configuration for Liberty https://review.openstack.org/227839 | 20:03 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Updates the lint check to ignore templates https://review.openstack.org/231101 | 20:04 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Updated the neutron l3HA tool to use v3 https://review.openstack.org/229053 | 20:05 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Updated the repo-build process https://review.openstack.org/230716 | 20:06 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update rabbitmq-server to v3.5.6-1 https://review.openstack.org/233700 | 20:06 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Switch from MySQL-python to PyMySQL https://review.openstack.org/233172 | 20:07 |
| BjoernT | Hey, did anyone test ldap by chance in Kilo ? | 20:07 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement Neutron LBAAS using haproxy https://review.openstack.org/220365 | 20:07 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: V-51391: Initialize AIDE https://review.openstack.org/234264 | 20:08 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Redirect "apt-get install -y" stdin to /dev/null https://review.openstack.org/233331 | 20:08 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Set Keystone endpoints to be versionless https://review.openstack.org/205192 | 20:08 |
| *** mgoddard has quit IRC | 20:10 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement cinder venv support https://review.openstack.org/225463 | 20:10 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement keystone venv support https://review.openstack.org/229513 | 20:11 |
| *** mgoddard has joined #openstack-ansible | 20:11 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement neutron venv support https://review.openstack.org/230726 | 20:12 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement horizon venv support https://review.openstack.org/229226 | 20:12 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement swift venv support https://review.openstack.org/230733 | 20:13 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement heat venv support https://review.openstack.org/229225 | 20:14 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement glance venv support https://review.openstack.org/229221 | 20:15 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement ceilometer venv support https://review.openstack.org/229212 | 20:15 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Seperated out Telemetry Alarming (Aodh) https://review.openstack.org/232224 | 20:16 |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement aodh venv support https://review.openstack.org/233401 | 20:16 |
| *** jmccrory has quit IRC | 20:17 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Implement nova venv support https://review.openstack.org/230727 | 20:17 |
| *** jmccrory has joined #openstack-ansible | 20:19 | |
| *** abitha has quit IRC | 20:21 | |
| sigmavirus24 | odyssey4me: cloudnull do you think it would be beneficial to use openstack-announce to announce osa releases? | 20:21 |
| odyssey4me | sigmavirus24: yes, and I think some smart work to automate the production of release notes from commit messages would be good too | 20:24 |
| sigmavirus24 | odyssey4me: I expect there's already automation in the openstack release managers channel | 20:25 |
| sigmavirus24 | they can probably share some things with you/us | 20:25 |
| odyssey4me | there's a lot we could do to cut down some of the manual work and keep people informed | 20:25 |
| * sigmavirus24 nods | 20:25 | |
| odyssey4me | sigmavirus24: will you be at the summit? I'd like to let some of the creative thought take root and maybe hack a few things. | 20:26 |
| *** jwagner is now known as jwagner_away | 20:26 | |
| sigmavirus24 | odyssey4me: I will not | 20:27 |
| *** jwagner_away is now known as jwagner | 20:28 | |
| odyssey4me | sigmavirus24: :( then we'll have to arrange another time - for now I'm out | 20:28 |
| odyssey4me | We can chat on the morrow. | 20:29 |
| sigmavirus24 | sounds good | 20:29 |
| odyssey4me | night all | 20:29 |
| *** cloudtrainme has quit IRC | 20:34 | |
| *** cloudtrainme has joined #openstack-ansible | 20:35 | |
| *** elo has quit IRC | 20:35 | |
| *** spotz_zzz is now known as spotz | 20:39 | |
| *** then3rd has joined #openstack-ansible | 20:41 | |
| *** cloudtrainme has quit IRC | 20:48 | |
| *** phalmos has quit IRC | 20:51 | |
| *** cloudtrainme has joined #openstack-ansible | 20:52 | |
| openstackgerrit | Merged openstack/openstack-ansible: Add novnc console support https://review.openstack.org/232657 | 20:55 |
| openstackgerrit | Merged openstack/openstack-ansible: Install spice-html5 from source https://review.openstack.org/232697 | 20:58 |
| openstackgerrit | Merged openstack/openstack-ansible: Update Glance Configuration for Liberty https://review.openstack.org/229967 | 20:58 |
| *** subscope has quit IRC | 21:02 | |
| *** cloudtrainme has quit IRC | 21:04 | |
| *** mgoddard has quit IRC | 21:05 | |
| *** cloudtrainme has joined #openstack-ansible | 21:07 | |
| *** jwagner is now known as jwagner_away | 21:08 | |
| *** spotz is now known as spotz_zzz | 21:12 | |
| *** jwagner_away is now known as jwagner | 21:14 | |
| *** woodard_ has joined #openstack-ansible | 21:16 | |
| mhayden | any way to make pep8 happy with a python heredoc that goes over 80 chars in width? | 21:17 |
| *** ggillies has quit IRC | 21:18 | |
| *** ggillies has joined #openstack-ansible | 21:19 | |
| *** woodard has quit IRC | 21:19 | |
| *** woodard_ has quit IRC | 21:20 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Docs overhaul https://review.openstack.org/234439 | 21:24 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-security: Docs overhaul https://review.openstack.org/234439 | 21:26 |
| cloudnull | mhayden: you'd need to skip the check | 21:30 |
| mhayden | cloudnull: life finds a way | 21:33 |
| mhayden | :P | 21:33 |
| *** jwagner is now known as jwagner_away | 21:33 | |
| mhayden | (i watched jurassic park over the weekend) | 21:33 |
| cloudnull | indeed it does | 21:33 |
| *** spotz_zzz is now known as spotz | 21:33 | |
| mhayden | so all of the controls are in the repo now | 21:34 |
| mhayden | now the bribing for reviews begins :P | 21:34 |
| * cloudnull will review for beer | 21:35 | |
| cloudnull | :) | 21:35 |
| openstackgerrit | Merged openstack/openstack-ansible-security: V-3850{2,3,4}: Ownership/mode of /etc/shadow https://review.openstack.org/232087 | 21:45 |
| openstackgerrit | Merged openstack/openstack-ansible-security: V-38621: System clock sync https://review.openstack.org/233209 | 21:52 |
| BjoernT | cloudnull: Why didn't you mention this earlier, we can fix that. That actually goes bidirectional | 21:55 |
| cloudnull | what ? | 21:55 |
| cloudnull | the beer thing . i thought that was well known :) | 21:56 |
| *** daneyon has joined #openstack-ansible | 21:58 | |
| BjoernT | no it wasn't | 22:03 |
| *** elo has joined #openstack-ansible | 22:06 | |
| -cloudnull- cloudnull will do reviews for beer :) | 22:06 | |
| cloudnull | BjoernT: now it is :) | 22:06 |
| stevelle | my free OSAS quota for today has been reached as well :) | 22:07 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:08 | |
| cloudnull | im out take care | 22:31 |
| *** spotz is now known as spotz_zzz | 22:32 | |
| BjoernT | later | 22:34 |
| *** darrenc is now known as darrenc_afk | 22:35 | |
| *** sdake has quit IRC | 22:41 | |
| *** daneyon has quit IRC | 22:41 | |
| *** k_stev1 has quit IRC | 22:44 | |
| *** woodard has joined #openstack-ansible | 22:55 | |
| *** woodard has quit IRC | 22:55 | |
| *** tiagogomes_ has joined #openstack-ansible | 22:56 | |
| *** tiagogomes_ has quit IRC | 22:56 | |
| *** woodard has joined #openstack-ansible | 22:56 | |
| *** KLevenstein has quit IRC | 23:04 | |
| *** cloudtrainme has quit IRC | 23:17 | |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible: Implement Neutron LBAAS using haproxy https://review.openstack.org/220365 | 23:30 |
| *** alop has quit IRC | 23:33 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!