*** markvoelker has joined #openstack-ansible | 00:02 | |
*** BjoernT has quit IRC | 00:11 | |
*** shoutm has quit IRC | 00:23 | |
*** shoutm has joined #openstack-ansible | 00:25 | |
*** alejandrito has joined #openstack-ansible | 00:32 | |
*** agireud has joined #openstack-ansible | 00:42 | |
*** elo has quit IRC | 00:50 | |
*** shoutm_ has joined #openstack-ansible | 01:00 | |
*** alejandrito has quit IRC | 01:01 | |
*** shoutm has quit IRC | 01:02 | |
*** skamithi1 has joined #openstack-ansible | 01:55 | |
*** kerwin_bai has joined #openstack-ansible | 02:00 | |
*** shoutm_ has quit IRC | 02:04 | |
*** alop has quit IRC | 02:07 | |
*** shoutm has joined #openstack-ansible | 02:14 | |
*** elo has joined #openstack-ansible | 02:45 | |
*** tlian has quit IRC | 02:54 | |
*** darrenc is now known as darrenc_afk | 02:54 | |
*** elo has quit IRC | 02:56 | |
*** _hanhart has joined #openstack-ansible | 02:58 | |
*** skamithi13 has quit IRC | 03:02 | |
*** _hanhart has quit IRC | 03:11 | |
*** darrenc_afk is now known as darrenc | 03:12 | |
*** sdake has joined #openstack-ansible | 03:12 | |
*** markvoelker has quit IRC | 03:27 | |
*** skamithi13 has joined #openstack-ansible | 03:28 | |
*** skamithi1 has left #openstack-ansible | 03:44 | |
*** markvoelker has joined #openstack-ansible | 04:27 | |
pellaeon | Has anyone seen my question regarding neutron-ha-tool.py ? | 04:30 |
---|---|---|
*** markvoelker has quit IRC | 04:32 | |
*** kerwin_bai has quit IRC | 04:56 | |
*** elo has joined #openstack-ansible | 05:03 | |
*** shoutm has quit IRC | 05:08 | |
*** shoutm has joined #openstack-ansible | 05:14 | |
*** elo has quit IRC | 05:20 | |
*** shoutm_ has joined #openstack-ansible | 05:45 | |
*** shoutm has quit IRC | 05:49 | |
*** elo has joined #openstack-ansible | 05:50 | |
*** markvoelker has joined #openstack-ansible | 05:59 | |
*** openstackgerrit has quit IRC | 06:01 | |
*** openstackgerrit has joined #openstack-ansible | 06:01 | |
*** markvoelker has quit IRC | 06:05 | |
*** harlowja has quit IRC | 06:19 | |
*** elo has quit IRC | 06:46 | |
*** elo has joined #openstack-ansible | 06:53 | |
*** elo has quit IRC | 06:55 | |
*** elo has joined #openstack-ansible | 06:58 | |
*** gus has quit IRC | 07:00 | |
*** neillc has quit IRC | 07:00 | |
*** neillc has joined #openstack-ansible | 07:01 | |
*** gus has joined #openstack-ansible | 07:02 | |
*** kukacz has joined #openstack-ansible | 07:04 | |
*** kukacz has quit IRC | 07:07 | |
*** elo has quit IRC | 07:09 | |
*** kukacz has joined #openstack-ansible | 07:09 | |
*** neilus has joined #openstack-ansible | 07:20 | |
*** elo has joined #openstack-ansible | 07:23 | |
*** gparaskevas has joined #openstack-ansible | 07:38 | |
*** pabelanger has quit IRC | 07:41 | |
*** pabelanger has joined #openstack-ansible | 07:42 | |
*** elo has quit IRC | 07:48 | |
*** sdake has quit IRC | 07:53 | |
*** markvoelker has joined #openstack-ansible | 08:00 | |
*** markvoelker has quit IRC | 08:05 | |
*** elo has joined #openstack-ansible | 08:12 | |
*** elo has quit IRC | 08:13 | |
*** openstackgerrit has quit IRC | 08:16 | |
*** openstackgerrit has joined #openstack-ansible | 08:16 | |
*** shoutm_ has quit IRC | 08:39 | |
*** shoutm has joined #openstack-ansible | 08:41 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Allow Neutron Agent prevent_arp_spoofing to be configurable https://review.openstack.org/227721 | 08:50 |
odyssey4me | pellaeon I seem to have missed it. What's up? | 08:58 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Configure HAProxy SSL frontends with cipher suite https://review.openstack.org/227730 | 09:09 |
*** shoutm has quit IRC | 09:31 | |
*** markvoelker has joined #openstack-ansible | 10:01 | |
*** markvoelker has quit IRC | 10:06 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Fix for keystone LDAP pkg missing https://review.openstack.org/226740 | 10:10 |
odyssey4me | mancdaz are you winning yet? :) | 10:12 |
mancdaz | odyssey4me yeah | 10:12 |
mancdaz | just trying to work out where in the ansibles to insert the things | 10:12 |
mancdaz | but I have a working setup | 10:12 |
odyssey4me | how specific does it need to be? | 10:13 |
mancdaz | odyssey4me I need to set some sysctl inside the containers | 10:13 |
odyssey4me | ie is it general, or do you have to use the mac or ip value? | 10:13 |
mancdaz | odyssey4me you use the mac address of the interface | 10:14 |
odyssey4me | ah ok - for all containers? | 10:14 |
mancdaz | which you get from /sys | 10:14 |
mancdaz | yeah all containers | 10:14 |
mancdaz | we are already using the sysctl module on the hosts | 10:14 |
mancdaz | but not in containers | 10:14 |
odyssey4me | oh bother, in that case it's not a static setting is it? it changes every time the container restarts | 10:14 |
mancdaz | odyssey4me that's fine I've got that bit sorted | 10:14 |
mancdaz | I just need to set arp_notify in proc in all the containers | 10:15 |
odyssey4me | well, we can easily do it on the container creation - but that won't handle existing containers | 10:15 |
mancdaz | odyssey4me where would we do it on container creation? | 10:16 |
odyssey4me | we do some stuff before building the containers here: https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_hosts/tasks/lxc_cache_preparation.yml | 10:16 |
odyssey4me | that preps the 'cache' aka container base 'image' | 10:17 |
odyssey4me | the containers actually get created here: https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/tasks/container_create.yml | 10:17 |
mancdaz | odyssey4me I think we'd be better running it as a task after container creation | 10:17 |
mancdaz | at some point where it will also get executed on normal runs | 10:17 |
odyssey4me | then as a post task in the playbooks would be best | 10:18 |
mancdaz | odyssey4me post what? | 10:18 |
mancdaz | I also need to get a change to the container interface file (inside the container), but on existing containers | 10:19 |
mancdaz | I know we set that up in the container create | 10:19 |
odyssey4me | well, actually a pre_task in the playbooks will work - it just needs to be executed if 'not is_metal | bool' | 10:19 |
mancdaz | ok | 10:19 |
odyssey4me | the playbooks for the services execute after the container is created, so that works | 10:20 |
odyssey4me | it feels a bit wasteful though, but this is the price we pay for trying to reduce the downtime during upgrades | 10:20 |
mancdaz | odyssey4me what's wasteful? | 10:20 |
odyssey4me | well, lots of repeated code | 10:21 |
mancdaz | odyssey4me is there not some base common thing that gets included everywhere? | 10:21 |
odyssey4me | I'm wondering that myself - but I think that's only done on container creation | 10:21 |
odyssey4me | so https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/tasks/container_create.yml#L142-L154 gets executed whenever someone runs the setup-hosts play | 10:23 |
odyssey4me | it will restart the container if something in the config changes | 10:23 |
odyssey4me | so that would be the best play/role to actually include this in - it'd be one entry and would execute on creation, but also handle existing containers | 10:24 |
odyssey4me | let me take a look at the container interface ini file - it seems that it can be edited on the host, which is great | 10:25 |
mancdaz | odyssey4me I think that's for the host side of the verth pair | 10:25 |
odyssey4me | mancdaz yeah, so this is more like what you'll end up having to do: https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/tasks/container_create.yml#L102-L113 | 10:27 |
mancdaz | so for the network interface inside the container the post up can be added here: https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/defaults/main.yml#L57-L77 | 10:28 |
mancdaz | but that only happens on container creation | 10:28 |
mancdaz | for the sysctl stuff, yeah I guess a crufty shellout using lxc_container | 10:28 |
mancdaz | I'd rather use the sysctl module in a normal run though | 10:29 |
odyssey4me | actually, that's executed here and will execute on existing containers too https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/tasks/container_create.yml#L102-L113 | 10:29 |
odyssey4me | it'll require that the setup-hosts playbook is run, but it will execute on existing containers | 10:29 |
mancdaz | oh that runs on existing containers? | 10:29 |
mancdaz | ok then that works | 10:29 |
odyssey4me | yup | 10:29 |
mancdaz | for both greenfield and upgrades | 10:30 |
mancdaz | so | 10:30 |
mancdaz | how to use the sysclt module inside all containers without writing a new task out inside each role | 10:30 |
odyssey4me | and the crufty shellout sucks, but it'll also execute inside the containers without repeating a task across all the playbooks | 10:30 |
odyssey4me | the alternative is to use the sysctl module in each of the plays | 10:31 |
mancdaz | odyssey4me right, that's where I was hoping there was a common play to dump it in | 10:31 |
odyssey4me | considering that this is a common setting across all containers, I'd suggest having it in the container create | 10:32 |
odyssey4me | it just means you won't be able to use the sysctl module | 10:32 |
mancdaz | hrm | 10:33 |
mancdaz | odyssey4me or create a common playbook and include it from all other playbooks? | 10:35 |
odyssey4me | ie a role? ;) | 10:35 |
mancdaz | sure, a common role | 10:36 |
mancdaz | how come we don't have one? there were common things in juno no? | 10:36 |
odyssey4me | lxc_container_create is kind-of a common role I suppose - let's look at the play that consumes it | 10:38 |
mancdaz | setup-hosts ? | 10:38 |
odyssey4me | yeah, it targets all containers: https://github.com/openstack/openstack-ansible/blob/master/playbooks/lxc-containers-create.yml#L17 | 10:38 |
mancdaz | can I add it as a post task? | 10:39 |
mancdaz | since ssh will be up by then | 10:40 |
mancdaz | or just another task after the create containers one | 10:40 |
odyssey4me | add a task to https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/tasks/container_create.yml but don't delegate it to the host? | 10:40 |
odyssey4me | hmm, that ssh check needs to run before you do something in the container - perhaps it is best to add it as a post task | 10:40 |
mancdaz | odyssey4me why not a second task in here: https://github.com/openstack/openstack-ansible/blob/master/playbooks/lxc-containers-create.yml#L17 | 10:41 |
odyssey4me | yep, that's what I'm thinking too | 10:42 |
odyssey4me | either that or the ssh wait task must get pulled into the role and the sysctl run after it | 10:42 |
odyssey4me | we need to do some analysis of all this - it seems like we're doing work all over the place and could do better at making it more intuitive and easier to figure out/understand | 10:44 |
openstackgerrit | Merged openstack/openstack-ansible: Do not use default LXC network in gate https://review.openstack.org/215040 | 10:47 |
odyssey4me | woohoo - mattt please backport https://review.openstack.org/215040 to kilo :) | 10:47 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Cinder to Liberty RC1 https://review.openstack.org/227205 | 10:49 |
*** neilus has quit IRC | 10:50 | |
*** neilus has joined #openstack-ansible | 10:52 | |
openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: Do not use default LXC network in gate https://review.openstack.org/227765 | 10:53 |
*** fawadkhaliq has joined #openstack-ansible | 11:00 | |
*** elo has joined #openstack-ansible | 11:00 | |
*** elo has quit IRC | 11:00 | |
evrardjp | hello | 11:04 |
evrardjp | sorry to drop in your conversation but changes to the containers sysctl should be done in the container create role | 11:04 |
evrardjp | it's more readable that way | 11:04 |
evrardjp | we just need to mention in the docs that people wanting to upgrade need to run the setup-hosts again | 11:05 |
evrardjp | then we could think of cleaning up some other stuff | 11:05 |
mancdaz | evrardjp problem is that the containers aren't up (ssh) until that role is finished | 11:06 |
mancdaz | so I probably need to add it as a post task | 11:07 |
mancdaz | tryin to do it in the middle of that role means I can't use sysctl module | 11:07 |
evrardjp | oh you mean for using the ansible sysctl module? | 11:08 |
mancdaz | evrardjp right | 11:08 |
evrardjp | it seems more logical to me to drop it here https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/tasks/container_create.yml#L78 | 11:10 |
mancdaz | evrardjp to edit a setting in /proc inside the container? | 11:11 |
evrardjp | weren't we speaking of post-up scripts that does an ip link set? | 11:12 |
mancdaz | evrardjp no, I know where I need to put that one | 11:12 |
mancdaz | but we also need to edit a sysctl setting inside the containers | 11:12 |
mancdaz | evrardjp https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/lxc_container_create/defaults/main.yml#L71 | 11:13 |
evrardjp | it's the equivalent of two things: writing a sysctl file in the correct folder (which can be done in the task above) and doing a sysctl -w, which can also be done | 11:14 |
evrardjp | mancdaz: yeah, except I'd make it generic for this case: you can have a new loop that sets the sysctl -w on link up ;) | 11:14 |
mancdaz | evrardjp or just drop it into sysctl.conf using the sysctl module, so it persists properly | 11:15 |
mancdaz | adding it as a post up hook feels a bit dirty | 11:16 |
evrardjp | saddly that's often ubuntu's way | 11:16 |
evrardjp | but it seems weird to have a post task to wait for ssh, then doing a task improving the arp handling | 11:17 |
evrardjp | and doing before won't work, because ssh isn't reactive yet | 11:18 |
evrardjp | so you'll have to wait for it to be up to be improved in the future | 11:18 |
mancdaz | evrardjp well at that point we don't have any problems with the arp cache because the containers are up for the first time | 11:18 |
evrardjp | I was thinking upgrades, but upgrades won't have problems because you should be able to reach the containers | 11:19 |
evrardjp | so yeah, sorry for the discussion ;) | 11:19 |
mancdaz | it will affect upgrades too, you're right | 11:19 |
mancdaz | but upgrades also call this task (according to odyssey4me ) because it's part of setup-hosts | 11:19 |
odyssey4me | mancdaz so why not move the ssh wait task into the role, then add the sysctl change after it? | 11:20 |
evrardjp | that would be fine too | 11:20 |
mancdaz | odyssey4me what difference does that make? | 11:20 |
odyssey4me | all the tasks will be inside the role - it's more intuitive than having some in the role and some in the play | 11:21 |
evrardjp | it makes more sense for the role itself, playbook will just handle the orchestration of the roles... it also bring more independance of the play | 11:21 |
evrardjp | role* | 11:21 |
mancdaz | odyssey4me that's the pattern across the board though | 11:21 |
odyssey4me | functionally it's no different | 11:21 |
mancdaz | I'm not going to change it for this one role/commit | 11:21 |
mancdaz | it needs to be part of a larger restructure | 11:21 |
mancdaz | blueprint or w/e | 11:21 |
odyssey4me | mancdaz so the ssh wait in the other plays is due to changes that are specific to the role implemented by the play - ie aa profile changes, extra network interfaces, etc | 11:22 |
evrardjp | indeed | 11:22 |
odyssey4me | but yeah, either way is fine - I do think that we should take the time to evaluate all the roles an dplays and consider how we could do better | 11:22 |
mancdaz | odyssey4me I still think it's unrelated to the fix, and I'm aware we generally don't like restructures being thrown in as part of something else :) | 11:23 |
odyssey4me | I have put up a suggested summit session for it :) | 11:23 |
mancdaz | odyssey4me cool | 11:23 |
mancdaz | lemme test this then, and I'll get a review up | 11:23 |
odyssey4me | mancdaz fair enough :) | 11:23 |
evrardjp | when I see all this, I think we should have a something to define post-up scripts | 11:24 |
odyssey4me | mancdaz it probably also makes sense for the 'wait for ssh' tasks to only execute if there actually was a change resulting in a restart before it | 11:25 |
evrardjp | odyssey4me: you mean it could even be a handler? | 11:26 |
odyssey4me | evrardjp no, I mean that https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-ceilometer-install.yml#L21-L29 should have a registered result | 11:27 |
evrardjp | that's what I expected ;) | 11:27 |
evrardjp | it was just to make sure we speak the same language | 11:27 |
odyssey4me | and https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-ceilometer-install.yml#L35-L46 should only execute if the result was a change | 11:27 |
odyssey4me | :) it's important to verify if you think there could be a misunderstanding | 11:28 |
evrardjp | that's why I said that ;) | 11:28 |
evrardjp | more than one way to do it, but sometimes the more documented way isn't the best (like this case) | 11:29 |
evrardjp | quick question | 11:29 |
evrardjp | why are we using delay? | 11:29 |
evrardjp | instead of timeout? | 11:29 |
evrardjp | delay should be really small (it's just containers here) | 11:30 |
evrardjp | so I would even neglect delay | 11:30 |
mancdaz | evrardjp I think timeout is implied, maybe 5 seconds default iirc | 11:30 |
mattt | odyssey4me: https://review.openstack.org/#/c/221009/4/specs/mitaka/gate-split.rst | 11:30 |
evrardjp | maybe there is a good reason, it's just for my understanding | 11:30 |
mancdaz | but yeah, maybe not much point in a delay | 11:30 |
mattt | odyssey4me: sorry if i missed it ... but say someone submits a keystone role change, how many gates do we do ? | 11:30 |
odyssey4me | evrardjp the delay was fixed and made variable on request by another contributor working with vagrant - he needed a bigger delay | 11:32 |
odyssey4me | the default timeout if 5 mins | 11:32 |
evrardjp | that could be long | 11:32 |
mancdaz | odyssey4me surely the delay is how long to wait before you start polling? | 11:32 |
mancdaz | 5 mins? | 11:32 |
evrardjp | mancdaz: no | 11:32 |
odyssey4me | in testing I found that it seems to fail much more prematurely, possibly due to some sort of tcp half open state | 11:32 |
mancdaz | ok I misunderstood then | 11:32 |
evrardjp | the timeout is 5 minutes, the delay is 5 sec | 11:33 |
evrardjp | oh wait | 11:33 |
evrardjp | you maybe right | 11:33 |
evrardjp | let me chek | 11:33 |
odyssey4me | mancdaz yes, the delay is how long to wait before polling, the timeout is how long after you start polling to wait before declaring failure | 11:33 |
evrardjp | check* | 11:33 |
mancdaz | odyssey4me so surely we don't wait 5 kinutes before testing to see if ssh is up | 11:33 |
odyssey4me | mattt I didn't specify any scoping in the gate split - but I do think that scoping would be useful and will try it out in practise | 11:34 |
evrardjp | ssh delay is 5 seconds by default, connect_timeout is 5 seconds, timeout is 5 minutes | 11:34 |
odyssey4me | I mean there's no point in checking whether rabbitmq high availability works when none of the code relating to rabbit was changed, right | 11:34 |
mancdaz | afk | 11:34 |
odyssey4me | evrardjp what's the diff between connect_timeout and timeout? | 11:35 |
evrardjp | connect_timeout is the number of seconds to wait for a connection to happen before retrying | 11:35 |
evrardjp | while timeout is the task timeout | 11:35 |
evrardjp | the maximum number of seconds to wait for before task fails | 11:36 |
evrardjp | so right now, with the task retries, it tries every 5 seconds until 300 x the number of retries before task fails | 11:37 |
evrardjp | so if you reduce the timeout and/or the retries, the task will fail more rapidly | 11:38 |
evrardjp | which is sometimes good | 11:38 |
evrardjp | maybe we should introduce the connect_timeout and/or timeout in the tasks, but that's not a priority right now | 11:39 |
*** markvoelker has joined #openstack-ansible | 11:47 | |
*** tlian has joined #openstack-ansible | 11:49 | |
*** markvoelker has quit IRC | 11:52 | |
*** skamithi14 has joined #openstack-ansible | 11:55 | |
*** skamithi13 has quit IRC | 11:58 | |
*** tiagogomes has quit IRC | 11:59 | |
*** tiagogomes has joined #openstack-ansible | 11:59 | |
*** fawadkhaliq has quit IRC | 12:01 | |
pellaeon | odyssey4me: repost: | 12:01 |
pellaeon | 21:42 < pellaeon> Hello, my ./neutron-ha-tool.py --l3-agent-check always fail with: | 12:01 |
pellaeon | 21:42 < pellaeon> ERROR {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}} | 12:01 |
pellaeon | 21:43 < pellaeon> which seems to be caused by using keystone v3 API as v2 | 12:02 |
pellaeon | 21:44 < pellaeon> in openrc OS_AUTH_URL=http://172.29.236.19:5000/v3 | 12:02 |
pellaeon | 21:45 < pellaeon> use neutron-ha-tool.py with -d shows that it make requests to http://172.29.236.19:5000/v3/tokens | 12:02 |
pellaeon | 21:47 < pellaeon> I just upgraded from older kilo branch to the latest kilo branch | 12:02 |
pellaeon | 21:48 < pellaeon> before that I removed my dedicated network host and use infra1-3 instead | 12:02 |
pellaeon | I think the problem happens after I move network host to infra1-3, but i'm not sure, I did not notice the problem at that time | 12:03 |
odyssey4me | pellaeon interesting - the tool instantiates a v2 api connection with keystone, so whatever is in the openrc doesn't matter: https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/os_neutron/files/neutron-ha-tool.py#L92 | 12:04 |
odyssey4me | but we should probably fix that to work with the v3 api | 12:04 |
tiagogomes | hi, just wondering, is there any reason why l3_ha is disabled by default? | 12:05 |
odyssey4me | for liberty we might be able to remove that altogether and use the native neutron ha scheduling - we need to check if it actually works now | 12:05 |
odyssey4me | tiagogomes because it didn't work when we tried it with kilo | 12:06 |
tiagogomes | :( | 12:06 |
odyssey4me | we need to check again | 12:06 |
tiagogomes | but it is an OpenStack bug or ansible ? | 12:06 |
tiagogomes | well, openstack | 12:07 |
odyssey4me | the issue was upstream in neutron, but it may be fixed now | 12:07 |
odyssey4me | pellaeon can you please register a bug for that? | 12:07 |
odyssey4me | ie the issue you're seeing | 12:08 |
pellaeon | odyssey4me: um, things in openrc should matter, isn't it ? since it writes auth_url=os.environ['OS_AUTH_URL'] | 12:09 |
evrardjp | interesting tool that I didn't know of :D | 12:09 |
pellaeon | and OS_AUTH_URL is loaded from openrc | 12:10 |
odyssey4me | pellaeon yeah, that might be it - but keystone is usually better at redirecting | 12:10 |
odyssey4me | if you can log a bug for it we can make sure to figure it out - we need to put some time into verifying that everything is now using unversioned url's for liberty - but also to see what should be used for kilo | 12:11 |
pellaeon | yes I can file a bug | 12:12 |
odyssey4me | thanks pellaeon - if you do figure out a solution then please note it in the bug comments to help anyone who is working on triage | 12:12 |
pellaeon | so there seems to be something wrong with my keystone containers? | 12:13 |
pellaeon | since it doesn't redirect | 12:13 |
*** alejandrito has joined #openstack-ansible | 12:13 | |
odyssey4me | not necessarily | 12:14 |
odyssey4me | pellaeon if you change the openrc to point at one of your keystone v2 endpoints, does it work? | 12:14 |
pellaeon | let me try | 12:14 |
odyssey4me | if we continue to carry that tool then I'd rather that we made it work regardless of the endpoint version | 12:15 |
pellaeon | yes, changing OS_AUTH_URL to http://172.29.236.19:5000/v2.0 will make neutron-ha-tool.py work again | 12:17 |
*** fawadkhaliq has joined #openstack-ansible | 12:17 | |
*** skamithi14 has quit IRC | 12:22 | |
odyssey4me | pellaeon ok, so that gives you a workaround for now at least | 12:25 |
pellaeon | won't it break something else? | 12:26 |
mhayden | buenos dias | 12:26 |
pellaeon | https://bugs.launchpad.net/openstack-ansible/+bug/1499708 here's the bug report | 12:27 |
openstack | Launchpad bug 1499708 in openstack-ansible "Migrate neutron-ha-tool.py to use Keystone API v3" [Undecided,New] | 12:27 |
odyssey4me | pellaeon yeah, so we can't just set that across the board as that affects other things | 12:27 |
odyssey4me | the keystone v3 api is preferred by keystone and the v2 api will be deprecated, probably in the M cycle | 12:28 |
odyssey4me | so we need to make all the things work with the v3 api instead | 12:28 |
pellaeon | evrardjp: are you referring to neutron-ha-tool.py? this is invoked in neutron agents container by cron | 12:28 |
odyssey4me | I'm busy doing tests to verify whether unversioned URI's still let everything work, so that the api version used can rather be determined by the service/client. | 12:29 |
evrardjp | ok, didn't know it pellaeon. Thanks for the info | 12:29 |
*** markvoelker has joined #openstack-ansible | 12:31 | |
pellaeon | I can probably modify the script to work with v3 though | 12:31 |
pellaeon | now it seems to be the only way and the ultimate solution | 12:31 |
*** skamithi13 has joined #openstack-ansible | 12:34 | |
pellaeon | aha | 12:34 |
pellaeon | I have misunderstood something | 12:34 |
*** alejandrito has quit IRC | 12:36 | |
pellaeon | "2.0" in L92 means neutron API v2, not keystone API v2 | 12:36 |
odyssey4me | oh right, silly me | 12:37 |
*** agireud has quit IRC | 12:37 | |
*** alejandrito has joined #openstack-ansible | 12:38 | |
odyssey4me | well, the bits that will be needed for it to work against the v3 API are the other environment variables you should see in openrc | 12:38 |
odyssey4me | the domain related variables | 12:38 |
odyssey4me | if you're able to work it out and submit a patch for it, that'd be grand! | 12:38 |
odyssey4me | o/ mhayden how goes it with you this fine firday? | 12:39 |
odyssey4me | *friday | 12:39 |
mhayden | not too bad, how about you? | 12:39 |
pellaeon | I'll try to figure it out and submit a patch :-) | 12:39 |
odyssey4me | pellaeon patches are always welcome :) | 12:40 |
odyssey4me | mhayden not too bad, working through nova for liberty rc1 at the moment - trying to figure out what goes and what stays | 12:40 |
* mhayden is reading the Project CameraShy report and about to bump the security-hardening spec again ;) | 12:43 | |
*** fawadkhaliq has quit IRC | 12:45 | |
*** fawadkhaliq has joined #openstack-ansible | 12:45 | |
*** fawadkhaliq has quit IRC | 12:45 | |
*** woodard has joined #openstack-ansible | 12:47 | |
*** woodard has quit IRC | 12:47 | |
*** woodard has joined #openstack-ansible | 12:48 | |
*** shoutm has joined #openstack-ansible | 12:57 | |
*** pradk has joined #openstack-ansible | 12:58 | |
odyssey4me | mhayden if you can do the final recommended tweaks from dstanek then I think https://review.openstack.org/223717 is ready to roll | 13:07 |
* mhayden does a little dance | 13:07 | |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible: Add SSL/TLS listener to RabbitMQ https://review.openstack.org/223717 | 13:12 |
mhayden | hopefully that will make it good to go | 13:12 |
*** gparaskevas has quit IRC | 13:14 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Nova to Liberty RC1 https://review.openstack.org/227839 | 13:18 |
*** spotz_zzz is now known as spotz | 13:20 | |
*** shoutm has quit IRC | 13:23 | |
*** gparaskevas has joined #openstack-ansible | 13:31 | |
*** sdake has joined #openstack-ansible | 13:34 | |
*** sdake_ has joined #openstack-ansible | 13:37 | |
mancdaz | odyssey4me ok in testing, the patch I'm working on ran about 3 minutes faster than current master, which still has the full flush | 13:37 |
odyssey4me | mancdaz that sounds great! | 13:38 |
mancdaz | odyssey4me I'll get cloudnull's review updated with that patchset | 13:38 |
odyssey4me | that'll make it equivalent but without the disruption to other containers | 13:38 |
odyssey4me | sweet, thanks | 13:39 |
mancdaz | odyssey4me and importantly, without the disruption to the static arp entries that the linuxbridge agent adds on the compute hosts | 13:39 |
*** sdake has quit IRC | 13:39 | |
odyssey4me | ah yes mancdaz, that is the big upgrade disruption issue | 13:40 |
odyssey4me | and quite an issue overall when running the plays | 13:40 |
odyssey4me | (considering that the flush was running every time) | 13:40 |
*** KLevenstein has joined #openstack-ansible | 13:42 | |
mhayden | whoa, just flipped my default view in gerrit to the "new screen" | 13:42 |
mhayden | handy as heck | 13:42 |
odyssey4me | hehe, yeah - it takes a little getting used to but I like it | 13:43 |
odyssey4me | the comment history is probably the biggest win | 13:43 |
mhayden | yeah, it's quite nice | 13:43 |
mattt | i always go back to the old view :-/ | 13:44 |
openstackgerrit | Darren Birkett proposed openstack/openstack-ansible: Removes over zealous arp cache flushing https://review.openstack.org/225367 | 13:45 |
mancdaz | Apsu ^ | 13:45 |
mhayden | mattt: i still think you're cool anyway | 13:45 |
mattt | thanks mang | 13:45 |
*** jmckind has joined #openstack-ansible | 13:45 | |
odyssey4me | mancdaz do you mind if I rebase that patch quickly to have it include the hpcloud-b4 fixer? | 13:52 |
odyssey4me | I suppose for now I'll leave it - the patch might get iterated a little more | 13:52 |
*** skamithi13 has quit IRC | 13:55 | |
*** skamithi13 has joined #openstack-ansible | 13:55 | |
*** sdake has joined #openstack-ansible | 14:03 | |
*** sdake_ has quit IRC | 14:07 | |
*** Mudpuppy has joined #openstack-ansible | 14:07 | |
*** neilus has quit IRC | 14:12 | |
openstackgerrit | Merged openstack/openstack-ansible: Do not use default LXC network in gate https://review.openstack.org/227765 | 14:12 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Fix for keystone LDAP pkg missing https://review.openstack.org/226740 | 14:13 |
*** fxpester has quit IRC | 14:14 | |
openstackgerrit | Merged openstack/openstack-ansible: Put horizon in its own process https://review.openstack.org/227570 | 14:20 |
*** kukacz has quit IRC | 14:26 | |
cloudnull | morning | 14:32 |
*** galstrom_zzz is now known as galstrom | 14:34 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 14:41 |
*** fawadkhaliq has joined #openstack-ansible | 14:43 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 14:44 |
odyssey4me | o/ cloudnull | 14:44 |
*** galstrom is now known as galstrom_zzz | 14:44 | |
*** galstrom_zzz is now known as galstrom | 14:45 | |
evrardjp | I don't see much reviews around this commit: https://review.openstack.org/#/c/218818/ | 14:47 |
evrardjp | is this bad or good? | 14:47 |
evrardjp | it's maybe too disruptive | 14:48 |
cloudnull | evrardjp: no its really quite good. :) | 14:49 |
odyssey4me | evrardjp what do you think about putting that into a seperate repo? | 14:49 |
evrardjp | I have two question | 14:50 |
evrardjp | questions* | 14:50 |
odyssey4me | have keepalived in its own role repo | 14:50 |
cloudnull | was meaning to come back to that but was dealing with the general gate issues for afew days and then i forgot. | 14:50 |
*** e-vad has joined #openstack-ansible | 14:51 | |
evrardjp | ok first, do you think I should make this facultative? for example, even if you have multiple hosts, you don't nessarily use keepalived? | 14:51 |
evrardjp | like defining a variable haproxy_use_keepalived: False by default | 14:51 |
cloudnull | odyssey4me: i think we can get there. IMO the role changes are good as is, it provides functionality people have been asking for, and we can look at migrating the roles out soonish. | 14:52 |
evrardjp | odyssey4me: it's already in a separate repo, because it's in ansible-galaxy | 14:52 |
evrardjp | I just dropped the latest edition here | 14:52 |
mancdaz | odyssey4me I think that gate is going as slowly as the last patchset :( | 14:52 |
cloudnull | ohhh. can we add the galaxy reference to the role requirements? | 14:52 |
cloudnull | evrardjp: ^ | 14:52 |
odyssey4me | evrardjp what repo is it in? | 14:53 |
evrardjp | I'm not sure cloudnull, what do you mean? | 14:53 |
cloudnull | IE https://github.com/openstack/openstack-ansible/blob/master/ansible-role-requirements.yml.example but the file would be https://github.com/openstack/openstack-ansible/blob/master/ansible-role-requirements.yml instead ? | 14:53 |
evrardjp | you know this is a question I asked in a weekly meeting? | 14:54 |
cloudnull | maybe im a bad person for bringing it up... sorry :( | 14:54 |
evrardjp | and it was mentionned to do it the current way while this role separation wasn't decided | 14:54 |
evrardjp | it's no problem | 14:54 |
evrardjp | just to make sure | 14:54 |
odyssey4me | evrardjp yep, but since then that spec has merged | 14:54 |
cloudnull | https://github.com/openstack/openstack-ansible-specs/blob/master/specs/mitaka/independent-role-repositories.rst | 14:54 |
odyssey4me | so we can now create an openstack repo for it if you'd like, but cloudnull is right - the review is ready and looks like it works | 14:55 |
odyssey4me | we can pull it out a bit later | 14:55 |
mancdaz | presumably the roles will be pulled put one at a time? | 14:55 |
cloudnull | odyssey4me: if that role already exists in the galaxy we dont need to carry it in the openstack big tent | 14:55 |
mancdaz | if so, doesn't it make sense for this to be the first? | 14:55 |
cloudnull | we can just use the upstream one. | 14:55 |
odyssey4me | mancdaz yeah, each role split out needs its own spec | 14:56 |
evrardjp | that seems more logical | 14:56 |
evrardjp | cloudnull: ^ | 14:56 |
cloudnull | what we have here is the same as whats in your upstream galaxy role evrardjp? | 14:56 |
odyssey4me | cloudnull yep, that's true - unless there is some value seen in having the role in the big tent | 14:56 |
evrardjp | cloudnull: yup | 14:56 |
cloudnull | my 2cents, but I dont think theres any need to duplicate the role | 14:56 |
evrardjp | there was initially on first release because it was BSD and not Apache2 license | 14:57 |
odyssey4me | in that case I would rather say that the review should change to a documentation addition which tells a user how to use the role from galaxy with OSA | 14:57 |
cloudnull | evrardjp: this one https://github.com/evrardjp/ansible-keepalived ? | 14:57 |
evrardjp | while I think it wasn't really an issue, I moved to Apache2, so it's not really an issue | 14:57 |
evrardjp | whoops sorry for me repeating myself | 14:57 |
evrardjp | yup cloudnull | 14:58 |
evrardjp | so for the ansible-role-requirements | 14:58 |
evrardjp | I'd use: | 14:58 |
odyssey4me | we don't need to carry extra code, but a doc ref for how to use your role would be wonderful | 14:58 |
cloudnull | all we need to do is add an entry like so http://docs.ansible.com/ansible/galaxy.html#advanced-control-over-role-requirements-files | 14:59 |
evrardjp | yup, without forgetting to set name: keepalived | 15:00 |
cloudnull | indeed | 15:00 |
evrardjp | I'm also using path: roles/ just to make sure it's not put on a default folder location | 15:00 |
cloudnull | http://cdn.pasteraw.com/cmmnlruhubc3x2esuostwa7k1se6o0l < so that should work | 15:01 |
evrardjp | that's a way to do it | 15:01 |
evrardjp | what I use is this: http://paste.openstack.org/show/474046/ | 15:02 |
cloudnull | evrardjp: i'd not set the path, that way its resolved into the root ansible role namespace and allows for future upgrades of the repo without dealing with the untracked role. | 15:02 |
evrardjp | it doesn't stick the version | 15:02 |
evrardjp | cloudnull: right | 15:03 |
cloudnull | idk if its best to use the galaxy api or go direct github ? | 15:03 |
cloudnull | i have no opinnions on that either way | 15:04 |
evrardjp | galaxy maps to last version by default, and also changes the name of the repo | 15:04 |
evrardjp | but if you set name: and version: it's really not that important | 15:04 |
odyssey4me | can you specify the version or sha if you only use galaxy api? | 15:04 |
cloudnull | idk? | 15:05 |
odyssey4me | the sha is useful as a lot of roles don't have tags | 15:05 |
evrardjp | I never tried to specify the sha with galaxy | 15:05 |
evrardjp | I'll try | 15:05 |
evrardjp | the other advantage of galaxy is dependance handling | 15:06 |
evrardjp | when you're using ansible-galaxy with a galaxy role (not a zip) I'm pretty sure it will pull down the dependencies mentionned in the meta | 15:06 |
evrardjp | not a zip for example* | 15:07 |
odyssey4me | mancdaz https://review.openstack.org/225367 is running tempest right now, so it's within time - it's slow due to running on hpcloud - this is a fairly normal run time for hpcloud instances | 15:09 |
mancdaz | odyssey4me really? it's getting close to 90 minutes | 15:09 |
mancdaz | oh it's done | 15:10 |
odyssey4me | it just finished successfully :) | 15:10 |
mancdaz | :D | 15:10 |
mancdaz | woot | 15:10 |
openstackgerrit | Merged openstack/openstack-ansible: Add ebtables to neutron agent configuration https://review.openstack.org/217103 | 15:10 |
cloudnull | ++ evrardjp is right, dep handling seems to be better with direct galaxy integration. | 15:10 |
cloudnull | "Roles pulled from galaxy work as with other SCM sourced roles above. To download a role with dependencies, and automatically install those dependencies, the role must be uploaded to the Ansible Galaxy website." | 15:10 |
cloudnull | via http://docs.ansible.com/ansible/galaxy.html | 15:11 |
evrardjp | so using tags with ansible-galaxy installation by requirements file works | 15:11 |
evrardjp | I'm checking the sha now | 15:11 |
odyssey4me | now we just need another core review on https://review.openstack.org/225367 :) | 15:11 |
cloudnull | ^ that I'm +1 nice work tuning that up Apsu mancdaz . | 15:12 |
*** gparaskevas has quit IRC | 15:12 | |
cloudnull | no +2 for me, im the original author . | 15:12 |
evrardjp | evrardjp: it doesn't seem to work with commit sha if you're using galaxy (which makes sense) | 15:13 |
*** skamithi has joined #openstack-ansible | 15:13 | |
evrardjp | so if you want to pull something from galaxy that has tagged releases (or follow the master), you can use the shorter notation (evrardjp.keepalived) | 15:14 |
mancdaz | odyssey4me cloudnull 1:25 still feels long | 15:14 |
evrardjp | if you want to have one specific commit, then you should use the github complete link, with the commit' sha in version | 15:14 |
mancdaz | even for hp cloud, is that normal? | 15:14 |
cloudnull | mancdaz: it worked in hp cloud. working is the bar for success :) | 15:15 |
skamithi | kindly request some help troubleshooting my openstack-ansible config. http://pastebin.com/rmHxNEGu trying to setup vxlan tunneling using the .aio config. getting error with it building neutron.conf. i cannot find the dict object that has the "address" attribute..i must be blind or something.. any clues for me? or any further info needed? | 15:15 |
mancdaz | cloudnull well it worked once before too, by coming in just under 90 minutes | 15:15 |
cloudnull | if hp cloud , great success ! | 15:15 |
mancdaz | ha | 15:15 |
mancdaz | can we recheck | 15:16 |
mancdaz | at this point, before getting +2s | 15:16 |
mancdaz | oh | 15:16 |
evrardjp | skamithi could you give us your variables? | 15:17 |
odyssey4me | mancdaz 1:25 is around the normal time for some of the hpcloud runs - they're often 1:10-1:25 | 15:19 |
mancdaz | ok | 15:19 |
odyssey4me | mancdaz I wanted to rebase anyway - happy to do that? | 15:20 |
mancdaz | odyssey4me yup | 15:20 |
*** fawadkhaliq has quit IRC | 15:20 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Removes over zealous arp cache flushing https://review.openstack.org/225367 | 15:20 |
cloudnull | skamithi: seems like the containers or inventory is missing the address field ? what release are you running ? | 15:20 |
cloudnull | the address attribute is in the /etc/openstack_deploy/openstack_inventory.json file | 15:21 |
cloudnull | for the container under container_networks | 15:21 |
evrardjp | maybe I didn't get it but... is defining the sysctl to do the arp_notify after the ip link set the purpose? I thought ip link set send the gratuitous arp only if arp_notify is set beforehand | 15:22 |
skamithi | evrardjp, cloudnull: http://pastebin.com/0GF8mVAH (openstack_user_config.yml) | 15:23 |
skamithi | cloudnull: ah okay. thanks. i'll look under that container definition in that json file and see if the address is defined. | 15:23 |
skamithi | cloudnull: using kilo version | 15:24 |
mancdaz | evrardjp, setting the sysctl setting means that any future link sets will cause a grat arp (arp_notify) | 15:24 |
evrardjp | mancdaz: that's something I understand | 15:25 |
mancdaz | so I'm not sure the question about ordering | 15:25 |
mancdaz | dropping those two changes in means that every container stop/start will now ensure that the mac address table in the neighbours is updated immediately | 15:26 |
mancdaz | no arp_notify is sent at the time of the task to add thoe configs | 15:26 |
evrardjp | true, but before that, on the first deploy, aren't you interested by getting faster results in the first wait_for? | 15:26 |
evrardjp | but yeah, it's only the first one, not really important | 15:27 |
evrardjp | even more with the delay | 15:27 |
mancdaz | evrardjp no, there's no issue with the first one because there is no stale mac in the neighbour table | 15:27 |
mancdaz | since it's never been up | 15:27 |
evrardjp | ok | 15:27 |
evrardjp | I learned something about links today :) | 15:28 |
mancdaz | it's in subsequent stop/start events when the mac address changes that we were seeing issues | 15:28 |
evrardjp | ok | 15:28 |
evrardjp | skamithi could you also drop us your openstack_inventory.json? | 15:30 |
evrardjp | I think there is something wrong first with your used_ips | 15:30 |
evrardjp | if you're using comma, you're defining the start/end of the range | 15:30 |
evrardjp | maybe not related, but I'd like to check to see if that couldn't cause a messup in the inventory | 15:31 |
evrardjp | skamithi: also, your container networks seem to be your external one, instead of the internal one | 15:32 |
evrardjp | didn't you mean 192.168.50.0/24? | 15:33 |
*** elo has joined #openstack-ansible | 15:35 | |
skamithi | evardjp: didn't know used_ips defines a start and end range..good to know | 15:35 |
evrardjp | you can find more doc about how to configure the inventory here: http://docs.openstack.org/developer/openstack-ansible/install-guide/configure-networking.html | 15:36 |
skamithi | evrardjp: let me read that doc carefully. redo my config and blow away the json file and redo things..see if it works then. | 15:37 |
evrardjp | wait | 15:38 |
evrardjp | don't blow away your json file like that ;) | 15:38 |
skamithi | well its just in vagrant:) | 15:38 |
evrardjp | ok then :) | 15:38 |
odyssey4me | evrardjp skamithi is prepping a review for a full vagrant build of OSA | 15:38 |
evrardjp | yeah I remember now ;) | 15:39 |
odyssey4me | in his vast amount of spare time :p | 15:39 |
evrardjp | that's a modern time luxury! | 15:39 |
evrardjp | spare time \o/ | 15:39 |
*** skamithi13 has quit IRC | 15:41 | |
*** skamithi13 has joined #openstack-ansible | 15:42 | |
*** skamithi13 has quit IRC | 15:42 | |
*** skamithi13 has joined #openstack-ansible | 15:42 | |
*** skamithi13 has quit IRC | 15:43 | |
*** skamithi14 has joined #openstack-ansible | 15:43 | |
*** elo has quit IRC | 15:46 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible-specs: Add spec for OS_.* venv support https://review.openstack.org/227921 | 15:47 |
skamithi | odyssey4me: funny! was excited though when my workplace took one of my "spare time" projects from last year and paid me to work on it full time. https://support.cumulusnetworks.com/hc/en-us/articles/210452158 | 15:47 |
*** skamithi13 has joined #openstack-ansible | 15:48 | |
*** skamithi13 has quit IRC | 15:49 | |
*** skamithi14 has quit IRC | 15:49 | |
*** skamithi13 has joined #openstack-ansible | 15:49 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Update Keystone to Liberty RC1 https://review.openstack.org/226917 | 15:50 |
*** skamithi13 has quit IRC | 15:51 | |
*** skamithi13 has joined #openstack-ansible | 15:51 | |
Apsu | mancdaz: Why did you change from my $(< to $(cat? | 15:53 |
odyssey4me | skamithi wait, is that basically an implementation of some of the cisco style outputs for a linux server? | 15:53 |
Apsu | $(< doesn't call another process. I did it that way on purpose :P | 15:53 |
odyssey4me | thanks to mattt that job is running successfully on hpcloud-b4 :) | 15:55 |
odyssey4me | cloudnull ^ woot! | 15:55 |
*** sdake_ has joined #openstack-ansible | 15:55 | |
cloudnull | winning ! | 15:55 |
cloudnull | mattt: for PREZ! | 15:55 |
odyssey4me | Apsu I think mancdaz is gone now, but that would be a pretty quick and simple change | 15:56 |
odyssey4me | is not calling another process important? | 15:56 |
Apsu | It's not super important, it's just a Useless Use of Cat. | 15:57 |
skamithi | odyssey4me: started out troubleshooting linux networking on cumulus linux. pain in the ass using traditional linux tools. | 15:57 |
Apsu | Which is so common that it has a name, and is shamed :P | 15:57 |
Apsu | We're already doing clever things in bash, should use the tools we have. | 15:58 |
*** sdake has quit IRC | 15:58 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Fix for keystone LDAP pkg missing https://review.openstack.org/226740 | 15:59 |
evrardjp | skamithi: netshow...didn't know it. looks cool | 16:03 |
mattt | odyssey4me: woot! awesome | 16:04 |
evrardjp | yeah thanks mattt | 16:05 |
mattt | evrardjp: it was an easy fix, just none of us thought to get access to the instance to troubleshoot it :) | 16:05 |
evrardjp | making something run on hpcloud-b4 deserves AT LEAST a thanks from everyone | 16:05 |
evrardjp | or got the time to? ;) | 16:06 |
mattt | evrardjp: ha, yeah, i was allocated time to look at it, so that helps :P | 16:06 |
evrardjp | would you care to explain for the noobs like me? | 16:07 |
evrardjp | like for example, showing the commit that fixed it? | 16:08 |
odyssey4me | evrardjp https://review.openstack.org/215040 | 16:08 |
evrardjp | ok nice indeed :) | 16:08 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Fact caching will store all ansible facts https://review.openstack.org/225967 | 16:10 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Fact caching will store all ansible facts https://review.openstack.org/225967 | 16:11 |
*** galstrom is now known as galstrom_zzz | 16:11 | |
evrardjp | cloudnull: didn't see much performance improvement on my side | 16:13 |
odyssey4me | cloudnull perhaps change the co-authored-by to you? | 16:13 |
evrardjp | I tried that a few weeks ago | 16:13 |
odyssey4me | and the subject should perhaps be something like 'Implement fact caching' | 16:13 |
evrardjp | but it's still good to drop the json file somewhere, just to have it and understand more | 16:14 |
odyssey4me | and how big does the environment need to be before you start seeing the gains here? | 16:14 |
odyssey4me | evrardjp now that is entirely different, and a justification of its own - just having access to the ansible facts for troubleshooting is useful | 16:15 |
evrardjp | I don't know | 16:15 |
evrardjp | odyssey4me: yeah true | 16:15 |
evrardjp | I'll call it ansible_facts.json though | 16:15 |
*** tnarg has joined #openstack-ansible | 16:15 | |
evrardjp | I'd* | 16:15 |
evrardjp | I'd like to scale out my environment to give you an answer about the gains, but apparently my budget's wallet isn't strechable without more revenues first ;) | 16:17 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible-specs: Add spec for OS_.* venv support https://review.openstack.org/227921 | 16:19 |
cloudnull | evrardjp odyssey4me, i did an almost 200 node deployment the other day and will say that enabling fact caching helped a lot in terms of general speed when starting a playbook. | 16:22 |
*** galstrom_zzz is now known as galstrom | 16:22 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement fact caching https://review.openstack.org/225967 | 16:23 |
cloudnull | odyssey4me: updated title. | 16:23 |
odyssey4me | cloudnull good to know :) prior to that statement it was only an opinion | 16:23 |
cloudnull | we didnt have it on when we added roughly 150 compute nodes to the environment and it was crazy slow to gather all facts. I had remembered what bjoern had submitted enabled the caching using the json file and while fact gathering was slow at first all subsequent runs had no lag which was quite awesome | 16:25 |
*** sdake_ is now known as sdake | 16:26 | |
odyssey4me | cloudnull if you can add something to that effect in the bug then that'd be great :) | 16:26 |
*** agireud has joined #openstack-ansible | 16:30 | |
*** tnarg has quit IRC | 16:31 | |
mhayden | cloudnull: you may have my tiny +1 on virtualenvs ;) | 16:32 |
cloudnull | odyssey4me: https://bugs.launchpad.net/openstack-ansible/+bug/1498111 | 16:33 |
openstack | Launchpad bug 1498111 in openstack-ansible "Enable fact caching in ansible" [Wishlist,In progress] - Assigned to Bjoern Teipel (bjoern-teipel) | 16:33 |
odyssey4me | sweet, thanks cloudnull | 16:34 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement fact caching https://review.openstack.org/225967 | 16:35 |
cloudnull | tied it back to the fact gathering blueprint . | 16:36 |
openstackgerrit | Merged openstack/openstack-ansible: Add net.netfilter.nf_conntrack_max to Swift Storage https://review.openstack.org/226880 | 16:37 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Install irqbalance on hosts https://review.openstack.org/227942 | 16:38 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement fact caching https://review.openstack.org/225967 | 16:39 |
cloudnull | sorry for the spam | 16:39 |
cloudnull | now it properly linked. | 16:39 |
odyssey4me | cloudnull change the co-authored-by to you? | 16:41 |
odyssey4me | and rebase? | 16:41 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Implement fact caching https://review.openstack.org/225967 | 16:42 |
*** fawadkhaliq has joined #openstack-ansible | 16:42 | |
cloudnull | done | 16:42 |
*** elo has joined #openstack-ansible | 16:42 | |
*** harlowja has joined #openstack-ansible | 16:49 | |
*** galstrom is now known as galstrom_zzz | 16:53 | |
*** elo has quit IRC | 16:54 | |
*** elo has joined #openstack-ansible | 16:56 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Fixes log rotate issue https://review.openstack.org/227957 | 16:57 |
cloudnull | so this looks good post rebase | 16:59 |
cloudnull | https://review.openstack.org/#/c/225367/ | 16:59 |
cloudnull | need another core to do the needfuls | 17:00 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add ebtables to neutron agent configuration https://review.openstack.org/227963 | 17:00 |
*** skamithi has left #openstack-ansible | 17:00 | |
odyssey4me | yup, looks good | 17:00 |
*** skamithi has joined #openstack-ansible | 17:00 | |
odyssey4me | cloudnull I need a sanity check on https://review.openstack.org/227963 - not sure if I added ebtables in the right place to cover both the neutron agents containers and the compute hosts | 17:01 |
*** fawadkhaliq has quit IRC | 17:01 | |
cloudnull | odyssey4me: because the package lists collide in juno i'd add it to the https://github.com/openstack/openstack-ansible/blob/juno/rpc_deployment/vars/repo_packages/nova_libvirt.yml as that expressly runs on all compute nodes. | 17:03 |
odyssey4me | ok, so that will ensure that it covers both? | 17:03 |
cloudnull | it will | 17:04 |
*** galstrom_zzz is now known as galstrom | 17:04 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add ebtables to neutron agent configuration https://review.openstack.org/227963 | 17:04 |
odyssey4me | alright, updated - it's probably worth leaving that for some testing | 17:04 |
cloudnull | indeed. | 17:05 |
odyssey4me | I'm hesitent to add something like that knowing what it can break - but it's a CVE related fix | 17:05 |
cloudnull | this is an issue that we should get sorted for 11.2.3 if possible https://review.openstack.org/#/c/227957/ | 17:05 |
cloudnull | it kinda ties back to the on-metal testing that mattt was doing. but its something that i noticed in the OSIC lab. | 17:06 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Fixes log rotate issue https://review.openstack.org/227957 | 17:07 |
odyssey4me | ah, that does make sense | 17:07 |
cloudnull | add buug ref | 17:08 |
*** elo has quit IRC | 17:08 | |
odyssey4me | lgtm :) | 17:08 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Install nfs-common with nova-compute https://review.openstack.org/227969 | 17:11 |
*** abitha has joined #openstack-ansible | 17:11 | |
openstackgerrit | Merged openstack/openstack-ansible: Add Administration Capabilites to the Haproxy Stats GUI https://review.openstack.org/227042 | 17:11 |
openstackgerrit | Merged openstack/openstack-ansible: Add SSL/TLS listener to RabbitMQ https://review.openstack.org/223717 | 17:11 |
*** alop has joined #openstack-ansible | 17:15 | |
*** woodard has quit IRC | 17:16 | |
* mhayden hops for rabbit | 17:16 | |
*** woodard has joined #openstack-ansible | 17:16 | |
odyssey4me | cloudnull happy for 10.1.15 to release? | 17:23 |
cloudnull | i am though maybe we wait for some feedback for the CVE release ? | 17:24 |
odyssey4me | the ebtables thing? that'll be for the next tag | 17:25 |
odyssey4me | we won't get core votes in time for this one and there are other CVE's to cover for this one | 17:25 |
cloudnull | ok. | 17:25 |
*** elo has joined #openstack-ansible | 17:27 | |
odyssey4me | release done, wheels building | 17:37 |
*** skamithi13 has quit IRC | 17:51 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Updated juno for new dev work - 25 Sep 2015 https://review.openstack.org/227976 | 17:52 |
*** mrstanwell has joined #openstack-ansible | 17:53 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Updated juno for new dev work - 25 Sep 2015 https://review.openstack.org/227976 | 17:53 |
*** gparaskevas has joined #openstack-ansible | 17:58 | |
*** tiagogomes has quit IRC | 17:59 | |
mrstanwell | hi, folks. this should be an easy one: Has "openstack-ansible -e galera_ignore_cluster_state=true galera-install.yml" replaced "openstack-ansible galera-install.yml --tags galera-bootstrap" as the recommended maneuver for recovering after a reboot? Is there a more righter way to do it? | 18:05 |
openstackgerrit | Merged openstack/openstack-ansible: Update the AIO build convenience script https://review.openstack.org/227354 | 18:12 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Removes over zealous arp cache flushing https://review.openstack.org/225367 | 18:18 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated MariaDB to the new release version (10.0) https://review.openstack.org/178259 | 18:18 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add ebtables to neutron agent configuration https://review.openstack.org/227963 | 18:19 |
cloudnull | mrstanwell: your spot on. | 18:20 |
cloudnull | in https://review.openstack.org/#/c/178259/ we're looking to add back the specific function "--tags galera-bootstrap" with the ability to specify the cluster node. | 18:20 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Install irqbalance on hosts https://review.openstack.org/227942 | 18:20 |
mrstanwell | cloudnull: thanks! it looked like it worked right. but... well... openstack... | 18:21 |
cloudnull | that is always the case :) | 18:21 |
cloudnull | when in doubt, openstack. . . | 18:21 |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add policy changes required for OSSA-2015-018 / CVE-2015-5240 https://review.openstack.org/226872 | 18:21 |
mrstanwell | cloudnull: btw, I go on vacation and when I come back it's all renamed. And the world didn't end, even. I am in awe at your mad config mgmt skillz. | 18:22 |
cloudnull | it was a team effort :) | 18:23 |
cloudnull | but tyvm ! | 18:23 |
cloudnull | mattt: i owe you a beer, soda, lunch, dinner, whatever | 18:29 |
cloudnull | gates are passing in hpb4 =) | 18:30 |
*** brice has joined #openstack-ansible | 18:33 | |
odyssey4me | cloudnull the funny thing is that he had the source of the problem within an hour of looking at it | 18:34 |
odyssey4me | to think of all the time we've wasted with rechecks :p | 18:34 |
cloudnull | ++ :( | 18:35 |
cloudnull | but its better now. thus beers for everyone ! | 18:35 |
*** KLevenstein has quit IRC | 18:40 | |
*** alejandrito has quit IRC | 18:42 | |
*** dmsimard is now known as dmsimard|afk | 18:47 | |
*** skamithi13 has joined #openstack-ansible | 18:47 | |
*** galstrom is now known as galstrom_zzz | 18:48 | |
*** grumpylama has quit IRC | 18:49 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated MariaDB to the new release version (10.0) https://review.openstack.org/178259 | 18:52 |
*** grumpycat has joined #openstack-ansible | 18:57 | |
*** grumpycat is now known as Guest80969 | 18:57 | |
openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Install nfs-common with nova-compute https://review.openstack.org/227969 | 19:02 |
cloudnull | mhayden: can you cherry pick https://review.openstack.org/#/c/224304/ https://review.openstack.org/#/c/223717/ and https://review.openstack.org/#/c/224977/ to kilo ? | 19:04 |
mhayden | cloudnull: on my way | 19:05 |
openstackgerrit | Major Hayden proposed openstack/openstack-ansible: Update cached LXC image in place https://review.openstack.org/228005 | 19:05 |
odyssey4me | mhayden and https://review.openstack.org/226533 :) | 19:06 |
mhayden | cloudnull: it says "Code review - Error \n Cherry pick failed" on the last two | 19:06 |
mhayden | first went through | 19:07 |
cloudnull | :( likely a merge conflict | 19:07 |
odyssey4me | there's likely a dependent review that needs to go before it | 19:07 |
mhayden | that depends | 19:07 |
* mhayden attempted to make a joke | 19:07 | |
mhayden | yeah, last three all complained | 19:08 |
mhayden | odyssey4me: is that something i can fix? or do i need to wait on those others? | 19:08 |
odyssey4me | it's time you learned the cli tricks :) | 19:08 |
* mhayden gulps | 19:09 | |
odyssey4me | I'm shutting off for the night though - perhaps cloudnull can show you the way? | 19:09 |
mhayden | if there are docs, just point me to 'em | 19:09 |
mhayden | night, odyssey4me | 19:09 |
odyssey4me | effectively you need to figure out what changed the same files that were edited in the review | 19:10 |
odyssey4me | something changed one or more of the files in almost the same line (or the same line) | 19:11 |
cloudnull | what odyssey4me said . | 19:11 |
odyssey4me | once you've found the missing commit, then you have to decide whether it should be backported or not | 19:11 |
odyssey4me | so let's pick one and figure it out | 19:11 |
odyssey4me | https://review.openstack.org/#/c/224977/ only changed one file | 19:12 |
odyssey4me | compare https://github.com/openstack/openstack-ansible/commits/master/scripts/inventory-manage.py and https://github.com/openstack/openstack-ansible/commits/kilo/scripts/inventory-manage.py | 19:13 |
odyssey4me | easy enough to see https://github.com/openstack/openstack-ansible/commit/3962b9fb393f7c3658ba4f0202d70185cb33d8a2 was merged directly to the kilo branch | 19:14 |
mhayden | yeah, there's one commit different between them | 19:14 |
odyssey4me | it seems that'd be useful in master too, so let's forward port that | 19:15 |
odyssey4me | find it in https://review.openstack.org/#/q/is:watched+status:merged+branch:kilo,n,z and cherry-pick it to master | 19:15 |
mhayden | okay | 19:15 |
mhayden | https://review.openstack.org/#/c/223097/ <-- that one | 19:16 |
mhayden | so cherry pick that to master? | 19:16 |
odyssey4me | yep, except it appears that there was one in master already | 19:16 |
odyssey4me | select the change id in gerrit and you'll see all the matching change id's | 19:16 |
odyssey4me | the matching reviews with the same id | 19:17 |
cloudnull | looks like the rabbit ssl change is a simple doc update. | 19:17 |
odyssey4me | so you'll have to restore that one, then make it the same as the final commit in the kilo branch | 19:17 |
*** elo has quit IRC | 19:18 | |
cloudnull | odyssey4me mhayden i dont think https://review.openstack.org/#/c/223097/ should be in master. its a kilo specific change and if someone was going juno> to liberty they'd have to go through kilo first. | 19:19 |
odyssey4me | cloudnull, yes - now that I look at the details I agree | 19:19 |
cloudnull | i mean it doesnt hurt really, but it also is additionally debt that we dont need | 19:20 |
mhayden | by "restore that one", what do you mean? | 19:20 |
cloudnull | mhayden: i'd just fix the merge conflicts in the file and submit the cherrypick as is. | 19:20 |
cloudnull | no need to jump through that commit | 19:20 |
odyssey4me | mhayden so skip that train of thought, now we just go to the cli | 19:20 |
odyssey4me | get to your clone of openstack-ansible | 19:21 |
mhayden | so do i checkout kilo on my local box and apply my change, then git review? | 19:21 |
odyssey4me | yep | 19:21 |
mhayden | or well check it out so that the patch in conflict is right before mine? | 19:21 |
odyssey4me | well, hang on | 19:21 |
cloudnull | checkout kilo | 19:21 |
cloudnull | git cherry-pick -x d4d6b042df | 19:21 |
odyssey4me | you cherry pick your change onto origin/kilo with -x | 19:21 |
odyssey4me | what cloudnull said | 19:21 |
cloudnull | git status to see conflicts | 19:21 |
mhayden | ah and fix the conflicts after cherrypickin'? | 19:21 |
odyssey4me | yup | 19:22 |
mhayden | that kinda makes sense | 19:22 |
mhayden | thanks for the tutelage ;) | 19:22 |
cloudnull | once you fix the bits | 19:22 |
cloudnull | git cherry-pick --continue | 19:22 |
cloudnull | rather: git add <file> | 19:23 |
cloudnull | then: git cherry-pick --continue | 19:23 |
cloudnull | then: git review kilo | 19:23 |
cloudnull | finally: do the cherry-pick happy dance | 19:24 |
odyssey4me | not to be confused with the merge happy dance | 19:24 |
mhayden | imma take a crack at it | 19:25 |
cloudnull | exactly. | 19:25 |
cloudnull | ok im going to eat. BBL | 19:25 |
*** Mudpuppy has quit IRC | 19:27 | |
odyssey4me | ok, now I am really out | 19:27 |
odyssey4me | have a great w/end all :) | 19:27 |
*** galstrom_zzz is now known as galstrom | 19:34 | |
*** KLevenstein has joined #openstack-ansible | 19:47 | |
openstackgerrit | Merged openstack/openstack-ansible: Fix for keystone LDAP pkg missing https://review.openstack.org/226740 | 19:47 |
*** alejandrito has joined #openstack-ansible | 20:11 | |
*** woodard_ has joined #openstack-ansible | 20:11 | |
*** woodard has quit IRC | 20:15 | |
*** dmsimard|afk is now known as dmsimard | 20:15 | |
*** woodard_ has quit IRC | 20:18 | |
*** Mudpuppy has joined #openstack-ansible | 20:31 | |
*** galstrom is now known as galstrom_zzz | 20:39 | |
*** pradk has quit IRC | 20:40 | |
*** skamithi has left #openstack-ansible | 20:41 | |
*** elo has joined #openstack-ansible | 20:41 | |
*** alejandrito has quit IRC | 20:41 | |
*** alejandrito has joined #openstack-ansible | 20:46 | |
*** elo has quit IRC | 20:51 | |
*** skamithi13 has quit IRC | 20:58 | |
*** brice has quit IRC | 21:02 | |
*** sdake has quit IRC | 21:05 | |
*** galstrom_zzz is now known as galstrom | 21:07 | |
*** tlian has quit IRC | 21:11 | |
*** gparaskevas_ has joined #openstack-ansible | 21:14 | |
*** openstackgerrit has quit IRC | 21:16 | |
*** openstackgerrit has joined #openstack-ansible | 21:16 | |
*** gparaskevas_ has quit IRC | 21:18 | |
*** KLevenstein has quit IRC | 21:18 | |
*** jmckind has quit IRC | 21:28 | |
*** Mudpuppy has quit IRC | 21:28 | |
*** galstrom is now known as galstrom_zzz | 21:31 | |
*** galstrom_zzz is now known as galstrom | 21:32 | |
*** jhesketh has quit IRC | 21:39 | |
*** jhesketh has joined #openstack-ansible | 21:39 | |
*** Mudpuppy has joined #openstack-ansible | 21:39 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated MariaDB to the new release version (10.0) https://review.openstack.org/178259 | 21:40 |
*** spotz is now known as spotz_zzz | 21:44 | |
*** elo has joined #openstack-ansible | 21:53 | |
*** Mudpuppy has quit IRC | 21:55 | |
*** Mudpuppy has joined #openstack-ansible | 21:55 | |
*** openstackgerrit has quit IRC | 22:01 | |
*** openstackgerrit has joined #openstack-ansible | 22:01 | |
*** Mudpuppy has quit IRC | 22:06 | |
*** jlvillal has quit IRC | 22:06 | |
*** jlvillal has joined #openstack-ansible | 22:10 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated MariaDB to the new release version (10.0) https://review.openstack.org/178259 | 22:10 |
*** elo has quit IRC | 22:11 | |
*** alejandrito has quit IRC | 22:16 | |
*** elo has joined #openstack-ansible | 22:22 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated MariaDB to the new release version (10.0) https://review.openstack.org/178259 | 22:29 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Create config_template plugin https://review.openstack.org/228068 | 22:30 |
*** elo has quit IRC | 22:31 | |
*** openstack has joined #openstack-ansible | 22:35 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Adds the config_template to OpenStack Services https://review.openstack.org/228070 | 22:35 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to swift https://review.openstack.org/228071 | 22:36 |
*** elo has quit IRC | 22:36 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Adds the config_template to heat https://review.openstack.org/228072 | 22:36 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to galera_server https://review.openstack.org/228073 | 22:36 |
*** galstrom is now known as galstrom_zzz | 22:37 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Adds the config_template to glance https://review.openstack.org/228075 | 22:37 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Adds the config_template to keystone https://review.openstack.org/228076 | 22:37 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to tempest https://review.openstack.org/228077 | 22:37 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to neutron https://review.openstack.org/228078 | 22:38 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to nova https://review.openstack.org/228079 | 22:38 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Adds the config_template to cinder https://review.openstack.org/228080 | 22:39 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to galera_client https://review.openstack.org/228081 | 22:39 |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: adds the config_template to pip_lock_down https://review.openstack.org/228082 | 22:41 |
*** markvoelker has quit IRC | 22:46 | |
*** openstackstatus has quit IRC | 22:47 | |
*** openstack has joined #openstack-ansible | 22:50 | |
*** galstrom_zzz is now known as galstrom | 22:54 | |
openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: set unique galera server-id https://review.openstack.org/209404 | 22:54 |
*** mrstanwell has quit IRC | 22:57 | |
*** markvoelker has joined #openstack-ansible | 23:33 | |
*** galstrom is now known as galstrom_zzz | 23:49 | |
*** alop has quit IRC | 23:49 | |
*** sdake has joined #openstack-ansible | 23:49 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!