16:06:25 <odyssey4me> #startmeeting OpenStack-Ansible
16:06:25 <openstack> Meeting started Thu Apr 14 16:06:25 2016 UTC and is due to finish in 60 minutes.  The chair is odyssey4me. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:06:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:06:29 <openstack> The meeting name has been set to 'openstack_ansible'
16:06:57 <odyssey4me> #topic Agenda & Rollcall
16:07:16 <spotz> \o/
16:07:20 <mattt> \o
16:08:16 <automagically> o/
16:08:33 <odyssey4me> #link https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Agenda_for_next_meeting
16:08:39 <odyssey4me> #link http://eavesdrop.openstack.org/meetings/openstack_ansible/2016/openstack_ansible.2016-04-07-16.01.html
16:09:27 <odyssey4me> I had two action items last week - one to instrument all repositories for release notes. That's done in terms of our repo prep, I need to actually submit the infra patch to activate the jobs.
16:09:54 <odyssey4me> #action odyssey4me to submit patches to OpenStack-CI to activate release notes jobs for all IRR's
16:10:38 <odyssey4me> the other action item was to post the session details and send a notification to the ML
16:10:54 <odyssey4me> #link http://lists.openstack.org/pipermail/openstack-dev/2016-April/091760.html
16:10:55 <odyssey4me> done
16:11:16 <odyssey4me> #topic Newton Summit Planning
16:11:33 <cloudnull> o/
16:11:49 <odyssey4me> automagically you posted an etherpad for the one session - can you post it now to add to the meeting reference
16:12:08 <d34dh0r53> o/
16:12:10 <odyssey4me> has anyone else managed to prep an etherpad and is looking for early feedback?
16:12:21 <automagically> https://etherpad.openstack.org/p/openstack-ansible-newton-dynamic-inventory
16:12:37 <automagically> #link https://etherpad.openstack.org/p/openstack-ansible-newton-dynamic-inventory
16:14:15 * odyssey4me looking
16:16:05 <odyssey4me> looks good to me
16:16:30 <odyssey4me> I encourage everyone to take a peek and if you manage to do any investigation and put patches together ahead of time, make notes and add them to the etherpad
16:16:41 <odyssey4me> the more prepared we all are, the more productive the work session will be
16:17:36 <automagically> ++ to that ^
16:18:19 <cloudnull> ++
16:18:44 <odyssey4me> alright, I'll be doing most of my summit prep next week and hope to propose a few review for consideration and discussion
16:19:13 <odyssey4me> any other thoughts, comments, questions on the topic of the summit before we move on?
16:19:50 <evrardjp> none
16:19:57 <automagically> none here
16:20:41 <odyssey4me> alrighty then
16:20:53 <odyssey4me> #topic Release Planning and Decisions
16:21:17 <odyssey4me> We're scheduled to release 13.0.1, 12.0.11, 11.2.14 next week Thu.
16:21:41 <odyssey4me> The current state of the branch heads looks good. It seems like we have a rather healthy, stable base of code. :)
16:22:05 <odyssey4me> the only issue that's a bit of a thorn in the side seems to be https://bugs.launchpad.net/openstack-ansible/+bug/1569446
16:22:07 <openstack> Launchpad bug 1569446 in openstack-ansible "Secondary nodes fail to join galera cluster" [Critical,Confirmed] - Assigned to Darren Birkett (darren-birkett)
16:22:26 <odyssey4me> but it seems that mancdaz has managed to find a workaround that we can implement at least
16:22:28 <spotz> not very started but #link https://etherpad.openstack.org/p/openstack-ansible-newton-role-docs
16:22:35 <odyssey4me> so we should have something suitable for those releases
16:23:24 <odyssey4me> spotz the link command needs to be at the start of the line I think
16:23:25 <odyssey4me> ie
16:23:30 <odyssey4me> #link https://etherpad.openstack.org/p/openstack-ansible-newton-role-docs
16:23:49 <spotz> oops:)
16:23:59 <odyssey4me> :)
16:24:28 <odyssey4me> is anyone aware of any critical issues, or is anyone blocked on anything and needs assistance?
16:24:40 <odyssey4me> (related to any of the stable branches, or a master branch patch)
16:25:24 <mattt> i don't
16:25:30 <spotz> me neither
16:25:43 <odyssey4me> alrighty
16:25:50 <odyssey4me> #topic Blueprint work
16:26:23 <odyssey4me> any updates for any of the feature work? michaelgugino, automagically, vdo ?
16:26:36 <odyssey4me> logan- ^
16:27:11 <automagically> Still waiting on an internal team that’s been working OVS/DVR to get me their diffs so I can compare them to what we’ve already got in a submitted patch
16:27:12 <logan-> will begin hacking on mitaka after the summit so next month perhaps ill have a review up for the calico stuff.
16:27:55 <odyssey4me> excellent logan- :) our network subsystem support is starting to really awrm up nicely :)
16:27:58 <odyssey4me> *warm
16:27:58 <evrardjp> Thanks for the work
16:28:23 <odyssey4me> automagically great to hear that your team is on it too :)
16:28:35 <odyssey4me> #topic Open Discussion
16:28:50 <odyssey4me> well, we have plenty of time to shoot the breeze
16:28:50 <stevelle> have any of the broken out roles gotten multiple gate tests or scenarios tested in a single gate proposed yet?
16:29:10 <odyssey4me> stevelle well, the galera_server role does multiple things
16:29:13 <stevelle> I haven't seen, but there are a fair number of roles now
16:29:17 <cloudnull> stevelle: the galera role will do an upgrade but its all in a single gate
16:29:37 <odyssey4me> stevelle and I think that the ironic role may too - IDK if you made it test both standalone and integrated cloudnull ?
16:29:50 <cloudnull> odyssey4me:  its all in a single gate
16:30:00 <stevelle> I'll peek at that then
16:30:02 <odyssey4me> yeah, all single job - but the preference is to pipeline things
16:30:17 <stevelle> afraid the galera upgrade isn't as comparable
16:30:23 <odyssey4me> ie if it's possible to build the env, test, reconfigure, test then that's what infra preferrs
16:30:24 <cloudnull> we need to get the timeout higher to do more
16:30:31 <stevelle> ^ thats a concern for me
16:30:45 <mattt> yep my nova + tempest functional test needs more time too :(
16:30:56 <odyssey4me> we may have to get more creative with keeping it under the timeout first
16:31:09 <cloudnull> i think we just need to make it longer.
16:31:17 <odyssey4me> for instance it may make sense for some role to not bother building containers and just deploy everything on the host
16:31:41 <stevelle> I haven't done a time analysis yet because I still don't have convergence on gnocchi working in the gate but it seems strange to take a half hour to build 3 containers, install galera and 2 os_* services
16:31:44 <cloudnull> i think that makes sense if the role doesnt have additional code paths when there is more than one instance
16:31:52 <odyssey4me> I also need to put in a patch for infra to mirror mariadb
16:32:03 <cloudnull> if the role does something else when there is n+1 then we need to test that
16:32:18 <odyssey4me> cloudnull agreed, if n+1 is needed then containers make sense
16:32:20 <evrardjp> do we have metrics different that just parsing logs?
16:32:54 <stevelle> I was planning to use tempest tests to verify a working deployment
16:32:56 <cloudnull> we have 1.5 hours on the integrated gate, we should get the same for the roles too
16:33:00 <jmccrory> need to collect additional logs for IRRs instead of just ansible output also
16:33:02 <evrardjp> to see the time each step takes in average
16:33:12 <cloudnull> +1 jmccrory
16:33:23 <stevelle> I won't have time to deploy tempest and run it right now
16:33:25 <odyssey4me> cloudnull infra is very reticent to increase timeouts
16:33:25 <cloudnull> it'd also be nice to collect conf files too
16:33:41 <odyssey4me> stevelle if you cull the containers, might that help?
16:33:48 <cloudnull> odyssey4me:  then we need external CI and we need to make it voting.
16:33:56 <odyssey4me> yeah, we can add a log collection job - I'll do a patch for that
16:33:57 <stevelle> odyssey4me: can't cull containers when I have 2 services using apache
16:34:00 <cloudnull> the infra roles are quick
16:34:14 <cloudnull> but things like nova require building a cluster.
16:34:20 <odyssey4me> #action odyssey4me to add review for the collection of IRR job logs
16:35:13 <cloudnull> which requires an image, networks, ect.
16:35:36 <odyssey4me> cloudnull external CI would be preferable, but that requires appropriate org sponsorship - I know that RPC may be ready to do that later in the cycle and automagically and I have spoken about it, but no hard plans yet.
16:35:54 <mattt> nova can probably go full bare metal
16:36:06 <automagically> I have agreement in principal within my org, but need to get the work prioritized
16:36:14 <cloudnull> so i think our only option is to get infra to give the jobs more time
16:36:16 <mattt> but it's easier putting things into containers
16:36:23 <odyssey4me> stevelle I think that perhaps we need to aim to adjust how we do the Apache bits so that we don't have them clashing
16:36:52 <evrardjp> mattt nested virt could work
16:37:12 <stevelle> odyssey4me: agreed, and I would like to see abt fronting w/ nginx also
16:37:27 <jmccrory> should apache be a separate role with horizon/keystone/whatever else providing vhost confs?
16:37:34 <stevelle> jmccrory: +1
16:37:36 <mattt> jmccrory: +1
16:37:39 <mattt> that would be ideal
16:37:45 <odyssey4me> jmccrory I would think so, yes.
16:37:47 <palendae> Maybe re-use an existing apache role there?
16:37:51 <evrardjp> I agree jmccrory
16:37:52 <spotz> +1 jmccrory
16:37:52 <jmccrory> looked like barbican wanted apache as well
16:38:06 <odyssey4me> There's likely a decent Apache role available in Galaxy that already supports multiple platforms and all that
16:38:07 <cloudnull> some projects have timeouts as large as 190
16:38:12 <spotz> jmccrory: They working on it last I was involved with them
16:38:25 <cloudnull> idk think its an issue to ask for a full hour
16:38:41 <stevelle> +1 reuse a reverse-proxy if provided, identify roles to manage them -- ideally with optional container
16:38:50 <odyssey4me> cloudnull I'll do a request for an increased timeout, or perhaps try and get a per-role setting so that we can customised it based on the specific role needs
16:39:08 <odyssey4me> but I do want to ensure that we think creatively and try to reduce the moving parts in the role tests
16:39:16 <evrardjp> odyssey4me what's the reason they are afraid?
16:39:22 <evrardjp> of increasing the timeout
16:39:27 <evrardjp> isn't it conditional on the repo?
16:39:37 <cloudnull> based on our project config we're not setting a timeout for our roles.
16:39:37 <odyssey4me> evrardjp not afriad, they just have limited resources and longer timeouts mean that resources are held for longer
16:39:47 <cloudnull> just for the intagrated build
16:40:18 <cloudnull> I think getting it set to 60 should be an easy proposition
16:40:55 <stevelle> the reality is that deploying and functionally testing that takes longer than unit testing.
16:41:01 <palendae> ^
16:41:15 <palendae> We're doing full deploys of stuff
16:41:16 <cloudnull> also i'd rather us *not* think too creative to minimize time -- I'd like the tests to be representative of what will be deployed.
16:41:16 <stevelle> our peg isn't round
16:41:45 <cloudnull> what stevelle said
16:42:16 <palendae> Do the Puppet modules use infra?
16:42:20 <cloudnull> they do .
16:42:29 <palendae> I'd think they'd run into similar problems
16:42:39 <cloudnull> their tests have a timeout of timeout: 60
16:42:39 <palendae> Though Puppet's language is more like a full programming language
16:42:41 <odyssey4me> cloudnull we have the integrated build for the full functional test which is representative, and I hope that we can do the gate split to increase coverage
16:43:02 <odyssey4me> puppet, for instance, has 3 scenarios tested in their integrated build
16:44:23 <odyssey4me> so anyway, I'll discuss the timeouts with infra and find an appropriate solution which can be applied per role
16:44:42 <odyssey4me> but we should be creative in ensuring that we aren't wasting the time that we have to also ensure quick feedback
16:44:58 <evrardjp> agree, best of both
16:45:35 <odyssey4me> also, less moving parts ensures a more focused test with a smaller chance of unrelated failures
16:45:48 <mattt> i think that is fair
16:46:10 <odyssey4me> #action odyssey4me to submit a patch to increase the job timeout for selected roles
16:46:20 <cloudnull> sorry lost connection for a min there.
16:47:34 <odyssey4me> #action odyssey4me to submit a patch to infra to mirror the mariadb repo to help reduce role job execution times, and reduce the risk of failure
16:48:16 <odyssey4me> alright, is there anything else I can help with to smooth the glide path to success? ;)
16:49:36 <cloudnull> haha
16:49:45 <odyssey4me> is there anything else that anyone would like to raise?
16:49:45 <cloudnull> you need to monitize your core competencies
16:49:53 <cloudnull> :p
16:49:56 <evrardjp> monitize?
16:50:07 <palendae> monetize*
16:50:08 <cloudnull> *monetize
16:50:30 <evrardjp> :D
16:51:09 <odyssey4me> alrighty, if that's all then we're done for today
16:51:10 <cloudnull> related to buzz words https://www.youtube.com/watch?v=GyV_UG60dD4
16:51:25 <odyssey4me> thank you all for participating - have an awesome day!
16:51:30 <cloudnull> have a good one
16:51:31 <odyssey4me> #endmeeting