16:00:00 <mhayden> #startmeeting OpenStack-Ansible
16:00:01 <openstack> Meeting started Thu Jul 21 16:00:00 2016 UTC and is due to finish in 60 minutes.  The chair is mhayden. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:04 <openstack> The meeting name has been set to 'openstack_ansible'
16:00:12 <mhayden> #topic Roll call
16:00:16 <odyssey4me> o/
16:00:35 <odyssey4me> I may need to leave early, so please excuse me when I do.
16:00:46 <stevelle> o/
16:00:50 <eil397_> o/
16:00:52 <jmccrory> o/
16:00:57 <cloudnull> o/
16:01:32 <automagically> o/
16:02:22 <prometheanfire> o/
16:02:33 <d34dh0r53> o/
16:03:16 <andymccr> o/
16:03:23 <BjoernT> o/
16:03:36 <asettle> o/
16:03:44 <mhayden> woot, i think we have a good group here
16:03:54 <mhayden> #topic Action items
16:04:12 <mhayden> the only action item was for odyssey4me to mail the list about the mascot, and that appears to be going well
16:04:33 <mhayden> since mascots are on the topics list, i guess we could take a few minutes now to talk that over
16:04:46 <mhayden> #topic OSA Mascot
16:05:00 <mhayden> i was hoping to use d34dh0r53 as the mascot, but alas, that breaks the rules ;)
16:05:08 <odyssey4me> we need to agree on the final list
16:05:10 <michaelgugino> hi all
16:05:25 <eil397_> am i right that "yak" in modern english related to it has some negative connatation ?
16:05:41 <odyssey4me> yeah, yak shaving is not a positive thing
16:05:44 <eil397_> s\related to it\related to IT\g
16:05:47 <mhayden> original email thread:
16:05:49 <mhayden> #link https://openstack.nimeyo.com/90412/openstack-dev-openstack-ansible-project-mascot?qa_q=openstack-ansible+mascot&show=90412#q90412
16:05:51 <eil397_> : - ) ok
16:05:51 <automagically> eil397_: Can be used as a synonym for vomiting
16:06:29 <cloudnull> im partial to the "Osa Eucharitid" :)
16:06:54 <cloudnull> but not against the  Cape Buffalo
16:07:10 <michaelgugino> Palm tree.  It's the beach, you're taking it easy.  OSA gives you more time to visit the beach, because you're not deploying openstack.
16:07:14 <mhayden> that Cape Buffalo looks good, means business
16:07:34 <odyssey4me> heh, happy with a palm tree too :)
16:08:32 <spotz> o? ate
16:08:38 <spotz> o/ late even:)
16:08:48 <andymccr> how many do we need for the list?
16:08:50 <mhayden> do we have an etherpad started for this?
16:09:12 <odyssey4me> andymccr 3-5
16:09:20 <eil397_> Osa looks cool - http://www.petsfoto.com/wp-content/uploads/2011/06/Insect-Life1.jpg
16:09:23 <michaelgugino> watcher submitted top 4, to account for duplicates and ones that might violate the rules.
16:09:27 <odyssey4me> no need for an etherpad, this meeting log is record which I'll use
16:09:35 <odyssey4me> but you can etherpad if you like
16:09:39 <odyssey4me> I gotta run - bbl
16:09:40 <spotz> mascot http://www.abbeyroaddals.com/images/hippiecatch.jpg !!!
16:09:51 <spotz> later odyssey4me
16:09:56 <jmccrory> i like the osa bug
16:09:56 <michaelgugino> most projects are picking animals I think, so I'm partial to something that's not an animal.
16:10:27 <mhayden> jmccrory: that osa bug looks like the insect version of the cape buffalo
16:10:30 <automagically> I most definitely do not have strong opinions on the matter
16:10:43 <eil397_> mhayden: right : - )
16:10:46 <cloudnull> ^ i also like that suggestion.
16:10:48 <mhayden> perhaps we could use a peacock, automagically? :)
16:11:05 <cloudnull> michaelgugino: re: not doing what everyone else is
16:11:06 <automagically> mhayden: Go for it, a bird, a bug, a buffalo….whatevs
16:11:49 <andymccr> im with automagically, no strong opinions really.
16:12:07 <eil397_> we can have one animal in list , one isect, one tree and...
16:12:13 <mhayden> can someone toss that stuff into an etherpad so we can vote and move things around?
16:12:26 <eil397_> s\isect\insect\g
16:13:10 <andymccr> all the suggestions that are being put forward seem appropriate - and we need a list of 3-5 so may as well submit them unless there are major issues with any of those.
16:14:07 <mhayden> odyssey4me: got what you need here? :)
16:14:29 <d34dh0r53> How about a camel since TMTOWTDI is possible with OSA?
16:15:52 <stevelle> As a last oddball suggestion, I'll throw out the Chanterelle as a mascot
16:16:04 <spotz> heeheh
16:16:18 <cloudnull> stevelle: thats a good one
16:16:43 <spotz> +1
16:16:50 <mhayden> https://etherpad.openstack.org/p/osa-mascot
16:17:25 <spotz> mhayden: You left off the dog!:)
16:18:18 <d34dh0r53> lol
16:18:22 <mhayden> oops
16:18:28 <mhayden> okay, can we move to the next topic for now?
16:18:41 <automagically> yes pleasew
16:18:44 <mhayden> haha okay
16:18:53 <mhayden> #topic wsgi vs uwsgi and Apache vs nginx role consistency
16:19:03 <mhayden> stevelle / cloudnull / odyssey4me: take it away!
16:19:20 <stevelle> Sure
16:20:04 <odyssey4me> back
16:20:09 <stevelle> We had a bit of discussion in the irc channel this week about this blueprint https://blueprints.launchpad.net/openstack-ansible/+spec/keystone-uwsgi
16:20:59 <stevelle> It was inspired by a change automagically proposed to fix Aodh. Upstream changed the way the API service is started.
16:21:09 <automagically> #link https://review.openstack.org/#/c/344555/
16:21:25 <automagically> #link https://review.openstack.org/#/c/343918/
16:21:40 <stevelle> To fix Aodh automagically proposed to run aodh-api under mod_wsgi with Apache, which conforms to how we run other wsgi services in OpenStack
16:21:58 <stevelle> All of those are run that way because this used to be the recommended means of running Keystone
16:22:06 <stevelle> particularly important with Federation
16:22:23 <stevelle> The Keystone community has changed their recommendation as noted in the blurprint
16:23:05 <cloudnull> stevelle: do we know if we lose our federation capabilities by moving away from what we curerntly have?
16:23:28 <stevelle> We seemed to agree in IRC at the time that we would like the WSGI apps to run similarly, so if we change Keystone away from Apache / mod_wsgi, we want to look at changing the others (Horizon, Gnocchi, Aodh in this case)
16:23:40 <stevelle> cloudnull: I don't know, haven't investigated that yet.
16:23:44 <cloudnull> ok
16:24:05 <automagically> Until we know if federation works, I think we should table moving all the roles to uwsgi
16:24:13 <michaelgugino> I think it's worth investigating, but I don't know if it's appropriate for this cycle.
16:24:13 <cloudnull> +1
16:24:14 <automagically> i.e. I’d rather be consistent for the time being
16:24:47 <stevelle> I did a proof of concept for running aodh-api under uwsgi just so we could look at it.
16:24:52 <stevelle> #link https://blueprints.launchpad.net/openstack-ansible/+spec/keystone-uwsgi
16:24:59 <stevelle> #link https://review.openstack.org/#/c/344555/
16:26:27 <stevelle> The PoC failed in the integrated gate due to the general merge issues.
16:26:36 <andymccr> id love to see all openstack api services using wsgi/uwsgi in a uniform way and not just these few - that's not really how it is right now though.
16:27:28 <stevelle> +1 but for Aodh and Gnocchi particularly they won't run under Werkzeug
16:28:05 <stevelle> Horizon also
16:28:27 <andymccr> sounds good to standardize esp if keystone have changed their recommendation
16:28:57 <michaelgugino> yes, uwsgi outperforms gevent/gunicorn/whatever they're using handily.
16:29:11 <mhayden> so it sounds like we've agreed that standardizing on uwsgi across the board makes sense?
16:29:17 <automagically> andymccr: I agree, but I think the question is how long to do we think it takes to get standardized across horizon, gnocchi, aodh, keystone
16:29:46 <michaelgugino> as a long term goal, but I don't think it's appropriate for this cycle.  We're a little far in, and I think we need to give greater attention to other areas, specifically Xenial and CentOS
16:29:50 <odyssey4me> mhayden where possible, yes - but the caution is that if it's not in by N3 or we don't think we can do it by N3 then it must carry over to the next cycle
16:29:51 <stevelle> based on the proposed change, it doesn't seem like it will be much of a challenge to move over the four.
16:30:03 <mhayden> odyssey4me: sounds good
16:30:21 <automagically> assuming that federation support can be maintained in keystone
16:30:31 <mhayden> #agreed Should begin using uwsgi wherever possible -- if not available by Newton-3, it needs to happen in O
16:30:32 <automagically> Do we have any gate checks around federation that can ensure we don’t break it?
16:30:40 <odyssey4me> automagically nope
16:31:10 <automagically> More short term, how do we want to unblock the aodh role gate?
16:32:03 <stevelle> With the uncertainty around federation, I'm inclined to say we should evaluate https://review.openstack.org/#/c/343918/
16:32:15 <automagically> And Newton-3 is 8/1 correct?
16:33:24 <automagically> Sorry, its 9/1
16:33:43 <cloudnull> stevelle:  +1
16:34:08 <andymccr> sounds like a good idea then we can evaluate a more long term approach later.
16:34:14 <cloudnull> the aodh gate needs to be unblocked.
16:34:20 <automagically> stevelle: +1 and michaelgugino +1 on needing to focus on Xenial
16:34:39 <automagically> So it sounds like we are decided
16:35:14 <stevelle> I'm probably not going to have time to do the development for all of the above including resolving federation by Aug 1, so seems clear to me
16:35:18 <mhayden> #agreed Unblock the aodh gate first, then worry about a long term approach later
16:35:28 <michaelgugino> +1
16:35:37 <stevelle> seems dangerous to expect it to be correct and merged by Sept 1
16:36:10 <stevelle> next topic?
16:36:11 <automagically> So, michaelgugino brought up Xenial, I’m having a hard time seeing the big picture on what work remains there
16:36:15 <mhayden> stevelle: woot
16:36:24 <mhayden> #topic Mid-cycle planning
16:36:50 <automagically> Can we #topic Xenial to-do list?
16:37:07 <mhayden> automagically: sure, i can do that next
16:37:11 <automagically> Awesome!
16:37:17 <mhayden> the meeting room is still good to go for the mid-cycle
16:37:25 <mhayden> the agenda is in the etherpad
16:37:33 <michaelgugino> is there an event bright or anything for the midcycle?
16:37:35 <mhayden> #link https://etherpad.openstack.org/p/osa-midcycle-newton
16:37:56 <mhayden> michaelgugino: not at the moment
16:37:59 <odyssey4me> michaelgugino nope, just the etherpad
16:38:07 <palendae> mhayden, Could you add the meeting room into the etherpad?
16:38:09 <mhayden> however, i need to make sure i have an accurate list so i can send the names down to our physical security folks
16:38:14 <mhayden> palendae: sure
16:38:22 <cloudnull> from what i can tell we need to do rally, babican, and zaqar to make xenial fully supporte.d
16:38:26 <spotz> michaelgugino: I'm going to make some eventbrites for dinners and put the links on the etherpad
16:38:40 <cloudnull> i've been deploying the integrated gate w/ xenial for a while now and it seems to work well.
16:38:46 <mhayden> palendae: added
16:38:47 <michaelgugino> I added my name as tentative, I should know by next week if I'm going, I don't think I'll have a problem getting the travel approved, but we'll see :)
16:38:54 <palendae> Speaking of our physical security: pokemon go is banned in the Castle, for those non-Rackers coming
16:39:11 <michaelgugino> omg
16:39:20 <mhayden> haha
16:39:23 <cloudnull> hahaha
16:39:25 <automagically> palendae: Well, than I better cancel my trip
16:39:29 <palendae> Don't get kicked out for something silly :)
16:39:32 <mhayden> okay, well please everyone make sure your name is in there
16:39:59 <mhayden> #topic Xenial to-do list
16:40:05 <cloudnull> michaelgugino: once the intgrated gate is unblocked it'd be great to get more testers on xenial things
16:40:31 <michaelgugino> yeah, I think we need more manual testing, lots of code paths not checked by default (like OVS, DVR) etc
16:40:38 <mhayden> cloudnull: i can try out xenial on some of my lab boxes
16:40:45 <cloudnull> -- from what i can tell we need to do rally, babican, and zaqar to make xenial fully "supported".
16:41:10 <odyssey4me> michaelgugino OVS testing is on by default for the os_neutron role right now
16:41:14 <automagically> Since those are non-integrated roles, do we let them slide if they are not done?
16:41:15 <cloudnull> -- i've been deploying the integrated gate w/ xenial for a while now and it seems to work well. but to michaelgugino point there are some manual code paths that need to be exercised.
16:41:24 <mhayden> brb in a sec
16:41:29 <automagically> And barbican looks to be done: https://review.openstack.org/#/c/341761/
16:41:49 <cloudnull> awesome.
16:42:00 <michaelgugino> that's good about OVS.  But yeah, just saying generally, I'm not up to date on all the different configurations.  Swift comes to mind, I know we're not checking all of the backing stores
16:42:02 * cloudnull moves card
16:42:06 <odyssey4me> yeah, primary focus is on the integrated roles - secondary focus are the role we want to integrate this cycle
16:42:51 <odyssey4me> michaelgugino the roles have the ability to do multiple scenario tests now, just like the neutron role
16:42:57 <cloudnull> cloudkitty is another one i'd like to see if we could optinally enable, similar to ironic.
16:43:04 <automagically> So should we have an action item to make sure that https://etherpad.openstack.org/p/openstack-ansible-newton-ubuntu16-04 is up to date
16:43:08 <odyssey4me> all it takes is a change to tox config, then an addition of the jobs in infra
16:43:30 <odyssey4me> so if you want to test more paths, please go ahead and add the tox config to make it happen
16:43:44 <michaelgugino> you mean add another tox environment?
16:43:50 <odyssey4me> we have experimental checks for xenial/centos in the integrated repo
16:44:16 <odyssey4me> due to the master branch brokenness I haven't manage to figure out whether it's working very well - because it works slightly differently
16:44:48 <odyssey4me> ie it uses tox
16:46:07 <automagically> #action automagically update xenial etherpad status
16:46:28 <automagically> #action mhayden test Xenial in his lab
16:46:33 <mhayden> automagically: that'd be most excellent
16:46:34 <mhayden> indeedy
16:46:40 <mhayden> the lab needs a reimage anyway
16:46:58 <odyssey4me> michaelgugino yes, either adapt the existing test to make it test more... or add a new tox env for a different test entirely
16:47:13 <automagically> Sounds like we have an action around figuring out the integrated gate issues for xenial
16:47:29 <odyssey4me> michaelgugino I would suggest trying to pipeline tests as far as possible, and only using multiple tox environments where the environments are structurally different
16:48:21 <michaelgugino> odyssey4me: is there an example, you mentioned OVS in neutron?
16:48:32 <mhayden> is there a plan to get a OSA integrated gate running on xenial?
16:48:38 <odyssey4me> michaelgugino https://github.com/openstack/openstack-ansible-os_neutron/commit/eb0bb51fafa01926a0d13021a1f97c333bbdabd8
16:48:49 <odyssey4me> mhayden there is already an experimental check
16:48:53 <mhayden> (about 11 minutes left in the meeting -- just a warning)
16:49:21 <odyssey4me> I'm not convinced yet that it's working because since implementing it the integrated gate has not been passing frequently.
16:49:38 <michaelgugino> that looks a little complicated, so I'll have to take some time to look it over.
16:49:40 <odyssey4me> so I'm waiting for that storm to blow over.
16:50:27 <automagically> michaelgugino: Would definitely appreciate your review of that
16:50:49 <odyssey4me> I can see some tox config optimisations which can be done now that it's in.
16:50:53 <odyssey4me> I'll drop a patch shortly.
16:51:35 <michaelgugino> ok, I've been a little busy recently, but I'm planning on getting back into the full swing here at OSA, more reviews, bugs, etc.  Just been a really busy month or so. :)
16:53:39 <mhayden> #topic Open floor
16:53:44 <mhayden> anyone have anything else to discuss?
16:53:48 <mhayden> only a few minutes left
16:54:03 <cloudnull> so i've noticed something with regard to gate testing like crazy. IDK if this is just me, but when I have callback plugins enabled the run is notably slower.
16:54:51 <cloudnull> Example http://cdn.pasteraw.com/gybn3gfnnum767fvnzbed2oyax66rxs
16:54:59 <cloudnull> thats just rerunning the keystone play on an already deployed cluster.
16:55:34 <cloudnull> I know we "use" them in the gate, but it wouls seem disabling them would be to our benifit in terms of testing speed.
16:55:35 <michaelgugino> I promised a nova-lxd patch last week.  I have it cut locally, but installing from pip doesn't work because they haven't published a newton package in pypi yet, so I need to install from source.
16:55:37 <automagically> ouch - time for some profiling
16:56:14 <michaelgugino> but, it's coming, I am working on it.  Next week is looking good (on travel this week)
16:56:41 <odyssey4me> cloudnull I think we should kill the profiling callback
16:56:53 <odyssey4me> it's actually annoying that it only reports the top 10 tasks
16:57:15 <cloudnull> the human readable one as well.
16:57:17 <automagically> Thanks to all of you who have been reviewing the long list of patchs for https://review.openstack.org/#/q/status:open+topic:bp/multi-rabbitmq-clusters
16:57:20 <automagically> Really appreciate it
16:57:21 <cloudnull> i mean we use it .
16:57:23 <jmccrory> i've also seen fact_caching cause a lot of problems in local deployments, lots of random slow downs. i'm wondering if that's causing some of the randomness of simple tasks to take 60+ seconds
16:57:29 <mhayden> why don't we remove both callbacks and see how the gate looks after?
16:57:33 <cloudnull> but it causes a delay in processing
16:58:10 <odyssey4me> maybe for the gate we should disable fact caching?
16:58:22 <mhayden> okay, let's take the caching/callback discussion to #openstack-ansible and close this thing up
16:58:22 <mhayden> thanks everyone!
16:58:24 <cloudnull> jmccrory: I too have been looking into that. I think a lot of those random slow downs are due to the ansible "smart" transport
16:58:28 <mhayden> #endmeeting