16:00:05 <b3rnard0> #startmeeting OpenStack Ansible Meeting
16:00:06 <openstack> Meeting started Thu Jan 29 16:00:05 2015 UTC and is due to finish in 60 minutes.  The chair is b3rnard0. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:09 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:00:21 <b3rnard0> #link https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Agenda_for_next_meeting
16:00:32 <b3rnard0> #topic RollCall
16:00:36 <mattt> \o
16:00:48 <cloudnull> present
16:00:56 <b3rnard0> hello
16:01:03 <rackertom> here
16:01:32 <claco> hola
16:01:35 <Sam-I-Am> yo
16:02:14 <hughsaunders> hey
16:02:27 <odyssey4me> o/
16:02:30 <b3rnard0> #topic What cherry-pick items do we need to sort out for icehouse/juno
16:02:57 <cloudnull> odyssey4me: you've been looking through the bp items .
16:03:02 <b3rnard0> #link https://review.openstack.org/#/q/status:open+project:stackforge/os-ansible-deployment,n,z
16:03:31 <cloudnull> do we have a good handle on what all needs to go in at this point.
16:04:04 <cloudnull> specifically in
16:04:06 <cloudnull> #link https://launchpad.net/openstack-ansible/+milestone/10.1.2
16:04:10 <odyssey4me> ok, anything that was targeted to a series should have a cherry-pick in flight
16:04:34 <odyssey4me> except for those that aren't marked in-progress
16:05:36 <cloudnull> kk.
16:05:45 <odyssey4me> those left either still have an unmerged patch waiting for review in master, or
16:05:58 <odyssey4me> have conflicts/dependancies which need to be sorted out
16:06:18 <cloudnull> so with that we have a couple of high priority items :
16:06:18 <cloudnull> https://bugs.launchpad.net/openstack-ansible/+bug/1415046
16:06:36 <odyssey4me> what hasn't been done is to trawl through all the bugs to see whether there are those which haven't yet been targeted at a milestone/series
16:07:12 <d34dh0r53> #here
16:07:41 <cloudnull> ok so, if you have time today, let go through them and make sure we're not missing anything.
16:08:29 <cloudnull> b3rnard0: action item.
16:08:55 * cloudnull trawl through all the bugs
16:09:07 <b3rnard0> #action cloudnull odyssey4me to go through bugs for targeting to milestones/series
16:09:46 <cloudnull> anythine else we want to point out on "What cherry-pick items do we need to sort out for icehouse/juno"
16:10:02 <odyssey4me> not from me
16:10:06 <cloudnull> ok, moving on.
16:10:09 <b3rnard0> #topic Review action items from last week
16:10:37 <cloudnull> so andymccr is out but has anyone else taken a stab at "investigate why rsyslog container doesn't send swift logs issue"
16:10:53 <cloudnull> this is related to d34dh0r53 To help andymccr on rsyslog issue
16:10:57 <odyssey4me> yep - I've got that sorted out in the patches that have been merged into master
16:11:30 <odyssey4me> it didn't affect icehouse as the way we created the folders & containers resulted in very permissive rights to the log files
16:11:44 <cloudnull> odyssey4me: tagged to "juno" ?
16:11:53 <odyssey4me> but for juno/master we changed that and rsyslog couldn't read some of the files
16:12:05 <odyssey4me> those patches are all merged into both master and juno :)
16:12:39 <odyssey4me> one outstanding patch for master which will need a backport: https://review.openstack.org/150483
16:12:51 <cloudnull> moving on to "Apsu Spike on F5 monitoring"
16:13:25 <hughsaunders> Apsu now wears gloves while handling F5s?
16:14:02 <Apsu> Gotta keep the fingers safe
16:14:49 <cloudnull> Apsu you had any time to look into the f5 monitoring ?
16:15:33 <Apsu> cloudnull: Been stuck on the API SSL story, so not really. I looked into it some the other day, and am considering pushing towards oslo middlware
16:15:42 <Apsu> For healthchecks that can be simple http status checks
16:16:02 <Apsu> There's some design work out there for it already, just needs progress.
16:16:12 <cloudnull> kk, so we'll carry that action item for now.
16:16:15 <Apsu> The alternative is to somehow shoehorn performant JSON into the F5.
16:16:28 <Apsu> Yeah, I have some good ideas for it.
16:16:38 <b3rnard0> #action Apsu Continue working on F5 monitoring spike
16:17:16 <cloudnull> so moving onto "cloudnull To target greater than wishlist next bugs into future milestones"
16:18:01 <cloudnull> which has been done a little bit but for our next sprint i believe we're going head long into gating so a lot of the wishlist items have been pushed for now.
16:18:50 <cloudnull> i skipped - "mattt To backport https://bugs.launchpad.net/openstack-ansible/+bug/1412762 into 10.1.3"
16:18:53 <cloudnull> my bad.
16:19:02 <odyssey4me> cloudnull: I just sorted that out
16:19:05 <odyssey4me> :)
16:19:20 <mattt> odyssey4me: the heat stuff ?
16:19:36 <odyssey4me> lol mattt nope - I just sorted out the bug targeting
16:19:42 <mattt> ah!
16:20:31 <b3rnard0> #info cloudnull To target greater than wishlist next bugs into future milestones -- per cloudnull: which has been done a little bit but for our next sprint i believe we're going head long into gating so a lot of the wishlist items have been pushed for now.
16:21:12 <cloudnull> so moving on, the tempest bits will all be a focus of ours real soon, so the targeting "next" makes sense.
16:21:42 <b3rnard0> #topic Blueprints
16:21:53 <b3rnard0> #link https://blueprints.launchpad.net/openstack-ansible/+spec/galaxy-roles
16:22:01 <b3rnard0> #link https://blueprints.launchpad.net/openstack-ansible/+spec/rackspace-namesake
16:22:09 <b3rnard0> #link https://blueprints.launchpad.net/openstack-ansible/+spec/inventory-cleanup
16:22:17 <b3rnard0> #link https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib
16:22:41 <cloudnull> we've been progressing on the genezied roles. and at the close of this sprint ill have a [WIP] to review. which I hope everyone will go pick on.
16:23:00 * odyssey4me nods
16:23:17 <cloudnull> is there anything that we want to talk about regarding these current BPs ?
16:23:22 <claco> where would the maas bits fall?
16:23:35 <odyssey4me> I've been testing it out and working through bits of it. It takes care of both the galaxy roles and rackspace-namesake blueprints nicely.
16:23:42 <b3rnard0> #info cloudnull:	we've been progressing on the genezied roles. and at the close of this sprint ill have a [WIP] to review. which I hope everyone will go pick on.
16:23:43 <cloudnull> they will be removed from the upstream stack .
16:23:46 <claco> they could likely be removed entirely using the same tinkerings miguel is doing for the tab stuff
16:24:12 <claco> seems like needs a bp proper to solidify that method
16:24:16 <cloudnull> i'd like to see up move the plays / roles into the rpc-maas repo
16:24:50 <odyssey4me> we'll need a volunteer to prepare a patch to convert rpc_support, maas and anything else rpc specific into a (or many) roles which can be installed by rpc after a standard os-ansible-deployment install
16:25:08 <cloudnull> ^ was writing that
16:25:34 <b3rnard0> odyssey4me: do you have a voluntold-er in mind?
16:25:40 <cloudnull> i think that we can build on what miguelgrinberg has done
16:26:05 <odyssey4me> ideally anyone but cloudnull - we need the ansible love spread around :)
16:26:05 <cloudnull> and do the same with rpc-support, rpc-maas and some of our rax specific scripts.
16:26:13 <cloudnull> +9000
16:26:46 <cloudnull> i am happy to help but would love to see someone else take the lead on that.
16:27:16 <claco> since he's not here to volunteer... I nominate miguelgrinberg
16:27:18 <claco> :-)
16:27:28 <odyssey4me> mattt would have been good, but is going on leave for two weeks from next week
16:27:56 <cloudnull> claco I second
16:28:07 <mattt> yeah i would happily take a stab but i'm only in mon/tue
16:28:33 <b3rnard0> #action miguelgrinberg to take the lead on: we'll need a volunteer to prepare a patch to convert rpc_support, maas and anything else rpc specific into a (or many) roles which can be installed by rpc after a standard os-ansible-deployment install
16:28:58 <b3rnard0> #action mattt will help miguelgrinberg before he leaves
16:29:33 <cloudnull> the other bp worth talking about is https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib
16:29:41 <mattt> :thumbsup:
16:29:44 <cloudnull> i know palendae has commented on it.
16:30:08 <palendae> Yep - no actual work done yet
16:30:09 <cloudnull> but has anyone else thought about it? is it a good idea/investmetn?
16:30:23 <cloudnull> do we care?
16:30:41 <palendae> I think it'd be nice, but could probably wait to be done
16:30:48 <mattt> i haven't read it so can't give any feedback
16:31:13 <hughsaunders> I would like a tool to query the inventory - list all groups for example
16:31:18 <odyssey4me> cloudnull: I agree it'd be nice - but I think that should be a target for later.
16:31:18 <hughsaunders> but not sure a DB is needed?
16:31:19 <Apsu> cloudnull: Just read through it. Seems interesting as a starting point to the ultimate scaling/managing goal
16:31:22 <b3rnard0> #action mattt to read https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib
16:31:30 <Apsu> Also seems like a simple task to convert to a lib/wrap
16:31:48 <odyssey4me> lol @ b3rnard0
16:31:53 <Apsu> The client tools using the lib/middleware connecting to storage/etc don't have to happen at the same time, obviously
16:31:58 <hughsaunders> #inprogress matt reading https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib
16:32:08 <d34dh0r53> haha
16:32:10 <mattt> seems like something worth considering, but not sure if it's pressing in light of what we have to do at the minute
16:32:19 <palendae> hughsaunders: I think the inventory-manage.py script is kind of that
16:32:34 <palendae> But it's a little basic, I don't know that it can show groups or such
16:32:38 <hughsaunders> #action hughsaunders to read inventory-manage.py
16:33:03 <cloudnull> ok, so lets push it for now as a nice to have and we'll revisit it sometime in the future.
16:33:04 <palendae> As I understood it, the DB was kind of an option
16:33:06 <Apsu> Theory is to be more DRY, and coalesce all that intelligence into a library
16:33:12 <palendae> Nothing we really *need* right now
16:33:21 <Apsu> Then -manage.py can be simple lib calls
16:33:26 <palendae> Yeah
16:33:49 <odyssey4me> perhaps someone would like to volunteer putting together some sort of PoC?
16:34:11 <b3rnard0> odyssey4me: i see a career in management for you :-)
16:34:26 <cloudnull> i nominate Apsu
16:34:27 <mattt> b3rnard0: the people he worked with didn't, hence him being here now
16:34:30 <odyssey4me> b3rnard0 them's fighting words :p
16:34:40 <Apsu> I have thus been nomernaded
16:34:43 <cloudnull> Apsu typie typie. make the go.
16:34:50 <Apsu> typey typey
16:34:59 <palendae> Link all that up to the F5 monitoring
16:35:00 <b3rnard0> #action Apsu perhaps someone would like to volunteer putting together some sort of PoC?
16:35:26 <cloudnull> moving on to bugs if nobody has anything else to add to these items.
16:35:46 <b3rnard0> #topic Bugs
16:35:57 <b3rnard0> #link https://bugs.launchpad.net/openstack-ansible
16:36:20 <b3rnard0> #link https://bugs.launchpad.net/openstack-ansible/+bugs?field.searchtext=&search=Search&field.status%3Alist=NEW&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branc
16:36:20 <b3rnard0> hes=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on&orderby=-importance&start=0
16:36:31 <cloudnull> b3rnard0 king of the link
16:36:32 <palendae> lol
16:36:39 <cloudnull> are there any bugs that need to be addressed, that have not been prioritized?
16:36:40 <b3rnard0> my link failed
16:36:43 <b3rnard0> lol
16:36:50 <Apsu> Yes, b3rnard0 not using a URL shortener.
16:36:53 <Sam-I-Am> tinyurl ftw
16:36:54 <cloudnull> #link https://bugs.launchpad.net/openstack-ansible/+bugs?search=Search&field.status=New
16:36:54 <Apsu> High priority
16:37:11 <b3rnard0> #action b3rnard0 to read about link shorteners
16:37:16 <hughsaunders> late comment: inventory management could be included in jasons support-ui?
16:37:56 <palendae> hughsaunders: Possibly! Though that's rax-specific
16:38:00 <cloudnull> hughsaunders: that might be an interesting take on that, but may be too rax specific
16:38:05 <palendae> But inventory management doesn't have to be
16:39:19 <odyssey4me> https://bugs.launchpad.net/openstack-ansible/+bug/1415411 I think is high importance
16:39:32 <odyssey4me> we have a mix of dynamic worker determination and hard coded workers
16:39:41 <odyssey4me> and some can be overridden
16:40:03 <odyssey4me> it'd be ideal to have them all dynamically determined, but also to allow them to be overridden by user variables
16:40:44 <cloudnull> that sounds like a sensible change.
16:40:53 <cloudnull> lets target that to juno .
16:41:06 <claco> the only other issue outstanding I can think of is the oslo.messaging issues
16:41:15 <odyssey4me> Apsu, I'll target https://bugs.launchpad.net/openstack-ansible/+bug/1399382 to you
16:41:42 <Apsu> odyssey4me: kk
16:42:09 <cloudnull> ok oslo messaging.
16:42:37 <cloudnull> specifically : https://bugs.launchpad.net/openstack-ansible/+bug/1406629
16:43:32 <cloudnull> oslo messaging 1.4.1 will satisfy juno requirements. https://github.com/openstack/requirements/blob/stable/juno/global-requirements.txt#L66
16:43:47 <claco> the main q was around upgrades for me
16:43:58 <claco> something code/person would need to force a downgrade of the package
16:43:59 <cloudnull> but pinning to 1.4.1 will force just about everyone to downgrade.
16:44:09 <claco> and is that really what we want, or do we ride it out until kilo?
16:44:47 <odyssey4me> or is there a workaround we can implement to help manage the situation a bit?
16:44:49 <cloudnull> from the looks of the project, if / when the patch to fix the uncontrollable q creation will not land until 1.6 +
16:45:36 <cloudnull> and when it does , it may not be a compatible change with juno.
16:45:44 <odyssey4me> perhaps there's a way to prefer 1.4.1, but leave a higher version in place if it's there?
16:46:14 <odyssey4me> ie new deployments will be fine, but existing deployments will stay as they are
16:46:19 <cloudnull> imo there are too many "if's" regarding oslo.messaging and hoping that a fix will come for juno.
16:46:20 <odyssey4me> (and can be manually fixed)
16:47:00 <cloudnull> if we pin to 1.4.1 existing deployments will already be satisifed because they have 1.5.1 installed.
16:47:26 <Apsu> Could carry the patch. Dirty but if the timeframe is awful, I'm not sure there's a great workaround besides monitoring channels/connection and restarting services if it blows up
16:47:45 <Apsu> Which is what our troubleshooting/monitoring strat currently is
16:47:46 <cloudnull> if they add additional services / infra nodes they will be in a situation where oslo messaging 1.4.1 and 1.5.1 are installed.
16:48:34 <odyssey4me> how disruptive is it to downgrade from 1.5 to 1.4.1?
16:48:52 <cloudnull> odyssey4me just about every os service will need to be restarted to use it.
16:48:57 <Apsu> Just to downgrade the package, not very unless post-install scripts restart things for you.
16:48:59 <Apsu> But ^
16:49:09 <Apsu> Gotta reload the python processes to use the new lib.
16:49:16 <cloudnull> ^ that one
16:49:29 <odyssey4me> ok, and if that's done in sequence per controller - what's the problem?
16:50:01 <cloudnull> there should be no problem , however we'll need to outline that process for deployers
16:50:03 <Apsu> I think cloudnull's point is more along the lines of the facts we have to A) keep track of what versions are where and B) schedule the maintenance to do so, in case things go bad
16:50:04 <cloudnull> as a release note
16:50:23 <odyssey4me> yeah - I'm definitely thinking a doc note
16:50:23 <Apsu> Which in the non-Rackspace world is just another facet of maintenance risk management.
16:50:32 <odyssey4me> and we should pin to 1.4.1
16:51:04 <odyssey4me> with the removal of all the rax stuff, we'll start getting new entrant users - better that they start on a good foot
16:51:55 <odyssey4me> we could pretty easily put together an 'upgrade' play to handle upgrades between major/minor versions
16:51:59 <Apsu> Should look into the mechanism of apt pinning/preferences for pinning to a lower version unless it's a downgrade.
16:52:05 <Apsu> Pretty sure you can do that, I've just forgotten how
16:52:18 <odyssey4me> Apsu we're not using apt for openstack code
16:52:45 <Apsu> odyssey4me: Then your suggestion of only pinning lower unless it's a downgrade probably isn't possible as a generic method.
16:52:51 <b3rnard0> #info there should be no problem, however we'll need to outline that process for deployers as a release note
16:52:57 <Apsu> But doc note, sure
16:53:02 <cloudnull> ok, lets put together a plan to pin it to 1.4.1 and then write up process for upgrading.
16:53:10 <cloudnull> b3rnard0 action item.
16:53:10 <claco> sigmavirus24 had some pip to force a downgrade as well
16:53:30 <claco> assuming this is a next target?
16:53:32 <sigmavirus24> claco: just `pip install oslo.messaging==1.4.1` will force a downgrade
16:53:42 <Apsu> yep
16:53:53 <b3rnard0> cloudnull: who is the volunteer
16:54:24 <cloudnull> ill step through that and see what it entails.
16:54:38 <cloudnull> and write up an doc note.
16:54:44 <b3rnard0> #action cloudnull plan to pin it to 1.4.1 and then write up process for upgrading.
16:54:48 <cloudnull> ok,
16:55:04 <cloudnull> so we have 5 min left. anything else someone wants to bring up ?
16:55:14 <claco> I like cheese.
16:55:23 <cloudnull> ^ is good
16:55:23 <b3rnard0> looks like we are pretty covered on the milestones with the earlier topics, so
16:55:26 <odyssey4me> it appears that d34dh0r53 and sigmavirus24 have done work on the issue itself and can perhaps help
16:55:32 <b3rnard0> #topic Open discussion
16:56:12 <sigmavirus24> odyssey4me: I was just there to help discover the issue. I also know how to make pip do bad things =P
16:56:15 <odyssey4me> #action b3rnard0 to prepare his whiskey cabinet for emptying later this year
16:56:22 <b3rnard0> odyssey4me: hah, def dark shades of management
16:56:32 <sigmavirus24> I'll be happy to help though anyway I can
16:56:42 <hughsaunders> odyssey4me: maybe b3rnard0 will share with us when we visit
16:56:46 <Apsu> sigmavirus24: You're helping enough elsewhere, stop volunteering.
16:56:49 <b3rnard0> odyssey4me: i have whiskey cart sir, it's all the rage with your favorite group of people
16:56:50 <Apsu> simma down!
16:57:10 <cloudnull> so i'm calling it.
16:57:15 <cloudnull> its been real its been nice
16:57:19 <hughsaunders> thanks cloudnull
16:57:22 <hughsaunders> laters
16:57:23 <mattt> ttyl
16:57:23 <odyssey4me> \o/
16:57:24 <b3rnard0> thanks everyone
16:57:31 <b3rnard0> #endmeeting