16:00:05 #startmeeting OpenStack Ansible Meeting 16:00:06 Meeting started Thu Jan 29 16:00:05 2015 UTC and is due to finish in 60 minutes. The chair is b3rnard0. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:07 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:09 The meeting name has been set to 'openstack_ansible_meeting' 16:00:21 #link https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Agenda_for_next_meeting 16:00:32 #topic RollCall 16:00:36 \o 16:00:48 present 16:00:56 hello 16:01:03 here 16:01:32 hola 16:01:35 yo 16:02:14 hey 16:02:27 o/ 16:02:30 #topic What cherry-pick items do we need to sort out for icehouse/juno 16:02:57 odyssey4me: you've been looking through the bp items . 16:03:02 #link https://review.openstack.org/#/q/status:open+project:stackforge/os-ansible-deployment,n,z 16:03:31 do we have a good handle on what all needs to go in at this point. 16:04:04 specifically in 16:04:06 #link https://launchpad.net/openstack-ansible/+milestone/10.1.2 16:04:10 ok, anything that was targeted to a series should have a cherry-pick in flight 16:04:34 except for those that aren't marked in-progress 16:05:36 kk. 16:05:45 those left either still have an unmerged patch waiting for review in master, or 16:05:58 have conflicts/dependancies which need to be sorted out 16:06:18 so with that we have a couple of high priority items : 16:06:18 https://bugs.launchpad.net/openstack-ansible/+bug/1415046 16:06:36 what hasn't been done is to trawl through all the bugs to see whether there are those which haven't yet been targeted at a milestone/series 16:07:12 #here 16:07:41 ok so, if you have time today, let go through them and make sure we're not missing anything. 16:08:29 b3rnard0: action item. 16:08:55 * cloudnull trawl through all the bugs 16:09:07 #action cloudnull odyssey4me to go through bugs for targeting to milestones/series 16:09:46 anythine else we want to point out on "What cherry-pick items do we need to sort out for icehouse/juno" 16:10:02 not from me 16:10:06 ok, moving on. 16:10:09 #topic Review action items from last week 16:10:37 so andymccr is out but has anyone else taken a stab at "investigate why rsyslog container doesn't send swift logs issue" 16:10:53 this is related to d34dh0r53 To help andymccr on rsyslog issue 16:10:57 yep - I've got that sorted out in the patches that have been merged into master 16:11:30 it didn't affect icehouse as the way we created the folders & containers resulted in very permissive rights to the log files 16:11:44 odyssey4me: tagged to "juno" ? 16:11:53 but for juno/master we changed that and rsyslog couldn't read some of the files 16:12:05 those patches are all merged into both master and juno :) 16:12:39 one outstanding patch for master which will need a backport: https://review.openstack.org/150483 16:12:51 moving on to "Apsu Spike on F5 monitoring" 16:13:25 Apsu now wears gloves while handling F5s? 16:14:02 Gotta keep the fingers safe 16:14:49 Apsu you had any time to look into the f5 monitoring ? 16:15:33 cloudnull: Been stuck on the API SSL story, so not really. I looked into it some the other day, and am considering pushing towards oslo middlware 16:15:42 For healthchecks that can be simple http status checks 16:16:02 There's some design work out there for it already, just needs progress. 16:16:12 kk, so we'll carry that action item for now. 16:16:15 The alternative is to somehow shoehorn performant JSON into the F5. 16:16:28 Yeah, I have some good ideas for it. 16:16:38 #action Apsu Continue working on F5 monitoring spike 16:17:16 so moving onto "cloudnull To target greater than wishlist next bugs into future milestones" 16:18:01 which has been done a little bit but for our next sprint i believe we're going head long into gating so a lot of the wishlist items have been pushed for now. 16:18:50 i skipped - "mattt To backport https://bugs.launchpad.net/openstack-ansible/+bug/1412762 into 10.1.3" 16:18:53 my bad. 16:19:02 cloudnull: I just sorted that out 16:19:05 :) 16:19:20 odyssey4me: the heat stuff ? 16:19:36 lol mattt nope - I just sorted out the bug targeting 16:19:42 ah! 16:20:31 #info cloudnull To target greater than wishlist next bugs into future milestones -- per cloudnull: which has been done a little bit but for our next sprint i believe we're going head long into gating so a lot of the wishlist items have been pushed for now. 16:21:12 so moving on, the tempest bits will all be a focus of ours real soon, so the targeting "next" makes sense. 16:21:42 #topic Blueprints 16:21:53 #link https://blueprints.launchpad.net/openstack-ansible/+spec/galaxy-roles 16:22:01 #link https://blueprints.launchpad.net/openstack-ansible/+spec/rackspace-namesake 16:22:09 #link https://blueprints.launchpad.net/openstack-ansible/+spec/inventory-cleanup 16:22:17 #link https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib 16:22:41 we've been progressing on the genezied roles. and at the close of this sprint ill have a [WIP] to review. which I hope everyone will go pick on. 16:23:00 * odyssey4me nods 16:23:17 is there anything that we want to talk about regarding these current BPs ? 16:23:22 where would the maas bits fall? 16:23:35 I've been testing it out and working through bits of it. It takes care of both the galaxy roles and rackspace-namesake blueprints nicely. 16:23:42 #info cloudnull: we've been progressing on the genezied roles. and at the close of this sprint ill have a [WIP] to review. which I hope everyone will go pick on. 16:23:43 they will be removed from the upstream stack . 16:23:46 they could likely be removed entirely using the same tinkerings miguel is doing for the tab stuff 16:24:12 seems like needs a bp proper to solidify that method 16:24:16 i'd like to see up move the plays / roles into the rpc-maas repo 16:24:50 we'll need a volunteer to prepare a patch to convert rpc_support, maas and anything else rpc specific into a (or many) roles which can be installed by rpc after a standard os-ansible-deployment install 16:25:08 ^ was writing that 16:25:34 odyssey4me: do you have a voluntold-er in mind? 16:25:40 i think that we can build on what miguelgrinberg has done 16:26:05 ideally anyone but cloudnull - we need the ansible love spread around :) 16:26:05 and do the same with rpc-support, rpc-maas and some of our rax specific scripts. 16:26:13 +9000 16:26:46 i am happy to help but would love to see someone else take the lead on that. 16:27:16 since he's not here to volunteer... I nominate miguelgrinberg 16:27:18 :-) 16:27:28 mattt would have been good, but is going on leave for two weeks from next week 16:27:56 claco I second 16:28:07 yeah i would happily take a stab but i'm only in mon/tue 16:28:33 #action miguelgrinberg to take the lead on: we'll need a volunteer to prepare a patch to convert rpc_support, maas and anything else rpc specific into a (or many) roles which can be installed by rpc after a standard os-ansible-deployment install 16:28:58 #action mattt will help miguelgrinberg before he leaves 16:29:33 the other bp worth talking about is https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib 16:29:41 :thumbsup: 16:29:44 i know palendae has commented on it. 16:30:08 Yep - no actual work done yet 16:30:09 but has anyone else thought about it? is it a good idea/investmetn? 16:30:23 do we care? 16:30:41 I think it'd be nice, but could probably wait to be done 16:30:48 i haven't read it so can't give any feedback 16:31:13 I would like a tool to query the inventory - list all groups for example 16:31:18 cloudnull: I agree it'd be nice - but I think that should be a target for later. 16:31:18 but not sure a DB is needed? 16:31:19 cloudnull: Just read through it. Seems interesting as a starting point to the ultimate scaling/managing goal 16:31:22 #action mattt to read https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib 16:31:30 Also seems like a simple task to convert to a lib/wrap 16:31:48 lol @ b3rnard0 16:31:53 The client tools using the lib/middleware connecting to storage/etc don't have to happen at the same time, obviously 16:31:58 #inprogress matt reading https://blueprints.launchpad.net/openstack-ansible/+spec/dynamic-inventory-lib 16:32:08 haha 16:32:10 seems like something worth considering, but not sure if it's pressing in light of what we have to do at the minute 16:32:19 hughsaunders: I think the inventory-manage.py script is kind of that 16:32:34 But it's a little basic, I don't know that it can show groups or such 16:32:38 #action hughsaunders to read inventory-manage.py 16:33:03 ok, so lets push it for now as a nice to have and we'll revisit it sometime in the future. 16:33:04 As I understood it, the DB was kind of an option 16:33:06 Theory is to be more DRY, and coalesce all that intelligence into a library 16:33:12 Nothing we really *need* right now 16:33:21 Then -manage.py can be simple lib calls 16:33:26 Yeah 16:33:49 perhaps someone would like to volunteer putting together some sort of PoC? 16:34:11 odyssey4me: i see a career in management for you :-) 16:34:26 i nominate Apsu 16:34:27 b3rnard0: the people he worked with didn't, hence him being here now 16:34:30 b3rnard0 them's fighting words :p 16:34:40 I have thus been nomernaded 16:34:43 Apsu typie typie. make the go. 16:34:50 typey typey 16:34:59 Link all that up to the F5 monitoring 16:35:00 #action Apsu perhaps someone would like to volunteer putting together some sort of PoC? 16:35:26 moving on to bugs if nobody has anything else to add to these items. 16:35:46 #topic Bugs 16:35:57 #link https://bugs.launchpad.net/openstack-ansible 16:36:20 #link https://bugs.launchpad.net/openstack-ansible/+bugs?field.searchtext=&search=Search&field.status%3Alist=NEW&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branc 16:36:20 hes=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on&orderby=-importance&start=0 16:36:31 b3rnard0 king of the link 16:36:32 lol 16:36:39 are there any bugs that need to be addressed, that have not been prioritized? 16:36:40 my link failed 16:36:43 lol 16:36:50 Yes, b3rnard0 not using a URL shortener. 16:36:53 tinyurl ftw 16:36:54 #link https://bugs.launchpad.net/openstack-ansible/+bugs?search=Search&field.status=New 16:36:54 High priority 16:37:11 #action b3rnard0 to read about link shorteners 16:37:16 late comment: inventory management could be included in jasons support-ui? 16:37:56 hughsaunders: Possibly! Though that's rax-specific 16:38:00 hughsaunders: that might be an interesting take on that, but may be too rax specific 16:38:05 But inventory management doesn't have to be 16:39:19 https://bugs.launchpad.net/openstack-ansible/+bug/1415411 I think is high importance 16:39:32 we have a mix of dynamic worker determination and hard coded workers 16:39:41 and some can be overridden 16:40:03 it'd be ideal to have them all dynamically determined, but also to allow them to be overridden by user variables 16:40:44 that sounds like a sensible change. 16:40:53 lets target that to juno . 16:41:06 the only other issue outstanding I can think of is the oslo.messaging issues 16:41:15 Apsu, I'll target https://bugs.launchpad.net/openstack-ansible/+bug/1399382 to you 16:41:42 odyssey4me: kk 16:42:09 ok oslo messaging. 16:42:37 specifically : https://bugs.launchpad.net/openstack-ansible/+bug/1406629 16:43:32 oslo messaging 1.4.1 will satisfy juno requirements. https://github.com/openstack/requirements/blob/stable/juno/global-requirements.txt#L66 16:43:47 the main q was around upgrades for me 16:43:58 something code/person would need to force a downgrade of the package 16:43:59 but pinning to 1.4.1 will force just about everyone to downgrade. 16:44:09 and is that really what we want, or do we ride it out until kilo? 16:44:47 or is there a workaround we can implement to help manage the situation a bit? 16:44:49 from the looks of the project, if / when the patch to fix the uncontrollable q creation will not land until 1.6 + 16:45:36 and when it does , it may not be a compatible change with juno. 16:45:44 perhaps there's a way to prefer 1.4.1, but leave a higher version in place if it's there? 16:46:14 ie new deployments will be fine, but existing deployments will stay as they are 16:46:19 imo there are too many "if's" regarding oslo.messaging and hoping that a fix will come for juno. 16:46:20 (and can be manually fixed) 16:47:00 if we pin to 1.4.1 existing deployments will already be satisifed because they have 1.5.1 installed. 16:47:26 Could carry the patch. Dirty but if the timeframe is awful, I'm not sure there's a great workaround besides monitoring channels/connection and restarting services if it blows up 16:47:45 Which is what our troubleshooting/monitoring strat currently is 16:47:46 if they add additional services / infra nodes they will be in a situation where oslo messaging 1.4.1 and 1.5.1 are installed. 16:48:34 how disruptive is it to downgrade from 1.5 to 1.4.1? 16:48:52 odyssey4me just about every os service will need to be restarted to use it. 16:48:57 Just to downgrade the package, not very unless post-install scripts restart things for you. 16:48:59 But ^ 16:49:09 Gotta reload the python processes to use the new lib. 16:49:16 ^ that one 16:49:29 ok, and if that's done in sequence per controller - what's the problem? 16:50:01 there should be no problem , however we'll need to outline that process for deployers 16:50:03 I think cloudnull's point is more along the lines of the facts we have to A) keep track of what versions are where and B) schedule the maintenance to do so, in case things go bad 16:50:04 as a release note 16:50:23 yeah - I'm definitely thinking a doc note 16:50:23 Which in the non-Rackspace world is just another facet of maintenance risk management. 16:50:32 and we should pin to 1.4.1 16:51:04 with the removal of all the rax stuff, we'll start getting new entrant users - better that they start on a good foot 16:51:55 we could pretty easily put together an 'upgrade' play to handle upgrades between major/minor versions 16:51:59 Should look into the mechanism of apt pinning/preferences for pinning to a lower version unless it's a downgrade. 16:52:05 Pretty sure you can do that, I've just forgotten how 16:52:18 Apsu we're not using apt for openstack code 16:52:45 odyssey4me: Then your suggestion of only pinning lower unless it's a downgrade probably isn't possible as a generic method. 16:52:51 #info there should be no problem, however we'll need to outline that process for deployers as a release note 16:52:57 But doc note, sure 16:53:02 ok, lets put together a plan to pin it to 1.4.1 and then write up process for upgrading. 16:53:10 b3rnard0 action item. 16:53:10 sigmavirus24 had some pip to force a downgrade as well 16:53:30 assuming this is a next target? 16:53:32 claco: just `pip install oslo.messaging==1.4.1` will force a downgrade 16:53:42 yep 16:53:53 cloudnull: who is the volunteer 16:54:24 ill step through that and see what it entails. 16:54:38 and write up an doc note. 16:54:44 #action cloudnull plan to pin it to 1.4.1 and then write up process for upgrading. 16:54:48 ok, 16:55:04 so we have 5 min left. anything else someone wants to bring up ? 16:55:14 I like cheese. 16:55:23 ^ is good 16:55:23 looks like we are pretty covered on the milestones with the earlier topics, so 16:55:26 it appears that d34dh0r53 and sigmavirus24 have done work on the issue itself and can perhaps help 16:55:32 #topic Open discussion 16:56:12 odyssey4me: I was just there to help discover the issue. I also know how to make pip do bad things =P 16:56:15 #action b3rnard0 to prepare his whiskey cabinet for emptying later this year 16:56:22 odyssey4me: hah, def dark shades of management 16:56:32 I'll be happy to help though anyway I can 16:56:42 odyssey4me: maybe b3rnard0 will share with us when we visit 16:56:46 sigmavirus24: You're helping enough elsewhere, stop volunteering. 16:56:49 odyssey4me: i have whiskey cart sir, it's all the rage with your favorite group of people 16:56:50 simma down! 16:57:10 so i'm calling it. 16:57:15 its been real its been nice 16:57:19 thanks cloudnull 16:57:22 laters 16:57:23 ttyl 16:57:23 \o/ 16:57:24 thanks everyone 16:57:31 #endmeeting