16:03:40 <odyssey4me> #startmeeting OpenStack-Ansible
16:03:41 <openstack> Meeting started Thu Mar 10 16:03:40 2016 UTC and is due to finish in 60 minutes.  The chair is odyssey4me. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:03:42 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:03:45 <openstack> The meeting name has been set to 'openstack_ansible'
16:03:46 <odyssey4me> #topic Roll call and Agenda
16:03:51 <automagically> o/
16:03:57 <odyssey4me> #link https://wiki.openstack.org/wiki/Meetings/openstack-ansible#Agenda_for_next_meeting
16:04:20 <izaakk> o/
16:04:43 <spotz> o/ only here for a few team outing in 10:)
16:05:17 <javeriak_> o/
16:05:37 <jmccrory> o/
16:05:50 <rromans> \o
16:05:51 <mattt> o/
16:05:55 <prometheanfire> \o
16:06:06 <raddaoui> o\
16:07:49 <michaelgugino> hello
16:08:10 <odyssey4me> #topic Review action items from last week
16:08:48 <odyssey4me> it looks to me like all action items were completed
16:09:17 <odyssey4me> moving on
16:09:20 <odyssey4me> #topic Newton Summit Planning
16:09:49 <odyssey4me> I added a topic proposal for general 'Project Housekeeping'
16:09:56 <odyssey4me> we could do it in the community day
16:10:13 <automagically> good addition
16:10:27 <odyssey4me> I've asked for 1 fishbowl, 8 work sessions and 1 community day
16:10:46 <odyssey4me> I'd be surprised if we get 8 work sessions, but it was worth asking. :)
16:11:35 <odyssey4me> In the fishbowl I figured that we can do a general update on the project, share Mitaka successes and get some feedback from anyone who wants to ask for anything the project doesn't currently do.
16:12:22 <odyssey4me> we'll have to see the final schedule and room allocations to figure out what would be most useful
16:12:33 <odyssey4me> any questions/comments before we move on?
16:13:02 <hughsaunders> o/ (late)
16:14:18 <odyssey4me> #topic Define core team expectations/Add non-Rackspace cores
16:14:32 <automagically> Thanks for all the feedback on that patch
16:14:50 <automagically> In reading through it, I think it might make sense for folks to submit updates directly
16:14:57 <odyssey4me> The rest of the core team still have not reviewed https://review.openstack.org/286858 - please can we have that done before COB tomorrow andymccr mattt hughsaunders d34dh0r53 cloudnull
16:15:20 <hughsaunders> Close-Of-Community surely?
16:15:31 <odyssey4me> ie by next week :p
16:15:55 <hughsaunders> I ignored it because it has 2x -1, but I can have a read
16:15:56 <odyssey4me> we need to revise and finalise that - I'd like to get it merged by the end of next week
16:16:26 <odyssey4me> #topic RoadMap update for the OpenStack Product Working Group
16:16:31 <odyssey4me> #link RoadMap update for the OpenStack Product Working Group
16:16:47 <odyssey4me> oops
16:16:54 <odyssey4me> #link https://etherpad.openstack.org/p/openstack-ansible-newton-roadmap-refresh
16:17:26 <odyssey4me> I'd like feedback on that please - this will be part of what is published by the Product Working Group
16:17:47 <odyssey4me> ideally we need to consider what we're thinking of doing next cycle and express it in terms of the high level themes
16:17:58 <odyssey4me> I welcome anyone who can express things better than I can. :)
16:18:07 <automagically> When would you like to have received all feedback?
16:18:32 <odyssey4me> I think by Tue next week ideally.
16:18:45 <odyssey4me> But we can stretch until Thu next week if need be.
16:19:06 <odyssey4me> jmccrory ping?
16:19:10 <jmccrory> hi
16:19:18 <odyssey4me> #topic Apt package lists for repo_server and repo_build
16:19:57 <jmccrory> so was looking into a bug about reducing number of packages installed in the repo_server role, there's about 40 right now
16:20:23 <odyssey4me> hmm, perhaps we should do something similar to what's been done with the pip packages
16:20:40 <jmccrory> cloudnull brought up a good idea in the review about separating out library packages required for actually building the wheels into repo_build
16:20:48 <jmccrory> https://review.openstack.org/#/c/289510/
16:21:03 <odyssey4me> ie the stuff needed for repo building should be in the repo_build role, and the stuff to get the repo_server functional for syncing content and stuff should be in the repo-server role
16:21:17 <automagically> That separation of responsibilities makes sense
16:21:19 <odyssey4me> heh, same thought :)
16:21:32 <jmccrory> think that would help separate concerns between the roles and making writing tests more clear. repo_server focused on nginx + syncing, repo_build git and wheel building
16:21:50 <odyssey4me> yep, makes sense to me - great initiative by the way!
16:22:01 <automagically> +1
16:22:03 <michaelgugino> +1
16:22:11 <odyssey4me> a lot of the python packages and apt package lists carry history and many of them may no longer be required
16:22:26 <cloudnull> o/ finally. :)
16:22:29 <jmccrory> yeah, noticed that too in testing and tried to comment on which are still needed and where
16:22:34 <logan-> i like the comments about why things are being installed instead of just a list of packages
16:22:43 <automagically> logan-: +2-
16:22:48 <odyssey4me> ++
16:22:53 <automagically> Whoops +20 rather
16:22:58 <odyssey4me> heh
16:23:26 <odyssey4me> we should possibly do something similar in the python package lists
16:24:01 <odyssey4me> happy to move on?
16:24:16 <jmccrory> yep, thanks for feedback
16:24:21 <odyssey4me> #topic Release Planning and Decisions
16:24:51 <odyssey4me> we have quite a hit list for mitaka going: https://launchpad.net/openstack-ansible/+milestone/13.0.0
16:25:26 <odyssey4me> most importantly though we need functional testing in all the roles
16:25:33 <odyssey4me> the current target is just convergence
16:25:48 <automagically> Looks like we should probably trim the blueprints list a bit
16:25:49 <odyssey4me> after that we need to actually do real tests
16:26:04 <automagically> Do we expect to get Desginate and the AIO gate work done
16:26:18 <odyssey4me> yeah, the AIO gate split is going to have to carry over
16:26:36 <odyssey4me> designate is actually very close to done, but I think it's going to have to slip too
16:26:54 <automagically> Doubtful we’d have testing done for designate in time
16:26:55 <odyssey4me> pretty much ll those will have to move to the next cycle
16:26:59 <odyssey4me> it's too late for new features now
16:27:27 <spotz> o/ off to outing
16:27:34 <mattt> spotz: do enjoy
16:27:41 <odyssey4me> I need to do some housekeeping on the milestone.
16:27:58 <odyssey4me> mattt do you want to discuss the functional testing based on your work so far?
16:28:16 <odyssey4me> we've come to realise that the testing gets very unwieldy very quickly
16:28:19 <mattt> odyssey4me: sure
16:28:28 <automagically> It sure does
16:28:41 <mattt> so as i'm sure everyone knows, we've been urgently getting a convergence test into each IRR
16:29:13 <mattt> which focuses for the most part on getting infra (rabbit/galera) stood up, keystone (required for services to register), and the role in question's service
16:29:34 <mattt> this has worked OK, but involves a ton of overrides needing to be set in the test playbooks, which is repeated from role to role
16:29:54 <mattt> so we need to find a way to abstract some of this stuff out of each role's test playbook so we can reuse the code from role to role
16:30:17 <mattt> right now, for example, each role's test playbook has to re-setup keystone and override a ton of things for that to work
16:30:33 <mattt> i've been trying to shim in openstack-extra's group_vars files but haven't gotten a build to go through yet
16:31:00 <mattt> s/openstack-extra/openstack-ansible/
16:31:28 <mattt> i've also been making some additional modifications to use a constraints file when installing packages in the test scenario
16:31:57 <mattt> otherwise, as we witnessed yesterday, some services will use untested dependencies which may or may not work (pysaml2 had a new release yesterday which broke the installation of keystone)
16:32:26 <automagically> While I like the technical goal of reuse, I think having the roles tests depend directly on assets in the OSA repo is likely to run counter to our goal of making them reusable outside the OSA playbook/inventory/group_vars structure
16:32:27 <mattt> anyway, things are currently in flux, there's no real pattern at the moment, but i think it's worthwhile getting this work done across all roles and we can then refactor to remove duplication and simplify things
16:32:45 <mattt> automagically: that is a good point
16:32:46 <cloudnull> automagically: +1
16:32:58 <automagically> Meaning, I’m not against finding refactoring patterns for the tests that work, I just want to be careful that they don’t adversely effect coupling
16:33:11 <automagically> In fact, ideally, the testing has shown us some paths to reduce coupling I believe
16:33:22 <michaelgugino> not a huge fan, but git submodules might work for this
16:33:25 <mattt> automagically: things are getting tightly coupled as it is, because we have to configure the tests to use the same SHAs in openstack-ansible
16:33:35 <odyssey4me> automagically well, not really - this is only for testing
16:33:37 <mattt> we may get to a point where we can just use master, but right now stuff doesn't work
16:33:56 <odyssey4me> the goal is to allow the role to be used in isolation, but the testing we do will be tests that combine our own roles
16:34:04 <automagically> Well, the SHAs coupling is to openstack in general more so than OSA
16:34:16 <jmccrory> mattt have a log or example of what's breaking when using master?
16:34:26 <odyssey4me> automagically yeah, but we need to manage the side effects of things breaking upstream
16:34:36 <automagically> no disagreement there ^
16:34:41 <odyssey4me> using the openstack-ansible sha's which update roughly every two eeks makes that simpler
16:34:49 <mattt> jmccrory: there was a change in keystone master which caused the installation of keystone to fail, it had something to do with creating a project now requires a domain, which we don't pass by default
16:35:02 <odyssey4me> so we focus on our work and fix brokenness from upstream changes in a predictable manner
16:35:09 <automagically> mattt: Hmm, more details on that would be good. I can prep a fix
16:35:14 <mattt> jmccrory: i tried a quick workaround by having the playbook pass a domain, but that didn't work ... anyway i didn't want to get too distracted chasing these master bugs and figured i'd use known SHAs for the time being
16:35:14 <automagically> Sounds like an easy item to work through
16:35:34 <automagically> or not so easy by the sound of your experience
16:35:34 <mattt> automagically: i can hunt down the review, will get it for you after this
16:35:38 <automagically> Thx
16:36:08 <odyssey4me> I think that having tox pull down the current branch from the openstack-ansible repo and re-using anything we can from there makes sense
16:36:18 <mattt> does it make sense for openstack-ansible to use roles which are tested against a different version of openstack services than openstack-ansible will be using?
16:36:28 <mattt> i just suspect we will constantly run into issues if we flip all tests to using master
16:36:31 <odyssey4me> it also provides a way where we can keep common testing vars there
16:37:26 <odyssey4me> mattt I agree absolutely. Anyone using the role for other purposes is welcome to set things up differently, but our tests should use the same SHA's as the openstack-ansible repo
16:37:39 <odyssey4me> it allows us to update them in one place, and maintain them in one place
16:37:43 <automagically> Using master is a benefit for testing, only once we have the tests running against the OSA SHAs
16:37:58 <automagically> i.e. useful for the long term, but not a current priority in my view
16:38:04 <odyssey4me> and also prevents us having our work disrupted by shenanigans in other project development cycles.
16:38:39 <mattt> yep because we will reach a point where the role needs modification to work with a project but then openstack-ansible will fail when it tries using that role since it's using an older version of the project
16:39:05 <automagically> mattt: That’s where role versioning needs to come in
16:39:27 <odyssey4me> automagically yep - if we get more people using the roles in that way, then fixes will naturally come in... but right now our current contributor base is quite small so we need to manage the side-effects of upstream development changes quite carefully
16:39:32 <automagically> FWIW, I’d love to have Liberty versions of the IRRs
16:39:48 <palendae> automagically: I think that's way too late at this point
16:39:56 <odyssey4me> automagically I would not be surprised if you could build liberty using the Mitaka roles
16:39:56 <palendae> Even if it would be nice
16:39:58 <mattt> so my suggestion is to plow on and get the tests all standardised across the roles
16:40:09 <mattt> w/ the necessary developer_mode changes that pull in upper constraints
16:40:11 <palendae> Also odyssey4me's probably right
16:40:21 <automagically> Can we attempt to standardise on the breakout of test-prep, container-create and test.yml?
16:40:28 <mattt> and once we're in that place we can sit down and figure out a better way to decouple from OSA (if that is the agreement) or how we can improve these
16:40:30 <automagically> As well as the developer_mode
16:40:36 <mattt> then i think we'll be in a good place to add more functional testing to each role
16:40:48 <mattt> automagically: i think that is a good idea
16:40:49 <odyssey4me> automagically yeah, I'd like to see the tests broken into more playbooks that are nicely named and have some sort of standard
16:41:00 <automagically> We seem to have an evolving standard
16:41:29 <mattt> cloudnull: appreciate your input too since you did a bunch of the testing on these IRRs
16:41:32 <odyssey4me> let's keep notes on this in https://etherpad.openstack.org/p/testing-os-roles please
16:41:56 <odyssey4me> let's use it to communicate challenges, the evolving standard, etc
16:42:09 <automagically> Yep, I added a link to Jimmy’s work on the test file decomposition there earlier
16:42:10 <mattt> odyssey4me: k
16:42:11 <automagically> https://review.openstack.org/#/c/288617/
16:42:39 <odyssey4me> so mattt you're working on figuring out how to pull in the SHA's from the openstack-ansible repo, right?
16:43:12 <mattt> odyssey4me: i am, but i am also out tomorrow
16:43:17 <mattt> so will need to pick up on that on monday
16:43:53 <odyssey4me> mattt :( if you can put in a review of where you're at, then hopefully someone else can pick up from where you're at
16:44:43 <odyssey4me> is anyone available to continue hacking on that during our evening?
16:44:58 <mattt> odyssey4me: well let's focus on getting things now in a sane place
16:45:08 <mattt> then we can focus on bringing in openstack-ansible's group_vars
16:45:15 <mattt> some may not even agree that's the right approach
16:45:16 <odyssey4me> fair enough
16:45:19 <mattt> so let's finish 1 thing then move onto that
16:46:01 <odyssey4me> so while mattt's working on that, can everyone else muck in to standardising having a functional convergence test that works the same way in all the roles?
16:46:49 <automagically> I could use some help figuring out what’s going on with https://review.openstack.org/#/c/290034/
16:47:02 <mattt> automagically: i can help you w/ that
16:47:08 <automagically> And could use more eyes on https://review.openstack.org/289454
16:47:44 <automagically> Once those convergence tests are operable and merged, I’m happy to add functional tests/assertions to those and other roles
16:47:52 <mattt> actually  https://review.openstack.org/289454 is the one i can help with, i was looking at that one already :)
16:48:20 <automagically> So heat is passing at least
16:48:28 <automagically> Horizon is the one I can’t see to get to pass convergence
16:49:21 <stevelle> Horizon will need a different inventory model b/c of keystone's apache
16:49:45 <stevelle> expecting it to have other challenges too
16:50:16 <automagically> stevelle: Thx, any comments on that one please drop em on https://review.openstack.org/#/c/290034/
16:50:21 <odyssey4me> ok, we have 10 mins left
16:50:23 <automagically> I should have time throughout the day to work on it
16:50:26 <odyssey4me> #topic Open Discussion
16:51:07 <automagically> Quick docs issue. Are folks generally happy with this pattern for IRR docs: http://docs-draft.openstack.org/55/290955/2/check/gate-openstack-ansible-os_keystone-docs/16c7d8d//doc/build/html/
16:51:24 <automagically> Where-in the defaults/main.yml ends up being directly sourced into the doc
16:51:37 <palendae> automagically: I think that's a good idea
16:51:58 <odyssey4me> automagically yeah, fantastic - at least while the docs are basic
16:52:00 <automagically> Should let us focus on making sure vars are explained in-situ
16:52:11 <odyssey4me> if we build out the docs then we can shift... but for now that makes much more sense
16:52:26 <jmccrory> looks good
16:52:53 <odyssey4me> love it
16:53:08 <automagically> Is there additional gate optimisation to be done, that we should be discussing?
16:53:22 <automagically> I know that was a focus of a fair bunch of work over the last week or two
16:53:37 <automagically> Seems pretty well ironed out at this point
16:54:27 <odyssey4me> it's getting better: https://review.openstack.org/#/q/project:%255Eopenstack/openstack-ansible.*+topic:gate-optimise
16:54:49 <palendae> I heard mention of some cirros random number generation behavior that might be affecting gate times, too
16:54:59 <odyssey4me> once the master patch for the new nova_api DB is merged, then https://review.openstack.org/290486 can be rechecked
16:55:03 <logan-> i figured i might run some new profiles in a week or two so we can see comparisons with new nodepools like osic/vexxhost and see how the other optimizations are affecting things
16:55:41 <stevelle> we should test against the standard affinity settings then
16:56:16 <odyssey4me> stevelle yeah, we should return it back to the right settings
16:56:37 <odyssey4me> not right now though - I need to get the apt mirror usage in, then the python mirrors too
16:56:50 <odyssey4me> and lastly I think perhaps also some improvements to the lxc cache prep
16:57:01 <odyssey4me> but that may have to be left until next cycle
16:57:11 <odyssey4me> it's a little late in the game to change how that works for this cycle
16:57:11 <raddaoui> I have one
16:57:20 <odyssey4me> raddaoui oh
16:57:22 <odyssey4me> ?
16:57:23 <raddaoui> I want to discuss this patch: https://review.openstack.org/#/c/289622/
16:58:00 <raddaoui> this patch allows to check for kernel modules if they are statically builtin in the kernel,
16:58:00 <raddaoui> loadable as modules or not compiled at all only against specific group hosts
16:58:00 <raddaoui> (networking, compute..).
16:58:42 <odyssey4me> let's chat about that in #openstack-ansible as we're out of time here
16:58:47 <odyssey4me> thanks all for attending
16:58:58 <raddaoui> okay
16:59:01 <odyssey4me> #endmeeting