15:00:35 <EmilienM> #startmeeting puppet-openstack
15:00:35 <openstack> Meeting started Tue Sep  8 15:00:35 2015 UTC and is due to finish in 60 minutes.  The chair is EmilienM. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:37 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:40 <openstack> The meeting name has been set to 'puppet_openstack'
15:00:41 <EmilienM> #link agenda: https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20150908
15:00:45 <EmilienM> o/
15:00:53 <iurygregory> こんにちは - hello o/
15:01:02 <angdraug> o/
15:01:03 <mwhahaha> o/
15:01:04 <richm> hello
15:01:30 <crinkle> o/
15:01:57 <EmilienM> ok, let's get start
15:02:01 <spredzy> o/
15:02:08 <EmilienM> #topic announcements
15:02:19 <xarses> hi
15:02:21 <EmilienM> #new approved spec: http://specs.openstack.org/openstack/puppet-openstack-specs/specs/liberty/support-for-keystone-domain-configuration.html
15:02:29 <EmilienM> #link new approved spec: http://specs.openstack.org/openstack/puppet-openstack-specs/specs/liberty/support-for-keystone-domain-configuration.html
15:02:38 <EmilienM> chem: good work ^
15:03:07 <EmilienM> #link Tokyo Summit etherpad: https://etherpad.openstack.org/p/HND-puppet
15:03:28 <EmilienM> see my thread on ML, I would like people to start thinking about topics that we will discuss
15:03:29 <chem> EmilienM: thanks :)
15:03:34 <EmilienM> so we can start working on agenda
15:03:58 <EmilienM> also, thank you for those who attended our sprint last week
15:04:10 <EmilienM> #link puppet sprint retrospective: https://etherpad.openstack.org/p/puppet-liberty-sprint-retrospective
15:04:28 <EmilienM> I'll let more time (some days) so people can feed the etherpad
15:04:44 <EmilienM> and during the next meeting we can do the retrospective
15:05:07 <EmilienM> #action emilien to wait more time and gathers data from sprint retrospective for next meeting
15:05:20 <_ody> o/
15:05:51 <EmilienM> #topic CI status
15:05:57 <EmilienM> just a quick status
15:06:10 <EmilienM> here is the updated situation of our jobs: https://etherpad.openstack.org/p/puppet-liberty-blocker
15:06:24 <EmilienM> still some issues with packaging, but much better on trusty
15:06:34 <EmilienM> so they'll vote again, for those which were disabled to vote
15:07:00 <EmilienM> I've heard this morning RDO has now a new repo beside others... So we are waiting a bit they stabilize that
15:07:10 <EmilienM> in the meantime, horizon & ceilometer modules have failing beaker jobs
15:07:38 <EmilienM> pabelanger is still working on multi node jobs
15:08:01 <EmilienM> and I'm still trying to make pass the puppet-openstack-integration jobs on trusty this week
15:08:22 <EmilienM> oh, I missed an announcement but anyway
15:08:31 <EmilienM> anteaya is working on having a developer doc
15:08:42 <EmilienM> #link puppet doc https://review.openstack.org/#/c/220555/
15:08:56 <EmilienM> so we will have docs.openstack.org/developer/puppet
15:09:09 <spredzy> EmilienM, how would that work with current wiki?
15:09:09 <EmilienM> and have content to explain how to contribute, doing reviews, testing, etc
15:09:19 <EmilienM> spredzy: the goal is to drop some wiki content
15:09:20 <spredzy> Everything will need to be backported there?
15:09:23 <EmilienM> yes
15:09:27 <EmilienM> not everything I think
15:09:29 <EmilienM> but some bits
15:09:41 <EmilienM> and it will be discussed in the reviews
15:09:52 <EmilienM> like, if you submit a patch, you'll have to explain what you want to drop from the wiki
15:10:00 <EmilienM> so we make sure not dropping everything
15:10:11 <spredzy> Ack. Hopefully we come out with a clear line between what goes in one and what goes in the ohter
15:10:31 <EmilienM> one good example is docs.openstack.org/developer/tempest
15:10:42 * spredzy checks
15:10:52 <EmilienM> let's move on
15:11:12 <EmilienM> #topic "SERVICE DEFAULT" parameter in puppet modules
15:11:16 <EmilienM> spredzy: o/
15:11:28 <EmilienM> #link SERICE_DEFAULT thread: http://lists.openstack.org/pipermail/openstack-dev/2015-September/073799.html
15:11:37 <spredzy> #link https://review.openstack.org/#/c/221005/
15:11:37 <EmilienM> also https://review.openstack.org/#/c/221005/
15:12:06 <spredzy> So now that all the necessary pieces have been merged we can use this feature
15:12:11 <EmilienM> spredzy: I WIPed the patch to make sure we discuss before
15:12:23 <EmilienM> spredzy: can you remind us a little of context?
15:12:29 <spredzy> the review mentioned above is to highlight the benefits of it
15:12:36 <EmilienM> I personally understand what's going on, but I'm not sure everyone does
15:13:05 <spredzy> So what it does is : if the default parameter value is '<SERVICE DEFAULT>' the cinder_config will ensure absent on that parameter
15:13:15 <spredzy> if it has a value will write it in the configuration fiel
15:13:20 <spredzy> file*
15:13:38 <EmilienM> so that means 'undef' default param will be replaced by <SERVICE DEFAULT> ?
15:13:42 <spredzy> This way we avoid the if/else cases for each value, and we are always in sync with upstream
15:13:50 <spredzy> EmilienM, yes
15:14:04 <spredzy> EmilienM, also the value that default to the upstream default will be changed
15:14:11 <spredzy> to <SERVICE DEFAULT>
15:14:35 <spredzy> in the review above the logging.pp shows greatly how we gain in readability
15:14:37 <EmilienM> I +1 the concept
15:14:54 <EmilienM> I'm just wondering if you're going to handle all defaults value
15:15:10 <spredzy> EmilienM, what do you mean?
15:15:47 <EmilienM> spredzy: we have a ton of default values
15:15:59 <EmilienM> a lot of 'undef' as default values
15:16:19 <EmilienM> so you're going to patch all modules using undef or false as default values to use this new string
15:16:26 <EmilienM> so it will makes sure it's absent
15:16:27 <EmilienM> right?
15:16:43 <spredzy> using undef, false or the upstream default
15:16:46 <spredzy> right
15:17:01 <EmilienM> +1
15:17:25 <chem> +1
15:17:33 <spredzy> cinder is the canary module, so other module won't be touched as of now
15:17:59 <spredzy> if the community agrees and after cinder has been prove to work after few days we can start moving everything
15:18:28 <EmilienM> spredzy: I guess silence means we can continue forward :)
15:18:41 <spredzy> EmilienM, yep Im done :)
15:18:45 <EmilienM> good
15:18:51 <EmilienM> so the decision is:
15:19:13 <EmilienM> we continue to work on puppet-cinder, gather feedback a little
15:19:18 <EmilienM> and continue in other modules
15:19:22 <EmilienM> after one or 2 weeks
15:19:35 <EmilienM> spredzy: keep us posted on ML
15:19:52 <spredzy> EmilienM, yes and will do
15:19:56 <EmilienM> spredzy: awesome
15:20:18 <EmilienM> #topic Status of keystone v3 work with domain naming
15:20:22 <EmilienM> richm o/
15:21:10 <richm> so in my attempts to get gildub's domain naming patches working with acceptance testing, I found several bugs that I introduced into the code in my original v3 work :-(
15:21:25 <EmilienM> the good news is, you found the solution :)
15:21:35 <richm> yes, so I would like to get some reviews
15:21:35 <EmilienM> #link https://bugs.launchpad.net/puppet-keystone/+bug/1492843
15:21:37 <openstack> Launchpad bug 1492843 in puppet-keystone "domain name from id lookups return empty" [High,In progress] - Assigned to Richard Megginson (rmeggins)
15:21:51 <richm> and 1492846 and 1492848
15:22:03 <EmilienM> ok so 221119, 221120 and 221121 are high priorities for review?
15:22:07 <richm> yes
15:22:12 <EmilienM> good to know
15:22:33 <richm> they need to be fixed in order that gildub's patches will pass acceptance
15:22:49 <richm> then I will rebase gildub's patches on top of these
15:22:55 <EmilienM> richm: once these 3 patches merged, would it make sense to backport them to stable/kilo?
15:23:31 <richm> EmilienM: I don't know - it will be difficult, and these bugs only affect deployments that are actually using domains, and setting a different default_domain_id
15:23:33 <EmilienM> richm: I guess you can already rebase them on top of the 3rd patch, so we can see the actual result and see the bug is fixed
15:23:39 <richm> EmilienM: right
15:23:43 <chem> richm: I'm trying them right now : it seems that they are not fixing, I'm checking if my test env is ok and that the error is the same
15:23:51 <EmilienM> richm: so backport is valid here
15:24:04 <EmilienM> I guess people using domains in stable/kilo might want to change the default domain id
15:24:18 <richm> EmilienM: ok
15:24:35 <EmilienM> chem, richm: I would like you to rebase gilles's patches on top of richm's fixes
15:24:38 <richm> so I will work on backport as well
15:24:44 <EmilienM> it will help reviewers and confirm it really works
15:24:52 <richm> I've already started working on it
15:25:04 <EmilienM> richm: if there is no code conflict, it should be easy but we never know...
15:25:08 <EmilienM> richm: cool
15:25:13 <richm> there is code conflict . . .
15:25:26 <EmilienM> #action people to review 221119, 221120 and 221121
15:25:41 <richm> conflicts with rebasing gildub's patches, and backporting
15:25:44 <EmilienM> #action richm to rebase gilles's patches on top of 221121
15:25:57 <EmilienM> richm: :(
15:26:01 <richm> then, I also found that indirection doesn't work as I thought it would
15:26:14 <richm> so all of that indirection code I added sucks performance big time
15:26:39 <richm> it turns out that it is much, much better to simply call openstack to get the id for each resource every time
15:26:43 <EmilienM> before cutting stable/liberty, we might need to warn people in the README
15:26:53 <EmilienM> that keystone v3 is still experimental
15:27:06 <EmilienM> except if we fix it before our release
15:27:10 <richm> after gildub's patches, it will be easy to remove the indirection and replace with osc calls
15:27:32 <EmilienM> richm: good, let's call osc each time
15:27:44 <EmilienM> if you have some demo of that?
15:27:52 <richm> some demo of what?
15:27:55 <EmilienM> I haven't tested personally
15:27:57 <EmilienM> of both cases
15:28:08 <EmilienM> like, logs or something visual about perfs
15:28:47 <richm> I don't have any logs - but the key thing is that calling Puppet::Resource.indirection.find(name) calls self.instances _every single time_
15:29:09 <EmilienM> and osc is faster than calls self.instances _every single time ?
15:29:31 <EmilienM> maybe there is something in Puppet we could report
15:29:37 <EmilienM> _ody, Hunner ^
15:30:29 <richm> for example, if you have thousands of users and thousands of tenants/projects, calling Puppet::Resource.indirection.find(user) or (tenant) will retrieve all of them, then create provider objects for all of them, then finally loop through all of them to find the one single matching one
15:30:46 <richm> that can't be good
15:30:57 <EmilienM> and osc client will just find it in the output result
15:31:16 <richm> osc will just retrieve the one single user or tenant
15:31:24 <EmilienM> ok it makes sense
15:31:32 <EmilienM> richm: have you filled a bug?
15:31:34 <richm> so, much less network traffic, internal puppet processing, etc.
15:31:38 <richm> EmilienM: no, not yet
15:31:58 <EmilienM> #action richm to write bug report about performances issues with indirection
15:32:38 <EmilienM> richm: lot of work here :)
15:32:46 <richm> yes
15:32:55 <EmilienM> richm: glad to see CI is useful at least
15:33:01 <richm> yes, very
15:33:13 <EmilienM> we have time for open discussion & bugs today !
15:33:17 <angdraug> yay!
15:33:18 <chem> note that indirection can still be useful when self.instances is not costly.
15:33:22 <EmilienM> richm: anything else?
15:33:30 <richm> EmilienM: no
15:33:33 <EmilienM> chem: right, but it seems it can be
15:33:40 <EmilienM> when having lot of resources
15:34:16 <chem> EmilienM: agreed, but it has to be case by case, not a general rule :)
15:34:26 <EmilienM> chem: got it
15:34:37 <richm> more of a guideline than a rule . . .
15:34:54 <EmilienM> maybe add a note in the documentation would be useful
15:35:19 <EmilienM> "if managing a lot keystone resources (users, tenants, etc) with puppet, be aware of perfs issues
15:35:32 <richm> "Don't use indirection with openstack resources unless you know what you're doing."
15:35:39 <EmilienM> lol
15:35:41 <EmilienM> good
15:35:46 <chem> +1
15:35:49 <iurygregory> +1
15:35:50 <EmilienM> richm: I think we can write it in the code directly
15:36:07 <EmilienM> ok, I'll open discussion in a few
15:36:07 <richm> where?
15:36:15 <EmilienM> where we will make the change
15:36:19 <_ody> EmilienM: I missed the context while getting some food.  I'll read scrollback and get back to you.
15:36:21 <richm> ok
15:36:22 <EmilienM> from our current code to the new one
15:36:30 <richm> ok
15:36:44 <EmilienM> #topic open-discussion
15:36:53 <EmilienM> not sure we need bug triage, regarding to the work done last week
15:37:03 <EmilienM> by the way, special kudos to mfisch & mwhahaha
15:37:26 <angdraug> we do, there's a couple of stuck commits we need to discuss
15:37:44 <angdraug> https://review.openstack.org/192721
15:37:51 <EmilienM> angdraug: oh yeah, I meant about bugs themselves
15:37:57 <EmilienM> sure you can send patches here
15:38:12 <angdraug> the patch ^ from degorenko is stuck for over a week now
15:38:37 <degorenko> o/ hey, also take a look on https://review.openstack.org/219284 and https://review.openstack.org/220090 please
15:38:56 <EmilienM> I think 1/ not much people know Sahara here 2/ a very few people is using the module here -> so it's hard for this patch to get merged
15:39:19 <EmilienM> we need to look what's going on with the patches
15:39:45 <degorenko> EmilienM, there is just a new services
15:40:41 <EmilienM> this is new features so I guess the only one thing we can do is ask people to review it
15:40:52 <angdraug> EmilienM: it's had +1 from Sahara PTL, I'm pretty sure Sahara cores are happy with it
15:41:08 <EmilienM> angdraug: cool, now we have to validate puppet
15:41:13 <angdraug> exactly
15:41:35 <degorenko> ^ right
15:41:36 <EmilienM> great feedback
15:42:02 <EmilienM> do we have anything else outstanding?
15:42:27 <angdraug> https://review.openstack.org/#/c/211986/
15:42:43 <EmilienM> I got some feedback yesterday on IRC about missing basic documentation 101 to build openstack cloud with puppet
15:42:50 <angdraug> nevermind, that last one just got merged )
15:42:53 <EmilienM> I was thinking at using future docs.openstack.org/developer/puppet for that
15:43:12 <spredzy> EmilienM, this is no easy task
15:43:19 <EmilienM> spredzy: it's not hard
15:43:29 <EmilienM> spredzy: I was thinking at documenting our CI manifest
15:43:46 <EmilienM> ie: https://github.com/openstack/puppet-openstack-integration/blob/master/fixtures/scenario001.pp
15:43:58 <EmilienM> which is a manifest you can run on your laptop and it will install openstack
15:44:31 <EmilienM> spredzy: I think beginners are looking for a place to start, and giving an example of manifest can be useful
15:44:39 <spredzy> EmilienM, well if that was just that you can simply throught that link a user will understand
15:44:40 <EmilienM> we already have examples in our modules
15:44:49 <EmilienM> but we might need a general manifest example
15:44:55 <spredzy> EmilienM, I amjust unsure if they expect that everything to be explains or not
15:45:02 <spredzy> ie need for rabbitmq, mysql, etc..
15:45:13 <EmilienM> spredzy: I think it's useful to document that ^
15:45:21 <EmilienM> because not everybody is familiar with it
15:45:34 <spredzy> But having  a all-in-one example is def a good start Id say
15:45:39 <EmilienM> and getting involved often requires to learn basic bits before sending a patch to our modules
15:45:54 <spredzy> take it run it openstack it is at the end
15:46:05 <EmilienM> yeah
15:46:26 <EmilienM> devstack already provides a tool to run openstack on your laptop
15:46:33 <EmilienM> having a manifest that can do that could be nice
15:46:55 <spredzy> +1
15:46:58 <EmilienM> the target here is really people who want to get involved
15:47:05 <EmilienM> but don't know how to start
15:47:10 <spredzy> I think osad (ansible) is also providing something like that
15:47:29 <EmilienM> nice
15:47:50 <EmilienM> ok, do we have anything else for today?
15:48:03 <EmilienM> raise your hand or I'll close the meeting in 1 minute
15:49:42 <EmilienM> thanks for attending
15:49:49 <EmilienM> #endmeeting