15:00:20 <crinkle> #startmeeting puppet-openstack
15:00:20 <openstack> Meeting started Tue Oct  6 15:00:20 2015 UTC and is due to finish in 60 minutes.  The chair is crinkle. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:21 <degorenko> o/
15:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:22 <sbadia> EmilienM: haha :)
15:00:24 <openstack> The meeting name has been set to 'puppet_openstack'
15:00:24 <sbadia> hello
15:00:29 <xarses> o/
15:00:30 <iberezovskiy> hi
15:00:30 <iurygregory> hello o/
15:00:37 <skolekonov> hi
15:00:43 <EmilienM> hello :)
15:00:45 <mwhahaha> hi
15:00:51 <angdraug> o/
15:01:03 <jasondotstar> o/
15:01:07 <crinkle> #topic Review past action items
15:01:09 <jasondotstar> decided to join.
15:01:15 <crinkle> puppet team to review https://review.openstack.org/#/c/226830/
15:01:30 <EmilienM> crinkle: thx for your reviews ^ we made good progress
15:01:30 <dilyin> hi
15:01:31 <crinkle> this went through a few more iterations since last week, I think it's close to done
15:01:38 <crinkle> EmilienM: yep thanks for bearing with me :)
15:01:48 <sbadia> wow a lot of reviews
15:01:49 <EmilienM> ahah, that's cool :)
15:01:56 <crinkle> all other action items look completed
15:02:12 <crinkle> #topic CI status
15:02:21 <crinkle> EmilienM: want to give a quick update since you're here?
15:02:25 <EmilienM> yeah so:
15:02:43 <EmilienM> just fyi, we have now integration jobs in a better shape, since we have less services running on both jobs
15:02:59 <EmilienM> the split was useful, and we should now carefuly interpret the job color
15:03:10 <iurygregory> nice =)
15:03:29 <EmilienM> crinkle: I'm pushing more documentation at the moment, so people can easily get onboarded
15:03:36 <crinkle> cool
15:03:45 <clayton> o/
15:03:51 <EmilienM> crinkle: I'm done for functional CI
15:03:53 <sbadia> nice, thx EmilienM
15:04:17 <crinkle> #topic Kilo release (should happen next week): please backport bugs or small features
15:04:40 <crinkle> contributors please make sure your patches are backported, reviewers please be reviewing patches for the stable/kilo branch
15:05:08 <crinkle> anything else we need to discuss on this?
15:05:13 <EmilienM> also, call for volunteers
15:05:24 <EmilienM> to help in release process (launchpad, release notes, etc)
15:05:34 <sbadia> I can help on this
15:05:39 <EmilienM> please ping me and I'll schedule it with interested folks
15:05:44 <jasondotstar> I'd like to help as well.
15:06:17 <crinkle> #info ping Emilien to get involved with the kilo release
15:06:23 <EmilienM> sbadia, jasondotstar: thanks, I'll come back to you
15:06:28 <crinkle> #action sbadia and jasondotstar signed up to help with kilo release
15:06:33 <sbadia> ack
15:06:37 <jasondotstar> ack
15:07:13 <crinkle> #topic ability to override variables of 'params' class (e.g. service/package names)
15:07:15 <crinkle> #link https://review.openstack.org/#/c/229918/
15:07:25 <crinkle> iberezovskiy: ?
15:07:31 <iberezovskiy> crinkle, thanks
15:07:32 <iberezovskiy> I want to discuss a problem with overriding default parameters which are hardcoded in 'params' classes.
15:07:36 <iberezovskiy> Sometimes it's useful to override these params if their calculation (which depends on OS) is not appropriate
15:07:42 <iberezovskiy> for users. They may want use antoher options for deployments (e.g. install packages from their custom mirrors
15:07:49 <iberezovskiy> or install debian package on Ubuntu and so on). Current implementation of 'params' class
15:07:54 <iberezovskiy> doesn't allow this functionality. I've prepared a patch with one of possible solution for this case
15:08:02 <iberezovskiy> I've added new 'globals' class with defaults from 'params', but with one important difference - we can change each varibale from that class.
15:08:31 <iberezovskiy> btw, Email thread was started by Thomas Goirand (zigo)
15:08:36 <iberezovskiy> #link https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg64891.html
15:08:44 <mwhahaha> So is there a reason we can't use the other method of overriding these values with the <| |> syntax like we already do?
15:08:53 <mwhahaha> is there something that is missing that requires the global.pp method?
15:09:27 <clayton> iberezovskiy: how does overriding via globals actually work in practice?
15:09:27 <mwhahaha> https://github.com/stackforge/fuel-library/blob/deb63f09df23170207310f06ca4e12785d018dc2/deployment/puppet/openstack/manifests/nova/controller.pp#L399
15:10:13 <xarses> it seems over kill instead of adding the debian defaults
15:10:18 <iberezovskiy> clayton, e.g. it work with mongodb module https://github.com/puppetlabs/puppetlabs-mongodb
15:10:30 <crinkle> instead of using a globals class we could just have class parameters default to using params.pp, that way they can be overridden with whatever the user wants
15:10:57 <crinkle> lots of classes have things like package_name => that can be overridden in that class for instance
15:10:59 <iberezovskiy> crinkle, my globals class actually use 'params' by default
15:11:04 <mwhahaha> xarses: the issue isn't debian defaults, it's trying to run debian packages on ubuntu
15:11:05 <clayton> iberezovskiy: couldn't you do the same thing by instantiating the params class?
15:11:18 <xarses> mwhahaha: k
15:11:37 <clayton> I'm in favor of the functionality, I'm just trying to understand this approach to the problem
15:11:48 <mwhahaha> iberezovskiy: i think the question is instead of yet another class called globals, it's turning params into a parameterized class like globals
15:12:06 <xarses> mwhahaha: ++
15:12:11 <crinkle> you can't have params be a parameterized class, because sometimes you have to inherit that class
15:12:19 <clayton> ah, nod, I see
15:12:32 <aglarendil> can we think of using hiera for this?
15:13:15 <clayton> aglarendil: I think you could just include the globals class and then set the parameters for it via hiera
15:13:16 <iberezovskiy> mwhahaha, in case of <| |> you need to override every resource where parameter that you need to override can be used
15:13:26 <mwhahaha> iberezovskiy: how many is that?
15:13:29 <dilyin> service default provider issues -> I guess we need to add confine to the pacemaker provider
15:13:48 <mwhahaha> iberezovskiy: just trying to understand the scope of the required changes, if it's one package that's not the end of the world
15:13:53 <mwhahaha> if it's 100s then globals might be better
15:14:13 <iberezovskiy> mwhahaha, a lot of thing are tied of service name for example
15:14:17 <crinkle> I think adding parameters to existing classes is a lot easier than adding a globals class
15:14:34 <clayton> crinkle: well, and we already have those parameters in many cases.
15:14:38 <mwhahaha> but with <||> we can just include a mos specific class that does <| |> for all of the mos packages/service names
15:14:41 <Hunner> The parse order for globals is usually globals -> params -> init/server/client/etc to allow "global" parameter changes (hense the name)
15:15:01 <clayton> mwhahaha: one downside to <||> in this case is that the collector silently fails if the package name ever changes.
15:15:07 <Hunner> <||> is good for overriding resource parameters, but not logic that uses variables
15:15:16 <crinkle> clayton: so what's the issue then? if users can already specify the debian package name if they want then aren't we done?
15:15:18 * EmilienM 
15:15:19 <Hunner> And yeah, silent failures as clayton says
15:15:33 <clayton> crinkle: I think I was agreeing with you :)
15:15:39 <crinkle> clayton: aha :)
15:15:56 <EmilienM> damn
15:15:58 <EmilienM> I think crinkle is right, I like the idea
15:17:17 <iberezovskiy> crinkle, so if we need to override all 'params', we need to split them by classes, isn't?
15:17:17 <Hunner> Is crinkles idea basically "add more base class parameters to override params values?"
15:17:34 <clayton> I believe so.  we already have it in place for a number of them
15:17:49 <angdraug> so back to mwhahaha's question, how much work would that be?
15:17:53 <iberezovskiy> crinkle, what about interclass dependencies?
15:18:38 <clayton> I think the thought at one point was to move away from interclass dependencies where services are referenced by name
15:18:41 <iberezovskiy> two classes requires one param, for one class it was overridden, for another wasn't
15:18:52 <mattymo1> like mysql service name?
15:19:34 <Hunner> interclass dependencies should be on the class, not resources inside the class. It makes for a cleaner graph and easier groking
15:19:50 <aglarendil> well, we could work with hiera hierarchy and put default parameters into default class and then fetch all hiera parameters at once
15:20:04 <aglarendil> like %{operatingsystem}.yaml
15:20:10 <crinkle> iberezovskiy: if two classes need the same value set then it should be up to the user to set it in both places
15:20:10 <aglarendil> and add one additional override.yaml
15:20:15 <clayton> Hunner: I think we need to move to handling them via named anchors personally, like this - https://review.openstack.org/#/c/216926/
15:20:19 <crinkle> the same as when we have to set a password in two different classes
15:21:18 <Hunner> clayton: That looks nice, and of course is needed for class containment
15:21:58 <clayton> it makes the dependencies a lot easier to deal with and requires less explicit ones
15:23:28 <angdraug> folks, time...
15:24:09 <crinkle> does anyone else have thoughts about adding a globals class?
15:24:16 <crinkle> or perhaps we should continue this on the mailing list?
15:24:36 <EmilienM> +1 for mailing list
15:24:40 <iurygregory> ML is fine i think , get feedback ;)
15:24:42 <iberezovskiy> let's continue in ML
15:24:45 <aglarendil> +1 for ML
15:24:46 <Hunner> If multiple independent classes need their params changed, globals is +1. If all classes can just inherit the base class, we could use that and not globals
15:24:59 <EmilienM> since it does not sound we have a decision
15:25:15 <crinkle> #action continue globals topic on mailing list
15:25:33 <crinkle> #topic Use OpenstackClient for nova providers auth
15:25:37 <crinkle> #link https://review.openstack.org/226862
15:25:42 <crinkle> #link https://review.openstack.org/229548
15:25:45 <degorenko> crinkle,  hey o/
15:25:46 <crinkle> degorenko: ?
15:25:48 <crinkle> hiya
15:25:52 <degorenko> so, my patch with a new auth based on openstack client for puppet nova is ready now, i need review
15:25:58 <degorenko> but there is one issue - nova_network provider will use old way auth based on nova client
15:26:04 <degorenko> because openstack client doesn't provide such possibility - to manage nova networks
15:26:10 <degorenko> so my question is: is it ok to mix two ways?
15:26:16 <degorenko> nova-network is deprecated so, it can be removed later, i think
15:26:21 <crinkle> i'm fine with it
15:26:29 <clayton> +1
15:26:49 <degorenko> then, guys, please review this: https://review.openstack.org/226862 and this one: https://review.openstack.org/229548
15:27:10 <crinkle> #action reviewers review 226862 and 229548
15:27:48 <crinkle> #topic blueprints: use-openstackclient-in-module-resources vs use-aviator-in-module-resources
15:28:03 <aglarendil> so I actually raised this topic
15:28:03 <crinkle> aglarendil: these were approved a while ago, did you have questions on them?
15:28:13 <EmilienM> w00t?
15:28:36 <aglarendil> well, actually, they contradict each other
15:28:50 <crinkle> yes, if you read the openstackclient one it overrides the aviator one
15:29:13 <aglarendil> then the other one should be obsoleted
15:29:17 <mattymo1> except the openstackclient isn't so amazing(
15:29:19 <aglarendil> but I think it should be vice versa
15:29:29 <EmilienM> #link http://specs.openstack.org/openstack/puppet-openstack-specs/specs/juno/use-openstackclient-in-aviator-resources.html
15:29:49 <aglarendil> we discussed ruby client for openstack providers in Atlanta
15:29:58 <aglarendil> and I though it was a really nice idea
15:30:06 <EmilienM> we have spent so much time about that, I don't think we will revert our current design.
15:30:26 <aglarendil> well, I think that ruby client is better
15:30:39 <aglarendil> but I failed to find an ML discussion on that
15:30:42 <EmilienM> aglarendil: crinkle did an awesome presentation in Vancouver about that
15:30:52 <crinkle> aglarendil: let me find the discussion
15:30:56 <EmilienM> aglarendil: a lot of things happened since Atlanta :-)
15:31:13 * EmilienM looking for crinkle's slides
15:31:29 <aglarendil> crinkle: EmilienM: okay, I will look into it and write an email to ML if I have any objections
15:31:36 <crinkle> #link https://groups.google.com/a/puppetlabs.com/forum/#!searchin/puppet-openstack/aviator$20openstackclient/puppet-openstack/GJwDHNAFVYw/ayN4cdg3EW0J
15:31:53 <EmilienM> #link http://www.slideshare.net/Colleen_Murphy/vancouver-presentation-48498077
15:32:06 <aglarendil> I personally think that ruby client will be much more effective, not to say that there is no ruby client for OpenStack which is kind of embarassing
15:32:09 <EmilienM> that's a tl;dr about why we did that
15:32:10 <aglarendil> at least for me :-)
15:32:16 <EmilienM> aglarendil: go ahead and write it !
15:32:26 <aglarendil> EmilienM: thanks, I will read it and write my comments
15:32:27 <crinkle> aglarendil: I think the deciding factor at the time was that we wanted keystone v3 support, which aviator didn't have and we didn't think would be added soon
15:32:56 <aglarendil> crinkle: the module has been abandoned for a while, but we could revive it actually
15:32:57 <EmilienM> and using openstackclient actually help OpenStackClient team to improve it and get short feedback loop
15:33:07 <crinkle> aglarendil: not the module, the library
15:33:10 <aglarendil> we have volunteers here - e.g. dilyin
15:33:32 <clayton> EmilienM: yeah, but distros don't update openstackclient very often
15:33:39 <dilyin> yes, I can take maintainership on the ruby-openstack library
15:33:52 <aglarendil> dilyin: or Aviator library which seems better
15:33:53 <EmilienM> clayton: all our code is tested against UCA & RDO
15:34:22 <clayton> EmilienM: I know, I'm just saying that getting changes upstream in openstack client has a long feedback loop before we can use the results.
15:34:22 <EmilienM> clayton: so we adjust the code to work with these distros
15:34:35 <EmilienM> clayton: sometimes we don't use the client because a feature is not supported.
15:34:49 <EmilienM> clayton: it's not that bad, we made lot of improvements
15:35:13 <EmilienM> clayton: do you have any blocker at this time?
15:35:23 <clayton> nope
15:35:50 <crinkle> aglarendil: will you bring this topic up on the mailing list?
15:35:56 <aglarendil> crinkle: yep, sure
15:35:59 <EmilienM> aglarendil: sorry to be that guy, but you're really late in the party to come and say "let's re-write everything in ruby"
15:36:19 <crinkle> #action aglarendil to bring up openstackclient vs aviator on mailing list
15:36:28 <EmilienM> we already tried all (or almost) solutions and we decided to use the osclient for some reasons.
15:36:33 <aglarendil> EmilienM: we can always discuss it and change it if we find more benfits for it than drawbacks
15:37:14 <crinkle> #topic service default provider issues
15:37:17 <crinkle> #link https://review.openstack.org/#/c/229095/
15:37:20 <crinkle> mwhahaha: ?
15:37:23 <mwhahaha> The issue I'm running into is that the "<SERVICE DEFAULT>" value is being put into configs and used for some reason as part of our acceptance/unit tests. I thought this was supposed to be handled correctly but I don't think it is.
15:37:23 <mwhahaha> It seems we only remove <SERVICE DEFAULT> in the exists? function within the openstack_config provider but we aren't checking if this is the value to prevent it from put into the configs.
15:38:15 <mwhahaha> see failures on that review
15:38:48 <clayton> I've got another review for service defaults that's also having acceptance failures that I can't seem to figure out, not sure if it's the same - https://review.openstack.org/#/c/230536/
15:39:43 <EmilienM> I wonder is spredzy is around
15:40:09 <sbadia> EmilienM: no he is in the train
15:40:48 <clayton> mwhahaha: My patch actually passed CI until I switched from a hard coded <SERVICE DEFAULT> to $::os_service_default.  no idea why that would matter.
15:40:59 <mwhahaha> me neither
15:41:20 <mwhahaha> does puppet interpret string vs a global variable differently?
15:41:38 <EmilienM> I've seen that failure too, in puppet-neutron
15:43:01 <mwhahaha> anyway just thought i'd mention it if anyone has any ideas
15:43:11 <mwhahaha> guess we can move on
15:43:12 <EmilienM> yeah, not sure the meeting is good place for debug
15:43:21 <EmilienM> but we need to pay attention about that
15:43:29 <crinkle> who was responsible for adding the fact? can we assign them responsibility for looking into this?
15:43:35 <mwhahaha> i added it
15:43:46 <crinkle> aha :)
15:43:55 <mwhahaha> but i don't know how the provider is properly handling the '<SERVICE DEFAULT>' stuff
15:43:55 <EmilienM> mwhahaha: you have to fix the world now !
15:43:58 <crinkle> i don't have any ideas
15:44:10 <mwhahaha> bunch of magic :/
15:44:12 <EmilienM> me neither but I'll take time to look later
15:44:13 <iurygregory> big problem =X
15:44:14 <clayton> crinkle: I don't think it's the fact, I've tested my patch locally and the fact seems to work fine
15:44:19 <clayton> I think it's an interaction issue
15:44:23 <crinkle> clayton: hmmm
15:45:03 <crinkle> mwhahaha: thanks for bringing it up, we'll help look into it
15:45:23 <crinkle> #topic openstack client timeouts and handling
15:45:26 <crinkle> mattymo, aglarendil, dilyin ?
15:45:35 <EmilienM> mwhahaha: we keep sync on ML about that ok?
15:45:38 <mwhahaha> k
15:47:03 <aglarendil> I think we should discuss all the excetoption handling stuff there. not only timeouts
15:47:28 <EmilienM> aglarendil: could you extend what you want to discuss exactly?
15:47:46 <aglarendil> EmilienM: there are many operational intermittent issues which lead to providers failures
15:47:46 <EmilienM> "exception handling stuff" is a bit general
15:48:01 <EmilienM> aglarendil: exemples?
15:48:02 <aglarendil> e.g. 503/401 errors which we could retry after a while
15:48:14 <aglarendil> other example is 404 on deletion of the resource
15:48:17 <aglarendil> and so on
15:48:24 <EmilienM> afik our providers already have timeouts handling
15:48:35 <EmilienM> glance_image for sure
15:48:47 <crinkle> it's built into the base provider
15:49:02 <EmilienM> in openstacklib, right
15:49:08 <aglarendil> EmilienM: I think I overlooked it somehow
15:49:46 <aglarendil> okay, which ML thread are we talking about? could you post a link?
15:49:53 <EmilienM> look https://github.com/openstack/puppet-openstacklib/blob/master/lib/puppet/provider/openstack.rb
15:50:21 <mattymo1> sorry I stepped away for a second
15:50:27 <EmilienM> aglarendil: I'm not sure we're talking about a ML thread now
15:50:49 <EmilienM> topics are appreciated when they are prepared, we have a short timing guys
15:50:55 <mattymo1> 1 is retry on certain codes.. the second is how we should treat the openstack client process if it hangs beyond timeout
15:50:57 <aglarendil> EmilienM: so you wrote a message to mwhahaha about the previous topic
15:51:12 <mattymo1> if the API doesn't ever respond, that process runs infinitely and eats all CPU and takes down the host
15:51:24 <EmilienM> aglarendil: I asked him to use ML to keep synchronised about our investigations
15:51:27 <mattymo1> should puppet be responsible for this or openstackclient code itself?
15:51:45 <EmilienM> yes
15:51:50 <aglarendil> mattymo1: I think we should not trust openstackclient code as we are operators
15:51:58 <EmilienM> openstackclient is not supposed to retry by itself
15:52:10 <aglarendil> and should wrap it with exceptions on our own
15:52:23 <EmilienM> aglarendil: trust? osclient is part of openstack, you need to trust it.
15:52:37 <crinkle> I think right now we only handle 401 or getting the message "Unable to establish connection", I think the providers would benefit from better error handling ane retries in http://git.openstack.org/cgit/openstack/puppet-openstacklib/tree/lib/puppet/provider/openstack.rb#n55
15:52:58 <aglarendil> EmilienM: should I cite House M. D. that everyone lies ?-)
15:53:36 <EmilienM> aglarendil: I would apprecaite more productive topics and discussion
15:53:55 <aglarendil> EmilienM: okay - I will rephrase it
15:54:09 <aglarendil> we do not control openstackclient completely and we should be able to
15:54:20 <EmilienM> we have 7 min left, not sure there is any decision we can take. If you have concerns about openstackclient usage, start an open discussion on the ML
15:54:23 <aglarendil> react properly on its misbehaviour or other intermittent issues
15:54:31 <aglarendil> okay, will do
15:54:49 <EmilienM> aglarendil: like crinkle said, you can look at http://git.openstack.org/cgit/openstack/puppet-openstacklib/tree/lib/puppet/provider/openstack.rb#n55 and see what is missing
15:54:52 <crinkle> aglarendil: I do see advantage in adding better error handling and retries, please feel free to make patches there
15:55:15 <clayton> aglarendil: we've put puppet-healthcheck between a lot of the web services and providers, to avoid issues with services not being up
15:55:29 <EmilienM> the feedback from operators would be useful to improve this provider
15:55:50 <aglarendil> EmilienM: I put it onto my drafts list - will follow up in ML
15:55:58 <crinkle> excellent
15:56:01 <EmilienM> aglarendil: cool thx
15:56:10 <crinkle> #topic Open Discussion, Bug and Review triage
15:56:30 <EmilienM> crinkle: I'm boarding now - thx again for leading the meeting
15:56:37 <crinkle> EmilienM: have a good flight!
15:56:43 <EmilienM> thx, see you in pdx
15:56:51 <crinkle> there are a few patches listed on https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20151006 to look at
15:57:01 <crinkle> any other patches or bugs folks want to bring up in the next couple minutes?
15:57:19 <degorenko> yep: https://review.openstack.org/219275 :)
15:57:40 <iurygregory> anyone good with acceptance can take a look at - https://review.openstack.org/#/c/208054/
15:57:41 <iurygregory> =)
15:58:17 * crinkle will look at those after the meeting
15:58:51 <aderyugin> one more: https://review.openstack.org/#/c/218234/
15:59:51 <crinkle> alright, thanks everyone
15:59:54 <crinkle> #endmeeting