15:00:14 <zns> #startmeeting satori
15:00:15 <openstack> Meeting started Mon Mar  3 15:00:14 2014 UTC and is due to finish in 60 minutes.  The chair is zns. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:18 <openstack> The meeting name has been set to 'satori'
15:00:48 <adrian_otto> good morning
15:00:59 <zns> Hi. Who's here?
15:01:02 <caleb_> o/
15:01:04 <zns> Hi adrian_otto
15:01:05 <adrian_otto> o/
15:01:08 <jasonpgignac1> o/
15:01:22 <hhoover> o/
15:01:22 <gondoi> o/
15:01:27 <samstav> o/
15:02:04 <zns> In a minute, we'll start with review of the action items from last time: http://eavesdrop.openstack.org/meetings/satori/2014/satori.2014-02-24-15.00.html
15:02:31 <zns> #topic Review Action Items
15:02:55 <zns> caleb_: would you start with an update of no. 1?
15:03:38 <caleb_> "cgroom include a base set of generic opinions to spec" - I updated the blueprint with text that says we will provide a set of base opinions.
15:03:48 <caleb_> #link https://wiki.openstack.org/wiki/Satori/OpinionsProposal
15:04:05 <caleb_> minor stuff really
15:04:11 <zns> Cool. Next is comments on that and then implementation. Any idea on when implementation can start?
15:04:21 <gusmaskowitz> o/  if it's not too late.
15:04:31 <caleb_> If we're agreed on the proposal I can start coding it today
15:04:35 <zns> Hi gusmaskowitz . And hi everyone else (explicitly) :-)
15:04:56 <zns> caleb_: sounds good. So comments would need to come in ASAP if anyone has any.
15:05:09 <jasonpgignac1> Are we providing info level opinions?
15:05:23 <jasonpgignac1> Or will that be a future change if we need it? I know we've discussed this offline.
15:05:45 <caleb_> jasonpgignac1: No `info` opinions right now. I'm still looking for a decent use case for those.
15:05:52 <jasonpgignac1> Agreed, then
15:06:17 <zns> jasonpgignac1: but we did say we would include a base set of generic ones in satori that can be enabled by the user and the ability to add your own through plugins.
15:06:22 <zns> Item 4 (2 and 3 included): I add the spec https://wiki.openstack.org/wiki/Satori/PluginsProposal, but not implemented yet.
15:06:50 <jasonpgignac1> zns: Yes, what I am referring to is an additional valid status of 'info' to go with 'ok', 'warning', and 'error'
15:06:58 <caleb_> I have an update on that too
15:07:20 <caleb_> We merged in some placeholder implementation
15:07:21 <caleb_> #link https://github.com/rackerlabs/satori/pull/12
15:07:25 <zns> I can also start implementing today. I know this one could take more comments; I have seen different plugin mechanisms in openstack
15:07:48 <caleb_> start by cleaning that up :)
15:08:04 <zns> Will do.
15:08:16 <caleb_> oh
15:08:17 <adrian_otto> it sounds like an opinion is actually a policy
15:08:30 <caleb_> and actually I just crossed two blueprints
15:08:31 <adrian_otto> is there a reason it is referred to differently?
15:08:44 <zns> I should probably also document that we're doing system-info plugins separately. Separate from discovery or collection plugins.
15:09:06 <caleb_> I did a partial implementation of https://blueprints.launchpad.net/satori/+spec/poc-system-info and zns is talking about https://blueprints.launchpad.net/satori/+spec/satori-plugin-support. Sorry for the confusion.
15:09:09 <zns> adrian_otto: referring to policy and opinion differently?
15:09:28 <adrian_otto> zns: yes, is there a difference?
15:09:32 <jasonpgignac1> adrian_otto: I could be wrong, but doesn't a policy imply a course of action?
15:09:48 <adrian_otto> the action may just be notify
15:09:50 <jasonpgignac1> Opinions are descriptive, policies are prescriptive?
15:09:55 <jasonpgignac1> Ah, ok.
15:10:04 <zns> We never had the term "policy" in mind... and we didn't really discuss the name.
15:10:36 <zns> Policy has an enforcement air about it... we've not included that in our scope for the project.
15:10:47 <jasonpgignac1> Is policy used elsewhere in openstack?
15:10:56 <zns> Rule is also a possibility.
15:10:58 <adrian_otto> ok, so I don't suggest we waste a lot of time discussing how things are named, but we might want to think about whatever is simple so people understand coming in with a more shallow learning curve
15:11:07 <adrian_otto> rule would be more obvious too
15:12:16 <caleb_> should we take an action to ask potential users or contributors about naming?
15:12:16 <gusmaskowitz> I discussed the satori concept with two techs this morning and the idea of applying an opinion onto some data you collect was a simple concept to grok.
15:12:17 <adrian_otto> "policy" will probably be used elsewhere in various command and control systems. I'm not aware of current use of that term.
15:12:53 <adrian_otto> ok, fair enough.
15:13:13 <zns> adrian_otto: fine with opinion? We can better document it.
15:13:46 <adrian_otto> this can be solved with additional descriptive text as we document.
15:14:19 <zns> #action caleb_ to better document the choice of the word opinion
15:14:27 <jasonpgignac1> Agreed. A 'Terminology' page is probably a good one to add to the docs todo list, in general
15:14:28 <samstav> Isn't there really two parts to the mechanism? The (1) data you collect adheres to some standard (we have referred to this sometimes as just "system info"), and (2) the *opinion* would be the output. Some of the terminology we are discussing might be defining both of these together
15:15:10 <zns> I've called the first part "collection" and the second "analysis". I think collection is a common term in this space.
15:15:11 <adrian_otto> jasonpgignac1: +1
15:15:31 <zns> Any volunteers to add a Terminology page?
15:15:45 <gusmaskowitz> gusmaskowitz I am
15:16:10 <gusmaskowitz> I am still queued to add some docs that I got help with last week, so happy to add a terminology page too.
15:16:11 <zns> And, to add another dimension, we have control plane discovery (APIs) and data plane discovery (logging on to the devices).
15:16:13 <caleb_> Is that IRC 3rd person?
15:16:31 <jasonpgignac1> I can take terminology.
15:16:38 <gusmaskowitz> I am a second language english speaker okay :)
15:16:40 <zns> #action gusmaskowitz Add Terminology page
15:16:43 <zns> Thanks, gusmaskowitz
15:16:44 <gusmaskowitz> Afrikaans for the win! :P
15:16:55 <jasonpgignac1> Ack, too slow. The foreigner got it.
15:16:57 <jasonpgignac1> ;)
15:17:20 <adrian_otto> gusmaskowitz: just pass it to jasonpgignac1 for a tune up when you have a draft up
15:17:46 <zns> Cool. Any more thoughts, questions on the opinions spec?
15:17:50 <adrian_otto> it's not hard to fix up something once you ahve a draft
15:18:23 <adrian_otto> question
15:18:30 <adrian_otto> are groups optional or required?
15:18:47 <caleb_> adrian_otto: required.
15:19:19 <zns> adrian_otto: groups?
15:19:28 <jasonpgignac1> Opinion groups, I think he means
15:19:35 <caleb_> zns: #link https://wiki.openstack.org/wiki/Satori/OpinionsProposal
15:19:40 <adrian_otto> opinions = {
15:19:40 <adrian_otto> 'opinion_group1': {
15:19:40 <adrian_otto> 'opinion1': {
15:19:50 <zns> Ah...
15:19:55 <adrian_otto> I suppose making them required simplifies the parsing logic
15:20:29 <samstav> Why numbers and not an array?
15:20:30 <jasonpgignac1> Any non-trivial implementation of Satori is likely to have enough opinions that organizing them into subject groups is likely to be helpful.
15:20:31 <zns> And it will provide a namespacing mechanism if there are multiple opinions plugins loaded.
15:20:36 <adrian_otto> are they likely to be grouped in the general case?
15:20:51 <caleb_> samstav: Names, not numbers so its a dict
15:21:17 <zns> I suppose they could be an array. Would we then need some kind of scoping field? Like applies-to: [DNS, ...]
15:21:50 <jasonpgignac1> I prefer the current structure.
15:21:50 <caleb_> zns: Don't like an array.
15:22:05 <caleb_> the `0` and `1 ` references are resource ids
15:22:05 <samstav> I didn't realize those numbers represented device ids
15:22:39 <caleb_> in reality those are probably UUIDs from nova
15:22:47 <caleb_> or the other services
15:22:52 <zns> Are there any rules to the group names? For example, can two plugins write opinions and put them in the same group?
15:23:07 <adrian_otto> caleb_ those will be uuids, so we should tweak the example to show that
15:23:23 <caleb_> #action caleb_ add UUIDs to the opinions example
15:23:47 <zns> Should they be URIs? I can see discovery span more than one region (or even cloud).
15:24:02 <caleb_> thats interesting
15:24:28 <caleb_> +1 to that idea. adrian_otto, what do you think?
15:24:53 <adrian_otto> caleb_: that's interesting… because...
15:25:10 <adrian_otto> having a URI there is a prevailing REST style (HATEOS)
15:25:23 <adrian_otto> but OpenStack does not currently use that style
15:25:47 <adrian_otto> the APIs are more loosely coupled tahn that
15:25:49 <caleb_> would using it create any problems within the community?
15:26:08 <adrian_otto> telying on knowledge of the relations rather than allowing machine parsed relations among members of a resource map
15:26:11 <zns> If we're discovering application topologies, I can see a common use case where you back up to swift on another provider or cloud. The resources should be globally locatable. It doesn't have to necessarily be in the ID, as long as a link somewhere has the full URI.
15:26:20 <jasonpgignac1> How about splitting the difference, and namespacing with an arbitrary id?
15:26:38 <caleb_> jasonpgignac1: then you need a lookup table of those IDs and the URI.
15:26:47 <adrian_otto> technically speaking I think the HATEOS style would make more sense for this use case
15:27:00 <zns> adrian_otto: so URIs?
15:27:02 <adrian_otto> where each node in the resource map has a URI
15:27:09 <jasonpgignac1> caleb_: Or you just use the information inside the data structure for actual discovery.
15:27:10 <adrian_otto> I suggest exploringthat first
15:27:28 <zns> If the resources are an array, then we don't need an id, just the id and URI in the body of each.
15:27:34 <adrian_otto> becuse it's easier to take that out later if you must… rather than adding it in as a necessity later.
15:27:45 <zns> Otherwise, we need a unique key for each resource.
15:27:48 <adrian_otto> zns: yes
15:28:10 <zns> But searching sucks. You have to loop over all of them to find the one you want.
15:28:14 <adrian_otto> so if you are persisting them, you can use a uuid as the id you use to persist them
15:28:42 <adrian_otto> otherwise, you can use a URI, and you will not have a compulsion to persist the ourput
15:29:14 <adrian_otto> because creation time is an input to the uuid generation algorithm
15:29:57 <zns> So URI might be the best choice for the id. I also think that we might discover a resource more than once (ex. a shared database used in two apps), so the ID should be the same in both instances. So my vote is for a URI.
15:29:58 <adrian_otto> and with a URI, you can make any scheme that
15:30:43 <adrian_otto> zns: +1, but expect that you may need to revisit that at some point
15:30:51 <zns> OK.
15:30:54 <caleb_> zns: +1
15:30:58 <zns> OK.
15:30:59 <caleb_> agreed that things can change
15:31:29 <zns> #agreed use URI to identify resources
15:31:37 <caleb_> it would be trivial to implement either way and we don't have any promised contract to uphold at this point
15:32:02 <zns> Back to opinion groups (recognizing we are slightly off agenda, but this is a good discussion. And I'll circle back to the agenda later).
15:32:11 <caleb_> #action caleb_ change opinions example to URI keys
15:32:31 <caleb_> zns: I think an opinion should be able to write into any namespace
15:32:41 <zns> So what are opinion groups? Are they fixed, namespaceable? Ex. satori:foo, tax:foo ?
15:32:45 <zns> rax
15:33:05 <zns> caleb_: an opinion module/plugin, right?
15:33:08 <adrian_otto> could groups also be accomplished using tags?
15:33:09 <caleb_> yes
15:33:23 <jasonpgignac1> I think the purpose of the opinion groups is semantic namespacing.
15:33:27 <adrian_otto> so you would tag the node oyu want to relate to a group
15:33:30 <caleb_> I'd recommend an opinion author stick to their own namespace unless they have a good reason to extend another.
15:34:00 <adrian_otto> or both tagging and grouping in the way that is currently proposed… could both be used in combination?
15:34:40 <jasonpgignac1> I think we are to add tagging it should be in combination - grouping is a useful for the purposes of parsing and iteration, both machine and human.
15:34:58 <samstav> jasonpgignac agreed
15:35:13 <zns> As a UI developer, I would like to have opinions have a uniquely referencable id. Otherwise, I'll probably have to generate my own on the client.
15:36:03 <jasonpgignac1> zns: Do you mean each instance of an opinion, or each class?
15:36:18 <jasonpgignac1> To clarify:
15:36:20 <caleb_> zns: would "opinion_group1"->"my_opinion" be considered unique?
15:36:20 <zns> jasonpgignac1: each opinion.
15:36:47 <zns> That's relevant because we have opinions showing under both opinions and resources in the spec.
15:36:47 <jasonpgignac1> Do you mean you want an id to apply to 'Opinion on whether or not there is too much foo', or...
15:36:59 <jasonpgignac1> An id to apply to 'On Server Y, there is X% too much foo'
15:37:31 <zns> jasonpgignac1: I'm not looking for a global ID always the same for that opion - to be clear. I just want the opions to have IDs within a discovery document.
15:38:03 <jasonpgignac1> I'm not partial to that, personally. Many users will simply store this as BSON, or a similar nested document.
15:38:28 <adrian_otto> zns: might one option to be to make a hash of that section of the JSON?
15:39:00 <jasonpgignac1> The individual opinions are part of a nested structure. If you as a developer wish to break up that structure into some normalized network of objects, I'd leave that to the vagueries of your implementation.
15:39:16 <zns> It makes it easy to reference in UI code and to reference in other systems (ex. an action list system that would generate tasks in response to specific opinions, especially warnings and errors).
15:39:18 <caleb_> To build on jasonpgignac1's idea. Should we add an ID to opinions. e.g: {'id': 'MYNAMESPACE00001', 'title': 'Foo Amount', 'status': 'Ok', 'description': '56% of foo'}
15:39:47 <caleb_> similar to pep8, pylint rules
15:39:50 <caleb_> they have IDs
15:40:26 <gondoi> caleb_: +1 i like it... although would we store those rules in the wiki... or actually i guess in the docs dir
15:40:27 <jasonpgignac1> caleb_ Yes, I like the RULE having an id, just nto the instance of its execution.
15:40:35 <zns> adrian_otto: a hash would work. That could then be used to calculate the ID. A hash that includes the message, group, and resource.
15:40:58 <adrian_otto> zns: but it might change if you include a timestamp in the body
15:41:22 <adrian_otto> so perhaps that would be somewhat restrictive
15:41:45 <jasonpgignac1> Then we can have a strictly defined namespace for publicly released/developed rules, and then set aside a numeric id block for use for private plugins implementation specific rules
15:42:21 <caleb_> I'm proposing a pylint-like consistent ID determined by opinion writer. zns and adrian_otto are discussing computer generated IDs. Which should we use?
15:42:32 <samstav> both
15:42:46 <zns> adrian_otto: that's OK. It depends on the use case. If the use case is to be able to reference an opinion from inside the discovery or from an external system, a hash would work. If the intent is to provide an identifier to the rule, then not. Those are two valid, but separate use cases.
15:43:12 <jasonpgignac1> Caleb_ implementation thumbs up. Autoid thumbs down - too specific to the use case.
15:43:16 <zns> I'm talking about the former. I think the pep-like rules conversation is the latter.
15:43:19 <adrian_otto> caleb_: yes, I am thinking that satori may find things that the opinion writer did not explicitly request by id
15:43:44 <adrian_otto> so until it does that, there may be no need for a generated id
15:43:57 <zns> caleb_: I think those are both valid use cases.
15:44:33 <caleb_> adrian_otto, zns: Can we agree to implement static now and revisit if a use case presents itself?
15:44:50 <adrian_otto> fine with me
15:45:03 <zns> adrian_otto: how would that happen? Anything we find would be written as a rule. And we're saying a rule should have an identifier (like PEP and pylint).
15:46:28 <adrian_otto> perhaps there is logic in one of the modules to look for additional things
15:46:31 <zns> caleb_: what is static? Is that the pep-like rule IDs or the opinion identifiers?
15:46:44 <adrian_otto> I suppose that might be discouraged for this reason
15:48:01 <samstav> static -> pep-like rule Ids
15:48:39 <zns> samstav: yes. And that would be hard-coded in the rule code.
15:49:08 <zns> So we're saying we don't want to generate IDs unique to each opinion generated? (yet)
15:49:17 <jasonpgignac1> zns: Agreed
15:49:20 <caleb_> thats my PoV
15:49:38 <zns> OK. Could we call that rule-id instead of id then?
15:49:48 <jasonpgignac1> zns: Agreed with that, too
15:49:51 <caleb_> +1
15:50:00 <samstav> I don't need feedback on this immediately, but I want to document two thoughts. (1) What if an opinion's logic gets updated? Does it change version, or does its ID change? and (2) What about opinions that will only be measured if certain configurations are found? e.g. opinions that will only be applied to certain topologies
15:50:02 <zns> Who's up for speccing that?
15:50:28 <caleb_> samstav: Create a blueprint
15:50:40 <caleb_> zns: I can update the existing spec with rule-id
15:50:56 <caleb_> time check: 10 minutes remaining
15:51:03 <jasonpgignac1> samstav: If the logic is updated to refine it, but it is still answering the same question, don't change the id. If it changes to answer a different question, change the id
15:51:03 <zns> #action caleb_ write spec for rule-ids in opinions (like PEP and PYLINT rule numbers) calling it [something-]id to allow for future opinion identifiers
15:51:22 <zns> Cool. Back to agenda then.
15:51:29 <zns> gusmaskowitz: item 5 update?
15:51:58 <gondoi> that applies to me as well...
15:52:03 <jasonpgignac1> And me.
15:52:05 <caleb_> 5-8 are all the same.
15:52:07 <zns> I also think 5,6,7 roll into 8
15:52:11 <gondoi> I sent in some super basic changes to the docs
15:52:12 <caleb_> We gave that task to EVERYBODY
15:52:13 <zns> Yep. All gusmaskowitz :-)
15:52:33 <gondoi> they described basic contribution steps and locations of code/docs
15:52:36 <gondoi> although
15:52:39 <zns> pinging gusmaskowitz
15:52:48 <jasonpgignac1> I compiled a list of the docs that we ought to write. I will submit that today.
15:52:53 <zns> samstav: while gusmaskowitz  wakes up?
15:52:57 <gusmaskowitz> I'm writing the docs. was doing that now actually sorry I was not here
15:52:57 <gondoi> now that we are merged into stackforge those need to change since they described github workflow
15:53:43 <zns> gondoi: agreed. Which reminds me - status update...
15:53:51 <zns> #info We're in stack forge!
15:53:57 <gondoi> WOOHOO
15:54:07 <zns> git review, everyone :-)
15:54:17 <caleb_> In summary, there has been activity from everybody, some has landed in the repo, some have not. Yes?
15:54:23 <zns> samstav: action 9?
15:54:29 <jasonpgignac1> #action jasonpgignac - I will doc how to submit with the new system
15:54:40 <samstav> I experimented with a catch all, disable via --debug.
15:54:46 <samstav> err
15:54:57 <zns> samstav: so being implemented?
15:55:14 <samstav> correct. This would be the standard mode
15:55:15 <caleb_> samstav: I think that should have a blueprint
15:55:42 <zns> samstav: I agree with caleb_ - would you create a blueprint?
15:55:55 <samstav> sure
15:56:07 <caleb_> samstav: Sorry, we should have pointed that out last week
15:56:31 <zns> So I'll push doc action items to next week as well...
15:56:41 <zns> #action gusmaskowitz will update docs for pypi and http://satori.readthedocs.org/en/latest/
15:56:46 <gusmaskowitz> yes
15:56:57 <gusmaskowitz> I got help on the docs from a "information developer" last week.
15:56:58 <zns> #action samstav catch exceptions in CLI so we don't stack trace (consider --debug mode allowing exceptions through to allow for debugging) + blueprint
15:57:11 <zns> #action zns implement plugins own establishing the connection to start with, but we provide a built-in ssh module that they can use
15:57:17 <gusmaskowitz> I did not know how to properly write what this is all about. Merging that info this week.
15:57:55 <zns> Another update. Guido updated pep-257 last week on our request and I'm going to write gating code for that in stackforge. Please mind your docstrings.
15:58:19 <zns> The update was no longer requiring the extra blank line in multi-line docstrings.
15:58:45 <caleb_> Everybody should go back over the how to contrib to OpenStack docs again to level-up your Gerrit skills
15:59:27 <caleb_> All good?
15:59:31 <zns> Also, the gates may not all be working 100% as I have not tested our stack forge config, so please allow some latitude while we firm that up.
15:59:39 <zns> caleb_: yes
15:59:45 <gondoi> #info https://wiki.openstack.org/wiki/Gerrit_Workflow
15:59:54 <zns> Allright, we're out of time. Thank you all for joining. Good discussion!
15:59:59 <jasonpgignac1> caleb_: Will do. Anyone who notices important gotchas to doc, please let me know
16:00:06 <zns> #endmeeting