16:00:59 <eglute> #startmeeting interopwg
16:01:00 <openstack> Meeting started Wed Aug 16 16:00:59 2017 UTC and is due to finish in 60 minutes.  The chair is eglute. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:01 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:04 <openstack> The meeting name has been set to 'interopwg'
16:01:06 <markvoelker> o/
16:01:13 <eglute> #chair markvoelker
16:01:14 <openstack> Current chairs: eglute markvoelker
16:01:18 <eglute> #chair hogepodge
16:01:18 <openstack> Current chairs: eglute hogepodge markvoelker
16:01:18 <zhipeng> o/
16:01:24 <eglute> Hello Everyone!
16:01:27 <eglute> #topic agenda
16:01:30 <ricolin> o/
16:01:47 <eglute> today's agenda, please update as needed: #link https://etherpad.openstack.org/p/InteropVertigo.11
16:02:01 <luzC> o/
16:02:09 <mugsie> o/
16:02:12 <eglute> #topic ptg
16:02:42 <eglute> PTG less than a month away! Please let us know you are coming by updating etherpad for PTG: https://etherpad.openstack.org/p/InteropDenver2017PTG
16:02:49 <eglute> also, please add proposed topics
16:03:14 <hogepodge> o/
16:03:16 <eglute> #topic  2017.09 Guideline
16:03:32 <hogepodge> I'm double booked this morning so my responses may be a bit delayed
16:04:13 <eglute> Since the August board meeting was cancelled,  we will ask the board to approve the next guideline during the September board meeting
16:04:29 <eglute> therefore, the next guideline will be 2017.09 rather than 2017.08
16:04:30 <catherineD> o/
16:04:40 * markvoelker makes note to rename it
16:04:58 <zhipeng> eglute could you explain a little bit on the reason for the naming ?
16:04:58 <eglute> #action markvoelker to rename 2017.08 to 2017.09
16:05:03 <zhipeng> 2017.09
16:05:15 <eglute> zhipeng we name the guidelines based on when they are approved,
16:05:21 <zhipeng> ah right
16:05:23 <zhipeng> :)thx
16:05:25 <eglute> year.month
16:05:29 <eglute> :)
16:05:49 <eglute> yeah, very simple convention, makes naming things easy
16:06:51 <eglute> thank you zhipeng for submitting bug report
16:06:53 <eglute> #link https://storyboard.openstack.org/#!/story/2001156
16:07:01 <eglute> good catch, wonder what happened there
16:07:41 <zhipeng> :)
16:07:47 <zhipeng> something we catch during testing
16:08:02 <eglute> we need to fix this, any takers?
16:08:08 <zhipeng> not sure if it is a bug or we misunderstand something
16:08:20 <zhipeng> eglute I could do it
16:08:22 <eglute> i think they should be unique
16:08:25 <eglute> zhipeng thank you!!
16:08:32 <catherineD> actually not
16:08:35 <zhipeng> so I could remove those duplicated capabilites ?
16:08:50 <hogepodge> If I understand some tests are overloaded, but that looks like an error. The fix should be to look at the code base and correspond the test names to the idempotent_id
16:08:50 <catherineD> in RefStack we do not match output to ID
16:08:59 <catherineD> we match them to the FQN
16:08:59 <eglute> #action zhipeng to fix duplicate test IDs
16:09:40 <zhipeng> okey I will submit a patch on this
16:09:47 <eglute> zhipeng what hogepodge said.
16:09:52 <eglute> zhipeng thank you!!!
16:10:11 <eglute> any other questions or comments on the next guideline?
16:10:12 <hogepodge> duplicated capabilities should not be removed. Now, we may be in a situation where if the tests were being skipped in testing we might want to flag them for the current guidelines.
16:10:30 <catherineD> see the explaination of FQN vs ID in https://bugs.launchpad.net/refstack/+bug/1709323
16:10:34 <openstack> Launchpad bug 1709323 in refstack "refstack-client removes tests which could be found by id (renamed tests)" [Undecided,New]
16:10:36 <zhipeng> hogepodge got it :)
16:10:50 <zhipeng> catherineD thx will take a look at it
16:11:13 <catherineD> these are actually testing 2 different feature using the same test method
16:11:26 <catherineD> it is like test the same thing in IPv4 vs IPv6
16:11:54 <eglute> catherineD but different tests should have different ids, correct?
16:12:11 <hogepodge> that's a case of overload
16:12:17 <catherineD> no ID is define at the test methid level
16:12:21 <hogepodge> eglute: in the code they don't
16:12:29 <catherineD> Interop-WG define test at the test case leve
16:12:33 <hogepodge> the tests use different input parameters.
16:12:51 <catherineD> see the explaination in https://bugs.launchpad.net/refstack/+bug/1709323
16:12:52 <openstack> Launchpad bug 1709323 in refstack "refstack-client removes tests which could be found by id (renamed tests)" [Undecided,New]
16:12:53 <hogepodge> We might want to ask the tempest team if we should introduce wrapper methods
16:13:07 <eglute> thanks catherineD
16:13:44 <eglute> anything else related to this?
16:14:01 <catherineD> RefStack dioes not use test ID for pass/fail evaluation at all . Tests are also mapping with FQM from testlist.
16:14:11 <eglute> thank you catherineD
16:14:16 <eglute> good to know
16:14:54 <eglute> #topic Extension programs
16:15:03 <eglute> thank you hogepodge for submitting patches
16:15:22 <eglute> hogepodge question: when do you want to get those approved? 2017.09 or 2018.01?
16:16:05 * markvoelker is leaning toward advisory by 2017.09 given that we have the extra month, personally
16:16:08 <hogepodge> eglute: I'd like to present them to the board as advisory for 2017.09, with the intention to refine them for official approval in 2018.01
16:16:15 <markvoelker> ++
16:16:24 <hogepodge> eglute: refine them with community and vendor feedback
16:17:14 <eglute> ok, then should the capabilities be as "advisory"? right now all are required.
16:17:20 <hogepodge> There's good discussion happening right now, which I want to encourage, but I'd also like them to be on track for board approval in early next year. I think that strikes a good balance between expediency and community review.
16:17:34 <hogepodge> eglute: probably. I struggled with how to handle this the first time around. But yes, I think that's the right way to go
16:17:45 <hogepodge> I can send up new patches.
16:18:27 <eglute> yeah, we should decide how new programs should be handled. advisory is good, and if someone wants early designate/heat badge, they should pass all the advisory capabilities
16:18:39 <hogepodge> I'd also like to bring up the notion of criteria. Right now the only criteria I have is PTL suggested, but I'm wondering if during the advisory period if we need a new set of criteria for extensions.
16:19:39 <markvoelker> hogepodge: I was noodling that a little this week too as I'm looking at verticals (for example: we've already been talking to OPNFV and I have some requirements suggested by ONAP now too)
16:19:49 <eglute> hogepodge is that what you call achievements?
16:20:52 <zhipeng> markvoelker seems like ONAP is on a different level from OpenStack ?
16:20:53 <hogepodge> eglute: yes
16:21:06 <markvoelker> eglute: see also: https://github.com/openstack/interop/blob/master/doc/source/schema/2.0.rst#criteria
16:21:25 <markvoelker> zhipeng: Yes, but they consume OpenStack via the multi-vim project.
16:22:04 <zhipeng> ah understood
16:22:31 <eglute> i think for criteria we should try to use the ones as we use for main guidelines, except as it relates to that particular project
16:23:10 <eglute> markvoelker hogepodge has something called "achievements" in the json, but i wasnt sure what it tied to in teh schema
16:23:39 <hogepodge> part of our confusing lexicon, we say achievements when a capability achieves the criteria
16:23:47 <hogepodge> so there's a synonym there
16:24:04 <eglute> hogepodge true
16:24:35 <eglute> hogepodge how is that tied to schema
16:24:45 <markvoelker> hogepodge: I had noticed you added the "ptl_proposed" achievements in those patches and was trying to think if using it as a way to define a source ("who wants this") made sense
16:25:17 <eglute> heh
16:25:47 <hogepodge> markvoelker: I couldn't think of any other criteria. I asked project leaders for APIs they, as project leaders, thought represented the interoperable set according to our requirements (not admin, single user, etc)
16:25:58 <eglute> perhaps ptls could score those capabilities on how important they are
16:26:10 <markvoelker> E.g. in the NFV vertical program case I might have some stuff we want to denote as "OPNFV needs this" and others as "ONAP needs this" or even specifically "ONAP multi-vim needs this".
16:26:14 <hogepodge> schema 2.0 was designed to allow for different sets of criteria to be used
16:26:49 <mugsie> PTLs should know what capabilities will need to be used across clouds
16:26:53 <markvoelker> But I'm not really sure that's what we want to use for achievements...my gut feel is to use similar criteria to what we use for Powered, but scope them to appropropriate audience
16:26:58 <hogepodge> and since the guideline is what's approved I put the criteria in the document so that they get pulled along with the approval. But I'm entirely happy to work on new criteria
16:27:08 <hogepodge> repurpose old, and so on
16:27:16 <eglute> markvoelker hogepodge i think it would be good to have some official documentation/justification for including capabilities in these new programs.
16:27:43 <markvoelker> E.g. if a thing is needed by both OPNFV and ONAP it's widely deployed (among clouds used for NFV)
16:27:48 <eglute> markvoelker i agree on " to use similar criteria to what we use for Powered, but scope them to appropropriate audience"
16:27:56 <markvoelker> (kinda poor example but you get the picture)
16:28:08 <hogepodge> eglute: I would like the guidelines to contain all of the information, but I'd happily document it elsewhere
16:28:25 <eglute> hogepodge i mean something similar to current scoring process
16:28:52 <eglute> we have all the capabilities scored, and comments on some why they were scored that way
16:29:00 <hogepodge> sure, I just don't think that the criteria for the core programs is 100% applicable to the extension or vertical programs
16:29:05 <eglute> so that next time we look, we have a reference point
16:29:23 <eglute> hogepodge not even if it is scoped down?
16:29:39 <hogepodge> eglute: scoped down, yes
16:30:37 <eglute> i think it would have to be scoped to that particular project. For example, for every cloud that deploys Heat, do most have X feature?
16:30:38 <hogepodge> I'm hearing that I need to update the two reviews and submit a third with proposed extension criteria?
16:32:08 <eglute> we need something that would justify why a set of capabilities are included
16:32:22 <hogepodge> It's just so difficult to measure something like that, and my inclination is to drop a criteria like that. It's an extension which is a special case, and since it's optional being more strict on what has to be there makes more sense. I don't want a bunch of extensions that only add things in name
16:32:47 <markvoelker> hogepodge: If we're going with the "same criteria, scoped to appropriate audience" then I'm not sure we need to doc them in a third patch...that's pretty much what we've presented to the BoD and community several times now.
16:32:49 <hogepodge> right now the justification is that project leaders have identified the capability as what's required for interoperable
16:33:22 <eglute> hogepodge i think it is good enough for getting them as advisory
16:33:42 <hogepodge> markvoelker: sure, but if we want external documentation that says something about why we chose a set of criteria for extensions we will have a one to many situation :-D
16:33:46 <eglute> but going forward, we will need some paper trail of why those particular capabilities were chosen
16:35:36 <markvoelker> hogepodge: If we're using the same criteria (but scoped) then we already have most of the doc though.
16:35:57 <hogepodge> markvoelker: ok
16:36:51 <markvoelker> E.g. I'd be fine with a smallish doc that basically boils down to something along the lines of "same as https://github.com/openstack/interop/blob/master/doc/source/process/CoreCriteria.rst but with the audience scoped to the project in question"
16:37:45 <eglute> +1.
16:38:20 <markvoelker> I'm not a big fan of "ptl_proposed" as I think it's too course.  The PTL probably has good reasons for proposing a capability, and I think if we use the core criteria (scoped) then it gives them a chance to back it with reasons rather than just being one person's opinion.
16:38:49 <hogepodge> markvoelker: ok
16:39:27 <hogepodge> (me saying ok like that makes it sound way more terse than I intend) :-D
16:39:33 <markvoelker> =)
16:40:04 <eglute> hogepodge i think we can work on those extra capabilities between now and next year, when the new programs become required
16:40:21 <eglute> i think we should be able to get it approved as advisory next month
16:41:10 <hogepodge> Especially since there's a conversation around heat that I was to give the full space to breathe
16:41:56 <eglute> heat: #link https://review.openstack.org/#/c/490648/
16:42:21 <eglute> i agree, good discussion there
16:43:58 <eglute> anybody want to share their thoughts on heat or designate programs?
16:44:43 <ricolin> eglute, yes thanks guys to propose heat out:)
16:45:11 <eglute> I think heat and designate are great add-ons to start with
16:45:29 <markvoelker> Generally it seems like a reasonable start on both fronts, though personally I'd like to spend some time picking through some tooling around some of them.  Just elbow grease+time. =)
16:45:31 <eglute> we do want as much discussion as possible at this stage
16:46:00 <eglute> markvoelker what kind of tooling do you think we should work on?
16:46:30 <hogepodge> I think that's the intent of advisory, and we're in a good place to fulfill that intent
16:47:19 <markvoelker> Well, just generally I'm looking for justifications that the capabilities are useful, and tooling is often a good indicator: e.g. if tools that use (for ex.) Designate support a bunch of API's but not others, maybe we need to look a little more closely at those that no tools use and see if they're actually good, useful candidates.
16:47:48 <markvoelker> The very fact that people are suggesting them is a good start, so just looking for some more backing for those opinions
16:47:52 <eglute> hogepodge i agree, good start for advisory
16:47:58 <eglute> markvoelker +1
16:48:03 <markvoelker> So basically the typical scoring process. =)
16:48:57 <mugsie> markvoelker: sure - thats what we did when we suggested our (Designates) list
16:49:02 <markvoelker> ++
16:50:39 <markvoelker> mugsie: did you folks keep notes or something along the way?  Would love to take a peek if so.
16:50:47 <eglute> mugsie thanks for working on the designate capabilities. we are just looking for additional documentation on why these particular capabilities were chosen. something like this: https://github.com/openstack/interop/blob/master/working_materials/scoring.txt
16:51:28 <mugsie> unfortunatly not- we did it in person in boston
16:51:32 <mugsie> All i have is https://etherpad.openstack.org/p/BOS-Forum-DNS-InterOp
16:51:37 <markvoelker> Ah, ok.  No worries.
16:51:37 <mugsie> and https://etherpad.openstack.org/p/DNS-InterOp-Capabilities
16:52:47 <eglute> so everyone ok with proposing heat and designate add-ons as advisory to the board in September?
16:53:15 <eglute> mugsie do any of the tests require admin? do you know?
16:53:30 <mugsie> they do not
16:53:45 <eglute> mugsie great, thank you
16:54:09 <mugsie> its is just 2 endpoints - creating / update / deleting a DNS zone and modifying Records
16:55:04 <eglute> mugsie thank you. seems like it covers essential interop for DNS
16:55:08 <hogepodge> time check five minutes
16:55:16 <eglute> ok, so everyone ok with proposing heat and designate add-ons as advisory to the board in September?
16:55:31 <markvoelker> Yep (pending the iterations hogepodge mentioned)
16:55:45 <hogepodge> I can get to work on those for next week
16:55:48 <eglute> +1
16:55:50 <eglute> thank you hogepodge
16:56:12 <hogepodge> mugsie: if you can offer any insight into scoring, that would be really great
16:56:50 <mugsie> sure - I will have a read of the methodolgy and comment in the review
16:56:56 <eglute> mugsie let us know if you would like to understand scoring better, i can help
16:57:07 <hogepodge> ricolin: same with you
16:57:14 <mugsie> great - if I have any questions, I will shout
16:57:27 <eglute> mugsie perfect!
16:57:35 <ricolin> hogepodge, sure
16:58:00 <eglute> since we are almost out of time, everyone, please be sure to review any outstanding patches
16:58:02 <hogepodge> Thank you everyone, I think this is a really bold and important step forward, and I'm inspired by the interest in seeing it be successful.
16:58:08 <eglute> as well as this RefStack patch:
16:58:13 <eglute> https://review.openstack.org/#/c/480298/
16:58:27 <eglute> hogepodge mugsie ricolin thank you so much for working on this
16:58:46 <eglute> any last words?
16:58:47 <markvoelker> Veryvery quickly on the NFV vertical front: last week I spoke with some folks from ONAP multi-vim.  They have a rough list of stuff they need from OpenStack here that folks may want to look over: https://wiki.onap.org/pages/viewpage.action?pageId=8229235
16:59:00 <ricolin> eglute, NP it takes entire heat team to make it:)
16:59:01 <markvoelker> I'll have that boiled down into actual API's by the time we get to Denver.
16:59:10 <eglute> thank you markvoelker
16:59:19 <eglute> ricolin very true!
16:59:49 <eglute> #action everyone look at https://wiki.onap.org/pages/viewpage.action?pageId=8229235 before next meeting
17:00:05 <eglute> ping us on interop channel or directly if needed.
17:00:08 <eglute> thanks everyone!
17:00:12 <eglute> #endmeeting