15:01:26 <zehicle_IRL> #startmeeting DefCore
15:01:27 <openstack> Meeting started Wed Jun  3 15:01:26 2015 UTC and is due to finish in 60 minutes.  The chair is zehicle_IRL. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:31 <openstack> The meeting name has been set to 'defcore'
15:01:37 <hogepodge> o/
15:01:44 <zehicle_IRL> #topic Flag.2
15:01:47 <zehicle_IRL> o/
15:01:53 <markvoelker> #link https://etherpad.openstack.org/p/DefCoreFlag.2
15:02:07 <zehicle_IRL> I've been building an agenda there
15:02:59 <markvoelker> First item on the agenda: the agenda.  I like it. =)
15:03:09 <kbaikov> Hello
15:03:22 <zehicle_IRL> always first for me - make sure we have all the topics
15:04:22 <markvoelker> zehicle_IRL: looks good to me.  Lots to do today.
15:04:23 <zehicle_IRL> top of the list, we made some changes to meeting format and structure [trial mode for this month]
15:04:31 <zehicle_IRL> 1) meetings on IRC
15:04:40 <zehicle_IRL> 2) new times w/ alternating schedule
15:05:30 <markvoelker> 3) Combined capabilities and process meetings into one meeting
15:05:37 <zehicle_IRL> The IRC should serve as the official attendee list - please make sure you o/ if you are here
15:06:17 <zehicle_IRL> questions about the governance change?
15:06:40 <kbaikov> what is o/ ?
15:06:44 <purp> o/
15:07:04 <zehicle_IRL> kbaikov, raising hand (present)
15:07:28 <kbaikov> o/
15:07:33 <frayedknot> o/
15:07:48 <eglute> o/
15:08:00 <zehicle_IRL> #topic Flag.2 Summit  Review
15:08:12 <purp> From #openstack-defcore, courtesy @markvoelker, the Meetbot quick reference guide: http://meetbot.debian.net/Manual.html#user-reference
15:08:32 <zehicle_IRL> #chair eglute
15:08:33 <openstack> Current chairs: eglute zehicle_IRL
15:08:42 <zehicle_IRL> #chair hogepodge
15:08:43 <openstack> Current chairs: eglute hogepodge zehicle_IRL
15:09:00 <zehicle_IRL> discussion or comments from the Summit?
15:09:15 <hogepodge> Lots of positive feedback all around.
15:09:17 <zehicle_IRL> #link https://etherpad.openstack.org/p/DefCoreFlag.1
15:09:27 <eglute> i think it was great, would like to see more working sessions next time
15:09:56 <zehicle_IRL> I was very happy w. the panel.  I think we had a lot of diverse perspectives.
15:10:19 * zehicle_IRL gives HIGH FIVES to everyone for level of attention and results around DefCore
15:10:23 <markvoelker> I think we've done everything on the todo list, except set a midcycle
15:10:27 <eglute> agreed!
15:10:36 <markvoelker> #link https://etherpad.openstack.org/p/DefCoreFlag.1
15:10:48 <markvoelker> ^^ etherpad from Vancouver with to-do list at the bottom
15:10:55 <purp> +1 working sessions, +10 face-to-face for progress and brainstorming
15:11:02 <zehicle_IRL> we ready to do that now or wait a bit?
15:11:14 <eglute> set midcycle? yes lets do it
15:11:14 <hogepodge> Can we think about matching up the midcycle with openstack-qa?
15:11:17 <zehicle_IRL> I think we could setup a poll for the week & location
15:11:26 <eglute> hogepodge when openstack-qa
15:11:27 <zehicle_IRL> hogepodge, +1
15:11:38 <hogepodge> That way we can work in tandem and get immediate help if needed.
15:11:39 <markvoelker> hodgepoge: When is the qa meetup?
15:11:46 <hogepodge> They haven't set it yet, which is the problem. :-/
15:11:47 <markvoelker> (and where)
15:11:47 <zehicle_IRL> if it's not too late.  I think that we need to meet sooner to get some of the work kicked off
15:11:59 <hogepodge> mtreinish might know
15:12:04 <zehicle_IRL> let's start with how long... 1 or 2 days?
15:12:27 <eglute> do you think we need 2 days?
15:12:53 <zehicle_IRL> we needed it last cycle.
15:12:57 <zehicle_IRL> what's the agenda?
15:13:00 <markvoelker> We filled up the last one pretty easily and deferred a lot to parking lot.  I think we need 2 days.
15:13:09 <hogepodge> I prefer 2 days. 1 to figure out what we need to do, the other to hammer out work ftf.
15:13:11 <eglute> ok, +1 on 2 days
15:13:31 <zehicle_IRL> we've got to deal with: flag process, networking, non-tempest tests
15:13:40 <zehicle_IRL> and also the mechanics of scoring new capabilities
15:13:50 <eglute> ok, 2 days then. where?
15:14:15 <Will__> Can we propose a week and ask QA if they will match our week?
15:14:48 <zehicle_IRL> assuming we are NOT changing the capabilities list at this time
15:15:07 <zehicle_IRL> are we expecting to have a lot of new capabilities to score?
15:15:20 * zehicle_IRL hopes that we do
15:15:44 <zehicle_IRL> eglute, assuming ATX, SATX or SJC
15:15:46 <markvoelker> #link https://github.com/openstack/defcore/blob/master/process/ProcessCycles.rst#flag-cycle-spring---summer-2015 Flag cycle objectives
15:15:53 <hogepodge> zehicle_IRL: not sure about capabilities, but I want to start adding tests (see strategic section later in agenda)
15:16:00 <zehicle_IRL> depending on timing, could look at OSCON (was not planning it this year)
15:16:24 <zehicle_IRL> hogepodge, if you add tests then doesn't that mean more capabilities?
15:16:43 <hogepodge> zehicle_IRL: or replacing tests in existing capabilities
15:16:51 <markvoelker> eglute: zehicle_IRL: VMware has hosted midcycles at our Palo Alto campus in the past, I'd be happy to investigate whether we have the capacity to host DefCore this time around.
15:17:16 <Shamail> hi. :x
15:17:32 <zehicle_IRL> markvoelker, that's a good option.  last time we did it at the foundation office and got the benefit of Staff input
15:17:55 <zehicle_IRL> there's a part of me that thinks we'll want them involved to get vendor feedback about flagging
15:17:55 <barrett> I can check about hosting at the Intel location in Santa Clara if folks are interested too.
15:18:04 <eglute> thank you markvoelker i can find out the same about hosting at one of the Rackspace offices
15:18:23 <eglute> how many people would we expect in person?
15:18:48 <Shamail> What is the proposed date?
15:18:49 <purp> Depends on dates. =\
15:19:12 <markvoelker> #action: eglute, markvoelker, barrett, et al to check into potential hosting for midcycle meetup with their corporate overlords
15:19:16 <purp> I'm out of the US mid-July through mid-August, so can't then. Otherwise, won't miss.
15:19:47 <zehicle_IRL> let's try this... +1 if you'd attend in person,+2 if you'd travel to attend
15:19:49 <purp> Unless y'all want to come to the lovely new HP building in Galway. Happy to host there. =D
15:19:55 <purp> +2
15:19:56 <markvoelker> +2
15:19:57 <zehicle_IRL> +2
15:20:05 <eglute> +2
15:20:10 <Shamail> +2 (but borderline)
15:20:44 <zehicle_IRL> one benefit of SJC is that I suspect we'll have people able to travel there to attend
15:21:07 <barrett> +2
15:21:10 <zehicle_IRL> I love ATX and SATX but they are not as much tech "I'll do business there anyway" destinations
15:21:22 <Shamail> Good point zehicle_IRL, it's much easier to justify SJC
15:21:22 <Will__> +2 also borderline
15:21:31 <markvoelker> So, suggestion: we said we wanted to do this early-ish in the cycle....
15:21:34 <hogepodge> openstack-qa was talking NY again, but that's the last I heard.
15:21:46 <markvoelker> How about those inquiring about venues look to see what's available in mid-late July and report back?  That way we can choose a date when we know we can get a venue.
15:22:00 <zehicle_IRL> NY is ok for me too.  I'm concerned that we need to be early in the cycle
15:22:14 <markvoelker> #action hogepodge to check with mtreinish about QA dates/places and whether it's possible to joint-host
15:22:17 <eglute> so how early is early
15:22:32 <zehicle_IRL> mid-late July will bump into OSCON if anyone has a conflict
15:22:50 <markvoelker> eglute: per https://etherpad.openstack.org/p/DefCoreFlag.1 within 6 weeks (ish)
15:23:01 <barrett> The week of Jly 20th is OSCON in Portland. Are any people planning to come to that? If so, Intel could host at our facilities in Portland.
15:23:05 <hogepodge> zehicle_IRL: I have a firm OSCON committment.
15:23:19 <zehicle_IRL> I'm not planning on OSCON right now
15:23:22 <eglute> i was not planning on OSCON this year
15:23:29 <zehicle_IRL> just calling it out as a travel block
15:23:43 <purp> markvoelker hogepodge: already reaching out to mtreinish about QA midcycle.
15:23:52 <markvoelker> purp: thanks
15:23:57 <barrett> Another date to keep in mind, there's a Board meeting in Austin on 7/28
15:24:00 <hogepodge> Yeah, we're sponsoring part of it and are also using it as our semi-annual offsite.
15:24:20 <hogepodge> whatever offsite means for a distribute team ;-)
15:24:40 * markvoelker notes that we're 24 minutes into the meeting now and have AI's for this topic...move on and continue discussion on ML?
15:24:41 <zehicle_IRL> could be interesting to work around the Board meeting
15:24:45 <Shamail> I'll be at OSCON
15:24:55 <zehicle_IRL> +1 to move on
15:24:59 <purp> +1
15:25:13 <zehicle_IRL> good start - we need to get this scheduled so people can plan
15:25:21 <eglute> +1
15:25:24 <barrett> +1
15:25:26 <zehicle_IRL> #topic Flag.1 v1.3 Schema
15:25:36 <zehicle_IRL> #topic Flag.2 v1.3 Schema
15:25:52 <zehicle_IRL> https://review.openstack.org/#/c/185158/
15:26:11 <markvoelker> #link https://review.openstack.org/#/c/185158/ Schema 1.3 review
15:26:16 <eglute> i have not had a chance to review the latest changes
15:26:23 <zehicle_IRL> quick review: Chris and I sat down and figured out how to make the schema changes less disruptive
15:26:45 <zehicle_IRL> markvoelker, been doing a good job making sure that we update reference materials with the change
15:27:13 <zehicle_IRL> overall, the change to the schema did not break the existing parser
15:27:24 <zehicle_IRL> there are some items that we're leaving in to deprecate later
15:27:40 <hogepodge> markvoelker: We can have a discussion about sha vs is in defcore room if you want, after meeting.
15:27:48 <hogepodge> s/in/id/
15:27:53 <markvoelker> hogepodge: sure
15:28:01 <zehicle_IRL> I'd like to have a time limit on the reviews because we need to get the new capabilities into the file too
15:28:07 <markvoelker> (or in gerrit)
15:28:07 <eglute> at first glance, the 1.3 looks good to me
15:28:39 <zehicle_IRL> I know that markvoelker and hogepodge need to resolve the git-id item
15:28:50 <zehicle_IRL> most of the changes were too meta information not the schema
15:28:59 <eglute> ok
15:29:06 <zehicle_IRL> markvoelker, you OK with a 48 hour turn around?
15:29:29 <markvoelker> #action markvoelker (et al) will review https://review.openstack.org/#/c/185158/ in the next 48 hours
15:29:30 <markvoelker> =)
15:29:30 <zehicle_IRL> we can continue to iterate on the rst support files
15:29:52 <hogepodge> for the git item, openstack tags releases, and take them seriously.
15:30:00 <zehicle_IRL> If you find issues, go ahead and push an update.
15:30:32 <hogepodge> so saying "4" has direct meeting as an ID, and while a sha exists that matches it and will never change, I'd like to allow something like "4" to be used.
15:30:43 <zehicle_IRL> before we move on, anyone have questions about the WHY?  this is 1.3 and not the more agressive 2.0 change?
15:30:46 <hogepodge> trusting qa team doesn't decide to move a tag.
15:31:17 <markvoelker> hogepodge: do all repositories of tests we're using use tags that way though?  I think right now it's basically tempest and in-tree tests, so presumably yes...
15:31:36 <zehicle_IRL> basically, we figured out we could add the flags under the tests
15:31:58 <markvoelker> hogepodge: with the governance changes, I think it's possible some might not.
15:32:00 <hogepodge> markvoelker: I don't know, but I've also come around to be strongly opposed to using anything but tempest (which is a debate for another week)
15:32:02 <zehicle_IRL> moving on...
15:32:23 <zehicle_IRL> hogepodge, that's a good discussion to have.  (later in the agenda)
15:32:44 <zehicle_IRL> #topic Flag.2 Capabilities Subdivsion
15:33:02 <zehicle_IRL> I did not see Van
15:33:12 <eglute> i think he is traveling this week
15:33:15 <zehicle_IRL> ok
15:33:40 <zehicle_IRL> it looked like subdividing the capabilities was really more mechanical
15:34:01 <zehicle_IRL> we just need the v1.3 schema in first
15:34:21 <eglute> do we want to move subdividing to next week's meeting?
15:34:31 <purp> +1
15:34:59 <markvoelker> +1 (makes sense for Van to be here to cover this since he was driving it)
15:35:01 <zehicle_IRL> 1 question about it - do we do it in .next first
15:35:51 <markvoelker> zehicle_IRL: I would think so.  2015.05 is already board-approved and immutable, no?
15:35:54 <zehicle_IRL> #idea future item: back propagate v1.3 schema and subcapabilities
15:36:13 <zehicle_IRL> markvoelker, we'd talked about having a 2015.06 version with the subcapabilities
15:36:39 <zehicle_IRL> let's focus on getting the changes in .next and then discuss backwards movement in the future
15:36:53 <markvoelker> hmm...my gut reaction would be to oppose issuing a new guideline outside of the timelines established in the process docs
15:36:57 <zehicle_IRL> for now, yes markvoelker , we should consider immutable
15:36:59 <markvoelker> But open to discussing it
15:37:25 <zehicle_IRL> markvoelker, we already told the board to expect another guideline since we were not able to make the subcaps change
15:37:33 <zehicle_IRL> it's sideways - there are no new additions
15:37:53 <markvoelker> zehicle_IRL: it's not hte Board I'm worried about, but people trying to get a logo agreement.
15:38:15 <markvoelker> We have not informed them, and in fact at the summit we told them that we were done with the catch-up quick release cycles.
15:38:20 <hogepodge> zehicle_IRL: the danger is in flagged tests. since we didn't bring them forward, we can be in a situation where defcore fails for many vendors because flags get timed out.
15:38:25 <zehicle_IRL> so far, there are no material deltas for vendors between the files
15:38:47 <zehicle_IRL> everyone OK w/ getting the changes in .next and then revisiting?
15:38:54 <markvoelker> yep
15:38:58 <eglute> +1
15:39:13 <hogepodge> zehicle_IRL: it's not insurmountable (two stepping it, bring all flags forward), just wanted to bring it up.
15:39:20 <hogepodge> (this is the flag cycle after all)
15:39:36 <eglute> flags! is that the next topic?
15:39:40 <zehicle_IRL> #topic Flag.2 Criteria for Adding Flags back
15:41:07 <eglute> we had this from our meeting in vancouver: "Types of flags: broken test, broken code (e.g. resize), and reasonable disagreement. Others?"
15:41:27 <zehicle_IRL> so, we need to have a set of criteria for flagging testst
15:41:37 <hogepodge> I think it's a matter of going back though the logs and submitting new patches grouped by reasons (common test code broken, etc).
15:41:43 <zehicle_IRL> discuss here for 10 minutes then create a patch for Hacking?
15:42:00 <hogepodge> Then seeing if we can fix in a reasonable time, or bring problems forward.
15:42:02 <zehicle_IRL> I think we've got the mechanics ready in the schema
15:42:30 <zehicle_IRL> what reasons are we going to accept for flagging tests.... brainstorm....
15:42:44 <hogepodge> (i.e., if the action is to fix a part of tempest, and we can do it in a week or two, not merge the flag patch for that problem and fix the tests)
15:43:09 <eglute> who would be responsible to fix teh tests? foundation, vendor, other?
15:43:58 <purp> Ugh. Unfortunately I have to drop now, and this was the topic I wanted most to be involved in.
15:44:02 <hogepodge> One would hope the vendors who raised the flags would be invested in fixing the tests, but we can't dictate technical solutions.
15:44:09 <zehicle_IRL> eglute, good question - not sure how to resolve it here
15:44:34 <zehicle_IRL> what do we want to accept as reasons to flag a test?
15:44:44 <hogepodge> So if there are no volunteers to fix something, the recourse should be to pul the flag back in.
15:44:48 * zehicle_IRL brainstorming, looking for input
15:45:04 <eglute> hogepodge i dont think we can ask vendor to fix it
15:45:04 * purp leaving at sucky time.
15:45:12 <zehicle_IRL> hogepodge, eglute I'd like to put that question aside for now
15:45:19 <eglute> purp we will be in the other channel later, find us there!
15:45:22 <hogepodge> eglute: we can ask, just not require.
15:45:28 <markvoelker> zehicle_IRL: strawman input: 1.) the test is for a capability that fails to meet DefCore Criteria (as set out in the CoreCriteria doc)
15:45:35 <hogepodge> eglute: vendor can always say "no"
15:45:47 <markvoelker> 2.) The test is buggy and fails to test the Criteria properly
15:46:04 <hogepodge> 3) upstream is broken
15:46:05 <markvoelker> 3.)  The test fails because other non-DefCore Criteria are also tested.
15:46:43 <eglute> hogepodge where there other flagged tests that didnt fall into one of the 4 categories listed?
15:46:50 <zehicle_IRL> #) reflects an implementation choice that is not universally accepted
15:47:20 <markvoelker> zehicle_URL: I think that falls into my #1 since one of our CoreCriteria is "widely deployed"
15:47:43 <markvoelker> (I'm thinking of some of the security-oriented flag requests we saw come in here)
15:47:47 <hogepodge> Where do policy options fall here?
15:47:49 <zehicle_IRL> I think there's a nuance where it's widely deployed different ways
15:48:05 <zehicle_IRL> #) security concerns raised
15:48:17 <markvoelker> hogepodge: if there are lots of different policy choices being made about a criteria, then it's not "widely deployed" in my book
15:48:19 <hogepodge> If nova gives you a file that specifies policies for APIs, and a vendor modifies the expected defaults, is that a reason to let them flag?
15:48:25 <eglute> also, when tests fail cause some other component used by openstack, if there is a bug with kvm or similar
15:48:30 <zehicle_IRL> markvoelker, I think that it will come back to us to CHOOSE which way will win
15:48:56 <zehicle_IRL> it's going to come back to interop.  for some things, we need to have consistent implementation
15:48:59 <hogepodge> markvoelker: but PTLs expressed strong opinions about why they set those policies, and I heard a couple of times "a policy change from our defaults breaks our intent and is not openstack)
15:49:07 <zehicle_IRL> I don't know WHICH things but we need to expect that to be a factor
15:49:25 <markvoelker> I'm not sure I agree that if you make something configurable and somebody configures it, you can't call it OpenStack.
15:49:33 <hogepodge> markvoelker: and they're now looking at technical solutions to prevent policy changes from not happening where they don't want them to happen
15:49:51 <zehicle_IRL> markvoelker, I think it's case by case
15:50:03 <zehicle_IRL> we also have to consider interop
15:50:23 * zehicle_IRL raises 10 minutes remaining flag
15:50:55 <zehicle_IRL> do we have enough to create a hacking patch?
15:51:02 <eglute> i think so
15:51:39 <markvoelker> zehicle_IRL: I suspect we have enough to start a patch and discuss/iterate on it in gerrit.  I would expect it may take a little more time to land though, as there seem to be a lot of opinions to discuss.
15:51:41 <zehicle_IRL> since this will take a while, I'd consider putting a -1 workflow on it UNLESS that will limit discussion
15:51:54 <markvoelker> zehicle_IRL: sure
15:52:04 <eglute> #action create hacking patch with flagging reasons
15:52:27 <markvoelker> eglute: who has that AI?
15:52:30 <Will__> +1 the -1
15:52:44 <zehicle_IRL> can we start w/ E### instead of D### ?  not sure of the D prefix
15:52:51 <eglute> if there are no volunteers, i will
15:52:57 <zehicle_IRL> eglute, thanks
15:53:13 <hogepodge> I'm confused about which patch that is. The 1.3 schema patch?
15:53:23 <eglute> HACKING doc patch i think
15:53:37 <zehicle_IRL> eglute, yes.  each rule would be it's own #
15:54:00 <eglute> +1
15:54:05 <eglute> next topic?
15:54:35 <zehicle_IRL> #topic Flag.2 strategic test planning
15:55:10 <hogepodge> I'd like for us to start thinking about what technical requirements we want out of tests. Right now it's just non-admin and api.
15:55:21 <eglute> is this for planning new tests?
15:55:28 <hogepodge> But that requires things like multiple images, multiple users, multiple tenants.
15:55:34 <zehicle_IRL> at some point, I'd like to get the admin tests included in our review.  they are relevant for private clouds
15:55:47 <hogepodge> Are those reasonable expectations in looking for interop?
15:55:54 <zehicle_IRL> hogepodge, so you are thinking of expanding tests for our uses cases?
15:56:12 <hogepodge> The tests also pull in network ids, etc.
15:56:20 * zehicle_IRL would like to see tests that are interop focused.
15:56:30 <eglute> +1 interop tests
15:56:48 <markvoelker> So, the idea of having separate tests for separate use cases did come up in Vancouver....
15:56:48 <zehicle_IRL> we are stressing the primary use--cases for Tempest by adding interop
15:56:56 <hogepodge> I'm thinking of coming up with a standard that expresses a "maximum knowledge for interoperability", then start identifying tests that match that and rewriting existing tests to meet it
15:57:01 <hogepodge> A long term initiative.
15:57:21 <zehicle_IRL> reasonable - do you think they could also pass the gate?
15:57:28 <hogepodge> To build on what we have.
15:57:38 <eglute> we are almost at time, do we need an action item for this?
15:57:45 <hogepodge> Yes, they'd be still in tempest
15:57:50 <zehicle_IRL> are gates a subset of those are are they a discrete set?
15:57:54 <hogepodge> eglute I'm working up a proposal.
15:57:58 <eglute> we can always move discussion to the defcore channel if we get kicked out
15:58:12 <zehicle_IRL> we can track it as a future item.  like driving to the f2f
15:58:21 <eglute> +1
15:58:29 <markvoelker> eglute: I'd suggest that we have an AI to take this up on the ML then discuss next week when we have a more concrete thing to look at.
15:58:37 <zehicle_IRL> that covers the primary business.
15:58:49 <zehicle_IRL> reminder for everyone to look at the v1.3 patch (tick tick tick)
15:59:00 <hogepodge> #action hogepodge test standards proposal
15:59:18 <hogepodge> (that's terribly worded) :-(
15:59:24 <zehicle_IRL> last words.....
15:59:45 <zehicle_IRL> #endmeeting