01:02:37 #startmeeting DefCore 01:02:38 Meeting started Thu Jun 11 01:02:37 2015 UTC and is due to finish in 60 minutes. The chair is markvoelker. Information about MeetBot at http://wiki.debian.org/MeetBot. 01:02:39 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 01:02:41 The meeting name has been set to 'defcore' 01:02:48 o/ 01:02:52 #chair eglute 01:02:53 Current chairs: eglute markvoelker 01:02:56 . 01:02:56 o/ 01:03:04 #chair zehicle 01:03:04 Current chairs: eglute markvoelker zehicle 01:03:05 thanks, it didnt like me then. 01:03:33 #link https://etherpad.openstack.org/p/DefCoreFlag.3 01:03:34 no problem....I'll turn it over to you to drive now. =) 01:03:40 thank you markvoelker 01:03:49 roll call pleasde 01:04:11 o/ 01:04:12 please indicate your time zone also, I'd like to make sure that we're accomplishing our objective 01:04:16 o/ Central 01:04:20 o/ CST 01:04:23 o/ EDT 01:04:34 o/ CST 01:04:38 o/ AEST - GMT+10 (Canberra Australia) 01:04:45 o/ PDT 01:05:03 PDT 01:05:13 o/ PDT 01:05:27 someone emailed the list and said this time does not work for him, he is in GMT +12 01:05:35 yy, that's Robert 01:05:44 y 01:06:04 ok, I'm thinking to follow the agenda on the ether pad 01:06:27 please make additions/changes there and we'll incorporate it 01:06:38 sounds good 01:06:42 #topic mid-cycle meeting 01:07:44 I'd like to have use close (or have a plan to close) the location & timing 01:07:51 do we have quorum to do that here? 01:07:54 Checked in with mtreinish; he's looking at late July/early August. Trying to sync with infra midcycle 01:08:27 We've got some block out times: OSCON & OpenStack BoD 01:08:32 zehicle regarding quorum, we had some people last week that said they would travel, some are not here right now 01:08:38 yy 01:08:55 I suspect that we cannot close it tonight. if so, can we get it down to two choices? 01:09:04 maybe 3 01:09:11 Seems reasonable. 01:09:20 my concern with waiting too long is that then we are at timeline July 27 (Summit - 3 months) 01:09:24 * zehicle gets his calendar out 01:09:53 BoD is on July 28th (Tuesday), oscon week before that 01:10:13 Week of July 6? 01:10:18 Too early? 01:10:19 I'm on vacation th week after 01:10:26 and traveling the week of July 6 01:10:47 So, 7/13 week 01:10:57 Could we plan it like this: 01:11:02 1) SJC week of 7/13 01:11:15 2) PDX week of 7/20 01:11:24 3) ATX week of 7/27 01:11:36 with a target dates of Wed-Thurs 01:11:40 works for me. 01:11:48 I can't make 13 Jul - 13 Aug. But we knew that. 01:12:01 so doodle to the mailing list with the dates? 01:12:11 that' 01:12:31 #action eglute send out doodle to the mailing list with date options 01:13:14 #topic v1.3 schema 01:13:40 so we merged the v1.3 schema with some discussion still potentially open 01:14:02 yes, i think there were some things that were missed. 01:14:14 markvoelker, were you OK w/ the latest HACKING or did we jump the gun? 01:14:23 lots of comments on https://review.openstack.org/#/c/185158/, so if something was missed, please submit a new patch 01:14:41 #link https://review.openstack.org/#/c/185158/ 01:14:52 So, I think the thing we may have overlooked was the bit where we removed the ability to drop tests completely... 01:15:08 i think there was at least one thing missed, need to find the new patch that was created that called it out 01:15:10 yeah 01:15:27 It looks like we've started the discussion on the new patch 01:15:28 #link https://review.openstack.org/#/c/189961/ 01:15:39 ^^ new patch 01:15:40 I wasn't sure if that was intentional or not though 01:16:02 not intentional! 01:16:04 yup, we need to be able to drop tests from next. I had proposed a change to do that, but it didn't make it in. 01:16:31 which is part of the point of next, to resolve outstanding flags. 01:16:32 Ok, so that should be relatively easy to correct 01:16:58 Shall I submit a patch for it, or does someone already have one in the works? 01:17:01 the previous patch ended up with too many comments and i could not tell that something was still not resolved. so yes, we need to fix it 01:17:25 markvoelker: you should feel free to do that 01:17:39 #action markvoelker submit a patch to fix process for removing tests 01:17:42 I don't have one in the works that's ready to go 01:17:43 I'd like to make sure that the patch w/ the flags stays about flags 01:17:50 +1 01:17:58 do we have tests that need to be removed? where did they go? 01:18:13 zehicle: Yes, see https://review.openstack.org/#/c/189961/ 01:18:15 zehicle: yes, there are tests that are hypervisor specific 01:18:43 soundn't we keep the tests but flag them? 01:18:54 and this https://review.openstack.org/#/c/189979/ 01:19:04 In next? No. 01:19:05 I thought that was the purpose of the flags? 01:19:09 i think the hypervisor one should be dropped, based on the discussions 01:19:13 I'm worried about having tests in the system that don't show up 01:19:27 zehicle: these are tests for capabilities that never should have been DefCore in the first place b/c they fail Core Criteria 01:19:38 +1 on that 01:19:39 So we could keep them...forever... 01:19:45 except that they would be flags 01:19:45 My understanding is that a required test is something we want, and that the actions on a flag are to fix or remove. 01:19:50 But that would just create clutter and we'd have to re-evaluate them every cycle 01:19:59 Whcih seems like a waste of resources 01:20:04 not if the flag reason was clear 01:20:11 OR... could we put them into a capability? 01:20:25 I'm worried that we're going to have them pop up every cycle for review 01:20:35 in hypervisor case, it would not meet the common use case 01:20:38 I'd rather make a decision about them one time and let it ride 01:20:45 confusing to end users. I've been telling vendors that a flag means a capability is required, but has a problematic test. 01:21:03 that still seems like a "compute-hypervisor-vendor-specific" capability 01:21:12 zehicle: so we had recently decided that we were going to re-evaluate all flagged tests at the beginning of every cycle. 01:21:12 then the cap is NOT required 01:21:13 zehicle: if that's the case I misunderstood the purpose then 01:21:13 This seems to point up the need for multiple flag types. No? 01:21:25 I' 01:21:27 So if we just leave them in the flag list indefinitely we're going to be re-evaling them indefinitely 01:21:40 And that list is going to get longer and longer 01:21:42 I'm happy to discuss.... my understanding was that we'd eventually list _every_ test 01:21:43 zehicle: I thought that a capability couldn't be removed in a cycle, but could in the next. 01:21:58 hogepodge, so we can flag them and move them 01:22:10 zehicle: but it shouldn't be in the required capability list 01:22:21 zehicle: I don't think it's feasible to list every test. That's a moving target. 01:22:25 Does it make sense to have capabilities that are driver specific? As in, if you are using a certain driver, within that context the expected behaviors are 01:22:28 zehicle: moving I'm ok with too 01:22:31 Especially when you consider non-tempest tests 01:22:58 markvoelker, eventually, it would be good to have them all tracked somehow since we want the data from all of them 01:23:23 dwalleck, we can have all sorts of capabilities. it's helpful to users to map functionality 01:23:24 zehicle: I'm saying that eventually will never come though....the tests are moving much faster than we are. 01:23:27 we just don't have to require them 01:23:36 we have a mechanism for listing all of the tests, it's the repository the test comes from 01:24:10 is this a topic we should leave for mid-cycle? 01:24:16 I'm OK w/ this suggestion - I have some concerns based on earlier postures 01:24:49 if we're thinking it's not a big deal then I'm OK to remove 01:24:58 since we have a lot of tests that are not covered 01:25:09 I just want to make sure that we don't keep trying to re-add them 01:25:10 i think for now, i vote +1 to remove 01:25:30 So, how about this: I'll just propose the patch and we can discuss it there. I'm happy to be overruled if folks think differently 01:25:44 markvoelker: +1 01:25:48 works for me 01:25:50 (in fact, I thought I had been overruled when the schema change with the procedure change landed! =p) 01:26:01 #action markvoelker to propose patch for test removal 01:26:06 no, we wanted to split the discussion 01:26:16 that's why we want to make sure to cover it here 01:26:32 because we suspected that the non v1.3 issues had not been resolved 01:26:40 +1 on splitting discussions in patches as appropriate. otherwise important issues get lost 01:27:02 based on discussion here, it seems like v1.3 is OK 01:27:09 I'll get that patch up tomorrow morning so we can get moving on it and have decision soonish so Chris's patches can move forward or be refactored 01:27:16 and we're aware that some of the issues in that patch need further discussion 01:27:24 yes 01:27:32 mission accomplished 01:27:53 other discussion on v1.3 patch and related skeltons unearthed? 01:28:19 * zehicle thinks we unblocked a lot of good discussion around flagging 01:28:20 need a proper validating schema? 01:28:20 the skeletons might be hiding in the next topic 01:28:32 Also, gate job to ensure json is parsable? 01:28:37 and correct? 01:28:42 #topic capabilities subdivisions 01:28:54 hogepodge, +1 01:29:12 getting more critical w/ each addition 01:29:53 Van was not going to make this meeting 01:30:01 I had two items related 01:30:10 right, i think we skipped this topic last week as well 01:30:12 1) who/when can we get the capabilities subsets done 01:30:15 ? 01:30:21 does someone have that list? 01:30:39 It's in a google sheet that I'm sure I can dig up a link for given enough time. 01:30:55 that has the correct subdivisions? 01:31:02 hogepodge, will doing that break all your patches? 01:31:02 Van and Catherine have been the primary contributors, and both aren't here (Catherine should be back next week) 01:31:14 #link https://docs.google.com/spreadsheets/d/15Fkt2n95WPPe7CYH-0WIKz3dhMeW3FW2gl5aSlamEeY/edit#gid=6 01:31:26 right, i think Van will be as well 01:31:43 zehicle: I'm not worried about the patches, they can be rewritten if the schema updates. I wanted to reclassify the tests for discussion, and figured those patches would be long lived and require rebasing 01:32:19 ok 01:32:48 so skip this topic for now? 01:32:50 so, can we get that subdivision done? 01:33:15 It's probably a matter of automation based on the list, so yes? 01:33:15 I'm worried about it causing downstream work if we delay 01:33:33 if you've got the material in that format 01:33:44 it's really not that much data to tweak 01:34:01 I'm thinking about 30 minutes tops 01:34:13 so, likely less effort than automation 01:34:39 the cap list is not the question - it's the test membership 01:34:49 that's what was missing from the earlier patch 01:35:00 hogepodge is this something you can work on with Van and Catherine? 01:35:12 I can 01:35:21 thank you hogepodge 01:35:44 ok, I see the list now 01:35:51 #action hogepodge work with Van and Catherine to subdivide capabilities 01:36:41 depending on the flags, they may actually be able to merge cleanly 01:37:10 the second part was just the we had a discussion about how to score capabilities in IRC 01:37:17 in the #openstack-defcore channel 01:38:01 the short version is that Van suggested a way to vote that may work 01:38:11 we'll need to try it sometime soon 01:38:18 if everyone is looking at the spreadsheet, voting should work 01:38:19 zehicle: got a link or a date when that happened? Was probably while I was on PTO and I'd love to read... 01:38:32 hogepodge, are there any new capabilities under consideration? 01:38:38 I can dig in eavesdrop if not 01:38:42 markvoelker i think that happened over the phone? 01:38:50 zehicle: not that I know of immediately 01:38:51 it was IRC 01:39:05 yesterday I think - around 4pm Central 01:39:32 Ok, I'll dig and post a link if I can turn it up before end of topic 01:39:33 was a big burst of activity including patches around that time 01:39:34 thanks 01:39:45 i missed then as well 01:39:48 I'd be glad to help with capabilities as well. I should have the bandwidth 01:39:54 wanted to make sure that people were are we'd dicussed it - we'll need to formally doc it before we try it 01:40:00 thank you dwalleck_ ! 01:40:22 dwalleck_ and hogepodge I think you've got all the data you need to subdivide from the spreadsheet 01:40:39 markvoelker: https://etherpad.openstack.org/p/defcore_scoring is a log of that conversation 01:40:45 #link http://eavesdrop.openstack.org/irclogs/%23openstack-defcore/%23openstack-defcore.2015-06-09.log.html#t2015-06-09T20:04:03 01:40:52 markvoelker: or that too :-) 01:41:00 #link https://etherpad.openstack.org/p/defcore_scoring 01:41:04 Thanks hogepodge 01:41:11 Thanks hogepodge 01:41:13 next topic.... 01:41:20 #review hacking file (flag tests) 01:41:24 #topic review hacking file (flag tests) 01:41:42 20 minutes remaining and this could be the rest.... 01:41:45 other topics first? 01:42:22 * zehicle reserves last few minutes to review the choice of time 01:42:33 i think we need to resolve this... dont think other topics will be short either 01:42:49 This is the one I missed last week, yes? 01:43:09 yes i think so 01:43:11 purp, it's a long runniung thread 01:43:20 there is a patch somewhere, let me find it 01:43:22 * purp is jealous of the stereo zehicles. 01:43:26 To be clear, are we talking about this? https://review.openstack.org/#/c/188661 01:43:39 yy 01:43:43 #link https://review.openstack.org/#/c/188661/ 01:43:43 #link https://review.openstack.org/#/c/188661 01:43:57 yes, thanks markvoelker 01:44:10 So IMHO there's not that much that's really controversial in the patch. I made a few suggestions in the review. 01:44:18 oh, can we cover #link https://review.openstack.org/#/c/182105/ first? 01:44:20 The big question for me is: is the list intended to be exhaustive? 01:44:44 zehicle: sure 01:44:45 markvoelker, I think so. that's a good catch 01:44:53 ok, back to the 2015A patch for a minute 01:45:08 #link https://review.openstack.org/#/c/182105/ 01:45:20 Oh, that's mine. 01:45:31 I wanted to discuss my objections and make sure that we were on the same page 01:45:47 yy, you were adding flag details to the 2015A process 01:46:07 to an extent, we did not add those details there on purpose 01:46:16 since they were left to DefCore to manage 01:46:33 I'm open to discussion that they should be part of the broader process 01:46:50 but it would have to be 2015B in that case and go back to the board 01:47:13 Yeah, I think this one predates some of the recent changes to HACKING that came in with schema 1.3. 01:47:25 Much of this is now covered there 01:47:40 you are correct markvoelker 01:47:48 E.g. rule D307 covers what is required when flagging a test 01:47:53 I'd like something to make flagging a harder thing to do 01:48:09 if that's covered in hacking, fine. 01:48:24 that was the goal of getting written down in hacking 01:48:41 * zehicle hacking is not the most obvious name for rules of engagement 01:48:53 hogepodge: D307 basically says "everything in the current schema's flagging section is required", so I think we're good there 01:49:09 I'm happy to obtain different opinions though. =) 01:49:14 markvoelker: I can't find what's in the schema though. I don't think it's documented 01:49:33 hogepodge: https://github.com/openstack/defcore/tree/master/schema 01:49:53 markvoelker: ah, ok 01:49:53 hogepodge, I moved it so we could track versions 01:50:08 since I was changing a lot of comments when I was fixing the readme 01:50:51 I just wanted to resolve if we felt there was a process change beyond hacking required 01:51:46 hacking needs more guidance along the lines of what I wrote, but is that part of what markvoelker is going to send up as part of his action item? 01:52:19 hogepodge: guidance about when a flag can be removed, or...? 01:52:22 I have no issues adding the rules 01:52:27 my concern was about where 01:52:47 2015B would be the right place if we wanted it at the board level 01:53:14 Hacking is right if we think we can hold the line 01:53:27 some flagging guidance should be in 2015B i think 01:53:32 but maybe not as detailed 01:53:40 zehicle: I'm of the opinion that this level of detail isn't of interest to the Board. But then, I'm not on the Board. =) 01:53:49 plus info about removing tests in next (or not) 01:53:50 +1 01:54:13 there are some items that the process leaves to defcore 01:54:24 hackings have always been a good "how do I work within this project" place to me 01:54:25 hogepodge: I'll definitely hit the removal of tests in .next in my patch 01:54:26 so sounds like everyone in favor of when to flag and what, when to remove to put hacking doc? 01:54:26 Agree. 01:54:36 test membership and flagging are both in that category 01:54:54 should we create a separate flagging process doc? 01:55:04 I 01:55:12 I'm ok w/ them in hacking - since it's central 01:55:17 ok 01:55:23 let's collect them there and see if we need more 01:55:23 eglute: I thought about that, but came around to kind of liking it in HACKING 01:55:35 five minute warning 01:55:47 as long as we all agree with the location, the name of the file works for me 01:55:48 Basically b/c I suspect the flagging process is an area where people not routinely involved in DefCore will want to interact with us 01:55:55 markvoelker ok, that works for me if we have consensus. most important to have them somewhere :) 01:56:04 and HACKING is, as dwalleck_ pointed out, where to look for info about how to interact with a project 01:56:17 easy enough to xlink in the readme 01:56:29 works for me then 01:56:34 ok, my topic on that patch took most of the time. sorry 01:56:39 +1 HACKING 01:56:53 markvoelker, +1 on adding a "list is comprehensive" statement 01:57:10 we don't want flag to be allowed because we did not think of a reason to block them 01:57:25 if a new reason surfaces then we can discuss 01:57:35 except for pixie dust 01:57:43 not up for discussion 01:58:10 flag neutron: because of pixie dust. 01:58:14 #topic is this time working? 01:58:31 just a check 01:58:32 zehicle: I'm also curious about the difference b/t D401 and D402. Didn't really seem to be one? 01:58:45 Hard for PDT ... family dinner hour. 01:59:09 did we get additional audience based on the time? 01:59:16 no, less people 01:59:17 markvoelker: I read D401 as "inadequately tests" and D402 as "test fails inappropriately" 01:59:20 time is ok for me...note that we have an AI to re-eval this in ~a month to see if it's working 01:59:27 dwalleck_, you are our rep from alternate time zone 01:59:36 dwalleck_ is in CST 01:59:40 yup 01:59:42 oh, sorry 01:59:52 That's pretty alternate. I've been there. 01:59:59 Though the nice thing about this time is that it's not possible to conflict with anything else :) 02:00:00 we do have someone from Australia 02:00:07 who was our 02:00:13 hughhalf: is in australian time zone 02:00:14 well, work conflict 02:00:17 purp: hrm. I guess I read both as "the test is borked". =) 02:00:23 it was hughhalf 02:00:29 hughhalf what do you think? 02:00:42 AEST - GMT+10 (Canberra Australia) 02:00:43 Sorry, stepped away, one mo please 02:00:55 I think you've answered the question 02:01:10 Doorbell rang! :) 02:01:15 Heh. 02:01:16 ok, we will try couple more times and then re-evaluate i think! 02:01:20 eglute, +1 02:01:27 any meetings following this? please tell us to vacate if so. 02:01:27 we agreed to give it a month 02:01:28 So yes, this time is ok for me 02:01:29 just wanted to test it at the end 02:01:35 not suggesting a change 02:01:55 * hughhalf nods 02:02:00 #action purp will ping Robert to see what times work better (due before end of June) 02:02:02 we're done 02:02:12 thank you everyone! 02:02:16 zehicle: did you want me to propose a new patchset for https://review.openstack.org/#/c/188661/3/HACKING.rst or did you want to? 02:02:24 any addition discussion, we'll be on regular channel 02:02:24 * hughhalf notest that fwiw a time that will work for Robert will likely work for most Oz folk too. 02:02:25 thanks all 02:02:37 markvoelker, go ahead 02:02:41 I was just getting it started 02:02:48 #endmeeting