16:01:15 <zehicle> #startmeeting defore
16:01:16 <openstack> Meeting started Wed Dec  2 16:01:15 2015 UTC and is due to finish in 60 minutes.  The chair is zehicle. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:20 <openstack> The meeting name has been set to 'defore'
16:01:26 <zehicle> #chair eglute
16:01:27 <markvoelker> o/
16:01:27 <openstack> Current chairs: eglute zehicle
16:01:32 <hogepodge> o/
16:01:37 <hogepodge> o/
16:01:39 <eglute> #topic agenda https://etherpad.openstack.org/p/DefCoreRing.4
16:01:47 <zehicle> #chair hogepodge
16:01:47 <openstack> Current chairs: eglute hogepodge zehicle
16:01:48 <Catherine1> O/
16:01:51 <zehicle> o/
16:01:55 <zehicle> #chair markvoelker
16:01:56 <openstack> Current chairs: eglute hogepodge markvoelker zehicle
16:02:45 <eglute> Hi Everyone, please review the agenda.
16:02:52 <eglute> please add/edit remove
16:03:58 <eglute> #topic should openstack require linux vms
16:04:27 <zehicle> #title defcore
16:04:39 <zehicle> sorry about messing up the title :?
16:05:03 <eglute> We had a lot of discussions about it, and we are also going to present this issue during the board meeting on Thursday
16:05:27 <eglute> I think we will live with that title for today :)
16:05:53 <markvoelker> So I guess the things we need to do are 1.) Finish up the doc and 2.) hogepodge was going to get this on the next TC agenda?
16:06:12 <markvoelker> Then wait for their feedback, make a final call on the review once we have it?
16:06:30 <markvoelker> (and follow-up actions obviously, if any)
16:06:36 <hogepodge> markvoelker: I'm writing up an agenda item right now for the meeting on the 8th
16:06:46 <markvoelker> hogepodge: thanks
16:07:00 <eglute> #meetingtopic defcore
16:08:01 <markvoelker> The board meeting is 12pm PST tomorrow, correct?  I'll make a point to dial in.
16:08:11 <zehicle> yes
16:08:15 <zehicle> we have 40 minutes on the agenda
16:10:06 <markvoelker> Ok, any other action needed on this topic right now then?  I think we just need to give ourselves an AI to review the latest changes to the doc from yesterday and make sure we've covered everything.
16:10:31 <eglute> not right now. unless there are any other comments on the issue that have not been voiced yet
16:10:50 <eglute> besides that, please provide feedback on the doc and possible actions
16:11:21 <zehicle> I suspect that we'll have a "get community feedback" AI from the BoD tomorrow
16:11:34 <eglute> well, we tried to get the community feedback
16:11:39 <eglute> and we got some
16:11:53 <hogepodge> I'd like to include a link to the doc to give the TC something to prep from.
16:12:00 <eglute> short of sending out a survey, i am not sure what else
16:12:07 <markvoelker> Perhaps we should point out in the doc that the review has been open for X amount of time and went out to the ML's on date X?
16:12:09 <eglute> hogepodge yes, you should
16:12:17 <eglute> markvoelker good point
16:12:18 <markvoelker> Just so the Board is aware.
16:12:22 <hogepodge> Or just include the full text in my mail, since that would probably be preferred to a number of people over viewing the google doc.
16:12:27 <eglute> right, we need to include it
16:12:39 <eglute> hogepodge you can provide the link, it is set to comments
16:12:56 <eglute> hogepodge or text/both
16:13:51 <eglute> do you think this issue has been communicated widely enough? we can ask foundation to send out a survey
16:14:18 <markvoelker> I'm not sure a survey is going to provide much more data than we already got, but I'm not opposed if they want to.
16:14:19 <hogepodge> I don't think we need a survey.
16:14:40 <hogepodge> It's a tool we don't want to use very often. There's a lot of survey fatigue in the community.
16:14:53 <eglute> agree
16:15:28 <markvoelker> Ok, so:
16:15:44 <markvoelker> #action everyone review/make final changes to doc
16:15:57 <markvoelker> #action markvoelker Add note about how we've solicited community feedback to doc
16:16:05 <markvoelker> Anything else?  Next topic?
16:16:21 <eglute> #topic Adjust scoring weights
16:16:49 <eglute> #link https://review.openstack.org/#/c/226980/
16:17:13 <eglute> this is something I would like to hear everyones opinion on
16:17:14 <markvoelker> Hmm...just realized I never added a second patchset to that one to show the impact of the changes on current scoring.  I'll do that.
16:17:37 <markvoelker> Would like to hear general feedback in the interim though.
16:17:39 <eglute> this RFC has been out for a while
16:18:01 <eglute> right, general feedback would be great. i would like us to change the scoring
16:18:18 <markvoelker> #action markvoelker Update patch to show potential scoring worksheet impact
16:18:23 <eglute> at least for the future direction
16:19:06 <eglute> does anyone else have an opinion on this patch right now
16:19:37 <zehicle> sorry - have not really doing a good review :(
16:19:42 <markvoelker> Please also note that part of this patch depends on this: https://review.openstack.org/#/c/226978/
16:19:52 <markvoelker> The latter changes the definitions of tools and clients a bit.
16:20:37 <eglute> i like the updated definitions
16:21:27 <markvoelker> Yeah, I think that one may be less controversial...it sort of came up repeatedly in the scoring we've been doing, so feels pretty natural.  But I may be biased. =p
16:22:06 <eglute> I agree with you... if everyone else approves, would like to get this one merged this week
16:22:11 <zehicle> I'm OK w/ this
16:22:46 <eglute> zehicle please comment on the patch as well
16:23:14 <eglute> regarding the weights- markvoelker will you submit a patch to show how it will affect current scores?
16:23:27 * zehicle done
16:23:33 <eglute> thanks zehicle
16:23:45 <markvoelker> YEah, I gave myself an # action on that above.  Should be able to knock that out today.
16:23:55 <eglute> thank you markvoelker
16:24:09 <markvoelker> Thought I'd done that weeks ago, so sorry for the lag. =)
16:24:14 <eglute> if there are no comments on the weights, we can move to the next topic
16:24:37 <markvoelker> One more thing
16:24:42 <eglute> ok
16:25:02 <markvoelker> If we like one or both of those changes, we should decide when we want them to take effect before landing them.
16:25:20 <markvoelker> I would assume at this point we'd let them take effect in 2016.07...
16:25:22 <eglute> I think we can land the tools definitions first
16:25:30 <markvoelker> (since we've already completed scoring for 2016.01)
16:25:36 <eglute> right, we would not re-score for 2016.1
16:25:59 <markvoelker> Should I just update both patches to remove the [RFC] and state in the commit message when they would take effect?
16:26:11 <eglute> yes, i think that would be good
16:26:37 <markvoelker> #action markvoelker Update patches to remove [RFC] and state that changes take effect in 2016.07 scoring
16:26:45 <eglute> thanks markvoelker
16:26:55 <eglute> next topic?
16:26:58 <markvoelker> sure
16:27:05 <eglute> #topic review patch https://review.openstack.org/#/c/246497/
16:27:18 <eglute> which looks like markvoelker has added +1
16:27:27 * markvoelker nods
16:27:29 <eglute> this is a small issue
16:27:39 <eglute> so, unless there are any objections, i will merge it now
16:28:26 <zehicle> catching up
16:28:39 <eglute> ok
16:29:26 <eglute> besides this, i didnt have a anything else on the agenda since i thought the linux issue will take up our time again. however, i would like to ask everyone to help out with the backlog
16:29:43 <eglute> we have some really old patches opened, some waiting for comments
16:29:46 <eglute> #topic backlog
16:30:34 <eglute> #link https://review.openstack.org/#/q/status:open+project:openstack/defcore,n,z
16:30:57 <markvoelker> This one could use a look or two.  hogepodge and a couple of RAX folks have chimed in, but we haven't come to consensus on the testing window...  https://review.openstack.org/#/c/232128/
16:31:27 <zehicle> where di we end up w/ https://review.openstack.org/#/c/233814/?
16:31:39 <hogepodge> I can add some comments.
16:32:00 <eglute> zehicle we are waiting on you i think
16:32:22 <hogepodge> In general, we want license agreements to last for a year, with the possibility of auto-renewal through submission of new test data within the year licensing period.
16:32:25 <eglute> there are a couple comments on re-wording https://review.openstack.org/#/c/233814/
16:33:04 <eglute> hogepodge so you are talking about optional auto-renewal?
16:33:25 <zehicle> I thought I'd addressed all the comments
16:33:32 <catherineD> hogepodge: what happen to the case when product upgrade to the newer openstack release ... is the license still good for the year?
16:34:34 <catherineD> say licesne was certified for Juno but product is now upgrade to Kilo .. is the license still good?
16:34:35 <hogepodge> We want the licenses to roll with the products, so if you upgrade to new releases and pass a recent guideline the renewal will carry forward.
16:35:09 <eglute> ^^^ interesting
16:35:15 <eglute> anyone knows whats going on
16:35:20 <catherineD> hogepodge: so vendor still needs to run tests against the update guideline
16:35:45 <eglute> ##meetingtopic defcore
16:35:52 <eglute> #meetingtopic defcore
16:35:56 <hogepodge> catherineD: yes
16:36:24 <catherineD> hogepodge: k thanks
16:36:50 <eglute> my only concern with rolling re-testing is requiring re-testing more than once a year
16:37:25 <catherineD> eglute: its does for vendors who keep up with openstack release in this case
16:37:25 <markvoelker> I think on that one I'd really like to hear from some other public cloud vendors.  The biggest impact of this is to them (and their consumers).
16:37:47 <dwalleck> It definitely encourages rolling DefCore testing into CI processes of public clouds
16:39:02 <SammyD> +1 dwalleck
16:39:18 <eglute> it does encourage extra testing, but i am still not a fan of adding extra burden of re-testing/re-certifying every two months or whatever the schedules might be for vendors
16:39:25 <zehicle> putting on a user hat - it's a major problem if a cloud goes out of DefCore compliance
16:39:28 <Rick_K> all: I'm new to this meeting, can you explain this license to me?  How it has an affect on a product when a new version comes out?
16:39:28 <hogepodge> the foundation doesn't want to be swamped with rolling test results, but having a way to check them with some frequency would help with license renewal
16:39:38 <catherineD> dwalleck: vendors most likely will have DefCore tests in their CI process ... the question is how often these results should be sent to RefStacke for certification purpose ...
16:40:00 <hogepodge> unrelated to the topic at hand, if you have any comments on the TC agenda item post leave them here. I'll send the e-mail after the meeting ends. https://etherpad.openstack.org/p/defcore-tc-dec-8-agenda-item
16:40:00 <Rick_K> all: or provide a link where i can read the details of it?
16:40:18 <catherineD> hogepodge: exactly ... RefStack should not be flooded with vendors interim test results ..
16:40:27 <markvoelker> Rick_K: sure, are you familiar with the Foundation's OpenStack Powered (TM) programs already?
16:40:40 <dwalleck> catherineD: Either continously or immediately running up to a push of public code
16:40:45 <hogepodge> Rick_K: This should get you up to speed #link http://www.openstack.org/brand/interop/
16:40:52 <Rick_K> got it, thanks
16:43:10 <eglute> since i am reading different opinions here, it would be really good if everyone commented on the patch as well
16:43:23 <eglute> https://review.openstack.org/#/c/232128/
16:43:45 <eglute> i will do so myself as well
16:44:13 <eglute> hogepodge would foundation want different schedules for different types of products?
16:44:54 <hogepodge> eglute: I think we want products to be tested annually, or when there is significant upgrade or code change
16:45:22 <eglute> hogepodge how would you define significant?
16:45:30 <hogepodge> new version, apis, and so on.
16:45:54 <markvoelker> hogepodge: new change to policy.json?
16:46:17 <markvoelker> hogepodge: new change to configs?  Playing devil's advoctate, those are one-liners that can break users.
16:46:22 <hogepodge> changing openstack version, changing apis
16:46:42 <eglute> #meetingtopic defcore
16:46:43 <markvoelker> And in public/some managed products, users don't get to determine when they consume those changes.
16:47:12 * markvoelker figures freenode must be getting attacked again
16:48:08 <eglute> yeah, keeps reseting to the previous meeting for some reason. will see where this meeting will be logged.
16:48:41 <Rick_K> all: on the testing, is this a question of the amount of workload on those that have to review the uploaded results, or on the vendors making the product to have to test relatively often?
16:49:25 <eglute> Rick_K both
16:49:33 <Rick_K> i c
16:49:43 <markvoelker> Rick_K: a little of both, but mostly the latter I think.  RefStack makes reviewing the results relatively easy (at the risk of putting words in hogepodge's mouth).
16:49:53 <eglute> that is why we are asking for feedback on https://review.openstack.org/#/c/232128/
16:49:57 <Rick_K> How quick is the review completed?
16:50:19 <Rick_K> eglute: got it, catching up, sorry if I am rehashing things a bit
16:50:20 <hogepodge> markvoelker: it makes reviewing results very easy. Night and day from what I was doing before
16:50:38 <hogepodge> markvoelker: it also puts a lot of trust in companies, because refstack results are really easy to cheat
16:50:44 <eglute> Rick_K no worries, hope you get a chance to review and provide feedback
16:50:58 <markvoelker> Rick_K: in my recent experience, we got feedback within a couple of days of submitting results.
16:51:13 <Rick_K> i c
16:51:23 <dwalleck> For a public cloud, the benefit to more continuous testing is that it surfaces issues immediately rather than finding it out every 6-12 months
16:51:37 <SammyD> +1 dwalleck
16:51:41 <hogepodge> Rick_K: I review the results and work with our web, legal, and marketing teams to send out the licenses and update the marketplace entries
16:51:51 <dwalleck> And the work to fix issues as they happen is almost certainly less than fixing them yearly
16:51:53 <eglute> dwalleck yes, but the public clouds can continuously test regardless
16:52:02 <hogepodge> two classifications are ok
16:52:35 <hogepodge> related to the cheating comment, I'd like for the guidelines and tests to evolve to actually be testable by consumers of clouds
16:52:46 <markvoelker> Just to point out, the patch sets up two schedules centered on when changes could break users...so it's essentially a user-centric approach
16:52:52 <hogepodge> so users and community don't have to trust results, they can check for themselves
16:53:25 <markvoelker> E.g. users of, say, a distro get to choose when they deploy updates or change configs.  Users of public clouds and some managed products don't, and may not really have any visibility into when those things change (unless it breaks them).
16:53:50 <hogepodge> right now our test suite doesn't allow that to be done easily (you need to tenant separated accounts and a large number of compute resources for each account, on one public cloud buying enough resources to pass defcore tests would cost me over $200/month)
16:54:10 <hogepodge> s/to tenant/two tenant/
16:54:31 <eglute> hogepodge so it is really the test issue. i think we had discussed this in the past, need to change tempest to be less greedy
16:55:05 * markvoelker glances at clock and notes we're down to about 5 minutes remaining
16:55:05 <hogepodge> eglute: I'm evaluating that right now with my own little cloud to see what tests we can target to slim down
16:55:45 <eglute> hogepodge let us know what you find
16:56:08 <eglute> besides waiting on more feedback from users, any other feedback on https://review.openstack.org/#/c/232128/ right now?
16:57:03 <eglute> if not, we can end the meeting 3 min early!
16:57:06 <eglute> or 2
16:57:24 <markvoelker> One other review of interest for DefCore folks: https://review.openstack.org/#/c/232371/
16:58:00 <eglute> i have looked at it, but did not dig in
16:58:06 <markvoelker> This is a glance spec that incorporates some of the feedback we've sent to glance concerning images and API versions in the past several months, so worth taking a look if you haven't.
16:58:08 <hogepodge> It's looking to build out a discoverable and interoperable api to capture all image upload use cases
16:59:31 <eglute> thanks markvoelker and hogepodge
16:59:38 <eglute> we are at time, thanks everyone
16:59:43 <eglute> #endmeeting