19:03:39 #startmeeting tc 19:03:39 Meeting started Thu Aug 14 19:03:39 2014 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:03:40 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:03:43 The meeting name has been set to 'tc' 19:03:55 This is an exceptional meeting to discuss the need for general changes in the Kilo release contents 19:04:00 o/ 19:04:11 is it really about kilo, or longer term? 19:04:11 The thread on the ML mentions two ideas in particular which I would like to explore (feel free to suggest others) 19:04:15 o/ 19:04:21 (strategic or tactical ourselves?) 19:04:26 annegentle_: well, longer term. But we need to discuss it now for Kilo 19:04:39 I think it's important we discuss specific suggestions here. 19:04:42 One idea is to freeze/reduce graduation to integrated release status (and potentially more incubation) until we get our shit together on the existing projects 19:04:53 Or as mordred said, "how do we scale our resources" 19:04:57 wait - no 19:05:01 how do we know when we have succesfully got "our shit together" ? 19:05:07 I disagree with that suggestion, please don't associate me with it 19:05:19 heh ok 19:05:19 The second is the idea that some projects are off-track and should be stopped. 19:05:23 or, as mordred said, "how do we ship good things" 19:05:25 :P 19:05:29 yes. associate me with that one 19:05:33 The deadline is that we need to come up with the projects that will be part of Kilo by September 16 19:05:49 And it's difficult to consider those graduation reviews until we made our minds on those questions 19:05:50 What drives that deadline? 19:05:54 We also have one incubation and one program request in progress, and our decision here affects the decision there as well 19:05:55 summit planning? 19:06:00 summit planning and elections 19:06:05 Ok 19:06:09 and PTl election deadlines as set by our charter 19:06:40 so. maybe mordred can statr wit hthe idea he likes to be associated with 19:06:43 so - re: the first one, I disagree beacuse I think it's based on an assumption that a large part of our shared resources scaling problems are tied to increase in projects 19:06:49 oh, was already typing the second thing 19:07:02 what we've seen is the opposite, really, trove and savana do not cause many problems at all 19:07:12 whereas nova and neutron are hard and takea lot of time 19:07:22 not really a measure for share resource like docs 19:07:22 so - while I appreciate that we should figure out cross-project scaling concerns 19:07:27 mordred: so you think it's not the nmber of things, it's the nature of htings ? 19:07:37 possibly 19:07:44 but I don't think there is a direct correlation 19:07:49 mordred: that's a decent framing but tough to measure 19:07:53 or that freezing somethign will necessarily solve something 19:08:07 so "Freeze" seem like solving a problem by inventing policy 19:08:07 mordred: i think the scaling of test resources is a direct correlation. as these projects add more tests, we'll see an amplification of any existing instabilities 19:08:10 and I think we have too much policy 19:08:18 and we should just be making decisions based on sanity 19:08:31 devananda: we ahve never experienced this being a problem ever 19:08:35 i think broadly speaking i'd agree with that. adding new projects certainly can strain resources, but it does not always do so, depending on the nature of the project, and the nature of what it brings to the table 19:08:43 jeblair: ++ 19:08:52 mordred: trove and sahara don't actually have a significant test suite in tempest yet 19:08:56 savanna is a great example... it came with a person who became infra-core for instance 19:09:00 sahara 19:09:01 dammit 19:09:05 So you don't find infra spending a lot of time hand holding new projects? 19:09:11 mikal: not really 19:09:26 I think that's what some people are worried about, the high infra workload 19:09:28 mordred: huh. except ironic. I feel like every time we ask infra for resources, there's push back. 19:09:36 But if its not new projects causing that, we're solving the wrong thing 19:09:38 the oslo team has set up many new repos and test jobs this cycle 19:09:39 so what would you consider as off-track projects ? projects that reinvent something rather than purely integrate 3rd party solutions ? and therefore bring specific complexity to the table? 19:09:42 devananda: you are a special bunny, because your requirements are very hard 19:09:45 mordred: ^ 19:09:59 ttx: well, can we get to some closure on this topic? this is just my opinion so far 19:10:04 mordred: I don't think I'm that special, but thank you 19:10:34 if new projects *were* causing strain, i'd rather raise the bar on what cross-project assistance they bring 19:10:36 i think what's true is that we don't have a lot of time to spend doing all the work ourselves anymore, but i also think that's less of an expectation now 19:10:37 rather than slow them down 19:10:39 devananda: we may not be giving you response times on requests that feel good, but I do not feel that your existence drains me 19:10:52 now we mostly try to work on setting new patterns so that projects can adopt them widely 19:11:01 yah 19:11:07 jeblair: i think you guys do an awesome job of it too :) 19:11:13 mordred: it's definitely true that the projects which reinvent something (Marconi, Heat, Ceilometer) rather than just integrating/provisioning existing solutions are up against additional complexity 19:11:45 mordred: ok, so which projects ARE a drain on you? I'm guessing, since it's not new ones, that you refer to nova and neutron here 19:11:58 mordred: but I don't know if that's actually what you mean 19:11:58 ttx: the issues with ceilometer right now aren't related to reinventing anything, afaik 19:11:59 for docs, we have strain, but we've purposely held some projects at bay, timing wise, but also, the projects have to bring their own docs resources 19:12:01 My belief was that they would overcome that complexity -- but maybe they were a bad idea to being with 19:12:05 devananda: yah. nova and neutron and gate breakages are the toughest issue 19:12:20 sometimes it's cinder that breaks the gate, one time it was swift 19:12:31 but that's the stressful and burn-out inducing stuff 19:12:56 ttx: ok - so it sounds like infra's concern is around stability in the "major" projects 19:13:02 annegentle_: +1 19:13:11 which leads to ... 19:13:17 what I REALLY am concerned about is the quality of our code 19:13:26 mordred: that 19:13:26 dhellmann: ceilometer was rearchitected several times over because it reinvents how to store metrics -- they may be on a good trail with time series now, but the fact is they had to rewrite themselves a number of times 19:13:29 and that we ship good stuff that solves needed problems 19:13:37 mordred: ++ 19:13:51 mordred: but your concern with quality has nothing to do with more and more scope? they seem related... 19:13:58 there's a proxy battle I want to mention first - or maybe proxy battle is the wrong word ... 19:14:07 ttx: no, there were schema changes, that's no the same thing. right *now* there is a rearchitecture going on for future improvements, but that's new this cycle 19:14:08 mordred: I certainly owrry that nva has way too many open bugs, but we haven't found a way to get people to work on that yet 19:14:11 annegentle_: i think they are related, just not _directly_ related 19:14:11 which is that I don't personally like telling my friends that I don't think their stuff is good 19:14:12 mordred: how do you propose we solve that ? 19:14:17 and I don't think any of us do 19:14:18 jeblair: correlated perhaps 19:14:23 and I like all of you people 19:14:29 i like you too 19:14:48 group irc hug. 19:14:49 so it's hard for me to say to flavio that I think marconi is a bad architecture and a bad idea, even though as a TC member I should be doing that 19:15:04 mordred: do you see any relationship between "making the code we ship better" (ie, nova, neutron) and "adding, or not adding,, more projects to the fold" ? 19:15:27 I don't think WE are judge and jury for quality necessarily, the tests and users are... but we're (TC) a governance bottle neck 19:15:31 devananda: I don't, because the crossover isn't that great - so not adding designate will not shift any resources to improving nova 19:15:37 right 19:15:46 annegentle_: I think I would like to change that 19:15:57 mordred: ok. so we're havign two conversations in parallel -again- :( 19:16:03 devananda: yeah. I blame ttx 19:16:06 heh 19:16:10 mordred: I realize that, but your increase in scope of OpenStack governance is what's tough. 19:16:11 * ttx looks the other way 19:16:13 he wouldn't let me finish topic one 19:16:23 annegentle_: indeed. and that is an excellent 19:16:25 point 19:16:46 it's entirely possible that the shared resource that doesn't scale is our ability to properly judge good software that we're shipping 19:16:52 * ttx lets mordred finish his point 19:17:07 ttx: oh, cat's out of the bag at this point 19:17:13 heh 19:17:42 but, in my view, there are general views on things that it seems most people agree with but we haven't figured out how to express 19:17:53 the topic of how we might decide such things in the future notwithstanding 19:18:07 jogo straw man to the 19:18:09 gah 19:18:25 jogo's straw man to the mailing list is a good example of this - although I dont' necessarily agree with all of it 19:18:46 mordred: one problem is.. we tend to judge quality at graduation time, not incubation time -- and after having forced people to jum through hoops for one or two incubation cycle, just saying, sorry, not good enough, is tough 19:18:52 so i think i heard that adding new projects isn't the big concern ... but the important issue is doubling down on quality, and dealing with projects we don't think are working out? 19:18:58 I think some of jogo's assumptions are faulty. 19:19:31 russellb: that's _my_ concern 19:19:31 russellb: ++ 19:19:36 I think its certainly true that there are projects I have concerns with, and I am sure most other TC members have concerns about something too (not nessesarily the same project as me) 19:19:39 OK. 19:19:40 ttx: didn't we decide to be lenient on incubation to grant "credibility" to attract contributors? maybe we should revisit that 19:19:49 I think that incubation should be the inclusive time, let many many many similar projects be incubated. then we hold the line at integrated so we don't pay so much "integration tax" in cross-project resources. 19:19:52 so, let's see if we can be concrete here ... 19:19:58 are there projects we need to have a future conversation about? 19:20:02 I have the impression that, at this point, everyone agrees the TC has a responsibility to ensure that we're shipping good quality software 19:20:04 some increased quality expectations for all projects? 19:20:05 russellb: ++. i'd clarify that i think we should be very selective about new projects, just not say we're going to stop altogether. 19:20:05 dhellmann: yeah, I think we focused a LOT early on on ensuring that we grow our community 19:20:11 annegentle_: erm, no? I am pretty sure that should happen on stackforge 19:20:16 I don't think that's actually a problem we need to actively attack now 19:20:16 russellb: future conversation about if they're off track? 19:20:28 jeblair: sure, i think that's always the case, and that we need to continue to refine our documentation expectations 19:20:31 annegentle_: the point of incubation is to /start/ the process of integrating things and pick one taht other projects should integrate with 19:20:32 devananda: ah yes incubation means past stackforge, right? 19:20:34 mikal: yeah. 19:20:36 annegentle_: right 19:20:44 I'm not sure the concerns about projects are always about quality 19:20:46 annegentle_: currently-incubating projects have gone through a lot to alig to our graduation requirements, so we'll have to be strong to tell them... you ticked all the boxes but we just realized this was a bad idea to begin with 19:20:49 so perhaps this doesn’t solve the actual scaling problem, but I wouldn’t mind making the integrated release only about infrastructure projects 19:20:50 Perhaps they're sometimes about overall direction for example 19:21:03 vishy: ++ 19:21:15 but yes, we're still not seeing that lots of incubation is problematic, but that certain projects and cross-projects are problematic? 19:21:17 i.e. anything that is best deployed on top of iaas doesn’t belong in the integrated release 19:21:21 vishy: I have a similarly but slightly different take - mainly because "infrastructure" is a bit vague 19:21:24 vishy: we'll never force you to ship my blog hosting software that way! 19:21:24 and we just do IaaS really well 19:21:37 vishy: yep that's what I'm voicing as well 19:21:51 vishy: I think I'd like to see us stop implemeting things that are data plane services 19:22:09 openstack's mission sattement says "massively scalable" - any project applying for incubation that can not already demonstrate that it is "massively scalable" 19:22:14 shouldn't be incubated, IMO 19:22:17 with the notable exception of swift, because that's a thing that seems to need to exist in clouds 19:22:18 my sniff test is that if I would be totally happy deploying something on top of nova for production use it doesn’t belong 19:22:22 ttx: I have experience with that conversation 19:22:26 as taht is the inflection point when we begin devotign cross-project resources to it 19:22:34 vishy: well, thing is - it's actually rEALLY nice to not have to run mysql inside of nova 19:22:40 and incubation is a "blessing" of a project in the eyes of the community 19:22:43 is there a non-integrated place for non-iaas, non-massive now? 19:22:48 (I think there's not) 19:22:59 mordred: for performance reasons? 19:23:01 vishy: a shared ops team can handle 10000 mysql databases well better tahn I can handle one (well, not me, I'm an expert and all) 19:23:09 vishy: for performance, and also for operational complexity reasons 19:23:16 but I think I'd call mysql "infrastructure" 19:23:20 mordred: but we don’t have anything like that 19:23:35 trove? 19:23:39 I _could_ run it in nova, but I'd rather not worry about any of the details and just get a mysql from my cloud 19:23:47 mordred: if you are referring to trove, it doesn’t pass my smell test 19:23:55 mordred: i think you are missing my point 19:24:00 i’m speaking as a deployer 19:24:01 vishy: quite possibly :) 19:24:05 if i go to deploy trove 19:24:06 vishy: ah, I'm not 19:24:12 I'm speaking as a cloud end user 19:24:15 I do not deploy clouds 19:24:17 I use them 19:24:22 i’m happy to deploy it on IaaS 19:24:25 i made the same assumption as mordred 19:24:34 IaaS vs PaaS ? :) 19:24:37 no 19:24:41 those terms are meaningless 19:24:46 so you're saying sure, Trove should exist, but not in OpenStack proper? 19:24:47 theya re marketing jargon that is not usefuk 19:24:55 it is in cloud “user-space” as i think of it 19:24:55 I still think it is super useful 19:25:04 I'm saying that trove is more useful to me in a cloud than swift is 19:25:04 I just don’t think it needs to be in the integrated release 19:25:14 i think if something is useful, we should do our best to foster it 19:25:14 cuz our scope is getting totally out of control 19:25:17 as an end user 19:25:20 and that thus far has been to bring it in to the release 19:25:26 vishy: i think there's the rub -- it is useful, and standardizing on it is useful, but it's hard to do that if it's not in openstack 19:25:33 jeblair: ++ 19:25:34 jeblair: +1 19:25:36 ++ 19:25:47 jeblair: +1 19:25:49 russellb: we can’t do everything though 19:25:54 russellb: foster - yes. include in the integrated release - maybe not. 19:25:57 we're clearly not doing everything :) 19:26:10 include in the release is the only thing we have, really 19:26:41 I think there ought to be a place for trove, sahara, and other things that naturally fit "on top of" openstack from a deployer's POV 19:26:45 I feel like this discussion is all over the map 19:26:50 +10000 19:26:51 that’s a problem though. I don’t think docker will ever be part of openstack but it is rapid ly becoming a standard for building applications 19:26:51 i'm not sure what we're trying to accomplish or answer right now 19:26:52 ++ 19:26:52 russellb: +1 19:26:59 mordred: at this point i've lost track of your original point 19:27:07 devananda: ++ 19:27:07 my point is that we're shipping crappy software in places 19:27:10 and I want to stop 19:27:19 agreed? 19:27:19 heh 19:27:20 and I don't want to do that by defining more policy 19:27:27 any suggestions? 19:27:30 (which is why I'm pushing back on vishy's point) 19:27:36 do we need better developers instead of fewer? 19:27:42 mordred so how do we force non-crappy software? 19:27:51 "all" we have to do is get to vishy's scope, which is where I'm at too 19:28:00 vishy: I think we need to be willing to take specific stands on specific issues rather than making policy to back us up 19:28:01 i think it's clear everyone *wants* that, and is trying 19:28:02 so 19:28:04 for instance 19:28:07 annegentle_: well imo they are different concerns 19:28:07 mordred: your point though is about the crappiness of major projects (noav, neutron, cinder, swift are the only four you mentioned breakages of) 19:28:14 annegentle_: well, I'd argue that some of the in-scope software isn't great, either, so I'm not sure that solves it. 19:28:21 annegentle_: there was a time when nova would fit into the “crappy” category imo 19:28:28 mordred: and you haven't addressed the wealth of recent project additions / applications 19:28:30 I think we should reject marconi, and I think we should deintegrate ceilometer, even though I like dhellmann, and I think most of us agree with that already so we should say it 19:28:33 heh vishy yeah I get that 19:28:47 I do not at all agree with that. 19:28:47 I think part of the problem with the large projects is the we've set the system up for folks to land their code in master at cost (including quality) 19:28:51 mordred: if that's the core, we need to get to it and discuss that 19:29:00 and I thnik we should be willing to be harsh, but individually harsh, on new suggestions 19:29:03 mordred: the problems in ceilometer are being fixed this cycle 19:29:18 The TC has also had some success at identifying specific bits of "crappy" and pushing on them to get fixed. nova-network / neutron for example. 19:29:22 dhellmann: that the new time-series effort is being spun up on the side indicates to me that they are not 19:29:30 mikal: right, i feel like that has been going well 19:29:32 markmcclain: ++ 19:29:35 if we are airing out things that shouldn’t be in, I’d point to sahara as well 19:29:40 So we could start identifying hero projects that affect quality and push on them harder 19:29:41 dhellmann: since one of the problems is blatantly ignoring existing good software in the space 19:29:43 mordred: I'll be happy to fill you in on how wrong you are when you have time 19:29:47 no, it is not being ignored 19:29:48 dhellmann: ok! 19:29:55 so there is the IaaS view, the IaaS+ view that doesn't include data plane except swift 19:29:55 and the current stuiation which is mostly: if you reached the bar of quality and can pretend to be IaaS+ you're in 19:29:55 quality being solely defines by our graduation criteria 19:29:56 * ttx is having network issues 19:29:59 there is a driver API in place 19:30:20 Although, if we're going to list things we should discuss as possibly dropping, I'm going to win friends and say that I think tripleo is concerning 19:30:42 i think it's fine if we want to discuss specific projects, but this meeting isn't that time 19:30:55 russellb: I agree, but felt left out 19:30:56 folks need time to go off and write up good detail on concerns, and be able to prepare a response 19:30:58 mikal: right. and my main point is that rather than defining IaaS or IaaS+ or whatever, we shoudl just bring those things up directly 19:31:01 russellb: +1 let's put down the torches and pitchforks 19:31:03 ok, I think I caght up, sorry network lag 19:31:05 russellb: +1 19:31:06 markmcclain: exactly 19:31:10 russellb: ++ 19:31:17 So, backing up a bit 19:31:19 * russellb puts up a "no pitchforks or torches" sign at the door 19:31:22 We think we have quality issues 19:31:33 We had some success with pushing on the nova-netowrk / neutron thing 19:31:43 Do we think that's a thing we could do more for other quality issues we see? 19:31:50 mordred: ok, so let's discuss the forst patr -- we should not approve Maconi. Why ? because it's data plane? Or because writing a queue is hard ? Or because writing a queue in Python is stupid ? 19:31:53 mikal: i think so 19:31:55 first part* 19:32:01 mikal: and i think that's perfectly in scope for the role we (TC) should play 19:32:08 russellb: because that's a very tangible, positive thing we could do 19:32:14 mikal: ++ 19:32:30 ttx: in my POV, all of the above, but that's just me 19:32:31 mikal: yes. I think the TC has a responsibility to ensure the quality of the code we're shipping 19:32:32 russellb: and which the board aint going to do for us -- we're the global oversight of quality as best as I can tell 19:32:40 mikal: whether that is done by "strongly encouraging" projects to clean up 19:32:42 ttx: I REALLY want to try to scale back places where we've NIH'd things 19:32:45 ttx: or because there are community issues around working with the team, which seems key to fixing some of the other issues 19:32:47 would it help to have an arch/code subset of TC that does a deep dive? if so we should apply retro all project like we did w/ grad requirements 19:32:51 mikal: or by rejecting / not accepting projects of dubios quality 19:32:53 mikal: absolutely, technical concerns are our domain. 19:32:57 Well... 19:33:09 We really only have one lever -- whether or not something is integrated / incubated 19:33:15 yep 19:33:23 I feel like we effectively said "fix this our you're out" with the networking thing 19:33:24 mordred: we have to decide in it's the addition of those issues that make us reject, or if it's one in particular 19:33:28 mikal: we could write a level factory ... 19:33:32 Which is a very very big stick, but hurtful to use 19:33:37 mikal: ++ 19:33:39 mikal: yes, pushing for change is a better first step than delisting projects 19:33:41 But we could experiment 19:33:48 What's the next most broken thing in openstack quality wise? 19:33:55 dhellmann: but pushing for change is only possible if we raise that threat 19:34:03 devananda: no, that's not true 19:34:09 dhellmann: i dont know of another way the TC can compel developers. what am i missing? 19:34:20 devananda: some teams do not shy away from saying that their code is broken (see ceilometer) while others do (see neutron) 19:34:24 Well... elephant in the room 19:34:33 We represent senior devs at a large portion of the openstack companies 19:34:43 Can you really not ring people's bosses and tell them to take a problem seriously? 19:34:47 dhellmann: I've think we've acknowledge there are some broken bits 19:34:51 Cause I would 19:35:02 markmcclain: true, it took a while to get some vendors on board with that, though 19:35:23 Let me pick on another example for a second 19:35:27 that's the issue is that we've stacked the incentives for vendors to push new code 19:35:29 mikal: but the carrot is such a juicy carrot (being incubated) 19:35:29 Nova is sad that cvells is unfinished 19:35:31 dhellmann: Some still aren't on board, but we're doing our best. 19:35:37 cells even 19:35:45 So, we're going to push on that to get it fixed 19:35:50 mikal: while "document cells" is not that juicy 19:35:53 markmcclain, mestery : yep, it has gotten better this cycle. My point is that some teams needed less incentive, that's all. 19:35:55 The lever we have is removing the code 19:36:02 dhellmann: Understood 19:36:21 The TC could be stepping in an publicly identifying cells completeness as an issue of concern if we wanted 19:36:30 Which would help drive resources to the problem 19:36:38 so on the neutron front we've been working up a plan to incubate features long before we merge them for integrated release 19:36:42 having 2 majorly different ways nova works is indeed sucky. 19:36:45 mikal: sure 19:36:50 mikal: do you need that "cover" for us to do that, or is the problem being addressed? 19:36:59 dhellmann: its not a perfect example 19:37:12 dhellmann: what I am trying to say is if the TC had identified that problem, not the nova-core team 19:37:16 mikal: ok, fair enough, if you were having trouble I would agree that the TC could back you 19:37:22 sure 19:37:23 Then the TC could have gone to the PTL and ask what cover was needed 19:37:42 "We've noticed that XXX is a big crap. What do we need to do to help you fix that?" 19:37:47 that's a much saner proposal than kicking the whole project out 19:37:49 mikal: the board of directors is pretty much constantly saying "where can we put resources to make things better". perhaps the tc giving them that feedback would be useful. 19:37:57 jeblair: ++ 19:37:58 jeblair: ++ 19:37:59 so is this discussion about what we can do with existing projects other than kick them out? trying to make sure we're still on topic.. 19:38:03 jeblair: ++ 19:38:19 russellb: I think it has become that - which I actually think is kinda nice 19:38:23 russellb: I think its about working out how we identify quality issues, and then how we get them fixed 19:38:27 mordred: I still want to understand if it's the addition of issues that would make us reject marconi, or a specific one that we would, starting from now, consider as inadmissible 19:38:29 cool, yeah. 19:38:29 to me, it's stop hiring coders and hire doc writers though... so many ways to solve the "what do we throw resources at" when you look at cross-project needs 19:38:31 russellb: the kicking out being the nuclear case 19:38:37 so we've gone from talking about marconi and ceilometer to cells 19:38:50 annegentle_: so that's anotehr good example 19:38:54 mikal: has that approach worked for marconi? 19:38:56 devananda: I think it might be the same thing 19:38:59 annegentle_: should the TC be pushing on PTLs to not merge undocumented features? 19:39:16 mikal: thing is, some things can't be fully documented until they're in production 19:39:19 mikal: but yes 19:39:31 mikal, annegentle_ : +1 19:39:35 mikal: we don't even have that hard of minimum requirements for docs 19:39:36 annegentle_: so we kind of do that with docimpact (well, try) 19:39:45 annegentle_: do we have data on the number of open docimpact bugs 19:39:45 ? 19:39:56 Or is the problem more systemic than that? 19:40:03 mikal: yes, we have systems in place, but if "high availability" is for example the biggest need from the board, then that's docs more than features 19:40:12 Ahhh, I get you 19:40:17 mikal: it's that it's not a one-to-one code-to-doc mapping 19:40:18 devananda: if a collaborative approach doesn't work with a team, that means there are other issues, and I think then it's appropriate to take the extreme measures you're proposing. but we should not reach for the eject button as a first step 19:40:19 mikal: right 19:40:45 dhellmann: ++ 19:40:51 dhellmann: i definitely agree it's not the first step, and don't think it's been used that way 19:40:55 What I'd like out of this meeting is -- do we do any change for Kilo. 19:41:03 I think a more and better feedback cycle is good 19:41:13 I guess another way to look at it is: if we're using 800 servers for infra, and that's quota-limited, then does it make sense to limit what we integrate for other quota-limiting reasons? 19:41:14 ttx: ++ 19:41:14 So... As a concrete proposal. Why don't we identify the three most important quality problems we see in openstack and would like fixed in kilo? 19:41:18 mordred: and more willingness on our part to tell projects "we have serious doubt about your code" 19:41:22 ttx: it sounds like I'm hearing "no, we should be more active in telling people how they suck and give the a chance to fix it" 19:41:24 when, in fact, we do 19:41:25 devananda: ++ 19:41:29 devananda: it is coming across right now that it is the only step people want to use 19:41:30 mikal: yes, something like the round of what we just did, but focused on quality issues 19:41:31 which also means 19:41:34 like, to be VERY direct about it 19:41:37 Do we make a decision that woul affec tthe current incubation/graduation 19:41:38 we need to make time to seriously look into projects 19:41:41 because it doesn't help anyone not being very direct 19:41:42 mikal: though that's not so different from what we did ... quality was included 19:41:42 mikal: ++ 19:41:42 before graduation 19:41:46 and I would say, even before incubation 19:41:47 I have to raise another cross project concern very relevant to neutron/kilo 19:41:47 russellb: yes, but a small list for the first go at it to keep the experiement conteolled 19:42:02 * mikal can't type at 5am it seems 19:42:06 mikal: ++ 19:42:10 mikal: understandable 19:42:20 sorry, got the time wrong 19:42:24 * markmc looks at eavesdrop 19:42:25 Its cool 19:42:26 This is important 19:42:37 Oh, I see, ignore me 19:42:40 mikal: yeah that follows what I suggested earlier 19:42:47 mordred: I'm fine with that (being very direct). But that means we still apply the same rules for Kilo incubation/integration, right? 19:42:54 dhellmann: i think, for some projects, after a few cycles of not making headway / seeing what appear to be the same problems, folks feel like that is the only resort left 19:43:17 ttx: our incubation rules are a minimum guideline; i never thought the understanding was if you check the boxes, you're in 19:43:19 devananda: yep, but I disagree that that applies to some of the projects that were proposed to be kicked out 19:43:24 mordred: also it seems a bit far away from "data plane is a bad idea" 19:43:27 ttx: yes - except I think ultimately, for my money, we need to be willing to go past the rules a little bit when we're looking at things 19:43:30 jeblair: ++ 19:43:52 ttx: we may get tehre - "data plane is a bad idea" is just another arbitrary policy 19:43:58 mordred: the requirements ALWAYS were the consensual and predictable rules 19:44:02 not a set of checkboxes 19:44:04 jeblair: right but that message isn't universally understood in the community 19:44:14 that communicates minimal expectations, not ALL of them 19:44:20 otherwise we don't need to vote 19:44:31 yes. but to markmcclain's point, that consistently gets missed in this community 19:44:43 It's still OK to say "NO, just because" 19:44:47 yes 19:44:54 dhellmann: fair. I think we should have a discussion about each project 19:45:08 blueprint approval is another case where a minimal policy isn't understood to be reversable 19:45:10 devananda: ++ 19:45:12 It will piss off people, but we are all adults 19:45:27 the TC can also get its share of hate mail, like pTLs! 19:45:33 ttx: is it ok to just say no? I think there's a sense of "well, if they check all the boxes, we have to accept them" 19:45:39 ttx: perhaps we should be clear that we're willing to piss people off 19:45:44 ttx: we need to make sure it's not a constantly moving target, too, though 19:45:46 devananda: I think we need to make it clear to people that it's ok 19:45:47 devananda: definitely not imho 19:45:51 * reed fills in a request for asbestos suits 19:45:54 mordred: if there's consensus that it's for the best for openstack, sure 19:45:55 for us to just make a judgement call 19:45:57 ttx: and so people are looking for a policy to uphold, rather than have to make a decision, because not everyone has the time to invest in learning/assessing a project for themselves 19:46:20 reed: I'm allergic 19:46:38 here's the thing though. 19:46:42 we should probably make sure some subset is doing a deep dive on *every* project we look at ... 19:46:44 devananda: I think it will be hard to come back to a project today and tell them the whole idea was bad to begin with. 19:46:47 instead of assuming it happens 19:46:47 it turns out, diggign into a project that you didn't write and haven't deployed, deeply enough to decide "this is good" or "this is bad" is not often possible 19:46:53 and I think the only way we can do that 19:46:58 russellb: +1 19:46:59 because we incubated them on the premise that they were interesting 19:47:05 russellb: ++; also, i don't think it has to just be the tc 19:47:08 is get feedback from major operators (ie, our employers) about what works 19:47:10 and what doesn't 19:47:12 jeblair: true 19:47:20 but we need to be more clear that it's happening 19:47:22 which is why, on the ML, I suggested we set the bar to integration to be "is in production at scale" 19:47:25 for docs, I still document horizon, even though it's not IaaS. If I freed up those doc resources, would neutron be better documented? I'm not picking on those projects but just pointing out the priorities we're forcing by not narrowing scope. 19:47:29 some folks who are not in the tc have provided some really good feedback on new projects, and i appreciate that very much 19:47:36 jeblair: ++ 19:47:44 jeblair: ++ 19:47:48 jeblair: ++ 19:47:57 jeblair: ++ 19:47:59 jeblair: yes. I think we need that feedback. 19:48:00 devananda: ++ on getting feedback from operators 19:48:11 jeblair: I dont think we can make decisions without it, in fact 19:48:21 well, I think we _can_ 19:48:27 heh 19:48:29 I think we can make negative decisions without it 19:48:32 annegentle_: maybe we should have the board talk to companies to hire more writers to work with your team? 19:48:35 positive decisions are harder to make without it 19:48:36 sorry. i mean not well informed ones :) 19:48:48 devananda: I can make a decision that something is a bad idea with an operator 19:48:58 but blessing a thing without some proof points is much harder 19:49:03 right 19:49:05 dhellmann: yes, board members are quite willing, but it's still the scope problem and ratio of coders-who-dont-document 19:49:14 Do we also agree that we can't remove from the integrated release wihtout some form of failure to meet some conditions we've set? 19:49:22 ttx: ++ 19:49:25 ttx: ++ 19:49:29 ttx: wait, parsing double negative. 19:49:32 ttx: i don't agree with that :) 19:49:42 ttx: we have integrated and core projects that are not iaas 19:49:43 ttx: can you offer a strawman of such a failure of such a condition? 19:49:50 jeblair: we can still have unreasonable conditions :) 19:49:55 * devananda returns to formulating thoughts before typing quickly 19:49:57 I think we could remove them, but I think it would be a terrible idea to do it. 19:50:00 devananda: sure 19:50:00 devananda: IMO, failure to fill major gaps identified in project review 19:50:14 ttx: i think we can be arbitrary and capricous -- however, i think the suggested approach of trying to focus on specific quality issues with specific projects is the approach we should take. 19:50:22 jeblair: ++ 19:50:22 jeblair: ++ 19:50:24 yes. that 19:50:31 we review a project, identify gaps, expect a plan to fix it ... if that doesn't happen, removal seems like a reasonable thing to consider at that point 19:50:36 can and should are good words 19:50:37 russellb: is "not usable at a scale > 100 nodes" a sufficient gap? 19:50:42 devananda: mordred was mentoining deintegrating ceilometer. I don't think we can do it wthout ceilometer failing to meet the goals of a corrective action plan 19:50:49 Here's what's happening now though... couple of examples. Ceilometer is highly concerned at the docs teams reviews, thinking WE will prevent them from graduating. 19:50:51 and, frankly, those quality issues should not just apply to code, but how well the team is actually integrated with the rest of the project 19:50:52 ttx: I think we CAN 19:51:04 Same for neutron. As if the docs team is preventing them from graduating. 19:51:05 ttx: I think it's clear we SHOULD try to do somethign else 19:51:10 THIS is what concerns me. 19:51:23 the docs team cannot be responsible for a team's pass or fail. 19:51:28 it's the team's responsibility 19:51:34 annegentle_: ++ 19:51:36 mordred: ok 19:51:37 annegentle_: ++ 19:51:40 sure we CAN anything 19:51:44 annegentle_:+1 19:52:10 all of the cross project teams have what amounts to a pocket veto of any initiative of another team, though 19:52:12 ttx: agree with your earlier comment about corrective action plan, that's the sane approach to these things IMO 19:52:22 My point is, we SHOULD keep ceilometer in Kilo because they didn't fail at any challenge we communicated to them 19:52:26 ttx: I think we need to make it very clear that we CAN ... it's important because one day we may actually need to do it, and I don't think we want to spend a ton of time talking about whether we followed our own process first if we do 19:52:30 dhellmann: true, quality standards and so on 19:52:59 annegentle_: well, if I write bad docs that's my fault, but if you don't have enough resources to review what I write is that my fault? yours? someone else's? (I don't think it's yours) 19:53:00 dhellmann: that pocket veto is not nearly as strong as you might think ... I've tried and failed to use it several times 19:53:13 * dhellmann looks longingly at his cross-project testing patch to ifnra 19:53:42 dhellmann: reviews are also the team's responsibility especially for technical (cross project teams can't know everything about every project, unpossible) 19:54:01 dhellmann: that applies for infra and test too as far as I can tell 19:54:11 there's a sense of urgency here that I understand for evaluating graduation request 19:54:26 ttx: can you list again the requests coming in? 19:54:27 mordred: ack 19:54:27 OK, time check, 7 min left 19:54:28 do we need an extra exceptional meeting before condering incubations/graduations 19:54:28 or does this clarify what we SHOULD and SHOULD NOT do ? 19:54:28 not quite clear where it's coming from on the potential for de-integration 19:54:32 annegentle_: I can't +2 anything in infra or docs, though. I can review changes, but I can't actually move the work forward without help. 19:54:36 annegentle_: we've had several patches languish in -infra for month(s) with +1's from ironic cores 19:54:38 markmc: ++ 19:54:43 annegentle_: so i dont entirely agree there 19:54:51 maybe the urgency on that part wasn't intentional? 19:54:58 like, I do think we should always be open to de-integration discussions 19:54:58 OK so.. incubation: Manila. New program: rally. Graduation: Ironic, Marconi, Barbican (Designate is a bit young) 19:55:04 feel the way was always open 19:55:23 mordred: so as much as new projects aren't breakign the gate, they are bottlenecked on -infra to progress, eg. with test harnesses in devstack/tempest/config 19:55:29 yup 19:55:37 I don't think incubation in general should be held up 19:55:48 I feel like that provides a natural and non-policy based gating function 19:55:50 devananda: i don't want to ignore you if you have an issue to raise, but i think pursuing that discussion right now would be a distraction. can we resume that in -infra after this meeting? 19:56:00 ttx: for graduation should we get a subset of folks to dive through the programs that want to graduate? 19:56:08 has this discussion radically changed how we'll approach the graduation discussions? 19:56:08 markmcclain: we should still only incubate stuff we think is a good idea 19:56:17 ttx: yes 19:56:20 jeblair: sure. i dont mean to bring it up - merely responding to annegentle_'s comment and an earlier one from mordred. 19:56:21 not just the ones tha thave a devstack job 19:56:27 markmc: radically? no, not for me 19:56:31 devananda: dhellmann: that's why the scope and priorities matter so, so much to us. 19:56:32 not sure it's changed it at all for me 19:56:36 markmc: I would say no 19:56:37 we should always be open to evolving our requirements, but is there something more going on this time around ... ? 19:56:39 ok 19:57:00 devananda: I have no idea how he does it, but SergeyLukjanov joining and taking an active part in infra for non-sahara things was kinda huge ... and tips some scales in his favor in terms of making sure sahara is taken care of 19:57:02 care even more about quality before graduation? 19:57:11 is that the concrete thing? 19:57:18 russellb: i would hope so, yes 19:57:19 markmc: I think it's clearer that the requirements are just the minimal checkboxes the project has to tick, but it still needs to convince enough TC memebrs that it's good 19:57:24 if more projects wound up providing an infra person who did more than just care about that project, it would be AMAZING 19:57:27 russellb: though i have always cared about that 19:57:28 russellb, I think that's just a sign that we're learning 19:57:33 devananda: dhellmann: we are a quota-limited gate and there are even more docs reviewers than infra 19:57:34 markmc: agreed 19:57:36 I always thought like that, so it doesn't change my approach 19:57:38 have a little more confidence that saying no is 'okay' :) 19:57:40 mordred: ++ 19:57:44 i hope we learn and refine our expectations every cycle 19:57:45 ttx, yeah, same 19:57:52 annegentle_: "quota-limited"? 19:58:10 but yeah, it's OK to say no 19:58:15 russellb: do we have any more concrete means to guage the technical fitness of incubating projects, though? 19:58:17 mordred: ++ to projects providing contributors to the cross-project teams 19:58:18 we definitely said NO a lot more in the past 19:58:20 also, we've in this meeting tested vocally the idea of ejecting somethign and it seems that folks generally feel we should engage with the projects before doing so 19:58:32 russellb: I think I'd like to see that. and I think I've proposed one, but it hasn't gotten much traction 19:58:36 mordred: ++ 19:58:59 I don't think the conversation has made me necessarily more inclined to say no to any of these projects than I was before, though 19:59:13 trying to figure out what the conclusion here is 19:59:13 devananda: link? 19:59:19 mordred: yes, but I think the "no data plane" idea sets a hard limit 19:59:19 doesn't help that I suck and missed most of it :( 19:59:21 dhellmann: as in, you have a quota of this many doc changes can get in 19:59:29 ttx: I'm not sure we agreed to that 19:59:32 i don't know the conclusion either FWIW 19:59:36 ttx: nor did it seem to resonate with anyone 19:59:42 annegentle_: you limit how many changes a person can make? 19:59:51 mordred: should that be a guideline of yours, or try to be a common rule ? 19:59:55 dhellmann: we do, because we have few reviewers 19:59:57 mordred: i'd agree with that assesment; but i think the points that you made around that will influence my thinking about marconi 20:00:11 the question at the beginning was, what should we do about upcoming graduation requests? 20:00:13 ttx: it will be a guideline of mine ... but I will try for it to not be an arbitrary one 20:00:16 anyone have a summary of changes to that? 20:00:21 if any? 20:00:22 annegentle_: ok, it seems like there may be better ways to deal with that but I don't have the full background 20:00:28 dhellmann: ok, sure 20:00:30 we're about out of time here 20:00:35 mordred: it's easier to not incubate data plane things than to reject their graduation. 20:00:35 russellb: I believe we have none, other than an assertion that it's ok for us to say no just because 20:00:42 ttx: that is true 20:00:59 which is why I'd like us to explore it as a "integrated release" rule 20:01:09 mordred: ok, sure, fine with me. :) 20:01:13 one other thing from ttx's original email that we haven't fully explored: we _do_ need more overall project focused infra/qa/docs people 20:01:14 ttx: I'm just very wary of more explicit rules 20:01:18 jeblair: ++ 20:01:24 any team waiting for this room? we're over time 20:01:27 dhellmann: http://lists.openstack.org/pipermail/openstack-dev/2014-August/042962.html 20:01:28 i've tried to not focus on that because i think the urgency around this issue is the integration questions 20:01:40 and i don't think they are strictyl correlated. but it's still an issue we should work on 20:01:53 ttx, fwiw, I don't know what "no data plane" means, doesn't immediately resonate - maybe as a I catch up with the backlog it will become clear 20:01:53 russellb: normally no 20:02:00 OK, so .. no need for a meeting on Monday? 20:02:02 jeblair: you should really consider requiring liaisons; it has done wonders for oslo (though not perfect) 20:02:14 dhellmann: yeah, i've been mulling that around since your post 20:02:15 dhellmann: oh yes, I was going to reply to your ML post that we do teh same in docs with varying success 20:02:19 liaisons 20:02:40 jeblair: I'd be happy to chat if you'd like 20:02:41 and, having an infra-focused docs contributor has done WONDERS 20:02:44 +1 to all the major cross project efforts requiring a liasion 20:02:49 dhellmann: thanks 20:02:55 I think we generally need more people that care about the project as a whole but who live at project-level. 20:02:58 annegentle_: yes indeed! 20:03:04 :) 20:03:20 Those people take on security bugs, take on gate issues, take on project management... 20:03:21 ttx: yes, but that can be tough to balance 20:03:28 and I'd set that bar at incubation. if you aren't ready to contribute to infra and oslo, the team isn't diverse enough to incubate yet 20:03:31 take on liaisons 20:03:44 devananda: I think you're on to something there. 20:03:44 I sense there could be a metric of "most cross-project drag" for all programs 20:03:48 ttx: we do need more ... we're burning out the ones we have 20:03:56 russellb: ++ 20:03:58 neutron would be high. Barbican, who knows? How would we know? 20:04:22 sorry, I know we're over time. 20:04:23 basically I think we ave enough people on the vulnerability management team... but not enough people htat would work on a security patch 20:04:31 yep sorry 20:04:43 OK, so .. no need for a meeting on Monday? 20:04:44 dhellmann: you mean re: liasons, or the email/link i posted? 20:04:49 no meeting is fine with me 20:04:54 back to our regular schedlue on Tuesday ? 20:04:55 devananda: liaisons (I'll read the email after) 20:04:56 ttx: agree no monday meeting is necessary 20:04:58 let's take ideas to the list if any more come up 20:05:00 ttx: if we've concluded we're staying the course, then no need for a meeting 20:05:14 satying on course and it's ok to say NO 20:05:22 I'll post an idea about "cross project drag" to the ML 20:05:28 there's got to be some measure 20:05:33 OK thanks everyone 20:05:34 ttx: regular meeting on tuesday? if so, I wont plan on attending as I'll be somewhere on the road at that point 20:05:50 devananda: sure. You may want to give proxies in case things come to a vote 20:05:58 ack 20:06:00 We'll probably do Rall ynew program 20:06:06 #endmeeting