20:01:18 #startmeeting 20:01:19 o/ 20:01:19 Meeting started Tue Jun 12 20:01:18 2012 UTC. The chair is jbryce. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:35 6 of us...one more around? 20:01:44 notmyname is not able to attend today so I will be giving the swift update for him 20:01:52 chmouel: wrong meeting! 20:02:01 o/ 20:02:06 chmouel: one more hour to wait :P 20:02:16 devcamcar, heckj, vishy: around at all? 20:02:20 hey :) 20:02:24 vishy and anotherjesse are out 20:02:59 well jk0 makes 7 so we can get started 20:03:05 don't count me ... 20:03:16 I'm not a member any more - i'm merely a petitioner today 20:03:21 oops...habit 20:03:36 pvo: ? 20:03:36 jaypipes is your man 20:03:44 hello. 20:03:54 sorry for being late. 20:03:55 sweet 20:04:12 #topic Library/Gating projects 20:04:16 http://wiki.openstack.org/Governance/Proposed/LibraryProjectDefinition 20:04:28 ttx or mtaylor: want to intro this oen? 20:04:29 one 20:04:39 I can intro and let mtaylor elaborate 20:04:39 yeah 20:04:44 or ttx can 20:04:54 o/ 20:04:58 I'll do the historic intro 20:05:18 For client library projects we've always been a bit in between states 20:05:47 when they got split out of their "parent" project we decided that they would retain core status... 20:06:00 ... by being considered another deliverable of the same project 20:06:06 so same PTL, but also same release date 20:06:18 and same project in Launchpad, so same bugtracker 20:06:36 This creates a number of problems, so now is probably the time to change 20:06:44 mtaylor: go for it 20:07:06 so what we'd like to do, as I've written up above with a bunch of chatting with bcwaldon and ttx 20:07:28 is to divorce the release of the client libs from the normal release process a bit 20:07:38 and just release them whenever they need a release 20:08:04 in general, we seem to be in general agreement that any given version of a client lib should support both the current and the previous versions of the api 20:08:13 ++ 20:08:28 mtaylor: how far back? do we have a strategy for EOL-ing API versions? 20:08:34 took us a while to come up with somethign that we would both agree with 20:08:46 danwent: well, at the moment as far back as essex, because diablo is problematic 20:08:51 but, I'd argue indefinitely, tbh 20:08:56 mtaylor: 'essex' is not an api version 20:09:03 you can still use libmysqlclient to talk to v1.0 mysql if you can find a v1.0 mysql 20:09:18 bcwaldon: sorry. v2 and on, since v1 didn't have a fully impl, right? 20:09:27 mtaylor: are you talking about compute api? 20:09:34 mtaylor: but there's definitely a maintenance burden to supporting all API versions. 20:09:47 mtaylor: compute 1.1 api was renamed to 2 20:10:08 danwent: that there is - so I'm purely talking about theory - I think we probably do want to think about an EOL policy for practicality sake 20:10:24 hold on meow, mtaylor, do you have anything else to say w.r.t. explanation, or can we move forward with feedback? 20:10:24 I'd like to remove support for Quantum v1.x APIs fairly agressively… maybe even in Folsom. 20:10:34 but, the thing is - thinking about it from an end user's perspective, if they hear "my cloud provider is running openstack" - 20:10:39 bcwaldon: good point, i'll hold off :) 20:11:05 then it really should be enough for them to just grab the openstack library and be assured that they can talk to that cloud 20:11:18 the discussion on deprecation doesn't really affect the proposed plan anyway 20:11:20 without having to know whether or not the provider is running essex or garbanzo 20:11:26 it actually doesn't 20:11:40 well, it does a LITTLE 20:11:49 so- amongst the points are: 20:12:01 the question at hand is about the creation of new categories of projects to cover for a separate release scheme for client library projects 20:12:21 and additionally recognize gating projects as a category of openstack projects as well 20:12:25 indeed. sorry, I got caught up in technical details :) 20:12:27 ttx: ok, let's talk about that first, then we can circle back to backward compat questions. 20:12:34 separate from our strict definition of "core" 20:12:35 mtaylor, ttx: what are the problems this is triyng to solve? ttx referred to problems that have come up… 20:12:57 a couple - one of them is the branching problem 20:13:04 heckj: on my side, I'd say pressure to release cleint libraries at other moments that openstack core releases 20:13:15 than* 20:13:24 in that a stable/essex branch of python-*client doesn't make any sense from a release perspective for an end user 20:13:28 the other being what ttx just said 20:14:04 mtaylor, why not a stable/essex branch? it's what essex distros would ship 20:14:23 markmc: say you're a customer of rackspace or hp 20:14:24 additionally, sharing bugtracker is actaully a bit of pain that we'd remove :) 20:14:29 or ibm or at&t 20:14:32 jbryce: here now. 20:14:43 and you'd like to talk to instances in your cloud account 20:14:50 what version of the library should you install? 20:15:08 additionally, what if you have an account on rackspace AND hp AND at&t 20:15:08 mtaylor: I'm not getting the "a stable/essex branch of python-*client doesn't make any sense from a release perspective for an end user"… why doesn't it make sense? 20:15:10 for the late arrivals, we're discussing http://wiki.openstack.org/Governance/Proposed/LibraryProjectDefinition 20:15:21 and you would like to have a script which operates on all of those clouds 20:15:28 (such as the devstack gating scripts) 20:15:47 do you suggest that I should install two different versoins of python-*client to talk to the different clouds? 20:15:49 mtaylor, you might need to e.g. wait until Fedora 18 before being able to talk to a folsom based cloud provider 20:16:00 heckj: have you ever downloaded the "stable/whatever" version of libmysqlclient? :) 20:16:05 markmc: I'm installing from pypi 20:16:13 mtaylor, I'm giving the distro perspective 20:16:13 heckj: or have you downloaded the libmysqlclient0.15? 20:16:13 heckj: i think because the clients are talking to APIs not a specific version of the implementation 20:16:18 markmc: I know 20:16:34 markmc: I'm giving the "end user wants to talk to service" perspective 20:16:50 which I battle with every day as a very large customer of both hp and rackspace clouds :) 20:16:54 mtaylor, right, but both might be an option - stable/essex branch and more regular releases from master 20:17:13 jaypipes, mtaylor: then it's the same issue that Thierry is asserting - we'd like to have client library release versioned independently of the core projects, and not assert dependence on client library to core version 20:17:26 heckj: yes 20:17:35 heckj: exactly. 20:17:39 markmc: but it's not essex from a client perspective, it's api v X 20:18:01 which may or may not be different in essex from diablo or folsom 20:18:20 heckj: and the clients would be named not stable/essex, but python-glanceclient-2.x.x 20:18:41 heckj: with the number representing the last version of the API they speak to. 20:18:52 fwiw I was defending the same view as markmc until recently, but the idea of versioning client libraries by API support... and have pure <= backward compat makes the need for stable/* go away... and not having them simplifies versioning wrt PyPI publishing a lot 20:19:00 that's pretty appealing to me from a user perspective 20:19:16 jbryce: agree 20:19:27 I do want to bring up that it doesn't work 100% to version clients to match api versions directly 20:19:32 it means we'll need stronger back-compat testing than we have, but that's on our todo list 20:19:39 bcwaldon: why not? 20:19:40 bcwaldon: just the major API version.. 20:19:46 I just updated http://wiki.openstack.org/Governance/Proposed/LibraryProjectDefinition to add the list of problems that we're solving with this proposal 20:19:47 yeah- just major version 20:19:49 jaypipesThe versioning needs to belong to the library -- what if we need to rewrite a client lib without changing supported versions? 20:20:02 bcwaldon: +++ 20:20:10 bcwaldon: then the minor/revision numbers would change. 20:20:13 bcwaldon: so, I'd say that the Major of the client lib matches the major of the api 20:20:19 and what does it mean to release glanceclient v2, then add a bugfix for v1 support 20:20:23 jaypipes: that versioning is insane 20:20:28 mtaylor: no 20:20:43 I guess I mean "stable releases of the v2 glance client" 20:20:44 bcwaldon: I would argue that the major of the client lib is the contract assertion "I support X and earlier" 20:20:54 i.e. safe fixes to the v2 API support 20:20:58 but that version represents the librarary, not itscapabilities! 20:21:05 bcwaldon: you'd still increment the minor (or revision) number... it's a fix in the v1 area for the client that supports v2 20:21:06 what other projects do that? 20:21:12 adding v3 support does not qualify as a safe update to the v2 client 20:21:16 bcwaldon: every C library that versions itself sensibly 20:21:22 bcwaldon: every C library I know. 20:21:23 I'm 100% with bcwaldon here - the versioning of the library needs to be independent of it's capabilities. The whole point is to break this away from dependence on core project releases. 20:21:25 bcwaldon: standard libtool versioning 20:21:43 I'm not a C developer, so I don't have that bias 20:21:49 or experience, however you want to say it 20:21:50 no, the version of the library is only meaningful as an API contract assertation 20:22:03 i think the version is actually very easy to understand and pretty consistent with most server/client libs that i've seen 20:22:11 jbryce: example? 20:22:17 jbryce: ++ 20:22:22 we've already talked about mysql 20:22:29 I agree it's EXTREMELY obvious, but then we lose the versioning of the library itself 20:22:30 postgres 20:22:37 bcwaldon: how? 20:22:41 bcwaldon: did you read the proposal? 20:22:47 jbryce: ...yes 20:22:50 bcwaldon: there is versioning of the library itself 20:22:57 bcwaldon: all the other numbers 20:23:01 fwiw mtaylor proposed pure numbers as versions (1...2..3...) but I really thought that linking them to supported APi versions would make it a lot more meaningful 20:23:14 so let me post this case to you 20:23:23 mtaylor, libtool versioning is independent of project release versioning 20:23:35 if I want to completely rewrite my library, I'll probably increment the major version to indicate that 20:23:40 markmc: not project release. API version. 20:23:44 markmc: yes. that's what's great about it 20:23:51 if I completely rewrite glanceclient, I can't tell anybody about it! 20:23:57 bcwaldon: why? i don't have that bias...or experience 20:23:58 o/ 20:23:59 made it finally 20:24:02 bcwaldon: why would they care? 20:24:06 mtaylor: ++ 20:24:15 mtaylor: as a developer, I care a LOT 20:24:17 bcwaldon: why would I, as a consumer of the library and its api, care if you rewrote it? 20:24:21 bcwaldon: you shouldn't 20:24:23 versions are useful for the meaning we put in them 20:24:27 either it works when you call its api or it doesn't 20:24:29 mtaylor: do you care if you hav epython 2.6 vs 2.7? 20:24:35 bcwaldon: I care if the api changed 20:24:47 bcwaldon: I do not care if guido rewrite string.strip() 20:24:53 right. 20:24:56 as long as it behaves the same 20:24:56 mtaylor: I think you should 20:24:59 bcwaldon: no 20:25:00 actually 20:25:05 that's why apis are awesome 20:25:06 mtaylor: maybe not *you*, but you should be able to 20:25:07 I don't have to know 20:25:10 What if you changed the Python side of the calling interface but it still works with over-the-wire API v2? 20:25:16 great point 20:25:25 ijw: I would argue that you can't do that 20:25:31 mtaylor: and thats where you lose me 20:25:35 of course you care if an implementation is a compatible rewrite of the same API 20:25:35 add a new interface 20:25:40 mtaylor: if you want to tie my hands, I guess you can 20:25:42 mtaylor: but I will disown you 20:25:44 we cared about ksl being a rewrite of the same API 20:25:47 versioning schemes have whatever meaning we give to them 20:25:53 so, you might have 10000s of devs using your lib 20:25:54 markmc: +1 million 20:26:02 do you want to make them rewrite all of their code just because you had a whim? 20:26:05 that's crazy 20:26:07 jbryce: and I want to assign the most sane versioning 20:26:08 bcwaldon: I see your point. API-based versioning doesn't let you "show" that you made a major rewrite of the lib if it supports the same APIs 20:26:20 ttx: yes 20:26:28 The proposed plan supposes a bit of stability in the client libs 20:26:33 ttx: and major versions are supposed to indicate backwards incompatibile changes 20:26:43 or at least allow it 20:27:00 minor versions indicate a large, potentially risky update 20:27:03 bcwaldon: I think the idea here (re-reading the proposal) is that the first number is intended to indicate the API compatibiity, the second, third, etc - are whatever the client wants to use for it's versioning 20:27:08 micro versions indicate a safe update 20:27:20 heckj: I know what the proposal is 20:27:47 heckj: exactly 20:27:48 bcwaldon: would a era.api.revision numbering work better for you ? 20:27:55 again...the version means whatever we want it to mean 20:27:59 instead of api.revision ? 20:28:03 bcwaldon: so for example 2.5 getting a core API rewrite could go to 2.6? 20:28:29 ^^ that made no sense as a sentence 20:28:30 heckj: Error: "^" is not a valid command. 20:28:38 ttx: I dont really want 'api version' in my versioning scheme 20:28:38 heckj: :) 20:28:48 ok. can I propose something then? 20:28:49 doing a rewrite of the client library but supporting the same API would do something like go from 2.5 to 2.6 20:28:52 bcwaldon: I'd argue that's a bit developer-centric 20:28:54 mtaylor: shoot 20:28:58 The significance of the code version number isn't so much a significance of the number of lines of code changed. It's more an indicator of what reported bugs apply to the version you're looking at. 20:28:59 can we table the exact versioning scheme discussion for later 20:29:17 and get through the "we should release these in this general manner" bit for now 20:29:18 yes, it's actually not the core of the proposal either ;) 20:29:23 ttx: ++ personally i hope we have far more users than developers...otherwise we're all going to be in a tough place 20:29:29 and then we can circle back on what the major number should mean or not mean 20:29:36 mtaylor: good idea 20:29:39 because I TOTALLY hear your point 20:29:48 okie dokie 20:29:57 and I've got a slightly different one that I think we can sit down and work through so that we're both happy :) 20:29:59 the cenrtal part of the proposal is: 20:30:29 * Agree on a category of "openstack" project that is separate from core (which is the servers that get released every 6 months) 20:30:39 * Agree that those could be released separately 20:31:01 I think that any developer will need to read the "doc" README etc. prior to using a particular library to see if his app will be compatible with the target system. To some extent the actual versioning numbers have little meaning to me as a dev, except as a starting point perhaps 20:31:05 Then the secodn part is the abandon of stable/* branches and pure backwards compat 20:31:10 Then versioning 20:31:18 Then deprecation of API 20:31:31 ok 20:31:38 ttx: agree 20:31:52 I think the versioning discussion is actually not that important once you agree on the "second part" 20:31:52 is the vote bot running? 20:32:02 jbryce: should be. 20:32:13 it's more what is obvious by the version number and what is documented 20:32:22 /win 286 20:32:31 Does anyone have objections on the central part of the proposal 20:32:42 negative 20:32:42 The idea that there are openstack projects that are not core projects. 20:32:49 no objections. 20:32:49 not here 20:32:53 Libraries and Gating projects 20:33:00 notmyname: thoughts? 20:33:05 jaypipes: he's out today 20:33:07 that we stronglmy depend on to build core projects anyway 20:33:08 ah, k 20:33:18 jaypipes: I picked my day to present that ;) 20:33:23 heh 20:33:24 no objection 20:34:04 OK... second part then, the absence of need for stable/* branches and the idea that you should always be running the latest version of the client anyway 20:34:26 since that's what supports every non-completely-deprecated old version 20:34:39 ttx: how reasonable is that for distros? 20:35:02 always running the latest client, that is 20:35:06 bcwaldon: you have to take into account that there aren't so many fixes to client libs 20:35:08 for a distribution, you need a cut somewhere, and I want to be able to patch those specific pieces. Are you asserting all distributions should choose their own markers for that? 20:35:08 ttx: i mentioned this in an email to you all a while back. If in folsom I make big changes to the client, but someone discovers an important but small fix in what we release for essex, where do I commit that minor fix such that distros pick it up? 20:35:42 danwent: you won't be releasing clients as 'folsom' or 'essex' 20:35:47 ttx: assumption is that current state of client lib is in churn. 20:35:49 bcwaldon: otherwise it's quite unacceptable, and they would be doing their stable/essex branches anyway... but if you consider that there aren't that many updates anyway... 20:35:56 danwent: you'll bump the bugfix number and release again 20:35:57 do we have any limiters on complaints about documentation for non-core projects? :) 20:36:03 danwent: ^ exactly. I think we need this. I don't care if we call it stable/*, but we should have a branch or at least tag when we make a release. Doing otherwise is irresponsible 20:36:14 bcwaldon: fair, substitute "previous release" for essex, and "next release" for folsom. 20:36:45 bcwaldon: so we as developers we essentially keep an internal notion of a stable branch? 20:36:48 danwent: I would assume you could maintain the previous major release if you really wanted to and tag it 20:37:07 danwent: with glanceclient, I would only tag master when I want to release 20:37:21 heckj: the trick is that there are going to be a flow of releases (uploads to Pypi) 20:37:35 heckj: no strong marker like "essex" 20:38:24 bcwaldon: that makes sense. I guess my point is that we'll effectively be having a "stable" branch. So I guess i'm missing why we said we're getting rid of stable branches. 20:38:26 ttx: I'm fine with a tag on each release then, and not a branch - but there needs to be a marker in our code control that indicates where that match is. I don't relish using git commitish 20:38:40 danwent: how would we have 'stable' branches? 20:38:48 heckj: client libs are actually released by tagging, that's mtaylor's plan 20:39:03 danwent: there would be a single branch: master 20:39:36 bcwaldon: if i have one branch where I make major changes targeted for the next "big" release of the client, and I also have the ability to keep patching the last big release with minor fixes, to me that seems like having a master branch and a stable branch. 20:39:46 we basically only maintain the master..; that doesn't prevent people from having branches... or caollaborating in backporting 20:39:52 b/c those "minor bug fixes" would accumulate 20:39:57 danwent: I'm advocating for not keeping a 'previous relase' branch around 20:40:26 but the CI only tests a simple square matrix of combinations on every commit... not a cubic one 20:40:52 hmm, I mean, a set instead of a matrix :) 20:41:04 we're still talking about the client library projects ONLY here, right? 20:41:08 ttx: ok, i don't want to throw this discussion off track. my only point was that we may well be developing on train sequence of commits, while doing bugfix releases on another. 20:41:09 I think we're wandering off into arbitrary-land here 20:41:10 jaypipes: yes 20:41:25 ttx: get us back on track! :) 20:41:30 struggling 20:41:42 mtaylor just shot himself 20:41:45 we're obviously all in support of the overarching goal here 20:41:48 hehe 20:42:24 mtaylor: stil here ? Defend the absence of need for stable branches, that's your idea :) 20:42:39 ttx: yup 20:42:42 * markmc wonders whatever happened to the idea of proposals for the PPB being discussed on the mailing list first 20:43:14 it comes back down to user experience thing ... in what way do you release the old bugfixed client lib so that it's useful to the end user 20:44:07 sorry, is that a question to me? :) 20:44:43 mtaylor: i agree that's the question I'm getting at. 20:45:26 markmc: actually I don't think we'll close the proposal this week, it's more RFC at this stage 20:45:47 when we know how to write the final proposal, we'll push to list 20:45:55 ttx, cool, just think the various strands could be teased out more thoughtfully on list 20:46:21 markmc: in particular, got to know if the "cental part" is acceptable to a majority of PPB members or not 20:46:27 central* 20:47:05 ttx: none of my concerns are around the central part. 20:47:22 So if I summarize... 20:47:36 yes please :) 20:47:42 We can elaborate a plan based on the central part, with the following remaining pain points: 20:48:01 - need for "official" stable branches 20:48:29 - API or major-rewrite-driven versioning scheme 20:48:50 - room for deprecating old APIs 20:48:58 any other pain point ? 20:49:45 ttx: captures my comments 20:49:51 just a comment, as this is a "policy" decision... this needs to be globally applicable, not choose or not on a per project basis 20:50:04 ttx: matches to me 20:50:27 johnpur: that would apply to the new "Library" category of openstack projects. 20:50:40 but yes, for all of them. 20:50:50 right 20:51:23 In particular, it should probably include openstack-common once it is released as a separate lib 20:52:32 mtaylor: so that sums up the pain points you need to discuss on and off the ML this week 20:53:12 mtaylor: ideally we'll have a final proposal based on that feedback for next week ? 20:54:53 Corollary: devstack becomes a gating project, so it is no longer "not an official OpenStack project" as the website claims 20:55:04 all right...anything else for this week on this one? 20:57:18 ttx: yes. by next week we will have that 20:57:41 jbryce: looks like we're done for this week 20:57:55 thanks guys 20:58:00 #endmeeting