19:03:11 #startmeeting 19:03:12 Meeting started Tue Sep 20 19:03:11 2011 UTC. The chair is mtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:03:13 Useful Commands: #action #agreed #help #info #idea #link #topic. 19:04:05 o/ 19:05:57 o/ 19:06:58 o? 19:06:58 o/ 19:07:18 o⪐ 19:07:30 o no 19:07:32 I like o??? 19:07:55 #topic Actions from last week 19:08:04 #link http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-09-13-19.03.html 19:08:39 still nothing on those, sorry - been focused on getting diablo out the door 19:08:55 #action mtaylor Add how to contribute section to ci.openstack.org 19:09:02 #action mtaylor add packaging docs to ci.openstack.org 19:09:08 #action mtaylor infrastructure.openstack.org web config 19:09:15 #action mtaylor will work with carlp on setting up netstack CI hardware 19:09:32 #topic open discussion 19:09:55 soren, vishy and I (mostly soren and vishy) were talking earlier about pulling libvirt 0.9.5 into the openstack ppas 19:10:15 I've just about got a package ready there - but I'd LOVE it if you'd look at it soren 19:10:20 yehaw 9.5! 19:10:23 because there are a crap-ton of moving parts in there 19:10:33 I got a session approved for planning the CI for NetStack, I would love it if some or all of you could attend. 19:10:47 carlp: oh, I'll be there 19:11:27 mtaylor: Awesome. I figured getting everyone in the same room for an hour may help things along :) 19:11:40 yes. I think you are 100% correct on that 19:12:39 soren, ttx: also - need to chat at some point (again, sorry I'm annoying) about version numbering. folks want to upload things to PyPI at a frequency greater than full releases, and the current versioning scheme and the mechanism that producees it are - well - problematic for PyPI 19:13:36 I don't have a specific fix in mind - but I know the two of you have spent a LOT of time working out how that version sequencing works 19:13:55 mtaylor: yes, it's about the only thing that works from trunk to release PPAs 19:14:14 so that's the reason why it looks like it does -- not sure you can come up with an alternative that would work 19:14:19 but you can try :) 19:14:27 ttx: yup. heard and understood! :) 19:15:13 ttx: perhaps sitting down with a whiteboard and a LOT of beer might be the right choice there 19:15:23 If the solution is to drop the capability to upgrade packages seamlessly whatever their origin, I think I'd not agree 19:15:32 nope. I do not want that to be the solution 19:15:43 either 19:16:00 I have ignorance of PyPI... hmm... specificities. 19:16:16 but I do think that we need to be able to produce tarball artifacts that work outside of the context of debian versioning 19:16:16 so I miss part of the puzzle 19:16:33 ttx: the main thing is that nobody other than debian understands ~ version specifiers 19:16:50 the reason why stuff is versioned 2012.1~ is so that you don't do a version bump as the very last thing before a release 19:16:59 so when we depend on them, the version sequencing for our tarball releases becomes broken 19:17:30 it could be called 2012.1~prerelease~blabla so that people understand ~ 19:17:37 I totally undestand the reasoning there - I'm just saying that the mechanism isn't supported by all of the places where people would like for us to upload release artifacts (such as pypi) 19:18:06 it's that pypi and pip do not understand that 20112.1 > 2012.1~prerelease~blabla 19:18:12 afaik 19:18:14 nod. 19:18:44 mtaylor: we /could/ rename tarballs before uploading to ubuntu... the trick is that 2012.1~ enables us to have a painless release process 19:18:53 (that's pretty critical for ensuring clean debian-style upgrades. PyPI probably isn't worried about such OS-ish things.) 19:18:58 sice you don't regenerate anything one time at the very end 19:18:59 so, I think that where we are in trouble is that we're doing a GREAT job of getting things set for debian-based distro releases, but we sort of skip over the step of making sure our tarball sequencing is sane 19:19:40 mtaylor: the alternative is some fugly stuff like odd/even schemes + a version bump at the very end of the process that breaks stuff at the wors tmoment 19:19:51 and usually when everyone is sleeping and I have to release before 9am. 19:20:11 I've been there before. 19:20:31 well... as I think you know - I believe that any system in which a version bump breaks anything is a HUGE bug ... but I totally hear that pain and it's possible we have not fixed that in our codebase at the moment 19:20:59 but like I said- I don't have a solution - I just have a new use case/problem :) 19:21:21 mtaylor: not meant as a criticism of your work, but almost every piece of CI I have touched so far failed the first time I used it -- I'd rather that first time not be the release day. 19:21:48 that's release management common sense 19:22:07 does PyPI understand some alpha/beta/rc marker ? 19:22:09 mtaylor: It's not just the chance of actual breakage. 19:22:40 mtaylor: There are so many things that are massively more convenient by bumping early rather than late. 19:22:41 mtaylor: PyPI is just not meant to support beta versions, AFAICT 19:24:48 mtaylor: A good example is documentation. The job that publishes documentation will simply publish the docs from trunk, saying "this is the docs for the thing that will eventually become version X", and it can publish this at docs.openstack.org/X/ with a "work in progress" disclaimer. 19:24:57 so trying to (ab)use it to deliver frequent milestones sounds... difficult 19:25:35 mtaylor just told me he lost his network connection 19:25:41 As one of the very last things before release, we set "Final = True" to remove the disclaimer. 19:26:31 And whatever we push as the final version, automatically has its docs published in the right place. As the very next thing, we bump the version in the code to reflect the next planned version. 19:26:39 sorry - network droped me - I'm back 19:26:46 mtaylor: Do you have scrollback? 19:26:47 mtaylor: did you see the backlog ? 19:26:52 yes. I have - reading 19:26:55 Ok. 19:27:21 ok. yes. I agree with all of the words that you're saying. 19:27:40 The alternative involves either maintaining the upcoming version number somewhere outside the repository and pull from there, but keeping things in sync is just easier if there's nothing to keep in sync, because everything reads from the same place: the code. 19:28:08 Either that or having some sort of logic somewhere that's supposed to guess the next version. 19:28:30 ...but then we have to disable that when we're pushing the final revision (because that's actually the final revision, not the first revision of the next version). 19:28:35 It's mad. 19:29:08 It's funny, really, because you were the one who bumped the version early to begin with. I went "wait, what?", but later realised this is the only sensible way to do things. 19:29:32 ok. let's be clear ... I'm not criticising anyone's design or decisions - and yes indeed I did that 19:29:52 all I'm saying is - there is a desire from people to have things hit pypi more frequently than every six months 19:30:05 and the current system, for good or bad, does not support this as a goal 19:30:24 which means that as a goal should be re-assessed, or other elements of the system design should 19:30:37 which I believe is ripe for a conversation - probably one in person, I'm guessing 19:30:53 mtaylor: sure, I'll wait for a clear alternative 19:31:16 Where can I read about the versioning limitations that pypi imposes? 19:31:25 just saying that we are not using ~ just because we can... We use the right solution, and ~ just enabls us to do it conveniently. 19:31:32 I've not completely understood how our scheme is problematic for pypi. 19:31:49 soren: well, they are normal versioning limitations. nothing other than debian understands ~ as a version modifier 19:32:00 http://epydoc.sourceforge.net/stdlib/distutils.version.StrictVersion-class.html 19:32:00 so pypi does not grok that 2012.1~a < 2012.1 19:32:08 thanks :) 19:32:09 is that documentation operative? 19:32:20 i'm not positive, i've just been googling around during this conversation... 19:32:22 I believe so 19:32:24 mtaylor: yes, most others accept some 2012.1-alpha1 form though 19:32:34 they do 19:32:41 does PyPI support it ? 19:33:05 yes. check the link jeblair just posted. 0.5a1 19:33:05 because then it's just a matter of renaming before using in Debian, that's not the first upstream that needs to d othat 19:33:22 it looks like 5a sorts before 5 19:33:31 so can't we just replace the ~ with an a 19:33:32 way simpler than dropping the whole "name the version by the next release, not the previous one" discussion 19:33:36 and were golden? 19:33:50 right. I thnk that's all I'm getting at - tarballs/pypi need to be produced by those rules 19:34:31 potentially - we still hit the problem of our tarballs not being actually produced via version in setup.py but by renaming - which will cause some problems for the python setup.py sdist upload step 19:34:31 BUT 19:34:42 Yes, I respect that ~ is a debianism. I just want to make sure we don't throw the baby (pre-release versioning) with the bath water (~) 19:34:44 we can start from that versioning as a step one and see what we can come up with 19:34:50 ttx: oh god no 19:34:55 sorry if I was implying that 19:35:25 hopefully there is a pipy compatible way of doing the same kind of versioning 19:36:10 for the record: 19:36:25 #link http://epydoc.sourceforge.net/stdlib/distutils.version.StrictVersion-class.html 19:36:44 #link http://www.python.org/dev/peps/pep-0386/ 19:36:50 should probably keep up with that too 19:37:36 agree 19:38:01 Just read through the distutils code. 19:38:06 there also seems to be some history and narrative in there (which i haven't read yet) 19:38:07 'a' and 'b' are magic. 19:38:30 They're the only characters we can use for this. Anything else is rejected. 19:38:41 also you need a number 19:38:44 "a1" 19:38:46 interesting. no gammas or c's. :) 19:38:53 Right. 19:39:01 or e's for essex 19:39:09 :( 19:39:44 so you could replace ~ by a1 but that would look fun. 2012.1a1~e2 19:40:17 well - why e2? why not just 2012.1a20110944.1234 ? 19:40:17 you would probably drop the idea of naming the milestones e's, to just call them alphas 19:40:32 which sounds a bit scarier 19:40:47 mtaylor: That's invalid as well. 19:40:57 or just go the Flickr route and say "loves you" :) 19:40:59 mtaylor: No dots allowed after the 'a' or 'b'. 19:41:03 2012.1a201109441234 ? 19:41:10 mtaylor: because the milestones are "released" as nova-2011.3~d4.tar.gz currently 19:41:17 The reason is that we.. 19:41:20 right, what ttx said. 19:41:23 correct 19:41:37 how would you release them under a new scheme ? 19:41:45 When we switched to having to branches, we had to be able to tell them apart. 19:42:08 *two* branches. 19:42:16 I don't know - I need to think about it some... as I said earlier - I do not have this solved, merely wanted to bring up that it was something we needed to think about 19:42:20 Not "to branches". That's nonsense. 19:42:40 hehe. if that's the only nonsense anyone says in this channel today then I think we're doing well :) 19:43:04 mtaylor: but yes, we should discuss that 19:43:21 the current scheme was defined in Brussles with lots of aspirin 19:43:36 and lots of graphs 19:43:37 #agreed we should discuss versioning and how it relates to tarballs and PyPI 19:43:53 hm.. 19:43:58 I still have the drawings if you want to stress-test your solution 19:44:06 WEll, how's this: 19:44:45 On PyPI we either only ever publish stuff from trunk or the milestone branch. 19:45:02 yes. I agree with that 19:45:06 Or, we create two separate.. err.. I don't know the terminology.. "projects"? 19:45:12 +1 19:45:13 oh - wait. no. actually 19:45:15 One for trunk, one for milestone. 19:45:22 (+1 for the first suggestion) 19:45:29 this becomes a bit of a shitshow as well 19:45:37 mtaylor: if you like to be hurt: http://ubuntuone.com/p/vjd/ 19:45:56 #link http://tarekziade.wordpress.com/2009/07/03/dropping-pep-386-versions-comparison/ 19:46:02 because of pip-requires 19:46:03 BUT 19:46:20 if we only released milestones and releases to pypi (probably frequent enough?) 19:46:58 then we could release 2012.1~d4 as 2012.1b4 on pypi, no? 19:47:39 Yes. 19:47:42 so that pypi just always gets the latest milestone or release (or in the case of swift, every release) 19:47:53 I think that makes sense. 19:47:57 and then we don't have to worry about 2012.1~d4~2340239742.234 19:48:00 sweet 19:48:17 If anyone cares enough, we can add a nova-i-like-pain project with stuff from trunk. 19:48:21 yes 19:48:26 But meh. 19:48:40 but I think if they want that, then grabbing source and doing python setup.py develop is probably a better choice 19:48:54 * ttx takes a quick break before ppb meeting 19:48:59 ttx: Enjoy. 19:49:08 I would like to also do that. anybody got anything else for in here? 19:49:38 #link http://tarekziade.wordpress.com/2010/02/10/pep-345-and-386-accepted-summary-of-changes/ 19:49:47 mtaylor: I do the setup.py develop all the time - might be good to just suggest it as a standard pattern somewhere 19:49:52 #action mtaylor will propose something concrete based on milestone/releases going to PyPI using pep386 versioning 19:49:54 heckj: ++ 19:51:03 great. I need a sandwich 19:51:05 #endmeeting