16:06:17 <ewindisch> #startmeeting Python3
16:06:18 <openstack> Meeting started Thu Mar 21 16:06:17 2013 UTC.  The chair is ewindisch. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:06:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:06:21 <openstack> The meeting name has been set to 'python3'
16:06:23 <ewindisch> got distracted.
16:06:37 <ewindisch> who is here?
16:06:59 <Kharec> ewindisch: I'm here
16:07:05 <maksimov> o/ :)
16:07:09 <Kharec> :)
16:07:10 <markmcclain> here
16:07:31 <ewindisch> #topic getting started
16:07:48 <ewindisch> I'd like to discuss how we get started on this effort.
16:08:21 <ewindisch> first, my thoughts are that we should work on getting py3 compatibility in oslo
16:08:42 <ewindisch> as it is our library project and the barrier to anything else useing python3
16:08:44 <ewindisch> *using
16:08:53 <markmcclain> +1
16:08:53 <Kharec> Agreed.
16:09:08 <ewindisch> secondly, we can't make any progress at all without gating.
16:09:41 <ewindisch> agreed?
16:09:45 <Kharec> Gating?
16:09:57 <ewindisch> gate testing, i.e. jenkins
16:10:02 <Kharec> oh
16:10:06 <Kharec> agreed :)
16:10:10 <maksimov> is this an action item to follow up with infra team?
16:10:34 <markmcclain> should be able to add py33 target to tox and then they can setup a job
16:10:39 <ewindisch> it will be. I'd like to discuss what that follow-up will be.
16:10:49 <ewindisch> #topic gating python3
16:11:28 <ewindisch> markmcclain: I'm not sure that adding a py33 target is the right thing (also, py33 targets already exist in tox, the py* targets are implicit and part of tox itself)
16:12:12 <ewindisch> the first barrier is that the code we have doesn't work with Python3 already. We can't simply require that any code uploaded works in Python3.
16:12:47 <Kharec> So what can we do?
16:12:57 <ewindisch> I was speaking with jog0 yesterday. He told me what they're doing with pyflakes and I think their model can be adapted.
16:13:17 <ewindisch> they test the previous revision of a file, test the current version, and look for differences.
16:13:46 <Kharec> Sounds good
16:13:58 <ewindisch> I think we can do the same with '2to3'. It will be more binary, however. It will only make sure that a file is python3-compliant or not. It will only make sure that once a file becomes Python3-compatible, it doesn't regress.
16:14:54 <ewindisch> or maybe we can even use pyflakes /w python3 -- I haven't verified.
16:14:54 <Kharec> so adapting their method with 2to3 ?
16:14:59 <ewindisch> essentially.
16:15:03 <ewindisch> so now...
16:15:11 <markmcclain> I'm not sure 2to3 is something we want
16:15:16 <Kharec> maksimov: why?
16:15:22 <Kharec> oops
16:15:30 <Kharec> markmcclain: why?
16:16:05 <ewindisch> markmcclain: I mean only to run 2to3 determine if a file has "unconvertible python2 code"
16:16:24 <ewindisch> not to use the patchset output of it.
16:17:08 <ewindisch> but if we can get a python3 pyflakes test working, that might be better.
16:17:48 <ewindisch> #action ewindisch follow up with -infra to discuss gating
16:17:54 <rpodolyaka> I'd rather we supported both 2.6+ and 3.2+ using six or something like that
16:18:38 <ewindisch> rpodolyaka: I agree that we'll need to use six, or at least future imports.
16:18:43 <markmcclain> ok.. but 2to3 for checking just tells us the code cannot be converted.. .should we looking towards using something like six?
16:19:05 <ewindisch> rpodolyaka: but we need a way to gate and check that the code introduced, even with six, runs on Python3
16:19:28 <ewindisch> without breaking tests for code that isn't (yet) supposed to run on Python3
16:21:51 <ewindisch> I don't think six is such a magic bullet you can simply import it and say, "this code works in Python3 now".
16:22:08 <ewindisch> and six isn't a gate-check for jenkins
16:22:28 <ewindisch> but yes, I think it should be part of the strategy for making code compatible.
16:22:29 <markmcclain> agreed.. but I'm still not sure how you reliably check patchset deltas for py3 compatibility
16:22:35 <Kharec> Maybe jenkins jobs with 2to3 works better. I don't think six is the solution too.
16:23:02 <markmcclain> seems like the whole file would need to be checked
16:23:21 <ewindisch> markmcclain: if the previous version of a file passed a pyflakes or 2to3 check, then the current version should also pass. I.e. there shouldn't be a regression.
16:23:53 <markmcclain> ok.. so if the old version failed then the new version can fail and that is ok
16:23:55 <markmcclain> ?
16:24:13 <ewindisch> yes.
16:24:35 <ewindisch> so code that has never worked on python3 doesn't need to work with python3.
16:24:40 <ewindisch> once it works, it stays working.
16:25:11 <markmcclain> ok.. now I'm following
16:25:16 * markmcclain needs more coffee
16:25:23 <ewindisch> pyflakes tests would be better though. It will give a list of all the things broken in the code. The list of broken things in one version can be compared to the other version, which is much more granular.
16:26:28 <rpodolyaka> but can we really trust pyflakes? E. g. a module may use some third-party modules that are not Python 3 compatible
16:27:05 <markmcclain> rpodolyaka: doesn't execute code
16:27:08 <ewindisch> rpodolyaka: the reality is that we can't migrate any files to python3 that don't have working dependencies.
16:27:18 <ewindisch> so I'm not too worried about that
16:27:38 <ewindisch> or at least even if we did, there is little benefit
16:28:25 <ewindisch> the dependency problem is going to be a big one.
16:28:55 <ewindisch> want to switch to dependancies or python3 version questions? I'm afraid they might be tied at the hip.
16:29:13 <markmcclain> yeah let's switch
16:29:28 <ewindisch> before we do - does someone want to work on the 2to3 and/or pyflakes test?
16:30:05 <markmcclain> I can take a stab at it
16:30:28 <ewindisch> #action markmacclain to look at writing a py3 test
16:30:38 <ewindisch> #topic python3 version and dependencies
16:30:47 <ewindisch> so we need to figure out what version of python3 to support.
16:30:52 <rpodolyaka> 3.2+?
16:31:32 <maksimov> thought the idea was to target 3.3+
16:31:35 <ewindisch> My understanding is that 3.3+ is probably the only path that will allow you to write code that also runs on py26 and py27 without forking, or having really ugly constructs
16:31:51 <rpodolyaka> I think, 3.2 is going to be a mainline version in distros like debian for a while
16:32:01 <rpodolyaka> mainline of 3.x
16:32:11 <ewindisch> however, we should consider that our dependencies might support earlier versions of Python3, but not Python 3.3.
16:32:36 <mikeyp> Version will probably be driven by dependences more than anything else
16:33:06 <ewindisch> I'm inclined to think we target 3.3 and then retroactively add support for 3.2, if desirable or necessary. It probably will be necessary.
16:34:03 <ewindisch> Python 3.3 adds u'' back
16:35:09 <rpodolyaka> AFAIK, that's the only thing in 3.3 that makes both 2.x/3.x support eaesier
16:35:09 <ewindisch> mikeyp: perhaps we should evaluate what dependencies we already have in Oslo that are compatible with Python3 and see what versions they support.
16:35:58 <dripton> I'm willing to dig through Nova deps and make a list of which ones support Python 3 and which version.
16:36:23 <ewindisch> dripton: I'd prefer it were oslo deps (since nova depends on oslo)
16:36:23 <markmcclain> I think evaluating oslo is a good first step
16:36:37 <mikeyp> ewindsch: makes sense, that was where I thought we would start last year.
16:36:51 <ewindisch> dripton: will you do that then?
16:36:54 <dripton> okay, I'll dig through oslo deps.
16:37:05 <ewindisch> #action dripton to dig through oslo deps for version compat
16:37:42 <rpodolyaka> eventlet doesn't seem to work in 3.x, does it?
16:37:57 <ewindisch> rpodolyaka: no, neither does gevent.
16:38:09 <ewindisch> I'm honestly not sure what exists except Tulip
16:38:24 <ewindisch> (Guido's new pet project)
16:38:43 <dripton> Twisted doesn't work in 3 either, though some work has been done.  So basically no major async network libraries.
16:39:10 <ewindisch> to that point, I've thought that we might actually want to start with oslo.config
16:39:36 <ewindisch> it probably has the fewest horrible external dependencies (no eventlet) and is already out of incubation
16:40:11 <dripton> But what's the point of that, if we can't actually run openstack in Python 3?  Until we get eventlet ported, that just adds churn and work without really helping.
16:40:14 <ewindisch> plus we have the best chance of getting the an entire subproject/repository converted, which will be a good 'win'.
16:41:04 <markmcclain> yeah.. also limited in scope so it's a good place to learn
16:41:45 <ewindisch> dripton: my immediate goal for this effort is to facilitate the use of Python3 by new projects. Getting projects like Nova converted one day is noble, but we need to manage scope and take baby steps.
16:42:07 <dripton> ok, as long as it doesn't make Nova worse to make hypothetical projects better
16:42:38 <dripton> I'd be happiest with a big bang conversion, but we can't do that until we address dependencies.  Converting our code is the easy part.
16:43:21 <ewindisch> lets see what our dependencies look like and we can evaluate.
16:43:32 <ewindisch> but I agree that dependencies such as Eventlet and Paste are going to be painful.
16:43:57 <ewindisch> but there is enough work ahead that it doesn't have to be solved today.
16:44:09 <ewindisch> okay - anything else for version and deps?
16:44:24 <mikeyp> the PSF is sponsoring some Python 3 conversions - for eventlet, might be able to get some help there
16:44:56 <ewindisch> mikeyp: interesting. we should look into that.
16:45:14 <ewindisch> #action ewindisch to look at PSF sponsorship
16:45:54 <ewindisch> also, once the TCs are elected, we might need to *propose* a version of Python3 to support for them to bless.
16:46:10 <ewindisch> anyone else have anything to discuss before we go to open discussion?
16:46:36 <ewindisch> #topic open discussion
16:47:23 <dripton> I think 3.2 is probably a waste of effort, since Python 3 isn't used much in production yet.  Might as well start with 3.3.  (Different from the 2.x case, where 2.6 is still used by older systems like RHEL.)
16:47:55 <ewindisch> dripton: perhaps. It depends on the dependencies, I think.
16:48:19 <dripton> True, when I generate the list we'll see if anything works on 3.2 but not 3.3.  I doubt it but we'll see.
16:48:48 <ewindisch> yeah, I think forward compatibility is good once you get over the 2->3 hump
16:49:01 <dripton> Once a mainstream distro like Ubuntu launches with 3.x as default Python, that particular x becomes important for backward compatibility.
16:49:46 <ewindisch> Ubuntu already defaults to 3.2.3
16:49:59 <ewindisch> (ubuntu 12.10)
16:50:43 <dripton> 3.2.3 is included with Ubuntu but it's not the default Python; 2.7.x is.  A future Ubuntu will make 3.x the default and *that's* when it'll really be used.
16:51:14 <ewindisch> hmm, you're right.
16:51:22 <ewindisch> Ubuntu Raring will target Python 3.3
16:52:08 <ewindisch> also another gate we could do is to run unit tests with "python2.7 -3"
16:53:16 <dripton> Should I create https://wiki.openstack.org/wiki/Python3Deps, or is there a better place to put my list?
16:53:32 <ewindisch> dripton: that sounds like a good idea.
16:54:00 <dripton> I'll also create https://wiki.openstack.org/wiki/Python3 with a pointer to the Python3Deps, as a placeholder for future work.
16:54:11 <ewindisch> a more ambtious goal might be to just extract the dependencies and do a lookup against the hall of shame
16:54:27 <ewindisch> #action dripton create wiki pages
16:54:46 <dripton> I'm going to do that, but I fear that hall of shame might not be 100% accurate/complete so I might have to dive into project web pages, etc.
16:55:06 <ewindisch> acknowledged
16:55:14 <ewindisch> Finally, thank you everyone for helping with this effort.
16:55:56 <ewindisch> #endmeeting