How Python is Developed

National Science Foundation
May 11, 2004

A.M. Kuchling
www.amk.ca
amk @ amk.ca

Python will be examined as a detailed example. Other groups (Linux, Apache, etc.) have similar processes.

Python overview

Good software engineering practices

The code: Structure of core Python

The code: Version control

Why version control?

Version control needs network capability.

Python uses CVS. Perl uses Perforce; Linux uses Bitkeeper. Subversion is up-and-coming.

The code: Change notifications

Changes are sent to python-checkins mailing list 

Index: xmlrpclib.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/xmlrpclib.py,v
retrieving revision 1.22
diff -u -r1.22 xmlrpclib.py
--- xmlrpclib.py        1 Nov 2002 17:14:16 -0000       1.22
+++ xmlrpclib.py        9 Jan 2003 18:36:50 -0000
@@ -550,11 +550,12 @@
-    def __init__(self, encoding=None):
+    def __init__(self, encoding=None, allow_none=False):
         self.memo = {}
         self.data = None
         self.encoding = encoding
-
+        self.allow_none = allow_none
+

Purposes:

Python reviews after commit. Mozilla reviews before commit.

The code: Rules for committing changes

Usually, if you check out a copy of the CVS trunk, it will compile and run just fine.

Some projects have their CVS tree broken or difficult to use for long stretches (e.g. GNOME).

Source distribution includes 277 test scripts.

The code: Tracking bugs and patches

[Screenshot of the Python patch manager]

Planning: The python-dev list

python-dev is the mailing list where the developers of the Python core congregate.

Currently has ~600 subscribers, but most of them are lurkers.

~10 people perform the bulk of the work.
~40 people contribute intermittent assistance.
~60-100 offer opinions.

  • Lists are primary; there are dozens/hundreds of them.
  • IM not used much in Python, though the PSF directors use it Often used for assisting users, or for chats between a few individuals. Large meetings are difficult (time zones, keeping the meeting on track).
  • Sprints are face-to-face meetings, usu. at conferences, and often have an agenda. Developers work on a focused set of tasks.
  • Sprints often happen at conferences; can also be standalone.

    • Planning: Day-to-day and long-term

    van Rossum is the Benevolent Dictator For Life.

    There's an informal voting process inspired by Apache's voting scheme:

    Also described in PEP 10

    Torvalds is another BDFL. Perl has a BDFL role that rotates.

    Special Interest Groups (SIGs)

    Current and past SIGs:

    Python Enhancement Proposals (PEPs)

    Some example PEPS

         num  title                                        owner
     ---  -----                                        -----
 I     0  Index of Python Enhancement Proposals        Warsaw
 I     1  PEP Guidelines                               Warsaw, Hylton
 I     2  Procedure for Adding New Modules             Faassen
 I     3  Guidelines for Handling Bug Reports          Hylton
 I     6  Bug Fix Releases                             Aahz
 I     7  Style Guide for C Code                       GvR
 I     8  Style Guide for Python Code                  GvR, Warsaw
         ...
 SF  201  Lockstep Iteration                           Warsaw
 SF  202  List Comprehensions                          Warsaw
 SF  203  Augmented Assignments                        Wouters
         ...
 S   302  New Import Hooks                             JvR
 S   303  Extend divmod() for Multiple Divisors        Bellman
 S   320  Python 2.4 Release Schedule                  Warsaw, Hettinger
 S   327  Decimal Data Type                            Batista
 S   328  Imports: Multi-Line and Absolute/Relative    Aahz

    For each release, there's a PEP listing the new features and giving a planned schedule.

    Community: Scheduling

    Forks: Experimenting and adapting

    Anyone can take the Python code and make their own version.

    Well-known Python forks:

    Jython: reimplementation of Python in Java. Uses standard library from CPython. Dana Moore will talk about it later.

    IronPython: presented at recent PyCon; CLR implementation whose speed for some things is better than CPython.

    These forks let the community explore new implementation styles. Some forks die (e.g. Vyper), some find a user community (Stackless), some become parallel (Jython).

    Evolution in action: if CLR takes over the world, Python will adjust to it.

    Good software engineering practices

    Concluding points

    Loosely: you can't make people do things. Tightly: the distributed nature requires tools for good communication.

    SE practices: this is really in self-defense; projects which don't follow them don't survive very long.

    Questions?

    Detailed material is contained in the Python Developer's Guide at www.python.org/dev/.

    These slides:
    www.amk.ca/talks/python-dev

    Author's e-mail:
    amk @ amk.ca