pprocess

README.txt

75:4c74a30f60f8
2007-03-28 paulb [project @ 2007-03-28 23:52:25 by paulb] Added a test of two-way communication.
     1 Introduction
     2 ------------
     3 
     4 The pprocess module provides elementary support for parallel programming in
     5 Python using a fork-based process creation model in conjunction with a
     6 channel-based communications model implemented using socketpair and poll. On
     7 systems with multiple CPUs or multicore CPUs, processes should take advantage
     8 of as many CPUs or cores as the operating system permits.
     9 
    10 Quick Start
    11 -----------
    12 
    13 Try running the simple example:
    14 
    15 PYTHONPATH=. python examples/simple.py
    16 
    17 (A simple example which shows how a limited number of processes can be used to
    18 perform a parallel computation.)
    19 
    20 Or studying some elementary tests:
    21 
    22 PYTHONPATH=. python tests/create_loop.py
    23 PYTHONPATH=. python tests/start_loop.py
    24 
    25 (Simple loop demonstrations which use two different ways of creating and
    26 starting the parallel processes.)
    27 
    28 PYTHONPATH=. python tests/start_indexer.py <directory>
    29 
    30 (A text indexing demonstration, where <directory> should be a directory
    31 containing text files to be indexed, although HTML files will also work well
    32 enough. After indexing the files, a prompt will appear, words or word
    33 fragments can be entered, and matching words and their locations will be
    34 shown. Run the program without arguments to see more information.)
    35 
    36 Some examples are also found in the examples directory, notably the PyGmy
    37 raytracer modified to use pprocess:
    38 
    39 cd examples/PyGmy
    40 PYTHONPATH=../..:. python scene.py
    41 
    42 (This should produce a file called test.tif - a TIFF file containing a
    43 raytraced scene image.)
    44 
    45 Contact, Copyright and Licence Information
    46 ------------------------------------------
    47 
    48 No Web page has yet been made available for this work, but the author can be
    49 contacted at the following e-mail address:
    50 
    51 paul@boddie.org.uk
    52 
    53 Copyright and licence information can be found in the docs directory - see
    54 docs/COPYING.txt and docs/LICENCE.txt for more information.
    55 
    56 For the PyGmy raytracer example, different copyright and licence information
    57 is provided in the docs directory - see docs/COPYING-PyGmy.txt and
    58 docs/LICENCE-PyGmy.txt for more information.
    59 
    60 Dependencies
    61 ------------
    62 
    63 This software depends on standard library features which are stated as being
    64 available only on "UNIX"; it has only been tested on a GNU/Linux system.
    65 
    66 New in parallel 0.2.4 (Changes since parallel 0.2.3)
    67 ----------------------------------------------------
    68 
    69   * Set buffer sizes to zero for the file object wrappers around sockets: this
    70     may prevent deadlock issues.
    71 
    72 New in parallel 0.2.3 (Changes since parallel 0.2.2)
    73 ----------------------------------------------------
    74 
    75   * Added convenient message exchanges, offering methods handling common
    76     situations at the cost of having to define a subclass of Exchange.
    77   * Added a simple example of performing a parallel computation.
    78   * Improved the PyGmy raytracer example to use the newly added functionality.
    79 
    80 New in parallel 0.2.2 (Changes since parallel 0.2.1)
    81 ----------------------------------------------------
    82 
    83   * Changed the status testing in the Exchange class, potentially fixing the
    84     premature closure of channels before all data was read.
    85   * Fixed the PyGmy raytracer example's process accounting by relying on the
    86     possibly more reliable Exchange behaviour, whilst also preventing
    87     erroneous creation of "out of bounds" processes.
    88   * Added a removed attribute on the Exchange to record which channels were
    89     removed in the last call to the ready method.
    90 
    91 New in parallel 0.2.1 (Changes since parallel 0.2)
    92 --------------------------------------------------
    93 
    94   * Added a PyGmy raytracer example.
    95   * Updated copyright and licensing details (FSF address, additional works).
    96 
    97 New in parallel 0.2 (Changes since parallel 0.1)
    98 ------------------------------------------------
    99 
   100   * Changed the name of the included module from parallel to pprocess in order
   101     to avoid naming conflicts with PyParallel.
   102 
   103 Release Procedures
   104 ------------------
   105 
   106 Update the pprocess __version__ attribute.
   107 Change the version number and package filename/directory in the documentation.
   108 Update the release notes (see above).
   109 Check the release information in the PKG-INFO file.
   110 Tag, export.
   111 Archive, upload.
   112 Update PyPI.
   113 
   114 Making Packages
   115 ---------------
   116 
   117 To make Debian-based packages:
   118 
   119   1. Create new package directories under packages if necessary.
   120   2. Make a symbolic link in the distribution's root directory to keep the
   121      Debian tools happy:
   122 
   123      ln -s packages/ubuntu-hoary/python2.4-parallel-pprocess/debian/
   124 
   125   3. Run the package builder:
   126 
   127      dpkg-buildpackage -rfakeroot
   128 
   129   4. Locate and tidy up the packages in the parent directory of the
   130      distribution's root directory.