1 Introduction
2 ------------
3
4 The pprocess module provides elementary support for parallel programming in
5 Python using a fork-based process creation model in conjunction with a
6 channel-based communications model implemented using socketpair and poll. On
7 systems with multiple CPUs or multicore CPUs, processes should take advantage
8 of as many CPUs or cores as the operating system permits.
9
10 Quick Start
11 -----------
12
13 Try running the simple example:
14
15 PYTHONPATH=. python examples/simple.py
16
17 (A simple example which shows how a limited number of processes can be used to
18 perform a parallel computation.)
19
20 Or studying some elementary tests:
21
22 PYTHONPATH=. python tests/create_loop.py
23 PYTHONPATH=. python tests/start_loop.py
24
25 (Simple loop demonstrations which use two different ways of creating and
26 starting the parallel processes.)
27
28 PYTHONPATH=. python tests/start_indexer.py <directory>
29
30 (A text indexing demonstration, where <directory> should be a directory
31 containing text files to be indexed, although HTML files will also work well
32 enough. After indexing the files, a prompt will appear, words or word
33 fragments can be entered, and matching words and their locations will be
34 shown. Run the program without arguments to see more information.)
35
36 Some examples are also found in the examples directory, notably the PyGmy
37 raytracer modified to use pprocess:
38
39 cd examples/PyGmy
40 PYTHONPATH=../..:. python scene.py
41
42 (This should produce a file called test.tif - a TIFF file containing a
43 raytraced scene image.)
44
45 Contact, Copyright and Licence Information
46 ------------------------------------------
47
48 No Web page has yet been made available for this work, but the author can be
49 contacted at the following e-mail address:
50
51 paul@boddie.org.uk
52
53 Copyright and licence information can be found in the docs directory - see
54 docs/COPYING.txt, docs/lgpl-3.0.txt and docs/gpl-3.0.txt for more information.
55
56 For the PyGmy raytracer example, different copyright and licence information
57 is provided in the docs directory - see docs/COPYING-PyGmy.txt and
58 docs/LICENCE-PyGmy.txt for more information.
59
60 Dependencies
61 ------------
62
63 This software depends on standard library features which are stated as being
64 available only on "UNIX"; it has only been tested on a GNU/Linux system.
65
66 New in parallel 0.2.6 (Changes since parallel 0.2.5)
67 ----------------------------------------------------
68
69 * Added managed callables: wrappers around callables which cause them to be
70 automatically managed by the exchange from which they were acquired.
71 * Added MakeParallel: a wrapper instantiated around a normal function which
72 sends the result of that function over the supplied channel when invoked.
73 * Added a Map class which attempts to emulate the built-in map function,
74 along with a pmap function using this class.
75 * Extended and updated the examples.
76
77 New in parallel 0.2.5 (Changes since parallel 0.2.4)
78 ----------------------------------------------------
79
80 * Added a start method to the Exchange class for more convenient creation of
81 processes.
82 * Relicensed under the LGPL (version 3 or later) - this also fixes the
83 contradictory situation where the GPL was stated in the pprocess module
84 (which was not, in fact, the intention) and the LGPL was stated in the
85 documentation.
86
87 New in parallel 0.2.4 (Changes since parallel 0.2.3)
88 ----------------------------------------------------
89
90 * Set buffer sizes to zero for the file object wrappers around sockets: this
91 may prevent deadlock issues.
92
93 New in parallel 0.2.3 (Changes since parallel 0.2.2)
94 ----------------------------------------------------
95
96 * Added convenient message exchanges, offering methods handling common
97 situations at the cost of having to define a subclass of Exchange.
98 * Added a simple example of performing a parallel computation.
99 * Improved the PyGmy raytracer example to use the newly added functionality.
100
101 New in parallel 0.2.2 (Changes since parallel 0.2.1)
102 ----------------------------------------------------
103
104 * Changed the status testing in the Exchange class, potentially fixing the
105 premature closure of channels before all data was read.
106 * Fixed the PyGmy raytracer example's process accounting by relying on the
107 possibly more reliable Exchange behaviour, whilst also preventing
108 erroneous creation of "out of bounds" processes.
109 * Added a removed attribute on the Exchange to record which channels were
110 removed in the last call to the ready method.
111
112 New in parallel 0.2.1 (Changes since parallel 0.2)
113 --------------------------------------------------
114
115 * Added a PyGmy raytracer example.
116 * Updated copyright and licensing details (FSF address, additional works).
117
118 New in parallel 0.2 (Changes since parallel 0.1)
119 ------------------------------------------------
120
121 * Changed the name of the included module from parallel to pprocess in order
122 to avoid naming conflicts with PyParallel.
123
124 Release Procedures
125 ------------------
126
127 Update the pprocess __version__ attribute.
128 Change the version number and package filename/directory in the documentation.
129 Update the release notes (see above).
130 Check the release information in the PKG-INFO file.
131 Tag, export.
132 Archive, upload.
133 Update PyPI.
134
135 Making Packages
136 ---------------
137
138 To make Debian-based packages:
139
140 1. Create new package directories under packages if necessary.
141 2. Make a symbolic link in the distribution's root directory to keep the
142 Debian tools happy:
143
144 ln -s packages/ubuntu-hoary/python2.4-parallel-pprocess/debian/
145
146 3. Run the package builder:
147
148 dpkg-buildpackage -rfakeroot
149
150 4. Locate and tidy up the packages in the parent directory of the
151 distribution's root directory.