1.1 --- a/docs/reference.html Mon Jun 02 22:43:11 2008 +0000
1.2 +++ b/docs/reference.html Mon Jun 02 22:43:43 2008 +0000
1.3 @@ -32,6 +32,8 @@
1.4 <li><a href="#Map">Map-style Processing</a></li>
1.5 <li><a href="#Reusing">Reusing Processes and Channels</a></li>
1.6 <li><a href="#MakeReusable">Making Existing Functions Parallel and Reusable</a></li>
1.7 +<li><a href="#BackgroundCallable">Background Processes and Callables</a></li>
1.8 +<li><a href="#PersistentQueue">Background and Persistent Queues</a></li>
1.9 <li><a href="#Implementation">Implementation Notes</a></li>
1.10 </ul>
1.11
1.12 @@ -331,6 +333,67 @@
1.13 <p>This wraps the callable in a similar fashion to <code>MakeParallel</code>, but
1.14 provides the necessary mechanisms described above for reusable code.</p>
1.15
1.16 +<h2 id="BackgroundCallable">Background Processes and Callables</h2>
1.17 +
1.18 +<p>So far, all parallel computations have involved created processes which
1.19 +depend on the existence of the created process to collect results and to
1.20 +communicate with these created processes, preventing the created process from
1.21 +terminating, even if the created processes actually perform work and potentially
1.22 +create output which need not concern the process which created them. In order to
1.23 +separate creating and created processes, the concept of a background process
1.24 +(also known as a daemon process) is introduced.</p>
1.25 +
1.26 +<p>The <code>BackgroundCallable</code> class acts somewhat like the
1.27 +<code>manage</code> method on exchange-based objects, although no exchange is
1.28 +immediately involved, and instances of <code>BackgroundCallable</code> provide
1.29 +wrappers around existing parallel-aware callables which then be invoked in order
1.30 +to initiate a background computation in a created process. For example:</p>
1.31 +
1.32 +<pre>
1.33 +backgroundfn = pprocess.BackgroundCallable(address, fn)
1.34 +</pre>
1.35 +
1.36 +<p>This wraps the supplied callable (which can itself be the result of using
1.37 +<code>MakeParallel</code>), with the resulting wrapper lending itself to
1.38 +invocation like any other function. One distinguishing feature is that of the
1.39 +<code>address</code>: in order to contact the background process after
1.40 +invocation to (amongst other things) receive any result, a specific address
1.41 +must be given to define the contact point between the created process and any
1.42 +processes seeking to connect to it. Since these "persistent" communications
1.43 +employ special files (specifically UNIX-domain sockets), the address must be a
1.44 +suitable filename.</p>
1.45 +
1.46 +<h2 id="PersistentQueue">Background and Persistent Queues</h2>
1.47 +
1.48 +<p>Background processes employing persistent communications require adaptations
1.49 +of the facilities described in the sections above. For a single background
1.50 +process, the <code>BackgroundQueue</code> function is sufficient to create a
1.51 +queue-like object which can monitor the communications channel between the
1.52 +connecting process and a background process. For example:</p>
1.53 +
1.54 +<pre>
1.55 +queue = pprocess.BackgroundQueue(address)
1.56 +</pre>
1.57 +
1.58 +<p>This code will cause the process reachable via the given <code>address</code>
1.59 +to be contacted and any results made available via the created queue-like
1.60 +object.</p>
1.61 +
1.62 +<p>Where many background processes have been created, a single
1.63 +<code>PersistentQueue</code> object can monitor their communications by being
1.64 +connected to them all, as in the following example:</p>
1.65 +
1.66 +<pre>
1.67 +queue = pprocess.PersistentQueue()
1.68 +for address in addresses:
1.69 + queue.connect(address)
1.70 +</pre>
1.71 +
1.72 +<p>Here, the queue monitors all previously created processes whose addresses
1.73 +reside in the <code>addresses</code> sequence. Upon iterating over the queue,
1.74 +results will be taken from whichever process happens to have data available in
1.75 +no particular pre-defined order.</p>
1.76 +
1.77 <h2 id="Implementation">Implementation Notes</h2>
1.78
1.79 <h3>Signals and Waiting</h3>