1.1 --- a/docs/tutorial.html Fri Jun 05 01:46:07 2009 +0200
1.2 +++ b/docs/tutorial.html Fri Jun 05 21:23:25 2009 +0200
1.3 @@ -18,7 +18,12 @@
1.4 <ul>
1.5 <li><a href="#pmap">Converting Map-Style Code</a></li>
1.6 <li><a href="#Map">Converting Invocations to Parallel Operations</a></li>
1.7 -<li><a href="#Queue">Converting Arbitrarily-Ordered Invocations</a></li>
1.8 +<li><a href="#Queue">Converting Arbitrarily-Ordered Invocations</a>
1.9 + <ul>
1.10 + <li><a href="#Exchange">Replacing Queues with Exchanges</li></a>
1.11 + <li><a href="#channel">Using Channels in Callables</li></a>
1.12 + </ul>
1.13 +</li>
1.14 <li><a href="#create">Converting Inline Computations</a></li>
1.15 <li><a href="#MakeReusable">Reusing Processes in Parallel Programs</a></li>
1.16 <li><a href="#continuous">Supporting Continuous Processes in Parallel Programs</a></li>
1.17 @@ -309,6 +314,8 @@
1.18 taking advantage of the more relevant "positional" information emerging from
1.19 the queue.</p>
1.20
1.21 +<h3 id="Exchange">Replacing Queues with Exchanges</h3>
1.22 +
1.23 <p>We can take this example further, illustrating some of the mechanisms
1.24 employed by <code>pprocess</code>. Instead of collecting results in a queue,
1.25 we can define a class containing a method which is called when new results
1.26 @@ -382,6 +389,8 @@
1.27 other structure so that they may be processed and assigned to the correct
1.28 positions in the result array.</p>
1.29
1.30 +<h3 id="channel">Using Channels in Callables</h3>
1.31 +
1.32 <p>For the curious, we may remove some of the remaining conveniences of the
1.33 above program to expose other features of <code>pprocess</code>. First, we
1.34 define a slightly modified version of the <code>calculate</code> function:</p>
1.35 @@ -592,6 +601,85 @@
1.36 creating a new process for each computation and then discarding it, only to
1.37 create a new process for the next computation.</p>
1.38
1.39 +<h2 id="continuous">Supporting Continuous Processes in Parallel Programs</h2>
1.40 +
1.41 +<p>Although reusable processes offer the opportunity to invoke a callable over
1.42 +and over within the same created process, they do not fully support the
1.43 +potential of the underlying mechanisms in <code>pprocess</code>: created
1.44 +processes can communicate multiple values to the creating process and can
1.45 +theoretically run within the same callable forever.</p>
1.46 +
1.47 +<p>Consider this modified form of the <code>calculate</code> function:</p>
1.48 +
1.49 +<pre>
1.50 +def calculate(ch, i):
1.51 +
1.52 + """
1.53 + A supposedly time-consuming calculation on 'i'.
1.54 + """
1.55 +
1.56 + for j in range(0, N):
1.57 + time.sleep(delay)
1.58 + ch.send((i, j, i * N + j))
1.59 +</pre>
1.60 +
1.61 +<p>This function accepts a channel <code>ch</code> together with an argument
1.62 +<code>i</code> corresponding to an entire row of the input array, as opposed
1.63 +to having two arguments (<code>i</code> and <code>j</code>) corresponding to a
1.64 +single cell in the input array. In this function, a series of calculations are
1.65 +performed and a number of values are returned through the channel, without the
1.66 +function terminating until all values have been returned for the row data.</p>
1.67 +
1.68 +<p>To use this modified function, a modified version of the
1.69 +<a href="#simple_managed_queue">simple_managed_queue</a> program is used:</p>
1.70 +
1.71 +<pre>
1.72 + t = time.time()
1.73 +
1.74 + # Initialise the communications queue with a limit on the number of
1.75 + # channels/processes.
1.76 +
1.77 + queue = pprocess.Queue(limit=limit<strong>, continuous=1</strong>)
1.78 +
1.79 + # Initialise an array.
1.80 +
1.81 + results = [0] * N * N
1.82 +
1.83 + # Manage the calculate function.
1.84 +
1.85 + calc = queue.manage(<strong>calculate</strong>)
1.86 +
1.87 + # Perform the work.
1.88 +
1.89 + print "Calculating..."
1.90 + for i in range(0, N):
1.91 + <strong>calc(i)</strong>
1.92 +
1.93 + # Store the results as they arrive.
1.94 +
1.95 + print "Finishing..."
1.96 + for i, j, result in queue:
1.97 + results[i*N+j] = result
1.98 +
1.99 + # Show the results.
1.100 +
1.101 + for i in range(0, N):
1.102 + for result in results[i*N:i*N+N]:
1.103 + print result,
1.104 + print
1.105 +
1.106 + print "Time taken:", time.time() - t
1.107 +</pre>
1.108 +
1.109 +<p>(This code in context with <code>import</code> statements and functions is
1.110 +found in the <code>examples/simple_continuous_queue.py</code> file.)</p>
1.111 +
1.112 +<p>Although the inner loop in the work section relocated to the
1.113 +<code>calculate</code> function, the queue still receives outputs from that
1.114 +function with positional information and a result for the result array. Thus,
1.115 +no change is needed for the retrieval of the results: they arrive in the queue
1.116 +as before.</p>
1.117 +
1.118 <h2 id="BackgroundCallable">Performing Computations in Background Processes</h2>
1.119
1.120 <p>Occasionally, it is desirable to initiate time-consuming computations and to
1.121 @@ -714,10 +802,6 @@
1.122 be collected by the queue: a list containing all of the results produced in the
1.123 computation.</p>
1.124
1.125 -<h2 id="continuous">Supporting Continuous Processes in Parallel Programs</h2>
1.126 -
1.127 -<p><strong>To be written.</strong></p>
1.128 -
1.129 <h2 id="ManagingBackgroundProcesses">Managing Several Background Processes</h2>
1.130
1.131 <p>In the above example, a single background process was used to manage a number
1.132 @@ -887,11 +971,15 @@
1.133 <td>MakeParallel, Map, manage</td>
1.134 </tr>
1.135 <tr>
1.136 - <td rowspan="5">simple2</td>
1.137 + <td rowspan="6">simple2</td>
1.138 <td>simple_managed_queue</td>
1.139 <td>MakeParallel, Queue, manage</td>
1.140 </tr>
1.141 <tr>
1.142 + <td>simple_continuous_queue</td>
1.143 + <td>Queue, manage (continuous)</td>
1.144 + </tr>
1.145 + <tr>
1.146 <td>simple_managed</td>
1.147 <td>MakeParallel, Exchange (subclass), manage, finish</td>
1.148 </tr>