# HG changeset patch # User paulb # Date 1189885309 0 # Node ID 6949a574c55c566097ff1a2b551ea0b9e53f0119 # Parent 650cd5af09a5d2386d606cbff5c62cde6c8c50f3 [project @ 2007-09-15 19:41:49 by paulb] Changed the tutorial, adding usage of the Map class for programs which do not use the map built-in function. diff -r 650cd5af09a5 -r 6949a574c55c docs/tutorial.xhtml --- a/docs/tutorial.xhtml Sat Sep 15 19:41:09 2007 +0000 +++ b/docs/tutorial.xhtml Sat Sep 15 19:41:49 2007 +0000 @@ -1,7 +1,7 @@ - + pprocess - Tutorial @@ -13,32 +13,40 @@ in several processes. The most straightforward way of making a program parallel-aware - that is, where the program can take advantage of more than one processor to simultaneously process data - is to use the -pmap function. Consider the following Python code:

+pmap function.

+ +

Converting Map-Style Code

+ +

Consider a program using the built-in map function and a sequence of inputs:

     t = time.time()
 
+    # Initialise an array.
+
     sequence = []
     for i in range(0, N):
         for j in range(0, N):
             sequence.append((i, j))
 
+    # Perform the work.
+
     results = map(calculate, sequence)
 
-    print "Time taken:", time.time() - t
+    # Show the results.
+
     for i in range(0, N):
         for result in results[i*N:i*N+N]:
             print result,
         print
-
+ + print "Time taken:", time.time() - t

(This code in context with import statements and functions is found in the examples/simple_map.py file.)

-

Here, we initialise a sequence of inputs for an N by N grid, perform a -calculation on each element, then print the resulting sequence as a -grid. Since the map function performs the calculations sequentially, even if the calculate +

The principal features of this program involve the preparation of an array for input purposes, and the use of the map function to iterate over the combinations of i and j in the array. Even if the calculate function could be invoked independently for each input value, we have -to wait for each calculation to complete before initiating a new +to wait for each computation to complete before initiating a new one.

In order to reduce the processing time - to speed the code up, @@ -48,19 +56,25 @@ 

     t = time.time()
 
+    # Initialise an array.
+
     sequence = []
     for i in range(0, N):
         for j in range(0, N):
             sequence.append((i, j))
 
+    # Perform the work.
+
     results = pprocess.pmap(calculate, sequence, limit=limit)
 
-    print "Time taken:", time.time() - t
+    # Show the results.
+
     for i in range(0, N):
         for result in results[i*N:i*N+N]:
             print result,
         print
-
+ + print "Time taken:", time.time() - t

(This code in context with import statements and functions is found in the examples/simple_pmap.py file.)

@@ -68,5 +82,80 @@ function, and specifying the limit on the number of processes to be active at any given time, several calculations can now be performed in parallel.

+

Converting Invocations to Parallel Operations

+ +

Although some programs make natural use of the map function, others may employ an invocation in a nested loop. This may also be converted to a parallel program. Consider the following Python code:

+ +
+    t = time.time()
+
+    # Initialise an array.
+
+    results = []
+
+    # Perform the work.
+
+    print "Calculating..."
+    for i in range(0, N):
+        for j in range(0, N):
+            results.append(calculate(i, j))
+
+    # Show the results.
+
+    for i in range(0, N):
+        for result in results[i*N:i*N+N]:
+            print result,
+        print
+
+    print "Time taken:", time.time() - t
+ +

(This code in context with import statements and functions is found in the examples/simple1.py file.)

+ +

Here, a computation in the calculate function is performed for each combination of i and j +in the nested loop, returning a result value. However, we must wait for +the completion of this function for each element before moving on to +the next element, and this means that the computations are performed +sequentially. Consequently, on a system with more than one processor, +even if we could call calculate for more than one combination of i and j and have the computations executing at the same time, the above program will not take advantage of such capabilities.

+ +

In order to reduce the processing time - to speed the code up, +in other words - we can make this code use several processes instead of +just one. Here is the modified code:

+ +
+    t = time.time()
+
+    # Initialise the results using map with a limit on the number of
+    # channels/processes.
+
+    results = pprocess.Map(limit=limit)
+
+    # Wrap the calculate function and manage it.
+
+    calc = results.manage(pprocess.MakeParallel(calculate))
+
+    # Perform the work.
+
+    print "Calculating..."
+    for i in range(0, N):
+        for j in range(0, N):
+            calc(i, j)
+
+    # Show the results.
+
+    for i in range(0, N):
+        for result in results[i*N:i*N+N]:
+            print result,
+        print
+
+    print "Time taken:", time.time() - t
+ +

(This code in context with import statements and functions is found in the examples/simple_manage_map.py file.)

+ +

The principal changes in the above code involve the use of a pprocess.Map object to collect the results, and a version of the calculate function which is managed by the Map object. What the Map +object does is to arrange the results of computations such that +iterating over the object or accessing the object using list operations +provides the results in the same order as their corresponding inputs.

+