# HG changeset patch # User Paul Boddie # Date 1217116831 -7200 # Node ID 88ffc0d1c657a52c42a4aee4be5efd5d69ff620a # Parent 1f407246b42c616b39828b9f1d29491efad50130 Moved parts of the documentation into the docs directory. Rewrote the exceptions documentation. Introduced dynamic lambda objects since lambdas with defaults may employ a variation of default values, not a selection of one-time definitions. Made default parameter generation sensitive to the lambda case. Removed default parameters for lambdas from the image. Changed the known context in class invocations to instances of the new Undefined class (instead of Instance). Added a test of instance construction. Added a StoreContext instruction to support lambda objects. Make MakeObject an immediate instruction. diff -r 1f407246b42c -r 88ffc0d1c657 README.txt --- a/README.txt Sat Jul 26 01:16:02 2008 +0200 +++ b/README.txt Sun Jul 27 02:00:31 2008 +0200 @@ -24,209 +24,6 @@ * Functions, if they share compatible signatures, could share parameter list definitions. -Data Structures -=============== - -The fundamental "value type" is a pair of references: one pointing to the -referenced object represented by the interchangeable value; one referring to -the context of the referenced object, typically the object through which the -referenced object was acquired as an attribute. - -Value Layout ------------- - - 0 1 - object context - reference reference - -Acquiring Values ----------------- - -Values are acquired through name lookups and attribute access, yielding -the appropriate object reference together with a context reference as -indicated in the following table: - - Type of Access Context Notes - -------------- ------- ----- - - Local name Preserved Functions provide no context - - Global name Preserved Modules provide no context - - Class-originating Accessor Class accessor preserves the stored - attribute -or- context; instance accessor overrides - Preserved the stored context if it is null or - belongs to the instance's class - hierarchy - - Instance-originating Preserved Methods retain their original context - attribute - -There may be some scope for simplifying the above, to the detriment of Python -compatibility, since the unbound vs. bound methods situation can be confusing. - -Manipulating Values -------------------- - -According to the table describing value acquisition, different instructions -must implement different operations when acquiring values: - - Instruction Purpose Context Operations - ----------- ------- ------------------ - - LoadConst Load class, function, Combine null context with loaded - module, constant object - - LoadAddress Load attribute from Classes, functions and modules - known object stored as cause the loaded attribute to be - an attribute retrieved unchanged; whereas - constants (representing instances) - cause the constant to override the - attribute's own context (since all - attributes should belong to the - constant's class hierarchy) - - LoadAttr Load attribute from Attributes with null contexts or - instance stored as an contexts compatible with the - attribute instance cause loaded attributes - to combine the instance as context - with the object from the - attribute; other attributes have - their context preserved - - LoadAttrIndex Load attribute from Classes, functions and modules as - unknown object stored the unknown object accessor cause - as an attribute the loaded attribute to be - retrieved unchanged; whereas - instances cause the LoadAttr rules - to apply - -Consequently, a certain amount of run-time testing is required for both -LoadAttr and LoadAttrIndex. - -Objects -------- - -Since classes, functions and instances are all "objects", each must support -certain features and operations in the same way. - -The __class__ Attribute ------------------------ - -All objects support the __class__ attribute: - -Class: refers to the type class (type.__class__ also refers to the type class) -Function: refers to the function class -Instance: refers to the class instantiated to make the object - -Invocation ----------- - -The following actions need to be supported: - -Class: create instance, call __init__ with instance, return object -Function: call function body, return result -Instance: call __call__ method, return result - -Structure Layout ----------------- - -A suitable structure layout might be something like this: - - Identifier Address Type Object ... - - 0 1 2 3 4 - classcode invocation __class__ attribute ... - reference reference reference - -Here, the classcode refers to the attribute lookup table for the object. Since -classes and instances share the same classcode, they might resemble the -following: - -Class C: - - 0 1 2 3 4 - code for C __new__ class type attribute ... - reference reference reference - -Instance of C: - - 0 1 2 3 4 - code for C C.__call__ class C attribute ... - reference reference reference - (if exists) - -The __new__ reference would lead to code consisting of the following -instructions: - - create instance for C - call C.__init__(instance, ...) - return instance - -If C has a __call__ attribute, the invocation "slot" of C instances would -refer to the same thing as C.__call__. - -For functions, the same general layout applies: - -Function f: - - 0 1 2 3 4 - code for code class attribute ... - function reference function reference - reference - -Here, the code reference would lead to code for the function. Note that the -function locals are completely distinct from this structure and are not -comparable to attributes. - -For modules, there is no meaningful invocation reference: - -Module m: - - 0 1 2 3 4 - code for m (unused) module type attribute ... - reference (global) - reference - -Both classes and modules have code in their definitions, but this would be -generated in order and not referenced externally. - -Invocation Operation --------------------- - -Consequently, regardless of the object an invocation is always done as -follows: - - get invocation reference (at object+1) - jump to reference - -Additional preparation is necessary before the above code: positional -arguments must be saved to the parameter stack, and keyword arguments must be -resolved and saved to the appropriate position in the parameter stack. - -Attribute Operations --------------------- - -Attribute access needs to go through the attribute lookup table. Some -optimisations are possible and are described in the appropriate section. - -One important aspect of attribute access is the appropriate setting of the -context in the acquired attribute value. From the table describing the -acquisition of values, it is clear that the principal exception is that where -a class-originating attribute is accessed on an instance. Consequently, the -following algorithm could be employed once an attribute has been located: - - 1. If the attribute's context is a special value, indicating that it should - be replaced upon instance access, then proceed to the next step; - otherwise, acquire both the context and the object as they are. - - 2. If the accessor is an instance, use that as the value's context, acquiring - only the object from the attribute. - -Where accesses can be determined ahead of time (as discussed in the -optimisations section), the above algorithm may not necessarily be employed in -the generated code for some accesses. - Instruction Evaluation Model ============================ diff -r 1f407246b42c -r 88ffc0d1c657 docs/exceptions.txt --- a/docs/exceptions.txt Sat Jul 26 01:16:02 2008 +0200 +++ b/docs/exceptions.txt Sun Jul 27 02:00:31 2008 +0200 @@ -1,13 +1,15 @@ Exception Handling ------------------ -Where exceptions may be raised, the following rules are applied: +An exception handler stack is defined such that when a try...except or +try...finally block is entered, a new handler is defined. - 1. If exception labels exist, any raised exception causes a jump to the - handler label. +When an exception is raised, the program jumps to the most recently defined +handler. Inside the handler, the stack entry for the handler will be removed. - 2. If no exception labels exist, any raised exception causes the current - function to be terminated with an exception condition. +Depending on the nature of the handler and whether the exception is handled, +the program may jump to the next most recent handler, and so on. - 3. Where an invocation returns with an exception condition set, rules #1 and - #2 are applied. +If no handler is defined when an exception is raised or re-raised, the program +should terminate. This might be done by having a "handler #0" which explicitly +terminates the program. diff -r 1f407246b42c -r 88ffc0d1c657 docs/invocation.txt --- a/docs/invocation.txt Sat Jul 26 01:16:02 2008 +0200 +++ b/docs/invocation.txt Sun Jul 27 02:00:31 2008 +0200 @@ -19,7 +19,7 @@ Least expensive cases: - f(1, 2, 3) # put arguments on stack + f(1, 2, 3) # put arguments in frame # if f is not known, add arguments vs. parameters check f(1, 2) # to handle defaults, introduce default "filling" where # not enough arguments are given @@ -27,7 +27,7 @@ More expensive cases: - f(1, 2, c=3) # prepare stack using parameter details + f(1, 2, c=3) # prepare frame using parameter details # (provided c is a known parameter) # if f is not known, this is obviously done at run-time f(1, c=3) # as with the previous case, with default "filling" done @@ -89,6 +89,16 @@ 1 -> argument #3 2 -> argument #4 +Argument lists for classes: + + f(obj, 1, 2) # f known as C at compile-time + + f -> C.__new__ - don't get any context information + -> any __init__ method will be called from C.__new__ + obj -> argument #1 + 1 -> argument #2 + 2 -> argument #3 + Argument lists for unknown callables: f(obj, 1, 2) # f not known at compile-time diff -r 1f407246b42c -r 88ffc0d1c657 docs/structures.txt --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/structures.txt Sun Jul 27 02:00:31 2008 +0200 @@ -0,0 +1,203 @@ +Data Structures +=============== + +The fundamental "value type" is a pair of references: one pointing to the +referenced object represented by the interchangeable value; one referring to +the context of the referenced object, typically the object through which the +referenced object was acquired as an attribute. + +Value Layout +------------ + + 0 1 + object context + reference reference + +Acquiring Values +---------------- + +Values are acquired through name lookups and attribute access, yielding +the appropriate object reference together with a context reference as +indicated in the following table: + + Type of Access Context Notes + -------------- ------- ----- + + Local name Preserved Functions provide no context + + Global name Preserved Modules provide no context + + Class-originating Accessor Class accessor preserves the stored + attribute -or- context; instance accessor overrides + Preserved the stored context if it is null or + belongs to the instance's class + hierarchy + + Instance-originating Preserved Methods retain their original context + attribute + +There may be some scope for simplifying the above, to the detriment of Python +compatibility, since the unbound vs. bound methods situation can be confusing. + +Manipulating Values +------------------- + +According to the table describing value acquisition, different instructions +must implement different operations when acquiring values: + + Instruction Purpose Context Operations + ----------- ------- ------------------ + + LoadConst Load class, function, Combine null context with loaded + module, constant object + + LoadAddress Load attribute from Classes, functions and modules + known object stored as cause the loaded attribute to be + an attribute retrieved unchanged; whereas + constants (representing instances) + cause the constant to override the + attribute's own context (since all + attributes should belong to the + constant's class hierarchy) + + LoadAttr Load attribute from Attributes with null contexts or + instance stored as an contexts compatible with the + attribute instance cause loaded attributes + to combine the instance as context + with the object from the + attribute; other attributes have + their context preserved + + LoadAttrIndex Load attribute from Classes, functions and modules as + unknown object stored the unknown object accessor cause + as an attribute the loaded attribute to be + retrieved unchanged; whereas + instances cause the LoadAttr rules + to apply + +Consequently, a certain amount of run-time testing is required for both +LoadAttr and LoadAttrIndex. + +Objects +------- + +Since classes, functions and instances are all "objects", each must support +certain features and operations in the same way. + +The __class__ Attribute +----------------------- + +All objects support the __class__ attribute: + +Class: refers to the type class (type.__class__ also refers to the type class) +Function: refers to the function class +Instance: refers to the class instantiated to make the object + +Invocation +---------- + +The following actions need to be supported: + +Class: create instance, call __init__ with instance, return object +Function: call function body, return result +Instance: call __call__ method, return result + +Structure Layout +---------------- + +A suitable structure layout might be something like this: + + Identifier Address Type Object ... + + 0 1 2 3 4 + classcode invocation __class__ attribute ... + reference reference reference + +Here, the classcode refers to the attribute lookup table for the object. Since +classes and instances share the same classcode, they might resemble the +following: + +Class C: + + 0 1 2 3 4 + code for C __new__ class type attribute ... + reference reference reference + +Instance of C: + + 0 1 2 3 4 + code for C C.__call__ class C attribute ... + reference reference reference + (if exists) + +The __new__ reference would lead to code consisting of the following +instructions: + + create instance for C + call C.__init__(instance, ...) + return instance + +If C has a __call__ attribute, the invocation "slot" of C instances would +refer to the same thing as C.__call__. + +For functions, the same general layout applies: + +Function f: + + 0 1 2 3 4 + code for code class attribute ... + function reference function reference + reference + +Here, the code reference would lead to code for the function. Note that the +function locals are completely distinct from this structure and are not +comparable to attributes. Instead, attributes are reserved for default +parameter values. + +For modules, there is no meaningful invocation reference: + +Module m: + + 0 1 2 3 4 + code for m (unused) module type attribute ... + reference (global) + reference + +Both classes and modules have code in their definitions, but this would be +generated in order and not referenced externally. + +Invocation Operation +-------------------- + +Consequently, regardless of the object an invocation is always done as +follows: + + get invocation reference (at object+1) + jump to reference + +Additional preparation is necessary before the above code: positional +arguments must be saved to the parameter stack, and keyword arguments must be +resolved and saved to the appropriate position in the parameter stack. + +Attribute Operations +-------------------- + +Attribute access needs to go through the attribute lookup table. Some +optimisations are possible and are described in the appropriate section. + +One important aspect of attribute access is the appropriate setting of the +context in the acquired attribute value. From the table describing the +acquisition of values, it is clear that the principal exception is that where +a class-originating attribute is accessed on an instance. Consequently, the +following algorithm could be employed once an attribute has been located: + + 1. If the attribute's context is a special value, indicating that it should + be replaced upon instance access, then proceed to the next step; + otherwise, acquire both the context and the object as they are. + + 2. If the accessor is an instance, use that as the value's context, acquiring + only the object from the attribute. + +Where accesses can be determined ahead of time (as discussed in the +optimisations section), the above algorithm may not necessarily be employed in +the generated code for some accesses. diff -r 1f407246b42c -r 88ffc0d1c657 micropython/__init__.py --- a/micropython/__init__.py Sat Jul 26 01:16:02 2008 +0200 +++ b/micropython/__init__.py Sun Jul 27 02:00:31 2008 +0200 @@ -194,9 +194,11 @@ pos += 1 # Append any default values to the image. + # Only do this for named functions (not lambdas). - image += obj.default_attrs - pos += len(obj.default_attrs) + if obj.name is not None: + image += obj.default_attrs + pos += len(obj.default_attrs) # Append the function code to the image. diff -r 1f407246b42c -r 88ffc0d1c657 micropython/ast.py --- a/micropython/ast.py Sat Jul 26 01:16:02 2008 +0200 +++ b/micropython/ast.py Sun Jul 27 02:00:31 2008 +0200 @@ -305,7 +305,7 @@ "Return whether 'instruction' provides a simple input." - return isinstance(instruction, (LoadConst, LoadName, LoadTemp, LoadResult, LoadAddress)) + return isinstance(instruction, (LoadConst, LoadName, LoadTemp, LoadResult, LoadAddress, MakeObject)) def _is_simple_input_user(self, instruction): @@ -314,7 +314,7 @@ return isinstance(instruction, ( StoreTemp, StoreFrame, StoreResult, StoreException, # as the value being stored LoadAddressContext, LoadAttr, LoadAttrIndex, # as the object being referenced - StoreAttr, StoreAttrIndex # as the object being referenced + StoreAttr, StoreAttrIndex, StoreCallable # as the object being referenced )) def _is_input(self, instruction): @@ -409,7 +409,7 @@ if isinstance(target, Class): target = target.get_instantiator() - context = Instance() + context = Undefined() # A special context is chosen to avoid generating unnecessary # context loading and checking instructions. @@ -628,7 +628,7 @@ """ target, context, temp = self._generateCallFuncContext() - self._generateCallFuncArgs(target, context, args, node) + self._generateCallFuncArgs(target, context, temp, args, node) return temp def _generateCallFuncContext(self): @@ -665,12 +665,13 @@ return target, context, temp - def _generateCallFuncArgs(self, target, context, args, node): + def _generateCallFuncArgs(self, target, context, temp, args, node): """ - Given invocation 'target' and 'context' information, a list of nodes - representing the 'args' (arguments), generate instructions which load - the arguments for the invocation defined by the given 'node'. + Given invocation 'target' and 'context' information, the 'temp' + reference to the target, a list of nodes representing the 'args' + (arguments), generate instructions which load the arguments for the + invocation defined by the given 'node'. """ # Evaluate the arguments. @@ -819,15 +820,8 @@ "Too many arguments for %r: need at most %d arguments." % (target.name, nargs_max)) # Where defaults are involved, put them into the frame. - # Here, we use negative index values to visit the right hand end of - # the defaults list. - for pos in range(nargs_min, nargs_max): - if pos not in employed_positions: - self.new_op(LoadAddress(target.default_attrs[pos - nargs_min])) - self.new_op(StoreFrame(pos)) - - frame_pos += 1 + self._generateCallFuncDefaultArgs(target, temp, nargs_min, nargs_max, employed_positions) # Or generate instructions to do this at run-time. # NOTE: CheckFrame has to check the number of arguments and to fill in @@ -837,6 +831,32 @@ else: self.new_op(CheckFrame()) + def _generateCallFuncDefaultArgs(self, target, temp, nargs_min, nargs_max, employed_positions): + + """ + For the given 'target' and 'temp' reference to the target, generate + default arguments for those positions in the range 'nargs_min'... + 'nargs_max' which are not present in the 'employed_positions' + collection. + """ + + # Where a lambda is involved, construct a dynamic object to hold the + # defaults. + + dynamic = target.name is None + + # Here, we use negative index values to visit the right hand end of + # the defaults list. + + for pos in range(nargs_min, nargs_max): + if pos not in employed_positions: + if dynamic: + self.new_op(temp) + self.new_op(LoadAttr(target.default_attrs[pos - nargs_min])) + else: + self.new_op(LoadAddress(target.default_attrs[pos - nargs_min])) + self.new_op(StoreFrame(pos)) + def _doCallFunc(self, instruction): "Make the invocation." @@ -856,6 +876,44 @@ if instruction is not None: self.discard_temp(instruction) + def _generateFunctionDefaults(self, function): + + """ + Generate the default initialisation code for 'function', returning + a temporary storage reference if a dynamic object was created for the + function. + """ + + attr_to_default = zip(function.default_attrs, function.defaults) + if not attr_to_default: + return None + + # Where a lambda is involved, construct a dynamic object to hold the + # defaults. + + dynamic = function.name is None + + if dynamic: + self.new_op(MakeObject(len(attr_to_default))) + temp = self.get_temp() + + for attr, default in attr_to_default: + self.dispatch(default) + + self.record_value() + if dynamic: + self.new_op(temp) + self.new_op(StoreAttr(attr)) + else: + self.new_op(StoreAddress(attr)) + self.set_source() + self.discard_value() + + if dynamic: + return temp + else: + return None + def _visitName(self, node, classes): """ @@ -1087,18 +1145,6 @@ self.discard_temp(temp1) self.discard_temp(temp2) - def _generateFunctionDefaults(self, function): - - "Generate the default initialisation code for 'function'." - - for attr, default in zip(function.default_attrs, function.defaults): - self.dispatch(default) - - self.record_value() - self.new_op(StoreAddress(attr)) - self.set_source() - self.discard_value() - # Concrete visitor methods. def visitAdd(self, node): @@ -1375,9 +1421,21 @@ # outside. if self.unit is not node.unit: - self._generateFunctionDefaults(node.unit) + temp = self._generateFunctionDefaults(node.unit) self.new_op(LoadConst(node.unit)) + # Populate the new object required for the function. + + if temp is not None: + self.record_value() + self.new_op(temp) + self.new_op(StoreCallable()) + self.set_source() + self.discard_value() + + self.new_op(temp) + self.discard_temp(temp) + # Visiting of the code occurs when get_code is invoked on this node. else: diff -r 1f407246b42c -r 88ffc0d1c657 micropython/common.py --- a/micropython/common.py Sat Jul 26 01:16:02 2008 +0200 +++ b/micropython/common.py Sun Jul 27 02:00:31 2008 +0200 @@ -139,4 +139,10 @@ else: return self.parent.full_name() +class Undefined: + + "A special class of undefined values." + + pass + # vim: tabstop=4 expandtab shiftwidth=4 diff -r 1f407246b42c -r 88ffc0d1c657 micropython/inspect.py --- a/micropython/inspect.py Sat Jul 26 01:16:02 2008 +0200 +++ b/micropython/inspect.py Sun Jul 27 02:00:31 2008 +0200 @@ -300,6 +300,8 @@ self.store(name, function) else: self.store_lambda(function) + if node.defaults: + return None # NOTE: See lambda code in the ast module. return function diff -r 1f407246b42c -r 88ffc0d1c657 micropython/rsvp.py --- a/micropython/rsvp.py Sat Jul 26 01:16:02 2008 +0200 +++ b/micropython/rsvp.py Sun Jul 27 02:00:31 2008 +0200 @@ -162,7 +162,7 @@ class LoadAddress(Address): "Load the current value from the given fixed attribute address." class StoreAddress(Address): "Store the source value into the given fixed attribute address." class LoadAddressContext(Address): "Load the current value from the given fixed attribute address, making the current value the context." -class MakeObject(Instruction): "Make a new object. There isn't a complementary DropObject." +class MakeObject(Immediate): "Make a new object. There isn't a complementary DropObject." # Access to address-relative data. @@ -178,6 +178,7 @@ class StoreFrame(Immediate): "Store the current value as an argument for the parameter with the given position." class StoreFrameIndex(Immediate): "Store the current value as an argument for the parameter with the given index." class LoadCallable(Instruction): "Load the target of an invocation." +class StoreCallable(Instruction): "Store the source value into the object referenced by the current value." class LoadContext(Instruction): "Load the context of an invocation." class CheckFrame(Instruction): "Check the invocation frame and context for the target." class CheckSelf(Instruction): "Check the first argument of an invocation against the target." diff -r 1f407246b42c -r 88ffc0d1c657 tests/instance.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/tests/instance.py Sun Jul 27 02:00:31 2008 +0200 @@ -0,0 +1,13 @@ +#!/usr/bin/env python + +class C: + pass + +class D: + def __init__(self): + pass + +c = C() +d = D() + +# vim: tabstop=4 expandtab shiftwidth=4 diff -r 1f407246b42c -r 88ffc0d1c657 tests/lambda.py --- a/tests/lambda.py Sat Jul 26 01:16:02 2008 +0200 +++ b/tests/lambda.py Sun Jul 27 02:00:31 2008 +0200 @@ -1,18 +1,18 @@ #!/usr/bin/env python identity = lambda x: x +add_2 = lambda a, b=2: a + b -def f(): - return lambda a, b=2: a + b - -def f2(c): +def f(c): return lambda a, b=c: a + b def g(f, x): return f(x) +identity(1) +add_2(1) g(identity, 1) -g(f(), 1) -g(f2(3), 1) +g(add_2, 1) +g(f(3), 1) # vim: tabstop=4 expandtab shiftwidth=4