Lichen

Changeset

864:7783f93f4704
2019-01-17 Paul Boddie raw files shortlog changelog graph Merged changes from the default branch. trailing-data
     1.1 --- a/docs/wiki/APIs	Sun Jul 22 12:14:48 2018 +0200
     1.2 +++ b/docs/wiki/APIs	Thu Jan 17 14:23:38 2019 +0100
     1.3 @@ -1,25 +1,31 @@
     1.4  = APIs =
     1.5  
     1.6 -The principal application programming interfaces (APIs) within Lichen are described below.
     1.7 +The principal application programming interfaces (APIs) within Lichen are
     1.8 +described below.
     1.9  
    1.10  == Modules ==
    1.11  
    1.12 -When [[../Inspection|inspecting]] and [[../Translation|translating]] modules, common abstractions are used so that elements of the program are handled in similar ways. Various useful attributes and methods are provided by the `CommonModule` abstraction for use within `InspectedModule` and `TranslatedModule` methods:
    1.13 +When [[../Inspection|inspecting]] and [[../Translation|translating]] modules,
    1.14 +common abstractions are used so that elements of the program are handled in
    1.15 +similar ways. Various useful attributes and methods are provided by the
    1.16 +`CommonModule` abstraction for use within `InspectedModule` and
    1.17 +`TranslatedModule` methods:
    1.18  
    1.19  {{{#!table
    1.20  '''Attribute or Method''' || '''Purpose'''
    1.21  ==
    1.22 -`name` ||
    1.23 -An attribute providing the module name.
    1.24 +`name` || An attribute providing the module name.
    1.25  ==
    1.26 -`get_global_path(name)` ||
    1.27 -A method returning the qualified name of the given global name within the module being processed.
    1.28 +`get_global_path(name)` || A method returning the qualified name of the given
    1.29 +                        .. global name within the module being processed.
    1.30  ==
    1.31 -`get_namespace_path()` ||
    1.32 -A method returning the qualified name of the namespace being processed.
    1.33 +`get_namespace_path()`  || A method returning the qualified name of the
    1.34 +                        .. namespace being processed.
    1.35 +
    1.36   * For modules, this is the module name
    1.37 - * For classes, functions and methods, the path incorporates the module name and namespaces leading to the current namespace itself
    1.38 + * For classes, functions and methods, the path incorporates the module name
    1.39 +   and namespaces leading to the current namespace itself
    1.40  ==
    1.41 -`get_object_path(name)` ||
    1.42 -A method returning the qualified name of the given local name within the namespace being processed.
    1.43 +`get_object_path(name)` || A method returning the qualified name of the given
    1.44 +                        .. local name within the namespace being processed.
    1.45  }}}
     2.1 --- a/docs/wiki/Builtins	Sun Jul 22 12:14:48 2018 +0200
     2.2 +++ b/docs/wiki/Builtins	Thu Jan 17 14:23:38 2019 +0100
     2.3 @@ -1,36 +1,65 @@
     2.4  = Built-Ins =
     2.5  
     2.6 -The "built-ins" are a collection of special names that do not need to be explicitly imported. For example:
     2.7 +The "built-ins" are a collection of special names that do not need to be
     2.8 +explicitly imported. For example:
     2.9  
    2.10  {{{#!python numbers=disable
    2.11  biggest = max([23, 19, 27]) # max is a built-in function
    2.12  }}}
    2.13  
    2.14 -Such names are always available as if they were defined in the current module. However, they are provided by a package hierarchy that seeks to divide them into isolated areas of functionality that can be included or excluded depending on the facilities employed by each program. For example, complex numbers are provided in the `__builtins__.complex` module, but for any program not employing complex numbers, this module will be excluded and its functionality not appear in the final program. (The exclusion of modules is achieved using the hidden module functionality provided by the [[../Imports|import mechanism]].)
    2.15 +Such names are always available as if they were defined in the current module.
    2.16 +However, they are provided by a package hierarchy that seeks to divide them
    2.17 +into isolated areas of functionality that can be included or excluded
    2.18 +depending on the facilities employed by each program. For example, complex
    2.19 +numbers are provided in the `__builtins__.complex` module, but for any program
    2.20 +not employing complex numbers, this module will be excluded and its
    2.21 +functionality not appear in the final program. (The exclusion of modules is
    2.22 +achieved using the hidden module functionality provided by the
    2.23 +[[../Imports|import mechanism]].)
    2.24  
    2.25 -The principal, "top-level" module providing built-ins is the `__builtins__` module whose only role is to expose the actual built-in names. It does so by importing names directly from the different submodules, such as `__builtins__.complex`, so that attempts to import names from `__builtins__` may provide such attempts with the named objects. The `__builtins__` module looks like this in such cases:
    2.26 +The principal, "top-level" module providing built-ins is the `__builtins__`
    2.27 +module whose only role is to expose the actual built-in names. It does so by
    2.28 +importing names directly from the different submodules, such as
    2.29 +`__builtins__.complex`, so that attempts to import names from `__builtins__`
    2.30 +may provide such attempts with the named objects. The `__builtins__` module
    2.31 +looks like this in such cases:
    2.32  
    2.33  {{{#!python numbers=disable
    2.34  from __builtins__.complex import complex
    2.35  }}}
    2.36  
    2.37 -Accesses to built-in names use the same technique of importing a name (`complex`) rather than the module providing it (`__builtins__`). It is as if the following code would appear before usage of a built-in name:
    2.38 +Accesses to built-in names use the same technique of importing a name
    2.39 +(`complex`) rather than the module providing it (`__builtins__`). It is as if
    2.40 +the following code would appear before usage of a built-in name:
    2.41  
    2.42  {{{#!python numbers=disable
    2.43  from __builtins__ import complex
    2.44  }}}
    2.45  
    2.46 -Since it is the specific name that is being referenced, not the module, the other contents of the module can be ignored and the reference to the named object followed to its actual definition location. Thus, usage of `complex` causes `__builtins__.complex` to be included in the program so that it may provide the `complex` type.
    2.47 +Since it is the specific name that is being referenced, not the module, the
    2.48 +other contents of the module can be ignored and the reference to the named
    2.49 +object followed to its actual definition location. Thus, usage of `complex`
    2.50 +causes `__builtins__.complex` to be included in the program so that it may
    2.51 +provide the `complex` type.
    2.52  
    2.53 -Thus, it becomes possible to keep a module like `__builtins__` out of the program since its only role would be to hold references to other modules' objects, but such specific imports permit the module to be bypassed by just following the declared import relationships. However, consider the consequences of `__builtins__` being imported as follows:
    2.54 +Thus, it becomes possible to keep a module like `__builtins__` out of the
    2.55 +program since its only role would be to hold references to other modules'
    2.56 +objects, but such specific imports permit the module to be bypassed by just
    2.57 +following the declared import relationships. However, consider the
    2.58 +consequences of `__builtins__` being imported as follows:
    2.59  
    2.60  {{{#!python numbers=disable
    2.61  import __builtins__
    2.62  }}}
    2.63  
    2.64 -Its entire contents would then need to be exposed because it would then be possible to access any name provided by the module via the module. This was not the case with a specific name import because there was no way of referencing the module itself as a result of such an import.
    2.65 +Its entire contents would then need to be exposed because it would then be
    2.66 +possible to access any name provided by the module via the module. This was
    2.67 +not the case with a specific name import because there was no way of
    2.68 +referencing the module itself as a result of such an import.
    2.69  
    2.70 -This would then cause all of the referenced modules to be imported because it would no longer be possible to readily identify the modules that would actually be needed by the program. For example:
    2.71 +This would then cause all of the referenced modules to be imported because it
    2.72 +would no longer be possible to readily identify the modules that would
    2.73 +actually be needed by the program. For example:
    2.74  
    2.75  {{{#!python numbers=disable
    2.76  def get_things(obj):
    2.77 @@ -41,4 +70,8 @@
    2.78  get_things(__builtins__)
    2.79  }}}
    2.80  
    2.81 -Of course, no module in a program should be referencing the `__builtins__` module by explicitly importing it, anyway. Given that all the names provided by that module are already available without any need to perform an import operation, such an import operation would have rather limited additional benefits.
    2.82 +Of course, no module in a program should be referencing the `__builtins__`
    2.83 +module by explicitly importing it, anyway. Given that all the names provided
    2.84 +by that module are already available without any need to perform an import
    2.85 +operation, such an import operation would have rather limited additional
    2.86 +benefits.
     3.1 --- a/docs/wiki/Cache	Sun Jul 22 12:14:48 2018 +0200
     3.2 +++ b/docs/wiki/Cache	Thu Jan 17 14:23:38 2019 +0100
     3.3 @@ -1,6 +1,13 @@
     3.4  = Inspection Cache Files =
     3.5  
     3.6 -The results of inspection for each module are written out to cache files, and these files should be able to provide all the information that is gathered during inspection without having to inspect the source code again. One minor benefit of using cached data, instead of having to parse and inspect the source code for a given module, is that of a slightly reduced processing time for the inspection activity. However, an arguably greater benefit is that of being able to see the outcome of the activity as a summary of accumulated data.
     3.7 +The results of inspection for each module are written out to cache files, and
     3.8 +these files should be able to provide all the information that is gathered
     3.9 +during inspection without having to inspect the source code again. One minor
    3.10 +benefit of using cached data, instead of having to parse and inspect the
    3.11 +source code for a given module, is that of a slightly reduced processing time
    3.12 +for the inspection activity. However, an arguably greater benefit is that of
    3.13 +being able to see the outcome of the activity as a summary of accumulated
    3.14 +data.
    3.15  
    3.16  Each cache file has the following general format:
    3.17  
    3.18 @@ -34,7 +41,8 @@
    3.19  ...
    3.20  
    3.21  }}}
    3.22 -|| Special name and corresponding reference plus comma-separated list of usage namespaces
    3.23 +|| Special name and corresponding reference plus comma-separated list of usage
    3.24 +.. namespaces
    3.25  ==
    3.26  {{{
    3.27  members:
    3.28 @@ -115,7 +123,8 @@
    3.29  
    3.30  }}}
    3.31  ||
    3.32 -Comma-separated parameter definitions for each function, with each definition being of the form...
    3.33 +Comma-separated parameter definitions for each function, with each definition
    3.34 +being of the form...
    3.35  {{{
    3.36  <name>=<references>
    3.37  }}}
    3.38 @@ -144,7 +153,10 @@
    3.39  
    3.40  }}}
    3.41  ||
    3.42 -Attribute usage details for the given name in the given namespace, with usages being a semicolon-separated list of usage alternatives, each being a comma-separated list of attribute names or {} (meaning no attribute names used), attribute names employing ! if invoked
    3.43 +Attribute usage details for the given name in the given namespace, with usages
    3.44 +being a semicolon-separated list of usage alternatives, each being a
    3.45 +comma-separated list of attribute names or {} (meaning no attribute names
    3.46 +used), attribute names employing ! if invoked
    3.47  ==
    3.48  {{{
    3.49  attribute accesses:
    3.50 @@ -160,7 +172,9 @@
    3.51  ...
    3.52  
    3.53  }}}
    3.54 -|| Identity of the given attribute chain in the given namespace, with any unresolved attribute chain provided
    3.55 +||
    3.56 +Identity of the given attribute chain in the given namespace, with any
    3.57 +unresolved attribute chain provided
    3.58  ==
    3.59  {{{
    3.60  attribute access usage:
    3.61 @@ -168,7 +182,9 @@
    3.62  ...
    3.63  
    3.64  }}}
    3.65 -|| Indicates, for each access involving the given name and first attribute name in the given namespace, the definitions that may provide the name
    3.66 +||
    3.67 +Indicates, for each access involving the given name and first attribute name
    3.68 +in the given namespace, the definitions that may provide the name
    3.69  ==
    3.70  {{{
    3.71  attribute access-modifiers:
    3.72 @@ -176,7 +192,10 @@
    3.73  ...
    3.74  
    3.75  }}}
    3.76 -|| Indicates, for accesses involving the given name and first attribute name in the given namespace, the modifiers applying to each access, where = indicates assignment, ! indicates invocation, and _ indicates access
    3.77 +||
    3.78 +Indicates, for accesses involving the given name and first attribute name in
    3.79 +the given namespace, the modifiers applying to each access, where = indicates
    3.80 +assignment, ! indicates invocation, and _ indicates access
    3.81  ==
    3.82  {{{
    3.83  constant literals:
    3.84 @@ -184,7 +203,9 @@
    3.85  ...
    3.86  
    3.87  }}}
    3.88 -|| Describes a constant literal in the given namespace having the indicated type, encoding (if a string), and value
    3.89 +||
    3.90 +Describes a constant literal in the given namespace having the indicated type,
    3.91 +encoding (if a string), and value
    3.92  ==
    3.93  {{{
    3.94  constant values:
     4.1 --- a/docs/wiki/Changelog	Sun Jul 22 12:14:48 2018 +0200
     4.2 +++ b/docs/wiki/Changelog	Thu Jan 17 14:23:38 2019 +0100
     4.3 @@ -1,3 +1,4 @@
     4.4  = Changelog =
     4.5  
     4.6 -Currently, only an initial release has been made. See the repository history for change-related information.
     4.7 +Currently, only an initial release has been made. See the repository history
     4.8 +for change-related information.
     5.1 --- a/docs/wiki/Deduction	Sun Jul 22 12:14:48 2018 +0200
     5.2 +++ b/docs/wiki/Deduction	Thu Jan 17 14:23:38 2019 +0100
     5.3 @@ -1,41 +1,85 @@
     5.4  = Deducing Program Behaviour =
     5.5  
     5.6 -With information from [[../Inspection|inspection]] available, the isolated observations about operations in a program may now be combined with knowledge about the program's structure to produce deductions about the nature of each operation.
     5.7 +With information from [[../Inspection|inspection]] available, the isolated
     5.8 +observations about operations in a program may now be combined with knowledge
     5.9 +about the program's structure to produce deductions about the nature of each
    5.10 +operation.
    5.11  
    5.12  <<TableOfContents(2,3)>>
    5.13  
    5.14  == The Deduction Process ==
    5.15  
    5.16 -The deduction process takes observations made during the [[../Inspection|inspection process]] and attempts to form deductions about the behaviour of the program primarily in terms of the nature of the attribute '''accesses''', with their corresponding '''accessors''', featuring in the program. Where attributes are used in conjunction with names, accessors are name versions; where attributes are used in conjunction with other expressions, accessors are '''anonymous'''.
    5.17 +The deduction process takes observations made during the [[../Inspection|
    5.18 +inspection process]] and attempts to form deductions about the behaviour of
    5.19 +the program primarily in terms of the nature of the attribute '''accesses''',
    5.20 +with their corresponding '''accessors''', featuring in the program. Where
    5.21 +attributes are used in conjunction with names, accessors are name versions;
    5.22 +where attributes are used in conjunction with other expressions, accessors are
    5.23 +'''anonymous'''.
    5.24  
    5.25  === Indexes ===
    5.26  
    5.27 -For efficiency, indexes are defined to establish relationships between the following things:
    5.28 +For efficiency, indexes are defined to establish relationships between the
    5.29 +following things:
    5.30  
    5.31  {{{#!table
    5.32 -'''Indexes''' || '''Details'''
    5.33 +'''Indexes''' || '''From''' || '''To''' || '''Purpose'''
    5.34 +==
    5.35 +`access_index`
    5.36 +|| access operation
    5.37 +|| accessor (name version)
    5.38 +|| defining types at access locations
    5.39  ==
    5.40 -`access_index` || Which accessors (name versions) are involved with access operations
    5.41 +`access_index_rev`
    5.42 +|| accessor (name version)
    5.43 +|| access operations
    5.44 +|| determining whether names are used for accesses; establishing alias
    5.45 +.. information
    5.46  ==
    5.47 -`location_index` || Which attribute usage patterns are supported by accessors (name versions)
    5.48 +`location_index`
    5.49 +|| accessor (name version)
    5.50 +|| attribute usage patterns
    5.51 +|| deducing types for names
    5.52  ==
    5.53  `attr_class_types`<<BR>>`attr_instance_types`<<BR>>`attr_module_types`
    5.54 -|| Which types support which attribute names
    5.55 +|| attribute name and assignment state
    5.56 +|| class, instance, module types
    5.57 +|| determining types supporting name accesses and assignments
    5.58  ==
    5.59  `assigned_attrs`
    5.60 -|| Which usage patterns involve attribute assignment
    5.61 +|| attribute usage pattern
    5.62 +|| attribute assignment locations
    5.63 +|| determining possibly mutated attributes on types
    5.64 +==
    5.65 +`alias_index`
    5.66 +|| alias (name version)
    5.67 +|| accesses
    5.68 +|| determining the identities of aliases (name versions) from initialising
    5.69 +.. name or attribute accesses
    5.70 +==
    5.71 +`alias_index_rev`
    5.72 +|| access
    5.73 +|| aliases (name versions)
    5.74 +|| propagating updated information from accesses to aliases
    5.75 +}}}
    5.76 +
    5.77 +Various collections are also maintained:
    5.78 +
    5.79 +{{{#!table
    5.80 +'''Collections''' || '''Details''' || '''Purpose'''
    5.81  ==
    5.82  `reference_assignments`
    5.83 -|| Which accesses involve assignments
    5.84 +|| accesses involving assignments
    5.85 +|| constraining accessor types; adjusting access plans
    5.86  ==
    5.87  `reference_invocations`
    5.88 -|| Which accesses involve invocations
    5.89 -==
    5.90 -`alias_index`
    5.91 -|| Which names are aliases for other names, accesses or invocations
    5.92 +|| accesses involving invocations
    5.93 +|| converting access types to instantiation or invocation results
    5.94  }}}
    5.95  
    5.96 -The objective of deduction is to combine these indexes to establish new relationships between the different participants of these basic index relationships.
    5.97 +The objective of deduction is to combine these indexes to establish new
    5.98 +relationships between the different participants of these basic index
    5.99 +relationships.
   5.100  
   5.101  === Building Indexes ===
   5.102  
   5.103 @@ -138,7 +182,11 @@
   5.104  
   5.105  === Converting Usage to Types ===
   5.106  
   5.107 -A particularly important operation in the deduction process is that of converting attribute usage information to a set of types supporting such usage. Taking the mapping of attribute names to types, each attribute name provided by a usage observation can be applied, progressively narrowing the set of types available to support the complete attribute usage collection.
   5.108 +A particularly important operation in the deduction process is that of
   5.109 +converting attribute usage information to a set of types supporting such
   5.110 +usage. Taking the mapping of attribute names to types, each attribute name
   5.111 +provided by a usage observation can be applied, progressively narrowing the
   5.112 +set of types available to support the complete attribute usage collection.
   5.113  
   5.114  {{{#!graphviz
   5.115  //format=svg
   5.116 @@ -188,9 +236,14 @@
   5.117  }
   5.118  }}}
   5.119  
   5.120 -The types supporting attribute usage are the attribute '''providers'''. Where providers are classes, the affected accessors in the program may also be instances, since instances also support access to attributes of the instantiated class (and its ancestors).
   5.121 +The types supporting attribute usage are the attribute '''providers'''. Where
   5.122 +providers are classes, the affected accessors in the program may also be
   5.123 +instances, since instances also support access to attributes of the
   5.124 +instantiated class (and its ancestors).
   5.125  
   5.126 -Indexes mapping attributes to types must combine consideration of class and instance attributes in order to correctly identify instance providers. Consider the following example:
   5.127 +Indexes mapping attributes to types must combine consideration of class and
   5.128 +instance attributes in order to correctly identify instance providers.
   5.129 +Consider the following example:
   5.130  
   5.131  {{{#!graphviz
   5.132  //format=svg
   5.133 @@ -235,13 +288,23 @@
   5.134  }
   5.135  }}}
   5.136  
   5.137 -To recognise support by instance accessors for the usage pattern concerned, attribute details must be obtained from classes as well as instances. Note that the identified type cannot support such usage purely by providing class or instance attributes alone.
   5.138 +To recognise support by instance accessors for the usage pattern concerned,
   5.139 +attribute details must be obtained from classes as well as instances. Note
   5.140 +that the identified type cannot support such usage purely by providing class
   5.141 +or instance attributes alone.
   5.142  
   5.143  === Attribute Assignments ===
   5.144  
   5.145 -Since attribute access operations may occur as part of assignments, the nature of accesses is recorded during inspection. Combining the indexed information for assignment accesses allows the deducer to determine the most pessimistic effects on the program structure of such assignments.
   5.146 +Since attribute access operations may occur as part of assignments, the nature
   5.147 +of accesses is recorded during inspection. Combining the indexed information
   5.148 +for assignment accesses allows the deducer to determine the most pessimistic
   5.149 +effects on the program structure of such assignments.
   5.150  
   5.151 -Taking each attribute usage set employed by accessors involved in assignments, the types are deduced for such accessors, and each individual attribute known to be used in such assignments is then applied to the deduced types, '''mutating''' them in such a way that deductions may no longer assume a fixed identity for such attributes when obtained from these types.
   5.152 +Taking each attribute usage set employed by accessors involved in assignments,
   5.153 +the types are deduced for such accessors, and each individual attribute known
   5.154 +to be used in such assignments is then applied to the deduced types,
   5.155 +'''mutating''' them in such a way that deductions may no longer assume a fixed
   5.156 +identity for such attributes when obtained from these types.
   5.157  
   5.158  {{{#!graphviz
   5.159  //format=svg
   5.160 @@ -277,87 +340,172 @@
   5.161  
   5.162  In the context of a specific access, the types may be resolved further:
   5.163  
   5.164 - * Any name whose initialisation could be determined during inspection can be associated with its initialised type
   5.165 - * Any name referring to a constant object can be associated with the type of that object
   5.166 - * Usage of `self` in methods can result in only compatible class and instance types being retained from the types obtained from usage deductions
   5.167 + * Any name whose initialisation could be determined during inspection can be
   5.168 +   associated with its initialised type
   5.169 +
   5.170 + * Any name referring to a constant object can be associated with the type of
   5.171 +   that object
   5.172 +
   5.173 + * Usage of `self` in methods can result in only compatible class and instance
   5.174 +   types being retained from the types obtained from usage deductions
   5.175  
   5.176  === Reference Identification ===
   5.177  
   5.178 -The basic information about every accessor and accessed attribute in a program can now be combined to produce specific '''references''' to identities consistent with the inspection observations. Several different kinds of accessors and access situations exist:
   5.179 +The basic information about every accessor and accessed attribute in a program
   5.180 +can now be combined to produce specific '''references''' to identities
   5.181 +consistent with the inspection observations. Several different kinds of
   5.182 +accessors and access situations exist:
   5.183  
   5.184   * Name-based accesses involving attribute usage
   5.185 +
   5.186   * Aliases to names, possibly accompanied by accesses
   5.187 +
   5.188   * Anonymous accesses involving individual attributes
   5.189 +
   5.190   * Constant or previously-identified names, possibly accompanied by accesses
   5.191  
   5.192  === Aliases ===
   5.193  
   5.194 -Names initialised using other names or attribute accesses, or using the invocation of either of these things, are known as '''aliases'''. Information about aliases originates during inspection when such names are initialised with expression results within the program. During the resolution activity finalising the inspected details, this initialisation information is used to define relationships between aliases and other names and accesses.
   5.195 +Names initialised using other names or attribute accesses, or using the
   5.196 +invocation of either of these things, are known as '''aliases'''. Information
   5.197 +about aliases originates during inspection when such names are initialised
   5.198 +with expression results within the program. During the resolution activity
   5.199 +finalising the inspected details, this initialisation information is used to
   5.200 +define relationships between aliases and other names and accesses.
   5.201  
   5.202 -During deduction, attribute accesses are investigated and type information computed for both attribute accesses and accessors. The relationships defined between accesses and aliases can then be employed to propagate such deduced information to the aliases and to define appropriate type characteristics for them.
   5.203 +During deduction, attribute accesses are investigated and type information
   5.204 +computed for both attribute accesses and accessors. The relationships defined
   5.205 +between accesses and aliases can then be employed to propagate such deduced
   5.206 +information to the aliases and to define appropriate type characteristics for
   5.207 +them.
   5.208  
   5.209 -Where aliases are used as the basis of attribute accesses, this propagation refines the previously deduced information about the aliases and may also refine the details of the accesses with which the alias is involved.
   5.210 +Where aliases are used as the basis of attribute accesses, this propagation
   5.211 +refines the previously deduced information about the aliases and may also
   5.212 +refine the details of the accesses with which the alias is involved.
   5.213  
   5.214  === Invocation Target Suitability ===
   5.215  
   5.216 -Having identified accessed attributes, invocation information can be applied in cases where such attributes immediately participate in an invocation, comparing the specified argument details against the parameter details defined for the identified attribute, which must be a callable object for this technique to work. Where arguments do not appear to be suitable - either the number of arguments is incorrect or any keyword argument refer to non-existent parameters - the attribute providing the parameter details can be considered unsuitable for the access.
   5.217 +Having identified accessed attributes, invocation information can be applied
   5.218 +in cases where such attributes immediately participate in an invocation,
   5.219 +comparing the specified argument details against the parameter details defined
   5.220 +for the identified attribute, which must be a callable object for this
   5.221 +technique to work. Where arguments do not appear to be suitable - either the
   5.222 +number of arguments is incorrect or any keyword argument refer to non-existent
   5.223 +parameters - the attribute providing the parameter details can be considered
   5.224 +unsuitable for the access.
   5.225  
   5.226  === Classifying Accessors ===
   5.227  
   5.228 -Each accessor, being a particular version of a name, will have type information associated with it as a consequence of the deduction process. Such information takes the following forms:
   5.229 +Each accessor, being a particular version of a name, will have type
   5.230 +information associated with it as a consequence of the deduction process. Such
   5.231 +information takes the following forms:
   5.232  
   5.233 - * Provider types, indicating which types may provide the attributes used by the accessor
   5.234 + * Provider types, indicating which types may provide the attributes used by
   5.235 +   the accessor
   5.236 +
   5.237   * Accessor types, indicating which types will actually appear as the accessor
   5.238  
   5.239 -This information can be processed in a number of ways to produce the following:
   5.240 +This information can be processed in a number of ways to produce the
   5.241 +following:
   5.242  
   5.243 - * All types (from all kinds of type) of providers able to provide attributes via the accessor
   5.244 - * All types (from all kinds of type) of accessors compatible with the accessor
   5.245 + * All types (from all kinds of type) of providers able to provide attributes
   5.246 +   via the accessor
   5.247 +
   5.248 + * All types (from all kinds of type) of accessors compatible with the
   5.249 +   accessor
   5.250 +
   5.251   * The most general types of accessors compatible with the accessor
   5.252  
   5.253 -Where many types may be associated with an accessor, identifying the most general types involves eliminating those which are derived from others. Given that descendant types may not remove support for attributes provided by their ancestors, then where an ancestor type has been identified as a possible accessor, it should follow that all of its descendants may also have been identified as possible accessors. Since these descendants should be compatible, identifying them individually is unnecessary: merely specifying that the common ancestor or ''any'' descendant could provide an accessor is sufficient.
   5.254 +Where many types may be associated with an accessor, identifying the most
   5.255 +general types involves eliminating those which are derived from others. Given
   5.256 +that descendant types may not remove support for attributes provided by their
   5.257 +ancestors, then where an ancestor type has been identified as a possible
   5.258 +accessor, it should follow that all of its descendants may also have been
   5.259 +identified as possible accessors. Since these descendants should be
   5.260 +compatible, identifying them individually is unnecessary: merely specifying
   5.261 +that the common ancestor or ''any'' descendant could provide an accessor is
   5.262 +sufficient.
   5.263  
   5.264  ==== Defining Guards ====
   5.265  
   5.266 -A '''guard''' is a constraint supported by a run-time test indicating the compliance of an accessor with a particular type. Thus, where a single accessor type has been identified, a guard may be established for an accessor that tests the type of the accessor against a specific type. Where a single ''general'' accessor type is identified, a guard is established that tests the accessor against a "common" type: that is, an ancestor type with which other descendant types may comply.
   5.267 +A '''guard''' is a constraint supported by a run-time test indicating the
   5.268 +compliance of an accessor with a particular type. Thus, where a single
   5.269 +accessor type has been identified, a guard may be established for an accessor
   5.270 +that tests the type of the accessor against a specific type. Where a single
   5.271 +''general'' accessor type is identified, a guard is established that tests the
   5.272 +accessor against a "common" type: that is, an ancestor type with which other
   5.273 +descendant types may comply.
   5.274  
   5.275  === Classifying Accesses ===
   5.276  
   5.277 -At the point of classifying accesses, information is available about the following:
   5.278 +At the point of classifying accesses, information is available about the
   5.279 +following:
   5.280  
   5.281   * The accessors potentially involved in each access
   5.282 - * The types of accessors and the types providing attributes via those accessors
   5.283 +
   5.284 + * The types of accessors and the types providing attributes via those
   5.285 +   accessors
   5.286 +
   5.287   * Any guards applying to the accessors
   5.288 - * Whether an access is constrained by certain program characteristics and is thus guaranteed to be as deduced
   5.289 +
   5.290 + * Whether an access is constrained by certain program characteristics and is
   5.291 +   thus guaranteed to be as deduced
   5.292 +
   5.293   * The possible attributes referenced by the access
   5.294  
   5.295 -This information can be processed in a number of ways to produce the following:
   5.296 +This information can be processed in a number of ways to produce the
   5.297 +following:
   5.298  
   5.299   * The types of accessors, both general and specific, applying to each access
   5.300 - * The attributes that can be provided by each access, consolidating existing referenced attribute details
   5.301 +
   5.302 + * The attributes that can be provided by each access, consolidating existing
   5.303 +   referenced attribute details
   5.304 +
   5.305   * The general types providing the attributes
   5.306  
   5.307 -Since more than one accessor may be involved, information from all accessors must be combined to determine whether guard information still applies to the access, or whether it is possible for an accessor to be used that has an uncertain type at run-time.
   5.308 +Since more than one accessor may be involved, information from all accessors
   5.309 +must be combined to determine whether guard information still applies to the
   5.310 +access, or whether it is possible for an accessor to be used that has an
   5.311 +uncertain type at run-time.
   5.312  
   5.313  ==== Defining Tests ====
   5.314  
   5.315 -A '''test''' at the access level is a necessary check performed on an accessor before an access to determine whether it belongs to a certain type or group of types.
   5.316 +A '''test''' at the access level is a necessary check performed on an accessor
   5.317 +before an access to determine whether it belongs to a certain type or group of
   5.318 +types.
   5.319  
   5.320 -Where guards applying to accessors apply by extension to an access, it may not be enough to assume that the the attributes exposed by the accessor are the same as those considered acceptable through deduction. Therefore, attributes are obtained for the access using the applicable guard types as accessors. If this set of attributes does not include potentially accessible attributes (perhaps because the guard types are broadly defined and do not support all attribute usage), a test must be generated.
   5.321 +Where guards applying to accessors apply by extension to an access, it may not
   5.322 +be enough to assume that the the attributes exposed by the accessor are the
   5.323 +same as those considered acceptable through deduction. Therefore, attributes
   5.324 +are obtained for the access using the applicable guard types as accessors. If
   5.325 +this set of attributes does not include potentially accessible attributes
   5.326 +(perhaps because the guard types are broadly defined and do not support all
   5.327 +attribute usage), a test must be generated.
   5.328  
   5.329 -Where a single attribute provider can be identified, a test for a specific type is generated. Where a single general type can be identified as a provider, a test against a "common" type is generated. Tests involving the built-in `object` are not generated since it is the root of all classes and such tests would merely identify an accessor as a class. In all cases where a single, specific type cannot be tested for, the general attribute validation mechanism must be used instead.
   5.330 +Where a single attribute provider can be identified, a test for a specific
   5.331 +type is generated. Where a single general type can be identified as a
   5.332 +provider, a test against a "common" type is generated. Tests involving the
   5.333 +built-in `object` are not generated since it is the root of all classes and
   5.334 +such tests would merely identify an accessor as a class. In all cases where a
   5.335 +single, specific type cannot be tested for, the general attribute validation
   5.336 +mechanism must be used instead.
   5.337  
   5.338  == Preparing Access Descriptions ==
   5.339  
   5.340 -The accumulated deduced knowledge is directed towards producing access descriptions or plans which characterise each access in terms of the following:
   5.341 +The accumulated deduced knowledge is directed towards producing access
   5.342 +descriptions or plans which characterise each access in terms of the
   5.343 +following:
   5.344  
   5.345 - * The initial accessor, being the starting object for attribute accesses, unless a static accessor has been identified
   5.346 + * The initial accessor, being the starting object for attribute accesses,
   5.347 +   unless a static accessor has been identified
   5.348   * Details of any test required on the initial accessor
   5.349   * Details of any type employed by the test
   5.350   * Any identified static accessor (to be used as the initial accessor)
   5.351 - * Attributes needing to be traversed from the accessor that yield unambiguous objects
   5.352 + * Attributes needing to be traversed from the accessor that yield
   5.353 +   unambiguous objects
   5.354   * Access modes for each of the unambiguously-traversed attributes
   5.355 - * Remaining attributes needing to be tested and traversed (after having traversed the above attributes)
   5.356 + * Remaining attributes needing to be tested and traversed (after having
   5.357 +   traversed the above attributes)
   5.358   * Details of the context
   5.359   * Any test to apply to the context (to ensure its validity)
   5.360   * The method of obtaining the first attribute
   5.361 @@ -367,79 +515,191 @@
   5.362  
   5.363  === Characterising the Accessor ===
   5.364  
   5.365 -For a given access location, the referenced attribute details established during deduction are used to determine...
   5.366 +For a given access location, the referenced attribute details established
   5.367 +during deduction are used to determine...
   5.368  
   5.369 - * Whether the initial accessor is static, originating from a constant access or involving an identifiable static object
   5.370 + * Whether the initial accessor is static, originating from a constant access
   5.371 +   or involving an identifiable static object
   5.372 +
   5.373   * Whether the initial accessor is dynamic but has a known, deduced identity
   5.374  
   5.375 -Some useful information about the accessor and about the actual provider of the first accessed attribute can be defined:
   5.376 +Some useful information about the accessor and about the actual provider of
   5.377 +the first accessed attribute can be defined:
   5.378  
   5.379  || || '''Class''' || '''Instance''' || '''Module''' ||
   5.380  || '''Accessor''' || Class accessor || Instance accessor || Module accessor ||
   5.381  || '''Provider''' || Class provider || Instance provider || ||
   5.382  
   5.383 -Depending on which of these criteria are satisfied, some properties of the access operation can be established:
   5.384 +Depending on which of these criteria are satisfied, some properties of the
   5.385 +access operation can be established:
   5.386 +
   5.387 + * Object-relative accesses occur with class accessors or module accessors or
   5.388 +   when attributes are provided by instances
   5.389  
   5.390 - * Object-relative accesses occur with class accessors or module accessors or when attributes are provided by instances
   5.391 - * Class-relative accesses occur with instance accessors when attributes are provided by classes
   5.392 + * Class-relative accesses occur with instance accessors when attributes are
   5.393 +   provided by classes
   5.394  
   5.395 -Object-relative accesses merely involve obtaining attributes directly from accessors. Class-relative accesses involve obtaining the class of each accessor and then obtaining an attribute from that class.
   5.396 +Object-relative accesses merely involve obtaining attributes directly from
   5.397 +accessors. Class-relative accesses involve obtaining the class of each
   5.398 +accessor and then obtaining an attribute from that class.
   5.399  
   5.400  === Defining the First Access Method ===
   5.401  
   5.402 -For dynamic or unidentified accessors, the above access properties define the access method on the first attribute in the chain of attributes provided. However, any redefinition of the accessor to a static accessor may influence the first method. For static accessors, the first method is always object-relative since classes and modules do not offer transparent mechanisms to expose the attributes on their own classes.
   5.403 +For dynamic or unidentified accessors, the above access properties define the
   5.404 +access method on the first attribute in the chain of attributes provided.
   5.405 +However, any redefinition of the accessor to a static accessor may influence
   5.406 +the first method. For static accessors, the first method is always
   5.407 +object-relative since classes and modules do not offer transparent mechanisms
   5.408 +to expose the attributes on their own classes.
   5.409  
   5.410 -Static and identified, dynamic accessors should already be known to support the specified attributes. Other accessors require an access method to be used that also checks whether the accessor supports a given attribute.
   5.411 +Static and identified, dynamic accessors should already be known to support
   5.412 +the specified attributes. Other accessors require an access method to be used
   5.413 +that also checks whether the accessor supports a given attribute.
   5.414  
   5.415  === Redefining the Accessor ===
   5.416  
   5.417 -With knowledge about the initial accessor, the attributes involved in the access operation are then considered in the context of the accessor. For each attribute name in the chain described for an access, an attempt is made to obtain the details of the attribute of the given name from the accessor, determining whether these details can be used as an accessor to obtain subsequent attribute details.
   5.418 +With knowledge about the initial accessor, the attributes involved in the
   5.419 +access operation are then considered in the context of the accessor. For each
   5.420 +attribute name in the chain described for an access, an attempt is made to
   5.421 +obtain the details of the attribute of the given name from the accessor,
   5.422 +determining whether these details can be used as an accessor to obtain
   5.423 +subsequent attribute details.
   5.424  
   5.425 -Where more than one attribute identity is obtained, traversal is terminated: all remaining attributes must be traversed at run-time. If an attribute identified during traversal is static, the first access method changes accordingly.
   5.426 +Where more than one attribute identity is obtained, traversal is terminated:
   5.427 +all remaining attributes must be traversed at run-time. If an attribute
   5.428 +identified during traversal is static, the first access method changes
   5.429 +accordingly.
   5.430  
   5.431  === Defining the Final Access Method ===
   5.432  
   5.433 -An access can also involve an assignment to the accessed attribute or the invocation of the accessed attribute. Such operations change the nature of the access method used on the final attribute in a chain of attribute names.
   5.434 +An access can also involve an assignment to the accessed attribute or the
   5.435 +invocation of the accessed attribute. Such operations change the nature of the
   5.436 +access method used on the final attribute in a chain of attribute names.
   5.437  
   5.438  === Identifying Context Information ===
   5.439  
   5.440 -Final attribute accesses involving callables need to yield context information that can subsequently be used to invoke those callables. Where the nature of an accessed attribute is not known, a simplistic attempt can be made to look up all attributes stored using the attribute name in the program.
   5.441 +Final attribute accesses involving callables need to incorporate context
   5.442 +information that can subsequently be used to invoke those callables. Where the
   5.443 +nature of an accessed attribute is not known, a simplistic attempt can be made
   5.444 +to look up all attributes stored using the attribute name in the program.
   5.445 +Otherwise, with knowledge of the attribute, its details can be inspected to
   5.446 +determine if context information plays a role in the access.
   5.447 +
   5.448 +==== Context Testing ====
   5.449  
   5.450  Of particular interest are the following situations:
   5.451  
   5.452 - * Where class attributes are being accessed via instances, whether the attributes are all methods that can be bound upon access
   5.453 - * Where class attributes may be accessed via instances, whether any attributes could be methods
   5.454 + * Where class attributes are being accessed via instances, whether the
   5.455 +   attributes are all methods that are bound to the instances
   5.456 +
   5.457 + * Where class attributes ''may'' be accessed via instances, whether any
   5.458 +   attributes ''could be'' methods
   5.459 +
   5.460 +Such considerations dictate whether the context information originates from
   5.461 +the attribute or from the accessor and whether any run-time test is required
   5.462 +to determine this. Thus, for attributes in general:
   5.463  
   5.464 -Such considerations dictate whether the context information originates from the attribute or from the accessor and whether any run-time test is required to determine this.
   5.465 +{{{#!table
   5.466 +'''Accessor''' || '''Provider''' || '''Attributes'''
   5.467 +|| '''Effect on Context''' || '''Remark'''
   5.468 +==
   5.469 +Always instances || Always classes || Always methods
   5.470 +|| Replacement
   5.471 +|| Permit method calling using the instance as context
   5.472 +==
   5.473 +Always instances || Always classes || Sometimes methods
   5.474 +|| Test at run-time
   5.475 +|| Preserve original context for non-methods
   5.476 +==
   5.477 +Sometimes instances || Sometimes classes || Sometimes methods
   5.478 +|| Test at run-time
   5.479 +|| Preserve original context for non-methods, non-instance accessors
   5.480 +}}}
   5.481 +
   5.482 +In all other situations, the available context is ignored, with the attribute
   5.483 +itself providing any stored context information.
   5.484 +
   5.485 +==== Context Identity ====
   5.486 +
   5.487 +Where the context is ignored, no effort will be made to obtain or retain it in
   5.488 +the program for the access operation: it will be unset. Otherwise, the context
   5.489 +will be defined as one of the following:
   5.490 +
   5.491 + * The "base" or static accessor where this is also the accessor for the final
   5.492 +   access
   5.493 +
   5.494 + * The original (or initial) accessor where this is also the accessor for the
   5.495 +   final access
   5.496 +
   5.497 + * The final accessor, having been identified through attribute traversal
   5.498 +
   5.499 +Note that non-static accessors may be computed dynamically and thus need to be
   5.500 +stored temporarily for subsequent use. 
   5.501  
   5.502  == Preparing Instruction Plans ==
   5.503  
   5.504 -Instruction plans are sequences of program operations, encoded in a higher-level form than actual program instructions, that describe the steps needed to access attributes. Such sequences are produced from the details provided by individual access plans.
   5.505 +Instruction plans are sequences of program operations, encoded in a
   5.506 +higher-level form than actual program instructions, that describe the steps
   5.507 +needed to access attributes. Such sequences are produced from the details
   5.508 +provided by individual access plans.
   5.509  
   5.510  === Original Accessor ===
   5.511  
   5.512 -The expression providing the object from which the first attribute is obtained (or the only attribute if only one is specified) is known as the original accessor. Where this accessor can be identified, the expression describing it in the program can potentially be replaced with a static reference, and subsequent mentions of the accessor can potentially be replaced with such references or names used as expressions.
   5.513 +The expression providing the object from which the first attribute is obtained
   5.514 +(or the only attribute if only one is specified) is known as the original
   5.515 +accessor. Where this accessor can be identified, the expression describing it
   5.516 +in the program can potentially be replaced with a static reference, and
   5.517 +subsequent mentions of the accessor can potentially be replaced with such
   5.518 +references or names used as expressions.
   5.519  
   5.520 -|| '''Access Plan Information''' || '''Original Accessor''' ||
   5.521 -|| Static accessor identified || Identified accessor ||
   5.522 -|| Named accessor access, not invocation || Indicated name ||
   5.523 -|| Named accessor invocation, accessor known to provide the attribute || Indicated name ||
   5.524 -|| Named accessor invocation, accessor not known to provide the attribute || Accessor expression ||
   5.525 -|| Other accessors || Accessor expression ||
   5.526 +{{{#!table
   5.527 +'''Access Plan Information''' || '''Original Accessor'''
   5.528 +==
   5.529 +Static accessor identified
   5.530 +|| Identified accessor
   5.531 +==
   5.532 +Named accessor access, not invocation
   5.533 +|| Indicated name
   5.534 +==
   5.535 +Named accessor invocation, accessor known to provide the attribute
   5.536 +|| Indicated name
   5.537 +==
   5.538 +Named accessor invocation, accessor not known to provide the attribute
   5.539 +|| Accessor expression
   5.540 +==
   5.541 +Other accessors
   5.542 +|| Accessor expression
   5.543 +}}}
   5.544  
   5.545 -By using names or static references, the need to store the result of evaluating an accessor expression is eliminated because such labels can be inserted in the instruction sequence as required.
   5.546 +By using names or static references, the need to store the result of
   5.547 +evaluating an accessor expression is eliminated because such labels can be
   5.548 +inserted in the instruction sequence as required.
   5.549  
   5.550  === First Access Operation ===
   5.551  
   5.552 -Although it may be possible to convert accesses into single instructions that obtain attributes directly, many accesses will involve access operations that must consult an accessor object and obtain an attribute from that object, at least for the first attribute in a chain of attributes. This occurs when the access plan indicates the following situations:
   5.553 +Although it may be possible to convert accesses into single instructions that
   5.554 +obtain attributes directly, many accesses will involve access operations that
   5.555 +must consult an accessor object and obtain an attribute from that object, at
   5.556 +least for the first attribute in a chain of attributes. This occurs when the
   5.557 +access plan indicates the following situations:
   5.558  
   5.559 - * Final method is an access (meaning that an attribute cannot be directly obtained)
   5.560 - * Final method is an assignment (requiring the object whose attribute will be updated)
   5.561 + * Final method is an access (meaning that an attribute cannot be directly
   5.562 +   obtained)
   5.563 +
   5.564 + * Final method is an assignment (requiring the object whose attribute will be
   5.565 +   updated)
   5.566 +
   5.567   * Attributes (identified or otherwise) need traversing
   5.568  
   5.569  === Accessor Nature ===
   5.570  
   5.571 -Attribute assignments involve a single '''target accessor''' and potentially many other accessors, depending on how many distinct expressions are evaluated to yield the value to be set in the assignment. Such a target accessor will usually be derived from the evaluation of an expression, and in some cases the expression will be the result of an opaque operation such as the invocation of a function. In such cases, the target accessor is stored in a temporary variable for subsequent use in access operations.
   5.572 +Attribute assignments involve a single '''target accessor''' and potentially
   5.573 +many other accessors, depending on how many distinct expressions are evaluated
   5.574 +to yield the value to be set in the assignment. Such a target accessor will
   5.575 +usually be derived from the evaluation of an expression, and in some cases the
   5.576 +expression will be the result of an opaque operation such as the invocation of
   5.577 +a function. In such cases, the target accessor is stored in a temporary
   5.578 +variable for subsequent use in access operations.
   5.579  
   5.580  === Context ===
   5.581  
   5.582 @@ -453,6 +713,176 @@
   5.583  
   5.584  === Context Testing ===
   5.585  
   5.586 +=== Instruction Details ===
   5.587 +
   5.588 +The emitted instructions are as follows.
   5.589 +
   5.590 +==== Direct Load ====
   5.591 +
   5.592 +These instructions employ the attribute position for the supplied attribute
   5.593 +name.
   5.594 +
   5.595 +{{{#!table
   5.596 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.597 +==
   5.598 +`__load_via_class` || object, attribute name
   5.599 +|| Obtain class from object; load attribute from class at position
   5.600 +==
   5.601 +`__load_via_object` || object, attribute name
   5.602 +|| Load attribute from object at position
   5.603 +==
   5.604 +`__get_class_and_load` || object, attribute name
   5.605 +|| Obtain class from object if instance; load attribute from result at
   5.606 +.. position
   5.607 +}}}
   5.608 +
   5.609 +==== Direct Store ====
   5.610 +
   5.611 +These instructions employ the attribute position for the supplied attribute
   5.612 +name, storing an attribute value.
   5.613 +
   5.614 +{{{#!table
   5.615 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.616 +==
   5.617 +`__store_via_class` || object, attribute name, value
   5.618 +|| Obtain class from object; store attribute in class at position
   5.619 +==
   5.620 +`__store_via_object` || object, attribute name, value
   5.621 +|| Store attribute in object at position
   5.622 +}}}
   5.623 +
   5.624 +==== Checked Load ====
   5.625 +
   5.626 +These instructions employ the attribute position and code for the supplied
   5.627 +attribute name.
   5.628 +
   5.629 +{{{#!table
   5.630 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.631 +==
   5.632 +`__check_and_load_via_class` || object, attribute name
   5.633 +|| Obtain class from object; test for attribute and load or raise type error
   5.634 +==
   5.635 +`__check_and_load_via_object` || object, attribute name
   5.636 +|| Test for attribute and load or raise type error
   5.637 +==
   5.638 +`__check_and_load_via_any` || object, attribute name
   5.639 +|| Test for attribute and load or obtain class; test for attribute and load or
   5.640 +.. raise type error
   5.641 +}}}
   5.642 +
   5.643 +==== Checked Store ====
   5.644 +
   5.645 +These instructions employ the attribute position and code for the supplied
   5.646 +attribute name, storing an attribute value.
   5.647 +
   5.648 +{{{#!table
   5.649 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.650 +==
   5.651 +`__check_and_store_via_class` || object, attribute name, value
   5.652 +|| Raise type error
   5.653 +==
   5.654 +`__check_and_store_via_object` || object, attribute name, value
   5.655 +|| Test for attribute and store value or raise type error
   5.656 +==
   5.657 +`__check_and_store_via_any` || object, attribute name, value
   5.658 +|| Test for attribute and store value or raise type error
   5.659 +}}}
   5.660 +
   5.661 +==== Testing ====
   5.662 +
   5.663 +These instructions employ the special attribute position and code for the
   5.664 +supplied type name.
   5.665 +
   5.666 +{{{#!table
   5.667 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.668 +==
   5.669 +`__test_common_instance` || object, type
   5.670 +|| Obtain class from object; test conformance to type
   5.671 +==
   5.672 +`__test_common_object` || object, type
   5.673 +|| Test conformance to type or obtain class from object and test conformance
   5.674 +.. to type
   5.675 +==
   5.676 +`__test_common_type` || object, type
   5.677 +|| Test conformance to type
   5.678 +==
   5.679 +`__test_specific_instance` || object, type
   5.680 +|| Obtain class from object; test equivalence to type
   5.681 +==
   5.682 +`__test_specific_object` || object, type
   5.683 +|| Test equivalence to type or obtain class from object and test equivalence
   5.684 +.. to type
   5.685 +==
   5.686 +`__test_specific_type` || object, type
   5.687 +|| Test equivalence to type
   5.688 +}}}
   5.689 +
   5.690 +==== Static Load ====
   5.691 +
   5.692 +These instructions obtain references to static objects, in some cases
   5.693 +employing a supplied context.
   5.694 +
   5.695 +{{{#!table
   5.696 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.697 +==
   5.698 +`__load_static_ignore` || object
   5.699 +|| Load attribute populated with object, leaving the context unset
   5.700 +==
   5.701 +`__load_static_replace` || context, object
   5.702 +|| Load attribute populated with the context and object
   5.703 +==
   5.704 +`__load_static_test` || context, object
   5.705 +|| Load attribute populated with object; test context compatibility and set
   5.706 +.. the context
   5.707 +}}}
   5.708 +
   5.709 +==== Temporary Access ====
   5.710 +
   5.711 +These instructions access temporary values retained to perform the attribute
   5.712 +access. The temporary storage index is generated during program translation.
   5.713 +
   5.714 +{{{#!table
   5.715 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.716 +==
   5.717 +`__get_context` || (temporary)
   5.718 +|| Load the context stored in the temporary storage
   5.719 +==
   5.720 +`__set_accessor` || accessor
   5.721 +|| Store the accessor temporarily
   5.722 +==
   5.723 +`__set_context` || (temporary), context
   5.724 +|| Store the context in the temporary storage
   5.725 +==
   5.726 +`__set_private_context` || context
   5.727 +|| Store the context temporarily
   5.728 +==
   5.729 +`__set_target_accessor` || accessor
   5.730 +|| Store the assignment accessor temporarily
   5.731 +}}}
   5.732 +
   5.733 +==== Context Test ====
   5.734 +
   5.735 +These instructions perform tests on the available context object. The
   5.736 +temporary storage index is generated during program translation.
   5.737 +
   5.738 +{{{#!table
   5.739 +'''Instruction''' || '''Arguments''' || '''Operations'''
   5.740 +==
   5.741 +`__test_context_revert` || (temporary), context, attribute
   5.742 +|| Test compatibility of context; revert temporary to attribute context if
   5.743 +.. incompatible
   5.744 +==
   5.745 +`__test_context_static` || (temporary), context, value
   5.746 +|| Test compatibility of context; set temporary to specified context if
   5.747 +.. compatible
   5.748 +}}}
   5.749 +
   5.750  == Deduction Products ==
   5.751  
   5.752 -The deduction process should produce a complete catalogue of accessor and access references that may then be consulted by the [[../Translation|translation]] process needing to know the nature of any operation within the program. Central to the translation process's understanding of references is the '''attribute access plan''' for each reference which characterises each access and provides the basis for the formulation of the '''instruction plan''' used to replicate it in the final program.
   5.753 +The deduction process should produce a complete catalogue of accessor and
   5.754 +access references that may then be consulted by the [[../Translation|
   5.755 +translation]] process needing to know the nature of any operation within the
   5.756 +program. Central to the translation process's understanding of references is
   5.757 +the '''attribute access plan''' for each reference which characterises each
   5.758 +access and provides the basis for the formulation of the '''instruction
   5.759 +plan''' used to replicate it in the final program.
     6.1 --- a/docs/wiki/Design	Sun Jul 22 12:14:48 2018 +0200
     6.2 +++ b/docs/wiki/Design	Thu Jan 17 14:23:38 2019 +0100
     6.3 @@ -1,14 +1,28 @@
     6.4  = Design Decisions =
     6.5  
     6.6 -The Lichen language design involves some different choices to those taken in Python's design. Many of these choices are motivated by the following criteria:
     6.7 +The Lichen language design involves some different choices to those taken in
     6.8 +Python's design. Many of these choices are motivated by the following
     6.9 +criteria:
    6.10 +
    6.11 + * To simplify the language and to make what programs do easier to understand
    6.12 +   and to predict
    6.13 + * To make analysis of programs easier, particularly
    6.14 +   [[../Deduction|deductions]] about the nature of the code
    6.15 + * To simplify and otherwise reduce the [[../Representations|representations]]
    6.16 +   employed and the operations performed at run-time
    6.17  
    6.18 - * To simplify the language and to make what programs do easier to understand and to predict
    6.19 - * To make analysis of programs easier, particularly [[../Deduction|deductions]] about the nature of the code
    6.20 - * To simplify and otherwise reduce the [[../Representations|representations]] employed and the operations performed at run-time
    6.21 +Lichen is in many ways a restricted form of Python. In particular,
    6.22 +restrictions on the attribute names supported by each object help to clearly
    6.23 +define the object types in a program, allowing us to identify those objects
    6.24 +when they are used. Consequently, optimisations that can be employed in a
    6.25 +Lichen program become possible in situations where they would have been
    6.26 +difficult or demanding to employ in a Python program.
    6.27  
    6.28 -Lichen is in many ways a restricted form of Python. In particular, restrictions on the attribute names supported by each object help to clearly define the object types in a program, allowing us to identify those objects when they are used. Consequently, optimisations that can be employed in a Lichen program become possible in situations where they would have been difficult or demanding to employ in a Python program.
    6.29 -
    6.30 -Some design choices evoke memories of earlier forms of Python. Removing nested scopes simplifies the [[../Inspection|inspection]] of programs and run-time [[../Representations|representations]] and mechanisms. Other choices seek to remedy difficult or defective aspects of Python, notably the behaviour of Python's [[../Imports|import]] system.
    6.31 +Some design choices evoke memories of earlier forms of Python. Removing nested
    6.32 +scopes simplifies the [[../Inspection|inspection]] of programs and run-time
    6.33 +[[../Representations|representations]] and mechanisms. Other choices seek to
    6.34 +remedy difficult or defective aspects of Python, notably the behaviour of
    6.35 +Python's [[../Imports|import]] system.
    6.36  
    6.37  <<TableOfContents(2,3)>>
    6.38  
    6.39 @@ -32,7 +46,13 @@
    6.40  
    6.41  === Fixed Attribute Names ===
    6.42  
    6.43 -Attribute names are bound for classes through assignment in the class namespace, for modules in the module namespace, and for instances in methods through assignment to `self`. Class and instance attributes are propagated to descendant classes and instances of descendant classes respectively. Once bound, attributes can be modified, but new attributes cannot be bound by other means, such as the assignment of an attribute to an arbitrary object that would not already support such an attribute.
    6.44 +Attribute names are bound for classes through assignment in the class
    6.45 +namespace, for modules in the module namespace, and for instances in methods
    6.46 +through assignment to `self`. Class and instance attributes are propagated to
    6.47 +descendant classes and instances of descendant classes respectively. Once
    6.48 +bound, attributes can be modified, but new attributes cannot be bound by other
    6.49 +means, such as the assignment of an attribute to an arbitrary object that
    6.50 +would not already support such an attribute.
    6.51  
    6.52  {{{#!python numbers=disable
    6.53  class C:
    6.54 @@ -44,7 +64,10 @@
    6.55  C().y = 567 # not allowed (y not bound for C instances)
    6.56  }}}
    6.57  
    6.58 -Permitting the addition of attributes to objects would then require that such addition attempts be associated with particular objects, leading to a potentially iterative process involving object type deduction and modification, also causing imprecise results.
    6.59 +Permitting the addition of attributes to objects would then require that such
    6.60 +addition attempts be associated with particular objects, leading to a
    6.61 +potentially iterative process involving object type deduction and
    6.62 +modification, also causing imprecise results.
    6.63  
    6.64  === No Shadowing ===
    6.65  
    6.66 @@ -57,11 +80,15 @@
    6.67          self.a = 234 # not allowed (attribute shadows class attribute)
    6.68  }}}
    6.69  
    6.70 -Permitting this would oblige instances to support attributes that, when missing, are provided by consulting their classes but, when not missing, may also be provided directly by the instances themselves.
    6.71 +Permitting this would oblige instances to support attributes that, when
    6.72 +missing, are provided by consulting their classes but, when not missing, may
    6.73 +also be provided directly by the instances themselves.
    6.74  
    6.75  === No Dynamic Attributes ===
    6.76  
    6.77 -Instance attributes cannot be provided dynamically, such that any missing attribute would be supplied by a special method call to determine the attribute's presence and to retrieve its value.
    6.78 +Instance attributes cannot be provided dynamically, such that any missing
    6.79 +attribute would be supplied by a special method call to determine the
    6.80 +attribute's presence and to retrieve its value.
    6.81  
    6.82  {{{#!python numbers=disable
    6.83  class C:
    6.84 @@ -70,31 +97,41 @@
    6.85              return 123
    6.86  }}}
    6.87  
    6.88 -Permitting this would require object types to potentially support any attribute, undermining attempts to use attributes to identify objects.
    6.89 +Permitting this would require object types to potentially support any
    6.90 +attribute, undermining attempts to use attributes to identify objects.
    6.91  
    6.92  == Naming ==
    6.93  
    6.94  {{{#!table
    6.95  '''Lichen''' || '''Python''' || '''Rationale'''
    6.96  ==
    6.97 -Names may be local, global or built-in: nested namespaces must be initialised explicitly
    6.98 +Names may be local, global or built-in: nested namespaces must be initialised
    6.99 +explicitly
   6.100  || Names may also be non-local, permitting closures
   6.101  || Limited name scoping simplifies program inspection and run-time mechanisms
   6.102  ==
   6.103  `self` is a reserved name and is optional in method parameter lists
   6.104 -|| `self` is a naming convention, but the first method parameter must always refer to the accessed object
   6.105 -|| Reserving `self` assists deduction; making it optional is a consequence of the method binding behaviour
   6.106 +|| `self` is a naming convention, but the first method parameter must always
   6.107 +.. refer to the accessed object
   6.108 +|| Reserving `self` assists deduction; making it optional is a consequence of
   6.109 +.. the method binding behaviour
   6.110  ==
   6.111  Instance attributes can be initialised using `.name` parameter notation
   6.112 -|| [[https://stackoverflow.com/questions/1389180/automatically-initialize-instance-variables|Workarounds]] involving decorators and introspection are required for similar brevity
   6.113 +|| [[https://stackoverflow.com/questions/1389180/automatically-initialize-instance-variables|Workarounds]]
   6.114 +.. involving decorators and introspection are required for similar brevity
   6.115  || Initialiser notation eliminates duplication in program code and is convenient
   6.116  }}}
   6.117  
   6.118  === Traditional Local, Global and Built-In Scopes Only ===
   6.119  
   6.120 -Namespaces reside within a hierarchy within modules: classes containing classes or functions; functions containing other functions. Built-in names are exposed in all namespaces, global names are defined at the module level and are exposed in all namespaces within the module, locals are confined to the namespace in which they are defined.
   6.121 +Namespaces reside within a hierarchy within modules: classes containing
   6.122 +classes or functions; functions containing other functions. Built-in names are
   6.123 +exposed in all namespaces, global names are defined at the module level and
   6.124 +are exposed in all namespaces within the module, locals are confined to the
   6.125 +namespace in which they are defined.
   6.126  
   6.127 -However, locals are not inherited by namespaces from surrounding or enclosing namespaces.
   6.128 +However, locals are not inherited by namespaces from surrounding or enclosing
   6.129 +namespaces.
   6.130  
   6.131  {{{#!python numbers=disable
   6.132  def f(x):
   6.133 @@ -108,11 +145,16 @@
   6.134      return i
   6.135  }}}
   6.136  
   6.137 -Needing to access outer namespaces in order to access any referenced names complicates the way in which such dynamic namespaces would need to be managed. Although the default initialisation technique demonstrated above could be automated, explicit initialisation makes programs easier to follow and avoids mistakes involving globals having the same name.
   6.138 +Needing to access outer namespaces in order to access any referenced names
   6.139 +complicates the way in which such dynamic namespaces would need to be managed.
   6.140 +Although the default initialisation technique demonstrated above could be
   6.141 +automated, explicit initialisation makes programs easier to follow and avoids
   6.142 +mistakes involving globals having the same name.
   6.143  
   6.144  === Reserved Self ===
   6.145  
   6.146 -The `self` name can be omitted in method signatures, but in methods it is always initialised to the instance on which the method is operating.
   6.147 +The `self` name can be omitted in method signatures, but in methods it is
   6.148 +always initialised to the instance on which the method is operating.
   6.149  
   6.150  {{{#!python numbers=disable
   6.151  class C:
   6.152 @@ -120,11 +162,18 @@
   6.153          self.x = y # self is the instance
   6.154  }}}
   6.155  
   6.156 -The assumption in methods is that `self` must always be referring to an instance of the containing class or of a descendant class. This means that `self` cannot be initialised to another kind of value, which Python permits through the explicit invocation of a method with the inclusion of the affected instance as the first argument. Consequently, `self` becomes optional in the signature because it is not assigned in the same way as the other parameters.
   6.157 +The assumption in methods is that `self` must always be referring to an
   6.158 +instance of the containing class or of a descendant class. This means that
   6.159 +`self` cannot be initialised to another kind of value, which Python permits
   6.160 +through the explicit invocation of a method with the inclusion of the affected
   6.161 +instance as the first argument. Consequently, `self` becomes optional in the
   6.162 +signature because it is not assigned in the same way as the other parameters.
   6.163  
   6.164  === Instance Attribute Initialisers ===
   6.165  
   6.166 -In parameter lists, a special notation can be used to indicate that the given name is an instance attribute that will be assigned the argument value corresponding to the parameter concerned.
   6.167 +In parameter lists, a special notation can be used to indicate that the given
   6.168 +name is an instance attribute that will be assigned the argument value
   6.169 +corresponding to the parameter concerned.
   6.170  
   6.171  {{{#!python numbers=disable
   6.172  class C:
   6.173 @@ -132,7 +181,10 @@
   6.174          self.c = c # a traditional assignment using a parameter
   6.175  }}}
   6.176  
   6.177 -To use the notation, such dot-qualified parameters must appear only in the parameter lists of methods, not plain functions. The qualified parameters are represented as locals having the same name, and assignments to the corresponding instance attributes are inserted into the generated code.
   6.178 +To use the notation, such dot-qualified parameters must appear only in the
   6.179 +parameter lists of methods, not plain functions. The qualified parameters are
   6.180 +represented as locals having the same name, and assignments to the
   6.181 +corresponding instance attributes are inserted into the generated code.
   6.182  
   6.183  {{{#!python numbers=disable
   6.184  class C:
   6.185 @@ -147,7 +199,9 @@
   6.186          pass
   6.187  }}}
   6.188  
   6.189 -Naturally, `self`, being a reserved name in methods, can also be omitted from such parameter lists. Moreover, such initialising parameters can have default values.
   6.190 +Naturally, `self`, being a reserved name in methods, can also be omitted from
   6.191 +such parameter lists. Moreover, such initialising parameters can have default
   6.192 +values.
   6.193  
   6.194  {{{#!python numbers=disable
   6.195  class C:
   6.196 @@ -165,9 +219,13 @@
   6.197  {{{#!table
   6.198  '''Lichen''' || '''Python''' || '''Rationale'''
   6.199  ==
   6.200 -Class attributes are propagated to class hierarchy members during initialisation: rebinding class attributes does not affect descendant class attributes
   6.201 -|| Class attributes are propagated live to class hierarchy members and must be looked up by the run-time system if not provided by a given class
   6.202 -|| Initialisation-time propagation simplifies access operations and attribute table storage
   6.203 +Class attributes are propagated to class hierarchy members during
   6.204 +initialisation: rebinding class attributes does not affect descendant class
   6.205 +attributes
   6.206 +|| Class attributes are propagated live to class hierarchy members and must be
   6.207 +.. looked up by the run-time system if not provided by a given class
   6.208 +|| Initialisation-time propagation simplifies access operations and attribute
   6.209 +.. table storage
   6.210  ==
   6.211  Unbound methods must be bound using a special function taking an instance
   6.212  || Unbound methods may be called using an instance as first argument
   6.213 @@ -175,20 +233,24 @@
   6.214  ==
   6.215  Functions assigned to class attributes do not become unbound methods
   6.216  || Functions assigned to class attributes become unbound methods
   6.217 -|| Removing method assignment simplifies deduction: methods are always defined in place
   6.218 +|| Removing method assignment simplifies deduction: methods are always defined
   6.219 +.. in place
   6.220  ==
   6.221  Base classes must be well-defined
   6.222  || Base classes may be expressions
   6.223 -|| Well-defined base classes are required to establish a well-defined hierarchy of types
   6.224 +|| Well-defined base classes are required to establish a well-defined
   6.225 +.. hierarchy of types
   6.226  ==
   6.227  Classes may not be defined in functions
   6.228  || Classes may be defined in any kind of namespace
   6.229 -|| Forbidding classes in functions prevents the definition of countless class variants that are awkward to analyse
   6.230 +|| Forbidding classes in functions prevents the definition of countless class
   6.231 +.. variants that are awkward to analyse
   6.232  }}}
   6.233  
   6.234  === Inherited Class Attributes ===
   6.235  
   6.236 -Class attributes that are changed for a class do not change for that class's descendants.
   6.237 +Class attributes that are changed for a class do not change for that class's
   6.238 +descendants.
   6.239  
   6.240  {{{#!python numbers=disable
   6.241  class C:
   6.242 @@ -201,11 +263,20 @@
   6.243  print D.a # remains 123 in Lichen, becomes 456 in Python
   6.244  }}}
   6.245  
   6.246 -Permitting this requires indirection for all class attributes, requiring them to be treated differently from other kinds of attributes. Meanwhile, class attribute rebinding and the accessing of inherited attributes changed in this way is relatively rare.
   6.247 +Permitting this requires indirection for all class attributes, requiring them
   6.248 +to be treated differently from other kinds of attributes. Meanwhile, class
   6.249 +attribute rebinding and the accessing of inherited attributes changed in this
   6.250 +way is relatively rare.
   6.251  
   6.252  === Unbound Methods ===
   6.253  
   6.254 -Methods are defined on classes but are only available via instances: they are instance methods. Consequently, acquiring a method directly from a class and then invoking it should fail because the method will be unbound: the "context" of the method is not an instance. Furthermore, the Python technique of supplying an instance as the first argument in an invocation to bind the method to an instance, thus setting the context of the method, is not supported. See [[#ReservedSelf|"Reserved Self"]] for more information.
   6.255 +Methods are defined on classes but are only available via instances: they are
   6.256 +instance methods. Consequently, acquiring a method directly from a class and
   6.257 +then invoking it should fail because the method will be unbound: the "context"
   6.258 +of the method is not an instance. Furthermore, the Python technique of
   6.259 +supplying an instance as the first argument in an invocation to bind the
   6.260 +method to an instance, thus setting the context of the method, is not
   6.261 +supported. See [[#Reserved Self|"Reserved Self"]] for more information.
   6.262  
   6.263  {{{#!python numbers=disable
   6.264  class C:
   6.265 @@ -217,13 +288,24 @@
   6.266          get_using(C.f, self)(123) # binds C.f to self, then the result is called
   6.267  }}}
   6.268  
   6.269 -Binding methods to instances occurs when acquiring methods via instances or explicitly using the `get_using` built-in. The built-in checks the compatibility of the supplied method and instance. If compatible, it provides the bound method as its result.
   6.270 +Binding methods to instances occurs when acquiring methods via instances or
   6.271 +explicitly using the `get_using` built-in. The built-in checks the
   6.272 +compatibility of the supplied method and instance. If compatible, it provides
   6.273 +the bound method as its result.
   6.274  
   6.275 -Normal functions are callable without any further preparation, whereas unbound methods need the binding step to be performed and are not immediately callable. Were functions to become unbound methods upon assignment to a class attribute, they would need to be invalidated by having the preparation mechanism enabled on them. However, this invalidation would only be relevant to the specific case of assigning functions to classes and this would need to be tested for. Given the added complications, such functionality is arguably not worth supporting.
   6.276 +Normal functions are callable without any further preparation, whereas unbound
   6.277 +methods need the binding step to be performed and are not immediately
   6.278 +callable. Were functions to become unbound methods upon assignment to a class
   6.279 +attribute, they would need to be invalidated by having the preparation
   6.280 +mechanism enabled on them. However, this invalidation would only be relevant
   6.281 +to the specific case of assigning functions to classes and this would need to
   6.282 +be tested for. Given the added complications, such functionality is arguably
   6.283 +not worth supporting.
   6.284  
   6.285  === Assigning Functions to Class Attributes ===
   6.286  
   6.287 -Functions can be assigned to class attributes but do not become unbound methods as a result.
   6.288 +Functions can be assigned to class attributes but do not become unbound
   6.289 +methods as a result.
   6.290  
   6.291  {{{#!python numbers=disable
   6.292  class C:
   6.293 @@ -238,11 +320,18 @@
   6.294  C().f(123) # permitted: f has merely been exposed via C.f
   6.295  }}}
   6.296  
   6.297 -Methods are identified as such by their definition location, they contribute information about attributes to the class hierarchy, and they employ certain structure details at run-time to permit the binding of methods. Since functions can defined in arbitrary locations, no class hierarchy information is available, and a function could combine `self` with a range of attributes that are not compatible with any class to which the function might be assigned.
   6.298 +Methods are identified as such by their definition location, they contribute
   6.299 +information about attributes to the class hierarchy, and they employ certain
   6.300 +structure details at run-time to permit the binding of methods. Since
   6.301 +functions can defined in arbitrary locations, no class hierarchy information
   6.302 +is available, and a function could combine `self` with a range of attributes
   6.303 +that are not compatible with any class to which the function might be
   6.304 +assigned.
   6.305  
   6.306  === Well-Defined Base Classes ===
   6.307  
   6.308 -Base classes must be clearly identifiable as well-defined classes. This facilitates the cataloguing of program objects and further analysis on them.
   6.309 +Base classes must be clearly identifiable as well-defined classes. This
   6.310 +facilitates the cataloguing of program objects and further analysis on them.
   6.311  
   6.312  {{{#!python numbers=disable
   6.313  class C:
   6.314 @@ -255,11 +344,19 @@
   6.315      pass
   6.316  }}}
   6.317  
   6.318 -If base class identification could only be done reliably at run-time, class relationship information would be very limited without running the program or performing costly and potentially unreliable analysis. Indeed, programs employing such dynamic base classes are arguably resistant to analysis, which is contrary to the goals of a language like Lichen.
   6.319 +If base class identification could only be done reliably at run-time, class
   6.320 +relationship information would be very limited without running the program or
   6.321 +performing costly and potentially unreliable analysis. Indeed, programs
   6.322 +employing such dynamic base classes are arguably resistant to analysis, which
   6.323 +is contrary to the goals of a language like Lichen.
   6.324  
   6.325  === Class Definitions and Functions ===
   6.326  
   6.327 -Classes may not be defined in functions because functions provide dynamic namespaces, but Lichen relies on a static namespace hierarchy in order to clearly identify the principal objects in a program. If classes could be defined in functions, despite seemingly providing the same class over and over again on every invocation, a family of classes would, in fact, be defined.
   6.328 +Classes may not be defined in functions because functions provide dynamic
   6.329 +namespaces, but Lichen relies on a static namespace hierarchy in order to
   6.330 +clearly identify the principal objects in a program. If classes could be
   6.331 +defined in functions, despite seemingly providing the same class over and over
   6.332 +again on every invocation, a family of classes would, in fact, be defined.
   6.333  
   6.334  {{{#!python numbers=disable
   6.335  def f(x):
   6.336 @@ -268,7 +365,9 @@
   6.337      return f
   6.338  }}}
   6.339  
   6.340 -Moreover, issues of namespace nesting also arise, since the motivation for defining classes in functions would surely be to take advantage of local state to parameterise such classes.
   6.341 +Moreover, issues of namespace nesting also arise, since the motivation for
   6.342 +defining classes in functions would surely be to take advantage of local state
   6.343 +to parameterise such classes.
   6.344  
   6.345  == Modules and Packages ==
   6.346  
   6.347 @@ -276,25 +375,33 @@
   6.348  '''Lichen''' || '''Python''' || '''Rationale'''
   6.349  ==
   6.350  Modules are independent: package hierarchies are not traversed when importing
   6.351 -|| Modules exist in hierarchical namespaces: package roots must be imported before importing specific submodules
   6.352 -|| Eliminating module traversal permits more precise imports and reduces superfluous code
   6.353 +|| Modules exist in hierarchical namespaces: package roots must be imported
   6.354 +.. before importing specific submodules
   6.355 +|| Eliminating module traversal permits more precise imports and reduces
   6.356 +.. superfluous code
   6.357  ==
   6.358 -Only specific names can be imported from a module or package using the `from` statement
   6.359 +Only specific names can be imported from a module or package using the `from`
   6.360 +statement
   6.361  || Importing "all" from a package or module is permitted
   6.362 -|| Eliminating "all" imports simplifies the task of determining where names in use have come from
   6.363 +|| Eliminating "all" imports simplifies the task of determining where names in
   6.364 +.. use have come from
   6.365  ==
   6.366  Modules must be specified using absolute names
   6.367  || Imports can be absolute or relative
   6.368  || Using only absolute names simplifies the import mechanism
   6.369  ==
   6.370 -Modules are imported independently and their dependencies subsequently resolved
   6.371 +Modules are imported independently and their dependencies subsequently
   6.372 +resolved
   6.373  || Modules are imported as import statements are encountered
   6.374 -|| Statically-initialised objects can be used declaratively, although an initialisation order may still need establishing
   6.375 +|| Statically-initialised objects can be used declaratively, although an
   6.376 +.. initialisation order may still need establishing
   6.377  }}}
   6.378  
   6.379  === Independent Modules ===
   6.380  
   6.381 -The inclusion of modules in a program affects only explicitly-named modules: they do not have relationships implied by their naming that would cause such related modules to be included in a program.
   6.382 +The inclusion of modules in a program affects only explicitly-named modules:
   6.383 +they do not have relationships implied by their naming that would cause such
   6.384 +related modules to be included in a program.
   6.385  
   6.386  {{{#!python numbers=disable
   6.387  from compiler import consts # defines consts
   6.388 @@ -305,11 +412,15 @@
   6.389  consts # is defined
   6.390  }}}
   6.391  
   6.392 -Where modules should have relationships, they should be explicitly defined using `from` and `import` statements which target the exact modules required. In the above example, `compiler` is not routinely imported because modules within the `compiler` package have been requested.
   6.393 +Where modules should have relationships, they should be explicitly defined
   6.394 +using `from` and `import` statements which target the exact modules required.
   6.395 +In the above example, `compiler` is not routinely imported because modules
   6.396 +within the `compiler` package have been requested.
   6.397  
   6.398  === Specific Name Imports Only ===
   6.399  
   6.400 -Lichen, unlike Python, also does not support the special `__all__` module attribute.
   6.401 +Lichen, unlike Python, also does not support the special `__all__` module
   6.402 +attribute.
   6.403  
   6.404  {{{#!python numbers=disable
   6.405  from compiler import * # not permitted
   6.406 @@ -318,11 +429,22 @@
   6.407  interpreter # undefined in compiler (yet it might be thought to reside there) and in this module
   6.408  }}}
   6.409  
   6.410 -The `__all__` attribute supports `from ... import *` statements in Python, but without identifying the module or package involved and then consulting `__all__` in that module or package to discover which names might be involved (which might require the inspection of yet other modules or packages), the names imported cannot be known. Consequently, some names used elsewhere in the module performing the import might be assumed to be imported names when, in fact, they are unknown in both the importing and imported modules. Such uncertainty hinders the inspection of individual modules.
   6.411 +The `__all__` attribute supports `from ... import *` statements in Python, but
   6.412 +without identifying the module or package involved and then consulting
   6.413 +`__all__` in that module or package to discover which names might be involved
   6.414 +(which might require the inspection of yet other modules or packages), the
   6.415 +names imported cannot be known. Consequently, some names used elsewhere in the
   6.416 +module performing the import might be assumed to be imported names when, in
   6.417 +fact, they are unknown in both the importing and imported modules. Such
   6.418 +uncertainty hinders the inspection of individual modules.
   6.419  
   6.420  === Modules Imported Independently ===
   6.421  
   6.422 -When indicating an import using the `from` and `import` statements, the [[../Toolchain|toolchain]] does not attempt to immediately import other modules. Instead, the imports act as declarations of such other modules or names from other modules, resolved at a later stage. This permits mutual imports to a greater extent than in Python.
   6.423 +When indicating an import using the `from` and `import` statements, the
   6.424 +[[../Toolchain|toolchain]] does not attempt to immediately import other
   6.425 +modules. Instead, the imports act as declarations of such other modules or
   6.426 +names from other modules, resolved at a later stage. This permits mutual
   6.427 +imports to a greater extent than in Python.
   6.428  
   6.429  {{{#!python numbers=disable
   6.430  # Module M
   6.431 @@ -344,7 +466,10 @@
   6.432  import N
   6.433  }}}
   6.434  
   6.435 -Such flexibility is not usually needed, and circular importing usually indicates issues with program organisation. However, declarative imports can help to decouple modules and avoid combining import declaration and module initialisation order concerns.
   6.436 +Such flexibility is not usually needed, and circular importing usually
   6.437 +indicates issues with program organisation. However, declarative imports can
   6.438 +help to decouple modules and avoid combining import declaration and module
   6.439 +initialisation order concerns.
   6.440  
   6.441  == Syntax and Control-Flow ==
   6.442  
   6.443 @@ -353,11 +478,14 @@
   6.444  ==
   6.445  If expressions and comprehensions are not supported
   6.446  || If expressions and comprehensions are supported
   6.447 -|| Omitting such syntactic features simplifies program inspection and translation
   6.448 +|| Omitting such syntactic features simplifies program inspection and
   6.449 +.. translation
   6.450  ==
   6.451  The `with` statement is not supported
   6.452 -|| The `with` statement offers a mechanism for resource allocation and deallocation using context managers
   6.453 -|| This syntactic feature can be satisfactorily emulated using existing constructs
   6.454 +|| The `with` statement offers a mechanism for resource allocation and
   6.455 +.. deallocation using context managers
   6.456 +|| This syntactic feature can be satisfactorily emulated using existing
   6.457 +.. constructs
   6.458  ==
   6.459  Generators are not supported
   6.460  || Generators are supported
   6.461 @@ -369,12 +497,16 @@
   6.462  ==
   6.463  All parameters must be specified
   6.464  || Catch-all parameters (`*` and `**`) are supported
   6.465 -|| Omitting catch-all parameter population simplifies generic invocation handling
   6.466 +|| Omitting catch-all parameter population simplifies generic invocation
   6.467 +.. handling
   6.468  }}}
   6.469  
   6.470  === No If Expressions or Comprehensions ===
   6.471  
   6.472 -In order to support the classic [[WikiPedia:?:|ternary operator]], a construct was [[https://www.python.org/dev/peps/pep-0308/|added]] to the Python syntax that needed to avoid problems with the existing grammar and notation. Unfortunately, it reorders the components from the traditional form:
   6.473 +In order to support the classic [[WikiPedia:?:|ternary operator]], a construct
   6.474 +was [[https://www.python.org/dev/peps/pep-0308/|added]] to the Python syntax
   6.475 +that needed to avoid problems with the existing grammar and notation.
   6.476 +Unfortunately, it reorders the components from the traditional form:
   6.477  
   6.478  {{{#!python numbers=disable
   6.479  # Not valid in Lichen, only in Python.
   6.480 @@ -386,7 +518,9 @@
   6.481  true_result if (inner_true_result if condition else inner_false_result) else false_result
   6.482  }}}
   6.483  
   6.484 -Since if expressions may participate within expressions, they cannot be rewritten as if statements. Nor can they be rewritten as logical operator chains in general.
   6.485 +Since if expressions may participate within expressions, they cannot be
   6.486 +rewritten as if statements. Nor can they be rewritten as logical operator
   6.487 +chains in general.
   6.488  
   6.489  {{{#!python numbers=disable
   6.490  # Not valid in Lichen, only in Python.
   6.491 @@ -399,9 +533,17 @@
   6.492  a = x and 0 or 1 # not valid
   6.493  }}}
   6.494  
   6.495 -But in any case, it would be more of a motivation to support the functionality if a better syntax could be adopted instead. However, if expressions are not particularly important in Python, and despite enhancement requests over many years, everybody managed to live without them.
   6.496 +But in any case, it would be more of a motivation to support the functionality
   6.497 +if a better syntax could be adopted instead. However, if expressions are not
   6.498 +particularly important in Python, and despite enhancement requests over many
   6.499 +years, everybody managed to live without them.
   6.500  
   6.501 -List and generator comprehensions are more complicated but share some characteristics of if expressions: their syntax contradicts the typical conventions established by the rest of the Python language; they create implicit state that is perhaps most appropriately modelled by a separate function or similar object. Since Lichen does not support generators at all, it will obviously not support generator expressions.
   6.502 +List and generator comprehensions are more complicated but share some
   6.503 +characteristics of if expressions: their syntax contradicts the typical
   6.504 +conventions established by the rest of the Python language; they create
   6.505 +implicit state that is perhaps most appropriately modelled by a separate
   6.506 +function or similar object. Since Lichen does not support generators at all,
   6.507 +it will obviously not support generator expressions.
   6.508  
   6.509  Meanwhile, list comprehensions quickly encourage barely-readable programs:
   6.510  
   6.511 @@ -412,11 +554,21 @@
   6.512  a = [z for y in x if y for z in y if z]
   6.513  }}}
   6.514  
   6.515 -Supporting the creation of temporary functions to produce list comprehensions, while also hiding temporary names from the enclosing scope, adds complexity to the toolchain for situations where programmers would arguably be better creating their own functions and thus writing more readable programs.
   6.516 +Supporting the creation of temporary functions to produce list comprehensions,
   6.517 +while also hiding temporary names from the enclosing scope, adds complexity to
   6.518 +the toolchain for situations where programmers would arguably be better
   6.519 +creating their own functions and thus writing more readable programs.
   6.520  
   6.521  === No With Statement ===
   6.522  
   6.523 -The [[https://docs.python.org/2.7/reference/compound_stmts.html#the-with-statement|with statement]] introduced the concept of [[https://docs.python.org/2.7/reference/datamodel.html#context-managers|context managers]] in Python 2.5, with such objects supporting a [[https://docs.python.org/2.7/library/stdtypes.html#typecontextmanager|programming interface]] that aims to formalise certain conventions around resource management. For example:
   6.524 +The
   6.525 +[[https://docs.python.org/2.7/reference/compound_stmts.html#the-with-statement|with
   6.526 +statement]] introduced the concept of
   6.527 +[[https://docs.python.org/2.7/reference/datamodel.html#context-managers|context
   6.528 +managers]] in Python 2.5, with such objects supporting a
   6.529 +[[https://docs.python.org/2.7/library/stdtypes.html#typecontextmanager|programming
   6.530 +interface]] that aims to formalise certain conventions around resource
   6.531 +management. For example:
   6.532  
   6.533  {{{#!python numbers=disable
   6.534  # Not valid in Lichen, only in Python.
   6.535 @@ -426,7 +578,11 @@
   6.536          cursor.execute(query, args)
   6.537  }}}
   6.538  
   6.539 -Although this makes for readable code, it must be supported by objects which define the `__enter__` and `__exit__` special methods. Here, the `connect` method invoked in the first `with` statement must return such an object; similarly, the `cursor` method must also provide an object with such characteristics.
   6.540 +Although this makes for readable code, it must be supported by objects which
   6.541 +define the `__enter__` and `__exit__` special methods. Here, the `connect`
   6.542 +method invoked in the first `with` statement must return such an object;
   6.543 +similarly, the `cursor` method must also provide an object with such
   6.544 +characteristics.
   6.545  
   6.546  However, the "pre-with" solution is as follows:
   6.547  
   6.548 @@ -442,11 +598,23 @@
   6.549      connection.close()
   6.550  }}}
   6.551  
   6.552 -Although this seems less readable, its behaviour is more obvious because magic methods are not being called implicitly. Moreover, any parameterisation of the acts of resource deallocation or closure can be done in the `finally` clauses where such parameterisation would seem natural, rather than being specified through some kind of context manager initialisation arguments that must then be propagated to the magic methods so that they may take into consideration contextual information that is readily available in the place where the actual resource operations are being performed.
   6.553 +Although this seems less readable, its behaviour is more obvious because magic
   6.554 +methods are not being called implicitly. Moreover, any parameterisation of the
   6.555 +acts of resource deallocation or closure can be done in the `finally` clauses
   6.556 +where such parameterisation would seem natural, rather than being specified
   6.557 +through some kind of context manager initialisation arguments that must then
   6.558 +be propagated to the magic methods so that they may take into consideration
   6.559 +contextual information that is readily available in the place where the actual
   6.560 +resource operations are being performed.
   6.561  
   6.562  === No Generators ===
   6.563  
   6.564 -[[https://www.python.org/dev/peps/pep-0255/|Generators]] were [[https://docs.python.org/release/2.3/whatsnew/section-generators.html|added]] to Python in the 2.2 release and became fully part of the language in the 2.3 release. They offer a convenient way of writing iterator-like objects, capturing execution state instead of obliging the programmer to manage such state explicitly.
   6.565 +[[https://www.python.org/dev/peps/pep-0255/|Generators]] were
   6.566 +[[https://docs.python.org/release/2.3/whatsnew/section-generators.html|added]]
   6.567 +to Python in the 2.2 release and became fully part of the language in the 2.3
   6.568 +release. They offer a convenient way of writing iterator-like objects,
   6.569 +capturing execution state instead of obliging the programmer to manage such
   6.570 +state explicitly.
   6.571  
   6.572  {{{#!python numbers=disable
   6.573  # Not valid in Lichen, only in Python.
   6.574 @@ -477,11 +645,20 @@
   6.575      i += 1
   6.576  }}}
   6.577  
   6.578 -However, generators make additional demands on the mechanisms provided to support program execution. The encapsulation of the above example generator in a separate class illustrates the need for state that persists outside the execution of the routine providing the generator's results. Generators may look like functions, but they do not necessarily behave like them, leading to potential misunderstandings about their operation even if the code is superficially tidy and concise.
   6.579 +However, generators make additional demands on the mechanisms provided to
   6.580 +support program execution. The encapsulation of the above example generator in
   6.581 +a separate class illustrates the need for state that persists outside the
   6.582 +execution of the routine providing the generator's results. Generators may
   6.583 +look like functions, but they do not necessarily behave like them, leading to
   6.584 +potential misunderstandings about their operation even if the code is
   6.585 +superficially tidy and concise.
   6.586  
   6.587  === Positional and Keyword Arguments Only ===
   6.588  
   6.589 -When invoking callables, only positional arguments and keyword arguments can be used. Python also supports `*` and `**` arguments which respectively unpack sequences and mappings into the argument list, filling the list with sequence items (using `*`) and keywords (using `**`).
   6.590 +When invoking callables, only positional arguments and keyword arguments can
   6.591 +be used. Python also supports `*` and `**` arguments which respectively unpack
   6.592 +sequences and mappings into the argument list, filling the list with sequence
   6.593 +items (using `*`) and keywords (using `**`).
   6.594  
   6.595  {{{#!python numbers=disable
   6.596  def f(a, b, c, d):
   6.597 @@ -494,11 +671,15 @@
   6.598  f(2, 4, **m) # not permitted
   6.599  }}}
   6.600  
   6.601 -While convenient, such "unpacking" arguments obscure the communication between callables and undermine the safety provided by function and method signatures. They also require run-time support for the unpacking operations.
   6.602 +While convenient, such "unpacking" arguments obscure the communication between
   6.603 +callables and undermine the safety provided by function and method signatures.
   6.604 +They also require run-time support for the unpacking operations.
   6.605  
   6.606  === Positional Parameters Only ===
   6.607  
   6.608 -Similarly, signatures may only contain named parameters that correspond to arguments. Python supports `*` and `**` in parameter lists, too, which respectively accumulate superfluous positional and keyword arguments.
   6.609 +Similarly, signatures may only contain named parameters that correspond to
   6.610 +arguments. Python supports `*` and `**` in parameter lists, too, which
   6.611 +respectively accumulate superfluous positional and keyword arguments.
   6.612  
   6.613  {{{#!python numbers=disable
   6.614  def f(a, b, *args, **kw): # not permitted
   6.615 @@ -508,4 +689,11 @@
   6.616  f(1, 2, c=3, d=4)
   6.617  }}}
   6.618  
   6.619 -Such accumulation parameters can be useful for collecting arbitrary data and applying some of it within a callable. However, they can easily proliferate throughout a system and allow erroneous data to propagate far from its origin because such parameters permit the deferral of validation until the data needs to be accessed. Again, run-time support is required to marshal arguments into the appropriate parameter of this nature, but programmers could just write functions and methods that employ general sequence and mapping parameters explicitly instead.
   6.620 +Such accumulation parameters can be useful for collecting arbitrary data and
   6.621 +applying some of it within a callable. However, they can easily proliferate
   6.622 +throughout a system and allow erroneous data to propagate far from its origin
   6.623 +because such parameters permit the deferral of validation until the data needs
   6.624 +to be accessed. Again, run-time support is required to marshal arguments into
   6.625 +the appropriate parameter of this nature, but programmers could just write
   6.626 +functions and methods that employ general sequence and mapping parameters
   6.627 +explicitly instead.
     7.1 --- a/docs/wiki/FrontPage	Sun Jul 22 12:14:48 2018 +0200
     7.2 +++ b/docs/wiki/FrontPage	Thu Jan 17 14:23:38 2019 +0100
     7.3 @@ -2,41 +2,92 @@
     7.4  
     7.5  || [[/Downloads|Downloads]] || [[#Language|Language]] || [[#Toolchain|Toolchain]] || [[#Rationale|Rationale]] || [[#Documents|Documents]] ||
     7.6  
     7.7 -Lichen is both a Python-like [[/Design|language]] and a [[/Toolchain|toolchain]] for that language.
     7.8 +Lichen is both a Python-like [[/Design|language]] and a
     7.9 +[[/Toolchain|toolchain]] for that language.
    7.10  
    7.11  Some objectives:
    7.12  
    7.13 - * Perform analysis on programs to better understand program structure and behaviour
    7.14 + * Perform analysis on programs to better understand program structure and
    7.15 +   behaviour
    7.16   * Develop code generation capabilities
    7.17 - * Provide a platform for experimentation independent of existing Python language and library implementations
    7.18 + * Provide a platform for experimentation independent of existing Python
    7.19 +   language and library implementations
    7.20   * Provide independence from Python language evolution
    7.21   * Learn things about writing compilers
    7.22  
    7.23 -Despite building on a long [[/History|history]] of experimentation, Lichen still requires some [[/ToDo|work to be done]] for it to be more widely usable.
    7.24 +Despite building on a long [[/History|history]] of experimentation, Lichen
    7.25 +still requires some [[/ToDo|work to be done]] for it to be more widely usable.
    7.26  
    7.27  <<Anchor(Language)>>
    7.28  == Language ==
    7.29  
    7.30 -The Lichen language [[/Design|foregoes]] various dynamic aspects of Python to provide a foundation upon which more predictable programs can be built, while preserving essential functionality to make the core of the language seem very much "like Python" (thus yielding the name "Lichen"). The general syntax is largely identical to Python, with only certain syntactic constructs being deliberately unsupported, largely because the associated features are not desired.
    7.31 +The Lichen language [[/Design|foregoes]] various dynamic aspects of Python to
    7.32 +provide a foundation upon which more predictable programs can be built, while
    7.33 +preserving essential functionality to make the core of the language seem very
    7.34 +much "like Python" (thus yielding the name "Lichen"). The general syntax is
    7.35 +largely identical to Python, with only certain syntactic constructs being
    7.36 +deliberately unsupported, largely because the associated features are not
    7.37 +desired.
    7.38  
    7.39  <<Anchor(Toolchain)>>
    7.40  == Toolchain ==
    7.41  
    7.42 -The Lichen [[/Toolchain|toolchain]] employs existing tokeniser and parser software to obtain an abstract syntax tree which is then inspected to provide data to support deductions about the structure and nature of a given program. With the information obtained from these processes, a program is then constructed, consisting of a number of source files in the target compilation language (which is currently the C programming language). This generated program may be compiled and run, hopefully producing the results intended by the source program's authors.
    7.43 +The Lichen [[/Toolchain|toolchain]] employs existing tokeniser and parser
    7.44 +software to obtain an abstract syntax tree which is then inspected to provide
    7.45 +data to support deductions about the structure and nature of a given program.
    7.46 +With the information obtained from these processes, a program is then
    7.47 +constructed, consisting of a number of source files in the target compilation
    7.48 +language (which is currently the C programming language). This generated
    7.49 +program may be compiled and run, hopefully producing the results intended by
    7.50 +the source program's authors.
    7.51  
    7.52 -Lichen source files use the `.py` suffix since the language syntax is superficially compatible with Python, allowing text editors to provide highlighting and editing support for Lichen programs without the need to reconfigure such tools. However, an alternative, recommended suffix is likely to be introduced in future.
    7.53 +Lichen source files use the `.py` suffix since the language syntax is
    7.54 +superficially compatible with Python, allowing text editors to provide
    7.55 +highlighting and editing support for Lichen programs without the need to
    7.56 +reconfigure such tools. However, an alternative, recommended suffix is likely
    7.57 +to be introduced in future.
    7.58  
    7.59  <<Anchor(Library)>>
    7.60  == Library ==
    7.61  
    7.62 -Unlike other Python compilation efforts, Lichen programs employ a newly-written library that is distinct from the CPython standard library distribution and completely independent from the CPython extension and object libraries on which Python programs being run with CPython must depend. Thus, there is no dependency on any `libpython` for run-time functionality. Since the parts of the Python standard library that are written in Python tend to be rather variable in quality, there has been no real inclination to re-use modules from that particular source, noting that they would need modifying to be compatible with Lichen, anyway. However, rewriting such modules provides opportunities to "do things right": with some functionality being over twenty years old and in bad shape, this is arguably something that should have been done for Python, anyway.
    7.63 +Unlike other Python compilation efforts, Lichen programs employ a newly-written
    7.64 +library that is distinct from the CPython standard library distribution and
    7.65 +completely independent from the CPython extension and object libraries on which
    7.66 +Python programs being run with CPython must depend. Thus, there is no
    7.67 +dependency on any `libpython` for run-time functionality. Since the parts of
    7.68 +the Python standard library that are written in Python tend to be rather
    7.69 +variable in quality, there has been no real inclination to re-use modules from
    7.70 +that particular source, noting that they would need modifying to be compatible
    7.71 +with Lichen, anyway. However, rewriting such modules provides opportunities to
    7.72 +"do things right": with some functionality being over twenty years old and in
    7.73 +bad shape, this is arguably something that should have been done for Python,
    7.74 +anyway.
    7.75  
    7.76  <<Anchor(Rationale)>>
    7.77  == Rationale ==
    7.78  
    7.79 -Python has proven to be a readable, productive, comfortable-to-use and popular programming language. However, as it has accumulated features, the precise behaviour of programs making use of many of these features has become more difficult to predict. Features added to provide even more convenience to the programmer have often incurred run-time costs, introduced layers of indirection, and have made programs even more inscrutable. Instead of development tools reaching a point of being able to infer information about programs, it has been suggested that programmers annotate their programs in order to help tools understand those programs instead. Beyond superficial code style analysis and providing tooltips in integrated development environments, Python code analysis is often portrayed as a lost cause.
    7.80 +Python has proven to be a readable, productive, comfortable-to-use and popular
    7.81 +programming language. However, as it has accumulated features, the precise
    7.82 +behaviour of programs making use of many of these features has become more
    7.83 +difficult to predict. Features added to provide even more convenience to the
    7.84 +programmer have often incurred run-time costs, introduced layers of
    7.85 +indirection, and have made programs even more inscrutable. Instead of
    7.86 +development tools reaching a point of being able to infer information about
    7.87 +programs, it has been suggested that programmers annotate their programs in
    7.88 +order to help tools understand those programs instead. Beyond superficial code
    7.89 +style analysis and providing tooltips in integrated development environments,
    7.90 +Python code analysis is often portrayed as a lost cause.
    7.91  
    7.92 -By employing a refined language [[/Design|design]], Lichen aims to let each program define its [[/Structure|structure]] conveniently, be readily [[/Inspection|inspected]], and thus support [[/Deduction|deductions]] about the use of the program's objects by the code. The result should be more predictable programs that can be [[/Translation|translated]] to other, more efficient, [[/Representations|representations]]. The Lichen toolchain should be able to tell the programmer useful things about their programs that it may also be able to make use of itself. It not only aims to report information about programs that might be of interest to the developer, but it seeks to produce functioning, translated programs that can actually be run.
    7.93 +By employing a refined language [[/Design|design]], Lichen aims to let each
    7.94 +program define its [[/Structure|structure]] conveniently, be readily
    7.95 +[[/Inspection|inspected]], and thus support [[/Deduction|deductions]] about the
    7.96 +use of the program's objects by the code. The result should be more predictable
    7.97 +programs that can be [[/Translation|translated]] to other, more efficient,
    7.98 +[[/Representations|representations]]. The Lichen toolchain should be able to
    7.99 +tell the programmer useful things about their programs that it may also be able
   7.100 +to make use of itself. It not only aims to report information about programs
   7.101 +that might be of interest to the developer, but it seeks to produce
   7.102 +functioning, translated programs that can actually be run.
   7.103  
   7.104  <<Anchor(Documents)>>
   7.105  == Document Index ==
     8.1 --- a/docs/wiki/History	Sun Jul 22 12:14:48 2018 +0200
     8.2 +++ b/docs/wiki/History	Thu Jan 17 14:23:38 2019 +0100
     8.3 @@ -1,29 +1,115 @@
     8.4  = History =
     8.5  
     8.6 -Lichen is one of a number of experiments focused on the analysis of Python source code with potential reliability and performance improvements in mind. There have been a number of projects that can be regarded as its predecessors, the first of these being a project with the name "Analysis" that was inspired by the then-recent announcement of the [[https://dspace.mit.edu/handle/1721.1/16688|Starkiller]] tool, which attempted to compile Python to C++ (and was never publicly released). With the emergence of the [[http://pypy.org/|PyPy]] project, the notion of writing a Python implementation in a dialect of Python was also considered attractive.
     8.7 +Lichen is one of a number of experiments focused on the analysis of Python
     8.8 +source code with potential reliability and performance improvements in mind.
     8.9 +There have been a number of projects that can be regarded as its predecessors,
    8.10 +the first of these being a project with the name "Analysis" that was inspired
    8.11 +by the then-recent announcement of the
    8.12 +[[https://dspace.mit.edu/handle/1721.1/16688|Starkiller]] tool, which
    8.13 +attempted to compile Python to C++ (and was never publicly released). With the
    8.14 +emergence of the [[http://pypy.org/|PyPy]] project, the notion of writing a
    8.15 +Python implementation in a dialect of Python was also considered attractive.
    8.16  
    8.17  == Initial Experiments ==
    8.18  
    8.19 -After some experience gained, a successor to "Analysis" was attempted with the uninspired, but distinct, name of "analysis". This attempted to apply type inference and also generate C programs that could be compiled, albeit with a very simple run-time model that was not really amenable to further improvement. However, some useful concepts were explored, such as abstract syntax tree (AST) node annotations that could be used to direct code generation activities and to provide annotated source code and documentation summaries.
    8.20 +After some experience gained, a successor to "Analysis" was attempted with the
    8.21 +uninspired, but distinct, name of "analysis". This attempted to apply type
    8.22 +inference and also generate C programs that could be compiled, albeit with a
    8.23 +very simple run-time model that was not really amenable to further
    8.24 +improvement. However, some useful concepts were explored, such as abstract
    8.25 +syntax tree (AST) node annotations that could be used to direct code
    8.26 +generation activities and to provide annotated source code and documentation
    8.27 +summaries.
    8.28  
    8.29 -Various experiences with specialising functions and methods, plus general experience working with the AST led to a successor called "simplify", which attempted to extend specialisation to classes, thus introducing the notion of multiple forms of a single class whose attributes have differing characteristics. It also introduced the idea of a simplified instruction set for program operations, in contrast to Python bytecode whose instructions can represent relatively complicated operations, along with a simplified program representation that was intended to make program analysis easier. Issues around "data polymorphism" led to the realisation that attempting to identify specific types, in order to facilitate insights into program behaviour or optimisations of programs, was costly and not necessarily effective.
    8.30 +Various experiences with specialising functions and methods, plus general
    8.31 +experience working with the AST led to a successor called "simplify", which
    8.32 +attempted to extend specialisation to classes, thus introducing the notion of
    8.33 +multiple forms of a single class whose attributes have differing
    8.34 +characteristics. It also introduced the idea of a simplified instruction set
    8.35 +for program operations, in contrast to Python bytecode whose instructions can
    8.36 +represent relatively complicated operations, along with a simplified program
    8.37 +representation that was intended to make program analysis easier. Issues
    8.38 +around "data polymorphism" led to the realisation that attempting to identify
    8.39 +specific types, in order to facilitate insights into program behaviour or
    8.40 +optimisations of programs, was costly and not necessarily effective.
    8.41  
    8.42 -(Conversations with the author of [[http://shedskin.github.io/|Shed Skin]], a tool already available at this time, which compiles Python programs complying to certain restrictions to C++, were very useful in considering data polymorphism issues and others. Shed Skin must effectively be what Starkiller promised to be, and it has even been available in Debian for a number of years so that others can evaluate it for their own needs.)
    8.43 +(Conversations with the author of [[http://shedskin.github.io/|Shed Skin]], a
    8.44 +tool already available at this time, which compiles Python programs complying
    8.45 +to certain restrictions to C++, were very useful in considering data
    8.46 +polymorphism issues and others. Shed Skin must effectively be what Starkiller
    8.47 +promised to be, and it has even been available in Debian for a number of years
    8.48 +so that others can evaluate it for their own needs.)
    8.49  
    8.50  == Further Experiences ==
    8.51  
    8.52 -An alternative approach was taken in the next successor, "micropython" (not to be confused with any other, more popular, project of the same name), where the focus of analysis shifted to defining the attributes provided by program objects and then determining which objects might be used in different places in the program according to the attributes mentioned. It introduced a strategy for representing program objects at run-time, and continued the work on a simplified instruction set, defining its instructions in terms of the run-time structures. The ability to generate annotated source code summaries was enhanced, and the software could also generate programs in the simplified instruction set that could be executed using an interpreter. Although substantial progress was made, the increasing resistance experienced in adapting a growing system to further forms of optimisations, and the need to implement and test run-time functionality on a somewhat unique instruction set architecture meant that progress became slower over time.
    8.53 +An alternative approach was taken in the next successor, "micropython" (not to
    8.54 +be confused with any other, more popular, project of the same name), where the
    8.55 +focus of analysis shifted to defining the attributes provided by program
    8.56 +objects and then determining which objects might be used in different places
    8.57 +in the program according to the attributes mentioned. It introduced a strategy
    8.58 +for representing program objects at run-time, and continued the work on a
    8.59 +simplified instruction set, defining its instructions in terms of the run-time
    8.60 +structures. The ability to generate annotated source code summaries was
    8.61 +enhanced, and the software could also generate programs in the simplified
    8.62 +instruction set that could be executed using an interpreter. Although
    8.63 +substantial progress was made, the increasing resistance experienced in
    8.64 +adapting a growing system to further forms of optimisations, and the need to
    8.65 +implement and test run-time functionality on a somewhat unique instruction set
    8.66 +architecture meant that progress became slower over time.
    8.67  
    8.68 -The dependence of "micropython" on costly whole-program analysis to identify optimisations led to some further realisations. The notion of "attribute usage" had allowed program namespaces to have their names characterised, and the notion of well-defined structures had allowed the identification of specific object types throughout programs, albeit with a degree of understandable inaccuracy. Meanwhile, the run-time data model had relied on restrictions on structures to permit relatively efficient structure access operations even when object types cannot be identified. It thus appeared that such techniques might be almost as potent on their own as the combination of more costly techniques explored in earlier projects.
    8.69 +The dependence of "micropython" on costly whole-program analysis to identify
    8.70 +optimisations led to some further realisations. The notion of "attribute
    8.71 +usage" had allowed program namespaces to have their names characterised, and
    8.72 +the notion of well-defined structures had allowed the identification of
    8.73 +specific object types throughout programs, albeit with a degree of
    8.74 +understandable inaccuracy. Meanwhile, the run-time data model had relied on
    8.75 +restrictions on structures to permit relatively efficient structure access
    8.76 +operations even when object types cannot be identified. It thus appeared that
    8.77 +such techniques might be almost as potent on their own as the combination of
    8.78 +more costly techniques explored in earlier projects.
    8.79  
    8.80  == Current Work ==
    8.81  
    8.82 -It was with such realisations that a new project was effectively born. Tentatively called "!PythonLight" but renamed to "Lichen" as the code matured, the objectives now involved a simpler processing framework that merely attempted to catalogue structure members, to determine the origins of such members, and to record data flow within namespaces in order to determine attribute usage on names. The name "Lichen" can be pronounced in a way that is a contraction of "like Python", indicating that although the language being analysed is similar to Python, restrictions apply that make it not identical to it.
    8.83 +It was with such realisations that a new project was effectively born.
    8.84 +Tentatively called "!PythonLight" but renamed to "Lichen" as the code matured,
    8.85 +the objectives now involved a simpler processing framework that merely
    8.86 +attempted to catalogue structure members, to determine the origins of such
    8.87 +members, and to record data flow within namespaces in order to determine
    8.88 +attribute usage on names. The name "Lichen" can be pronounced in a way that is
    8.89 +a contraction of "like Python", indicating that although the language being
    8.90 +analysed is similar to Python, restrictions apply that make it not identical
    8.91 +to it.
    8.92  
    8.93 -Unlike previous efforts, instead of annotating the AST built by the Python parser, separate catalogues would record details of program operations for more convenient consumption by potentially separate analysis tools. It is worth emphasising that much of the processing work done to characterise program behaviour is effectively database work: searching for information that matches particular criteria. Emitting tabular data for perusal and potential use by other tools is a recognition of this, and it would not be unreasonable to expect future versions of tools of this nature to actually employ more conventional database management systems to facilitate program analysis.
    8.94 +Unlike previous efforts, instead of annotating the AST built by the Python
    8.95 +parser, separate catalogues would record details of program operations for
    8.96 +more convenient consumption by potentially separate analysis tools. It is
    8.97 +worth emphasising that much of the processing work done to characterise
    8.98 +program behaviour is effectively database work: searching for information that
    8.99 +matches particular criteria. Emitting tabular data for perusal and potential
   8.100 +use by other tools is a recognition of this, and it would not be unreasonable
   8.101 +to expect future versions of tools of this nature to actually employ more
   8.102 +conventional database management systems to facilitate program analysis.
   8.103  
   8.104 -Instead of targeting a project-specific virtual machine, the ultimate objective of processing was amended to generate a C program that could be compiled using existing tools. Although care must be taken to generate C source code, delegating the low-level work of actually producing an executable reduces the scope of the project and its associated demands.
   8.105 +Instead of targeting a project-specific virtual machine, the ultimate
   8.106 +objective of processing was amended to generate a C program that could be
   8.107 +compiled using existing tools. Although care must be taken to generate C
   8.108 +source code, delegating the low-level work of actually producing an executable
   8.109 +reduces the scope of the project and its associated demands.
   8.110  
   8.111 -Despite reasonable progress being made, certain features retained from Python proved to be a burden, notably the behaviour of Python's import system and the way in which programs can quickly accumulate superfluous code through that system's operation. To focus on genuinely important functionality and on producing working code, the project was [[../Restarted|restarted]] with various additional simplifications made to the supported language. Fortunately, this helped to drive progress, resulting in the development of a toolchain that can generate C programs for compilation and execution.
   8.112 +Despite reasonable progress being made, certain features retained from Python
   8.113 +proved to be a burden, notably the behaviour of Python's import system and the
   8.114 +way in which programs can quickly accumulate superfluous code through that
   8.115 +system's operation. To focus on genuinely important functionality and on
   8.116 +producing working code, the project was [[../Restarted|restarted]] with
   8.117 +various additional simplifications made to the supported language.
   8.118 +Fortunately, this helped to drive progress, resulting in the development of a
   8.119 +toolchain that can generate C programs for compilation and execution.
   8.120  
   8.121 -Lichen attempts to balance the goals of requiring relatively inexpensive analysis, offering meaningful insights into program behaviour, providing output in the form of working, translated, standalone programs, and encouraging further experimentation in the areas of language design, program analysis and optimisation. Much of the standard library is itself written in Lichen, even functionality that would not be written in Python for CPython, thus realising one of the original ambitions for this work and its predecessors.
   8.122 +Lichen attempts to balance the goals of requiring relatively inexpensive
   8.123 +analysis, offering meaningful insights into program behaviour, providing
   8.124 +output in the form of working, translated, standalone programs, and
   8.125 +encouraging further experimentation in the areas of language design, program
   8.126 +analysis and optimisation. Much of the standard library is itself written in
   8.127 +Lichen, even functionality that would not be written in Python for CPython,
   8.128 +thus realising one of the original ambitions for this work and its
   8.129 +predecessors.
     9.1 --- a/docs/wiki/Imports	Sun Jul 22 12:14:48 2018 +0200
     9.2 +++ b/docs/wiki/Imports	Thu Jan 17 14:23:38 2019 +0100
     9.3 @@ -1,17 +1,21 @@
     9.4  = Imports =
     9.5  
     9.6 -An '''import''' is a declaration of one or more names that are provided by another source file or '''module''':
     9.7 +An '''import''' is a declaration of one or more names that are provided by
     9.8 +another source file or '''module''':
     9.9  
    9.10    * `import` statements declare names that correspond to modules
    9.11    * `from` statements declare names provided by modules
    9.12  
    9.13 -Imports occur either through explicit import operations initiated by the `from` and `import` statements, or through implicit import operations occurring to satisfy the requirements of another kind of operation.
    9.14 +Imports occur either through explicit import operations initiated by the
    9.15 +`from` and `import` statements, or through implicit import operations
    9.16 +occurring to satisfy the requirements of another kind of operation.
    9.17  
    9.18  <<TableOfContents(2,3)>>
    9.19  
    9.20  == Packages and Submodules ==
    9.21  
    9.22 -A '''package''' is a collection of modules whose names are all prefixed by the package name. For example:
    9.23 +A '''package''' is a collection of modules whose names are all prefixed by the
    9.24 +package name. For example:
    9.25  
    9.26  {{{
    9.27  compiler
    9.28 @@ -19,30 +23,45 @@
    9.29  compiler.transformer
    9.30  }}}
    9.31  
    9.32 -Here, the `compiler` package is said to contain the `compiler.ast` and `compiler.transformer` modules.
    9.33 +Here, the `compiler` package is said to contain the `compiler.ast` and
    9.34 +`compiler.transformer` modules.
    9.35  
    9.36  === Defining Packages ===
    9.37  
    9.38 -The package root or top-level module is defined in a file called `__init__.py` inside the directory bearing the package's name, and this file provides a namespace for the top-level module. However, a package does not expose its member modules ('''submodules''') as members of its top-level module. Instead, the hierarchical relationship between a package and its submodules exists purely in the naming of those modules, and where submodules are imported they must be done so using their full names.
    9.39 +The package root or top-level module is defined in a file called `__init__.py`
    9.40 +inside the directory bearing the package's name, and this file provides a
    9.41 +namespace for the top-level module. However, a package does not expose its
    9.42 +member modules ('''submodules''') as members of its top-level module. Instead,
    9.43 +the hierarchical relationship between a package and its submodules exists
    9.44 +purely in the naming of those modules, and where submodules are imported they
    9.45 +must be done so using their full names.
    9.46  
    9.47 -Thus, relationships between packages and modules must be explicitly defined in module namespaces. For example, in the `compiler` module, the following would define relationships to the submodules:
    9.48 +Thus, relationships between packages and modules must be explicitly defined in
    9.49 +module namespaces. For example, in the `compiler` module, the following would
    9.50 +define relationships to the submodules:
    9.51  
    9.52  {{{
    9.53  from compiler.ast import ast
    9.54  from compiler.transformer import transformer
    9.55  }}}
    9.56  
    9.57 -Without such import statements, no attempt will be made upon importing `compiler` to access the submodules and automatically populate the package.
    9.58 +Without such import statements, no attempt will be made upon importing
    9.59 +`compiler` to access the submodules and automatically populate the package.
    9.60  
    9.61  === Accessing Submodules Directly ===
    9.62  
    9.63 -Importing of submodules from packages will not cause the package itself to be imported. For example:
    9.64 +Importing of submodules from packages will not cause the package itself to be
    9.65 +imported. For example:
    9.66  
    9.67  {{{
    9.68  import compiler.ast
    9.69  }}}
    9.70  
    9.71 -This initialises the name `ast` which refers to the `compiler.ast` module, but the `compiler` package and its top-level module will not be imported. Thus, submodules can be considered independent of their packages, although they may seek to import their package top-level module should they need to access objects provided by that module.
    9.72 +This initialises the name `ast` which refers to the `compiler.ast` module, but
    9.73 +the `compiler` package and its top-level module will not be imported. Thus,
    9.74 +submodules can be considered independent of their packages, although they may
    9.75 +seek to import their package top-level module should they need to access
    9.76 +objects provided by that module.
    9.77  
    9.78  == Implicit Imports ==
    9.79  
    9.80 @@ -56,11 +75,20 @@
    9.81  || Unary operators || `operator.unary` ||
    9.82  || Access to built-in name || `__builtins__` || (various modules in the [[../Builtins|built-ins]] package hierarchy) ||
    9.83  
    9.84 -Operator usage will cause a local name referring to an `operator` module function to be created, with the appropriate function being exposed by the `operator` module itself. However, the inspection process will seek to obtain a reference to the function in its actual definition location, ultimately referencing the function in one of the modules indicated above.
    9.85 +Operator usage will cause a local name referring to an `operator` module
    9.86 +function to be created, with the appropriate function being exposed by the
    9.87 +`operator` module itself. However, the inspection process will seek to obtain
    9.88 +a reference to the function in its actual definition location, ultimately
    9.89 +referencing the function in one of the modules indicated above.
    9.90  
    9.91  == Import Sequencing ==
    9.92  
    9.93 -In order to populate modules, other modules may themselves be required to provide names to a given module, and in turn these other modules may rely on yet more modules, and so on. One logical consequence of this is that circular imports become possible, but the resulting mutual dependencies may not be easily untangled without careful attention to the state of each of the participating modules. Consider the following situation:
    9.94 +In order to populate modules, other modules may themselves be required to
    9.95 +provide names to a given module, and in turn these other modules may rely on
    9.96 +yet more modules, and so on. One logical consequence of this is that circular
    9.97 +imports become possible, but the resulting mutual dependencies may not be
    9.98 +easily untangled without careful attention to the state of each of the
    9.99 +participating modules. Consider the following situation:
   9.100  
   9.101  {{{{#!table
   9.102  {{{#!graphviz
   9.103 @@ -116,25 +144,65 @@
   9.104  }}}
   9.105  }}}}
   9.106  
   9.107 -If modules were loaded upon being encountered in import statements, module A would not be completely processed when attempting to import from module B, and thus the import within module B of module A would only yield some information about module A. Consequently, the details of class D might not be available, and this would then have an impact on whether module B could even be completely processed itself.
   9.108 +If modules were loaded upon being encountered in import statements, module A
   9.109 +would not be completely processed when attempting to import from module B, and
   9.110 +thus the import within module B of module A would only yield some information
   9.111 +about module A. Consequently, the details of class D might not be available,
   9.112 +and this would then have an impact on whether module B could even be
   9.113 +completely processed itself.
   9.114  
   9.115 -The approach taken to generally deal with such situations is to defer resolution until all modules have been populated. Then, names are resolved with any names employing kinds of references specified as `<depends>` (instead of, for example, `<class>`) being resolved according to the recorded import dependencies.
   9.116 +The approach taken to generally deal with such situations is to defer
   9.117 +resolution until all modules have been populated. Then, names are resolved
   9.118 +with any names employing kinds of references specified as `<depends>` (instead
   9.119 +of, for example, `<class>`) being resolved according to the recorded import
   9.120 +dependencies.
   9.121  
   9.122 -Since the classes in one module may depend on those in other modules, it is not always possible to finalise the details of classes in a module context. And since modules may depend on each other, it is not always possible to finalise the details of classes until the details of all classes in a program are known.
   9.123 +Since the classes in one module may depend on those in other modules, it is
   9.124 +not always possible to finalise the details of classes in a module context.
   9.125 +And since modules may depend on each other, it is not always possible to
   9.126 +finalise the details of classes until the details of all classes in a program
   9.127 +are known.
   9.128  
   9.129  === Module Initialisation ===
   9.130  
   9.131 -Although static objects can be defined with interdependencies in a declarative fashion, the initialisation of objects in modules may require the availability of completely-initialised objects defined in other modules. Thus, an initialisation order needs to be established, with some modules being initialised before others, so that all modules do not encounter uninitialised names when they are expecting those names to provide valid objects.
   9.132 +Although static objects can be defined with interdependencies in a declarative
   9.133 +fashion, the initialisation of objects in modules may require the availability
   9.134 +of completely-initialised objects defined in other modules. Thus, an
   9.135 +initialisation order needs to be established, with some modules being
   9.136 +initialised before others, so that all modules do not encounter uninitialised
   9.137 +names when they are expecting those names to provide valid objects.
   9.138  
   9.139 -The most obvious example of a module requiring the initialisation of others before it is itself evaluated is, of course, the `__main__` module. Given that it may import instances defined as attribute on other modules, it clearly requires those modules to have been initialised and those instances to have been created. It would be absurd to consider running the body of the `__main__` module before such other modules. Similarly, such dependencies exist between other modules, and consequently, an appropriate initialisation ordering must be defined for them. In its entirety, then, a program must define a workable ordering for all of its modules, signalling a concrete error if no such ordering can be established.
   9.140 +The most obvious example of a module requiring the initialisation of others
   9.141 +before it is itself evaluated is, of course, the `__main__` module. Given that
   9.142 +it may import instances defined as attribute on other modules, it clearly
   9.143 +requires those modules to have been initialised and those instances to have
   9.144 +been created. It would be absurd to consider running the body of the
   9.145 +`__main__` module before such other modules. Similarly, such dependencies
   9.146 +exist between other modules, and consequently, an appropriate initialisation
   9.147 +ordering must be defined for them. In its entirety, then, a program must
   9.148 +define a workable ordering for all of its modules, signalling a concrete error
   9.149 +if no such ordering can be established.
   9.150  
   9.151  == Hidden Modules ==
   9.152  
   9.153 -Imports that do not obtain the imported module name itself, such as those initiated by the `from` statement and by implicit operations, keep the imported module '''hidden'''. Unless other operations expose hidden modules, they will remain hidden and may consequently be omitted from the final generated program: there would be no way of referencing such modules and they would therefore be unable to contribute their contents to the rest of the program.
   9.154 +Imports that do not obtain the imported module name itself, such as those
   9.155 +initiated by the `from` statement and by implicit operations, keep the
   9.156 +imported module '''hidden'''. Unless other operations expose hidden modules,
   9.157 +they will remain hidden and may consequently be omitted from the final
   9.158 +generated program: there would be no way of referencing such modules and they
   9.159 +would therefore be unable to contribute their contents to the rest of the
   9.160 +program.
   9.161  
   9.162 -However, where an object provided by a module is referenced, a module cannot remain hidden, since the provided object may depend on other parts of the module in order to function correctly. And since a provided object might reference or return other objects in the module, the general module contents must also be exposed.
   9.163 +However, where an object provided by a module is referenced, a module cannot
   9.164 +remain hidden, since the provided object may depend on other parts of the
   9.165 +module in order to function correctly. And since a provided object might
   9.166 +reference or return other objects in the module, the general module contents
   9.167 +must also be exposed.
   9.168  
   9.169 -Import dependencies are defined for namespaces indicating modules that are required by each namespace. By following dependency relationships, it is possible to determine the eventual target of an import and to potentially skip over modules that merely import and expose names. For example:
   9.170 +Import dependencies are defined for namespaces indicating modules that are
   9.171 +required by each namespace. By following dependency relationships, it is
   9.172 +possible to determine the eventual target of an import and to potentially skip
   9.173 +over modules that merely import and expose names. For example:
   9.174  
   9.175  {{{{#!table
   9.176  {{{#!graphviz
   9.177 @@ -184,4 +252,8 @@
   9.178  }}}
   9.179  }}}}
   9.180  
   9.181 -Here, B is never explicitly referenced, nor does it provide any referenced objects other than an imported name. Consequently, B is hidden and ultimately excluded from the final program. Such techniques are employed in the [[../Builtins|built-ins]] package hierarchy to reduce the amount of functionality employed by (and bundled in) a generated program.
   9.182 +Here, B is never explicitly referenced, nor does it provide any referenced
   9.183 +objects other than an imported name. Consequently, B is hidden and ultimately
   9.184 +excluded from the final program. Such techniques are employed in the
   9.185 +[[../Builtins|built-ins]] package hierarchy to reduce the amount of
   9.186 +functionality employed by (and bundled in) a generated program.
    10.1 --- a/docs/wiki/Inspection	Sun Jul 22 12:14:48 2018 +0200
    10.2 +++ b/docs/wiki/Inspection	Thu Jan 17 14:23:38 2019 +0100
    10.3 @@ -1,6 +1,12 @@
    10.4  = Inspecting Programs =
    10.5  
    10.6 -A program's source code is inspected module by module. As [[../Imports|imports]] of modules are encountered, other modules are added to the program. The inspection process for each module involves top-to-bottom traversal of the code using depth-first processing of the abstract syntax tree, with a stack of namespaces being employed to record definitions and names within namespaces are they are visited. Inspection continues until all modules in the program have been imported and inspected.
    10.7 +A program's source code is inspected module by module. As
    10.8 +[[../Imports|imports]] of modules are encountered, other modules are added to
    10.9 +the program. The inspection process for each module involves top-to-bottom
   10.10 +traversal of the code using depth-first processing of the abstract syntax
   10.11 +tree, with a stack of namespaces being employed to record definitions and
   10.12 +names within namespaces are they are visited. Inspection continues until all
   10.13 +modules in the program have been imported and inspected.
   10.14  
   10.15  <<TableOfContents(2,3)>>
   10.16  
   10.17 @@ -12,14 +18,20 @@
   10.18     * Identifying the names defined by each namespace
   10.19     * Recording relationships between namespaces
   10.20   * Describing the operations in each namespace
   10.21 -   * Identifying the names in each namespace and the attributes used by those names
   10.22 +   * Identifying the names in each namespace and the attributes used by those
   10.23 +     names
   10.24     * Recording details of assignments and invocations
   10.25  
   10.26 -The results of inspection are written out to [[../Cache|cache]] files, one for each module in the program.
   10.27 +The results of inspection are written out to [[../Cache|cache]] files, one for
   10.28 +each module in the program.
   10.29  
   10.30  === Program Units for Inspection ===
   10.31  
   10.32 -The `inspector` module performs much of the inspection work, relying on the `common` module for certain tasks, with the `modules` module providing the relevant module abstractions including those writing to and reading from cache files, and the `resolving` module completing each module's inspection. The `importer` module coordinates inspection across the whole program.
   10.33 +The `inspector` module performs much of the inspection work, relying on the
   10.34 +`common` module for certain tasks, with the `modules` module providing the
   10.35 +relevant module abstractions including those writing to and reading from cache
   10.36 +files, and the `resolving` module completing each module's inspection. The
   10.37 +`importer` module coordinates inspection across the whole program.
   10.38  
   10.39  === Name Identification ===
   10.40  
   10.41 @@ -31,15 +43,31 @@
   10.42   * Importing of modules
   10.43   * Importing of names from modules
   10.44  
   10.45 -The origins of names are discovered by considering local, global and built-in namespaces. Where a name is '''local''', it is defined in the same namespace. Where a name is '''global''', it is defined in the same module at the module level. Where a name is '''built-in''', it is defined in a module in the `__builtins__` package and exposed via the `__builtins__` namespace.
   10.46 +The origins of names are discovered by considering local, global and built-in
   10.47 +namespaces. Where a name is '''local''', it is defined in the same namespace.
   10.48 +Where a name is '''global''', it is defined in the same module at the module
   10.49 +level. Where a name is '''built-in''', it is defined in a module in the
   10.50 +`__builtins__` package and exposed via the `__builtins__` namespace.
   10.51  
   10.52 -Initial tentative identification of names will sort names into two categories: locals and external names. Global variables employed in function (or method) namespaces may not be defined when a function body is inspected, either within a module or being imported from another module, and so it is not initially possible to more specifically determine the nature of a name.
   10.53 +Initial tentative identification of names will sort names into two categories:
   10.54 +locals and external names. Global variables employed in function (or method)
   10.55 +namespaces may not be defined when a function body is inspected, either within
   10.56 +a module or being imported from another module, and so it is not initially
   10.57 +possible to more specifically determine the nature of a name.
   10.58  
   10.59 -These categories are later refined to distinguish between globals and built-in names as external names. The built-in origin of a name is only suggested when no locals or globals can provide the name, and the final identification of such names, plus other external names introduced as locals or globals via imports, will occur at the end of the inspection activity. Names that are unrecognised by then may be treated like unrecognised built-ins.
   10.60 +These categories are later refined to distinguish between globals and built-in
   10.61 +names as external names. The built-in origin of a name is only suggested when
   10.62 +no locals or globals can provide the name, and the final identification of
   10.63 +such names, plus other external names introduced as locals or globals via
   10.64 +imports, will occur at the end of the inspection activity. Names that are
   10.65 +unrecognised by then may be treated like unrecognised built-ins.
   10.66  
   10.67  === Name Restrictions and Resolution ===
   10.68  
   10.69 -'''Static''' objects, forming the fundamental [[../Structure|structure]] of the program, expose their names through the general structure hierarchy. Classes, which are defined as part of this structure, depend on well-defined names for any base classes they employ. For example:
   10.70 +'''Static''' objects, forming the fundamental [[../Structure|structure]] of
   10.71 +the program, expose their names through the general structure hierarchy.
   10.72 +Classes, which are defined as part of this structure, depend on well-defined
   10.73 +names for any base classes they employ. For example:
   10.74  
   10.75  {{{#!python numbers=disable
   10.76  class C(A): # depends on A being a name that can be resolved (and being a class)
   10.77 @@ -49,50 +77,83 @@
   10.78      ...
   10.79  }}}
   10.80  
   10.81 -Base classes must be identifiable and unambiguous. Since base classes may be imported, their identification may not necessarily occur immediately, but it must be possible once all information about a program is known and when all such dependencies are resolved.
   10.82 +Base classes must be identifiable and unambiguous. Since base classes may be
   10.83 +imported, their identification may not necessarily occur immediately, but it
   10.84 +must be possible once all information about a program is known and when all
   10.85 +such dependencies are resolved.
   10.86  
   10.87  === Attribute Identification ===
   10.88  
   10.89  Attributes are introduced to namespaces as follows:
   10.90  
   10.91 - * For classes and modules, the definition of names defines attributes in those namespaces
   10.92 - * Functions do not generally support attributes, although [[../Representations#Special_Members|representation-specific attributes]] do exist on functions
   10.93 - * For instances, assignments to attributes of the special `self` name, performed in methods, define attributes in all instances of the method's class
   10.94 + * For classes and modules, the definition of names defines attributes in
   10.95 +   those namespaces
   10.96 + * Functions do not generally support attributes, although
   10.97 +   [[../Representations#Special_Members|representation-specific attributes]]
   10.98 +   do exist on functions
   10.99 + * For instances, assignments to attributes of the special `self` name,
  10.100 +   performed in methods, define attributes in all instances of the method's
  10.101 +   class
  10.102  
  10.103 -Attributes are only supported on program objects if they have been defined or '''bound''' as described above. Any attempt to access or set an attribute on an object using a name that was not determined through the above process is considered an invalid operation. Thus, augmenting the attributes available on an object (so-called "monkeypatching") is not possible.
  10.104 +Attributes are only supported on program objects if they have been defined or
  10.105 +'''bound''' as described above. Any attempt to access or set an attribute on
  10.106 +an object using a name that was not determined through the above process is
  10.107 +considered an invalid operation. Thus, augmenting the attributes available on
  10.108 +an object (so-called "monkeypatching") is not possible.
  10.109  
  10.110 -When an attribute is accessed, the location of its use is recorded and ultimately associated with assignments of the name involved, just as is done for accesses of the plain name itself, but the attribute details are subsequently collected together for each assignment or '''version''' of the name. This is discussed below.
  10.111 +When an attribute is accessed, the location of its use is recorded and
  10.112 +ultimately associated with assignments of the name involved, just as is done
  10.113 +for accesses of the plain name itself, but the attribute details are
  10.114 +subsequently collected together for each assignment or '''version''' of the
  10.115 +name. This is discussed below.
  10.116  
  10.117  === Inherited Attributes ===
  10.118  
  10.119 -In order to support inheritance, a process of propagation makes class and instance attributes available to any given class from its ancestor classes according to a depth-first traversal of a class's base class hierarchy. Thus, for each class, given the definition of its base classes, a complete collection of class and instance attribute names can be determined. The actual mechanism of propagation occurs during the consolidation phase of inspection, principally because class bases are not generally immediately identifiable upon completing the inspection of any given class.
  10.120 +In order to support inheritance, a process of propagation makes class and
  10.121 +instance attributes available to any given class from its ancestor classes
  10.122 +according to a depth-first traversal of a class's base class hierarchy. Thus,
  10.123 +for each class, given the definition of its base classes, a complete
  10.124 +collection of class and instance attribute names can be determined. The actual
  10.125 +mechanism of propagation occurs during the consolidation phase of inspection,
  10.126 +principally because class bases are not generally immediately identifiable
  10.127 +upon completing the inspection of any given class.
  10.128  
  10.129  === Name Assignments and Accesses ===
  10.130  
  10.131 -Each assignment of a name defines a '''version''' of the name within a namespace. The location of this definition consists of the following:
  10.132 +Each assignment of a name defines a '''version''' of the name within a
  10.133 +namespace. The location of this definition consists of the following:
  10.134  
  10.135   * The namespace in which the assignment appears
  10.136   * The name itself
  10.137   * The version or assignment instance number
  10.138  
  10.139 -When a name is used, the location of its use is recorded and is ultimately associated with the assignments that may be providing the name at that location. This permits information about the type of the name to become available for each usage location. The location of each name usage consists of the following:
  10.140 +When a name is used, the location of its use is recorded and is ultimately
  10.141 +associated with the assignments that may be providing the name at that
  10.142 +location. This permits information about the type of the name to become
  10.143 +available for each usage location. The location of each name usage consists of
  10.144 +the following:
  10.145  
  10.146   * The namespace in which the name appears
  10.147   * The name itself
  10.148   * The access instance number
  10.149  
  10.150 -Name accesses are described as special cases of attribute accesses: where attributes would be indicated, none are specified.
  10.151 +Name accesses are described as special cases of attribute accesses: where
  10.152 +attributes would be indicated, none are specified.
  10.153  
  10.154  === Attribute Accesses ===
  10.155  
  10.156 -Attribute '''accesses''' are operations involving attributes where those attributes are obtained or set in conjunction with an '''accessor''' expression. They are recorded in terms of location tuples, each describing an access as follows:
  10.157 +Attribute '''accesses''' are operations involving attributes where those
  10.158 +attributes are obtained or set in conjunction with an '''accessor'''
  10.159 +expression. They are recorded in terms of location tuples, each describing an
  10.160 +access as follows:
  10.161  
  10.162   * The namespace in which the access occurs
  10.163   * Any named accessor or an anonymous accessor
  10.164   * The attribute (or chain of attributes) involved (omitted for name accesses)
  10.165   * The access instance number
  10.166  
  10.167 -As with name accesses, the access instance number distinguishes between different accesses employing the same details.
  10.168 +As with name accesses, the access instance number distinguishes between
  10.169 +different accesses employing the same details.
  10.170  
  10.171  Consider the following example:
  10.172  
  10.173 @@ -197,7 +258,11 @@
  10.174  }}}
  10.175  }}}}
  10.176  
  10.177 -Since the names involved may be provided by different name versions, accesses are counted independently. Meanwhile, a non-name or '''anonymous''' accessor may be involved in an access. Such anonymous accessors are independent and do not accumulate attribute usage because they potentially involve completely different objects.
  10.178 +Since the names involved may be provided by different name versions, accesses
  10.179 +are counted independently. Meanwhile, a non-name or '''anonymous''' accessor
  10.180 +may be involved in an access. Such anonymous accessors are independent and do
  10.181 +not accumulate attribute usage because they potentially involve completely
  10.182 +different objects.
  10.183  
  10.184  || '''Namespace''' || '''Name''' || '''Attribute''' || '''Access number''' ||
  10.185  || `module.f`      || `p`        || `a`             || 0 ||
  10.186 @@ -215,11 +280,19 @@
  10.187  || `module.f`      || `q`        || `{}`            || 0 ||
  10.188  || `module.f`      || `q`        || `{}`            || 1 ||
  10.189  
  10.190 -Here, the attribute field is left empty and indicates that name definitions are being described. Although the data is superficially similar to name accesses, it should be remembered that accesses employ an access number whereas accessors employ a name version, with such identifiers being different things.
  10.191 +Here, the attribute field is left empty and indicates that name definitions
  10.192 +are being described. Although the data is superficially similar to name
  10.193 +accesses, it should be remembered that accesses employ an access number
  10.194 +whereas accessors employ a name version, with such identifiers being different
  10.195 +things.
  10.196  
  10.197  === Names and Attribute Usage ===
  10.198  
  10.199 -Within each scope, the names employed by the program and the attributes used with those names are tracked. As the path of execution diverges and converges again through control-flow constructs, the tracking attempts to maintain the '''attribute usage''' associated with each name, or specifically each '''version''' of each name.
  10.200 +Within each scope, the names employed by the program and the attributes used
  10.201 +with those names are tracked. As the path of execution diverges and converges
  10.202 +again through control-flow constructs, the tracking attempts to maintain the
  10.203 +'''attribute usage''' associated with each name, or specifically each
  10.204 +'''version''' of each name.
  10.205  
  10.206  {{{{#!table
  10.207  <style="vertical-align: top">
  10.208 @@ -285,21 +358,45 @@
  10.209  }}}
  10.210  }}}}
  10.211  
  10.212 -The outcome of such tracking should be an indication of the attribute usage with each name based on the shortest routes that names can take through the control-flow structure. Such shortest-route usage defines the minimal selection of attributes that can be considered used with a name, and thus such usage defines the broadest selection of object types that can be identified as supporting such attributes. In the above example, the following minimal selections of attributes apply:
  10.213 +The outcome of such tracking should be an indication of the attribute usage
  10.214 +with each name based on the shortest routes that names can take through the
  10.215 +control-flow structure. Such shortest-route usage defines the minimal
  10.216 +selection of attributes that can be considered used with a name, and thus such
  10.217 +usage defines the broadest selection of object types that can be identified as
  10.218 +supporting such attributes. In the above example, the following minimal
  10.219 +selections of attributes apply:
  10.220  
  10.221  || '''Name Version''' || '''Minimal Set of Attributes''' || '''Types''' ||
  10.222  || `y` (version 0) || ''empty set'' || ''any object'' ||
  10.223  || `y` (version 1) || `a2` ||  ''only objects providing the single attribute'' ||
  10.224  
  10.225 -The assumption is made that any attribute used with a name is always provided by the object referenced by that name and that the correct execution of the code does not rely on an attribute being absent (and thus raising an exception). By defining usage for each name, the toolchain can determine whether any type can provide the attributes used with a name, producing a compile-time error if no type supports such usage.
  10.226 +The assumption is made that any attribute used with a name is always provided
  10.227 +by the object referenced by that name and that the correct execution of the
  10.228 +code does not rely on an attribute being absent (and thus raising an
  10.229 +exception). By defining usage for each name, the toolchain can determine
  10.230 +whether any type can provide the attributes used with a name, producing a
  10.231 +compile-time error if no type supports such usage.
  10.232  
  10.233 -It is possible that certain routes taken by names might define attribute usage that cannot be supported by types that do support the shortest-route usage, yet it might not be appropriate to forbid usage of such types with such names: the program logic may intend that such types do not visit the regions of the code that employ the attributes that such types cannot support. However, as a consequence, run-time tests will be required to prevent attribute accesses that are inappropriate for such types from taking place. In the above example, the following maximal selections apply:
  10.234 +It is possible that certain routes taken by names might define attribute usage
  10.235 +that cannot be supported by types that do support the shortest-route usage,
  10.236 +yet it might not be appropriate to forbid usage of such types with such names:
  10.237 +the program logic may intend that such types do not visit the regions of the
  10.238 +code that employ the attributes that such types cannot support. However, as a
  10.239 +consequence, run-time tests will be required to prevent attribute accesses
  10.240 +that are inappropriate for such types from taking place. In the above example,
  10.241 +the following maximal selections apply:
  10.242  
  10.243  || '''Name Version''' || '''Maximal Set of Attributes''' || '''Types''' ||
  10.244  || `y` (version 0) || `a1`, `a3` || ''only objects providing both attributes'' ||
  10.245  || `y` (version 1) || `a1`, `a2`, `a3` ||  ''only objects providing all three attributes'' ||
  10.246  
  10.247 -Tracking occurs separately in function or method namespaces and at a level combining the static namespaces in a module. This latter combination of namespaces permits the flow of global name details through class namespaces. Tracking employs special '''tracking names''' with which usage is associated, with globals and class attributes employing complete object paths to describe their names, whereas locals merely employ the plain names defined and used in local namespaces. Some examples follow:
  10.248 +Tracking occurs separately in function or method namespaces and at a level
  10.249 +combining the static namespaces in a module. This latter combination of
  10.250 +namespaces permits the flow of global name details through class namespaces.
  10.251 +Tracking employs special '''tracking names''' with which usage is associated,
  10.252 +with globals and class attributes employing complete object paths to describe
  10.253 +their names, whereas locals merely employ the plain names defined and used in
  10.254 +local namespaces. Some examples follow:
  10.255  
  10.256  || '''Namespace''' || '''Name''' || '''Name Scope''' || '''Tracking Name''' ||
  10.257  || `__main__` (module) || `x` || global || `__main__.x` ||
  10.258 @@ -309,14 +406,21 @@
  10.259  
  10.260  === Name Initialisation and Aliases ===
  10.261  
  10.262 -Each name version may be associated with a value upon assignment provided by an expression known as an '''initialiser'''. Such values may be used to inform the interpretation of operations on names, restricting the types involved to the initialised values. Some examples are as follows:
  10.263 +Each name version may be associated with a value upon assignment provided by
  10.264 +an expression known as an '''initialiser'''. Such values may be used to inform
  10.265 +the interpretation of operations on names, restricting the types involved to
  10.266 +the initialised values. Some examples are as follows:
  10.267  
  10.268  {{{#!python numbers=disable
  10.269  x = 123         # initialiser is a constant, indicating a known type
  10.270  x = classname() # initialiser refers to a class, indicating instantiation
  10.271  }}}
  10.272  
  10.273 -Names may also be assigned the values of other names, and such a name becomes an '''alias''' for such other names. Aliases can therefore be used to combine attribute usage observations across names. Other forms of aliases involve assigning attributes accessed via other names to any given name. Some examples are as follows:
  10.274 +Names may also be assigned the values of other names, and such a name becomes
  10.275 +an '''alias''' for such other names. Aliases can therefore be used to combine
  10.276 +attribute usage observations across names. Other forms of aliases involve
  10.277 +assigning attributes accessed via other names to any given name. Some examples
  10.278 +are as follows:
  10.279  
  10.280  {{{#!python numbers=disable
  10.281  y = x   # y 
  10.282 @@ -324,95 +428,180 @@
  10.283  z = x.q # z can be restricted to the types deduced for x.q
  10.284  }}}
  10.285  
  10.286 -Initialising values are retained for later resolution because their identities are not generally known until certain name resolution activities have taken place.
  10.287 +Initialising values are retained for later resolution because their identities
  10.288 +are not generally known until certain name resolution activities have taken
  10.289 +place.
  10.290  
  10.291  === Invocations ===
  10.292  
  10.293 -During inspection only limited information is available about the nature of invocations. However, some information will already be available about global names and these may be defined by classes. Thus, invocations that cause the instantiation of classes may be determined even during the inspection phase.
  10.294 +During inspection only limited information is available about the nature of
  10.295 +invocations. However, some information will already be available about global
  10.296 +names and these may be defined by classes. Thus, invocations that cause the
  10.297 +instantiation of classes may be determined even during the inspection phase.
  10.298  
  10.299 -Information about class instantiation is most useful in combination with the initialisation of names. When an assignment occurs, any initialising expression that provides enough information can be evaluated to see if it describes instantiation. If so, the nature of the instantiation can be associated with the name and used in the deduction process to constrain any usage of that name and its attributes. Such restrictions on the types associated with names are applied in the deduction phase.
  10.300 +Information about class instantiation is most useful in combination with the
  10.301 +initialisation of names. When an assignment occurs, any initialising
  10.302 +expression that provides enough information can be evaluated to see if it
  10.303 +describes instantiation. If so, the nature of the instantiation can be
  10.304 +associated with the name and used in the deduction process to constrain any
  10.305 +usage of that name and its attributes. Such restrictions on the types
  10.306 +associated with names are applied in the deduction phase.
  10.307  
  10.308  === Literals and Constants ===
  10.309  
  10.310 -Literal values or '''literals''' are specific values that appear in the program, typically representing numbers, character strings, mappings or collections. Literal numbers and strings are effectively '''constants''', meaning that their values are unambiguously defined and can eventually be referenced using a unique identifier that applies throughout the program and which can refer to an initialised object in any generated program. Initially, they are recorded for the namespace in which they appear. For example:
  10.311 +Literal values or '''literals''' are specific values that appear in the
  10.312 +program, typically representing numbers, character strings, mappings or
  10.313 +collections. Literal numbers and strings are effectively '''constants''',
  10.314 +meaning that their values are unambiguously defined and can eventually be
  10.315 +referenced using a unique identifier that applies throughout the program and
  10.316 +which can refer to an initialised object in any generated program. Initially,
  10.317 +they are recorded for the namespace in which they appear. For example:
  10.318  
  10.319  || '''Namespace''' || '''Identifier''' || '''Type''' || '''Encoding''' || '''Value''' ||
  10.320  || `__main__` || `__main__.$c0` || `__builtins__.str.string` || `iso-8859-15` || `'\xc6\xd8\xc5'` ||
  10.321  || `__main__` || `__main__.$c1` || `__builtins__.int.int` || || `123` ||
  10.322  
  10.323 -Since values may appear again in other namespaces, a mapping is generated from such local identifiers to common global identifiers for constants. Where such an identifier is derived from the value content, this can potentially occur immediately, although in practice (and in general) such a mapping will be generated after all modules have been inspected.
  10.324 +Since values may appear again in other namespaces, a mapping is generated from
  10.325 +such local identifiers to common global identifiers for constants. Where such
  10.326 +an identifier is derived from the value content, this can potentially occur
  10.327 +immediately, although in practice (and in general) such a mapping will be
  10.328 +generated after all modules have been inspected.
  10.329  
  10.330 -Collections and mappings may also be initialised using literal syntax but may contain non-constant information such as names or expressions whose values are unknown before the program is run. Such values can be represented as instantiation operations of the appropriate type in any generated program, and the instances of the type concerned can be associated with any names to which such literals are assigned. Special names are used to refer to literal types for collections and mappings. For example:
  10.331 +Collections and mappings may also be initialised using literal syntax but may
  10.332 +contain non-constant information such as names or expressions whose values are
  10.333 +unknown before the program is run. Such values can be represented as
  10.334 +instantiation operations of the appropriate type in any generated program, and
  10.335 +the instances of the type concerned can be associated with any names to which
  10.336 +such literals are assigned. Special names are used to refer to literal types
  10.337 +for collections and mappings. For example:
  10.338  
  10.339  || '''Identifier''' || '''Type''' ||
  10.340  || `$Ldict` || `__builtins__.dict.dict` ||
  10.341  || `$Llist` || `__builtins__.list.list` ||
  10.342  || `$Ltuple` || `__builtins__.tuple.tuple` ||
  10.343  
  10.344 -Such special names merely serve as convenient placeholders for the types of any literals mentioned in a module.
  10.345 +Such special names merely serve as convenient placeholders for the types of
  10.346 +any literals mentioned in a module.
  10.347  
  10.348  == Consolidating Inspection Details ==
  10.349  
  10.350 -As briefly discussed above, certain activities occur after information has been collected from an input program. Within each module, names that are external to namespaces are recorded in a '''name reference''' collection. These references identify unrecognised names but do not generally define their origins. Examples of name references are as follows:
  10.351 +As briefly discussed above, certain activities occur after information has
  10.352 +been collected from an input program. Within each module, names that are
  10.353 +external to namespaces are recorded in a '''name reference''' collection.
  10.354 +These references identify unrecognised names but do not generally define their
  10.355 +origins. Examples of name references are as follows:
  10.356  
  10.357  || '''Name''' || '''Identity''' ||
  10.358  || `__main__.repr` || `<deferred>:__builtins__.repr` ||
  10.359  || `__main__.sys` || `<module>:sys` ||
  10.360  
  10.361 -In order to obtain the final identities, deferred references may need to be resolved, yielding concrete references:
  10.362 +In order to obtain the final identities, deferred references may need to be
  10.363 +resolved, yielding concrete references:
  10.364  
  10.365  || '''Name''' || '''Identity''' ||
  10.366  || `__main__.repr` || `<function>:__builtins__.identity.repr` ||
  10.367  || `__main__.sys` || `<module>:sys` ||
  10.368  
  10.369 -This process of resolution cannot be performed immediately with only the knowledge available in a single module. Only with all program modules loaded can such resolution occur.
  10.370 +This process of resolution cannot be performed immediately with only the
  10.371 +knowledge available in a single module. Only with all program modules loaded
  10.372 +can such resolution occur.
  10.373  
  10.374  === Identifying Deferred References ===
  10.375  
  10.376 -After all modules are loaded, and with all objects present in the program at this point, each deferred reference defined within a module should ultimately yield an identity. Any failure to do so indicates usage of a name not defined in the program. The process of identifying them by resolving what each of them points to, is illustrated by the following example:
  10.377 +After all modules are loaded, and with all objects present in the program at
  10.378 +this point, each deferred reference defined within a module should ultimately
  10.379 +yield an identity. Any failure to do so indicates usage of a name not defined
  10.380 +in the program. The process of identifying them by resolving what each of them
  10.381 +points to, is illustrated by the following example:
  10.382  
  10.383  || '''Reference''' || '''Object Path to Search''' ||
  10.384  || `<depends>:__builtins__.repr` || `__builtins__.repr` ||
  10.385  || `<depends>:__builtins__.identity.repr` || `__builtins__.identity.repr` ||
  10.386  || `<function>:__builtins__.identity.repr` || ''identification has occurred'' ||
  10.387  
  10.388 -Initially, the origin information from the reference is used to find the details of the referenced object. However, in this case, the details yield another deferred reference that has not yet been resolved. Consequently, the origin information must be used to find the details of the object referenced in this case, finally yielding a reference with concrete object information. Deferred references are mutated to concrete references, thus changing the nature of such references throughout the accumulated data.
  10.389 +Initially, the origin information from the reference is used to find the
  10.390 +details of the referenced object. However, in this case, the details yield
  10.391 +another deferred reference that has not yet been resolved. Consequently, the
  10.392 +origin information must be used to find the details of the object referenced
  10.393 +in this case, finally yielding a reference with concrete object information.
  10.394 +Deferred references are mutated to concrete references, thus changing the
  10.395 +nature of such references throughout the accumulated data.
  10.396  
  10.397  === Identifying Module Dependencies ===
  10.398  
  10.399 -During deferred reference resolution, relationships are catalogued between modules in which deferred references are found and those providing the corresponding referenced objects. In addition, module ordering dependencies are established on the basis that some kinds of objects must be initialised before being used in other parts of the program. As a result, the modules providing such objects must themselves be initialised before the modules referencing such objects. More information on this topic can be found in the [[../Imports#Import_Sequencing|import sequencing]] documentation.
  10.400 +During deferred reference resolution, relationships are catalogued between
  10.401 +modules in which deferred references are found and those providing the
  10.402 +corresponding referenced objects. In addition, module ordering dependencies
  10.403 +are established on the basis that some kinds of objects must be initialised
  10.404 +before being used in other parts of the program. As a result, the modules
  10.405 +providing such objects must themselves be initialised before the modules
  10.406 +referencing such objects. More information on this topic can be found in the
  10.407 +[[../Imports#Import_Sequencing|import sequencing]] documentation.
  10.408  
  10.409  === Resolving Module Details ===
  10.410  
  10.411 -With all dependencies resolved, further work can be done with the details within each module. The `resolving` module provides the functionality to each module instance to perform such work.
  10.412 +With all dependencies resolved, further work can be done with the details
  10.413 +within each module. The `resolving` module provides the functionality to each
  10.414 +module instance to perform such work.
  10.415  
  10.416   1. Class bases can be identified.
  10.417 - 1. Special names (referring to built-in objects employed by certain operations) are resolved.
  10.418 - 1. Each access involving an external name is investigated to see if it provides an effectively constant object (as described below).
  10.419 - 1. Invocation references are converted to provide instantiation information, if appropriate.
  10.420 - 1. Initialisers are investigated to see if they provide concrete object details and can thus indicate the nature of a name.
  10.421 + 1. Special names (referring to built-in objects employed by certain
  10.422 +    operations) are resolved.
  10.423 + 1. Each access involving an external name is investigated to see if it
  10.424 +    provides an effectively constant object (as described below).
  10.425 + 1. Invocation references are converted to provide instantiation information,
  10.426 +    if appropriate.
  10.427 + 1. Initialisers are investigated to see if they provide concrete object
  10.428 +    details and can thus indicate the nature of a name.
  10.429   1. Constant value literals are associated with the appropriate types.
  10.430  
  10.431 -Thus, attribute details for each module and its classes are finalised. Meanwhile, modules that are still hidden are removed from the program.
  10.432 +Thus, attribute details for each module and its classes are finalised.
  10.433 +Meanwhile, modules that are still hidden are removed from the program.
  10.434  
  10.435  ==== Investigating Accesses ====
  10.436  
  10.437 -External names will not have been resolved during the inspection process, but with information about the whole program, it becomes possible to identify names and to determine whether attribute accesses involving those names can also be identified. Thus, name references from each namespace are investigated in turn as follows.
  10.438 +External names will not have been resolved during the inspection process, but
  10.439 +with information about the whole program, it becomes possible to identify
  10.440 +names and to determine whether attribute accesses involving those names can
  10.441 +also be identified. Thus, name references from each namespace are investigated
  10.442 +in turn as follows.
  10.443  
  10.444 -Each name referencing a static object is considered a constant access of that object. However, given a number of accompanying attributes describing an access, each attribute is added in turn, and the resulting object path identified. If the path identifies a constant object, the next attribute in the access chain is added and the resulting object path identified, and so on. The maximal collection of attributes identifying a constant object is then recorded as a constant access, with any remaining attributes in the access chain retained for traversal in a later phase.
  10.445 +Each name referencing a static object is considered a constant access of that
  10.446 +object. However, given a number of accompanying attributes describing an
  10.447 +access, each attribute is added in turn, and the resulting object path
  10.448 +identified. If the path identifies a constant object, the next attribute in
  10.449 +the access chain is added and the resulting object path identified, and so on.
  10.450 +The maximal collection of attributes identifying a constant object is then
  10.451 +recorded as a constant access, with any remaining attributes in the access
  10.452 +chain retained for traversal in a later phase.
  10.453  
  10.454 -Names yielding constant accesses do not need to have their identity deduced through the application of attribute usage.
  10.455 +Names yielding constant accesses do not need to have their identity deduced
  10.456 +through the application of attribute usage.
  10.457  
  10.458  ==== Investigating Initialisers ====
  10.459  
  10.460 -During inspection, initialisers will have been recorded in terms of expression results such as names, invocations, readily-identifiable instances, and attribute accesses. With details of the whole program plus constant accesses having been made available, such results may be definitively identified and associated with initialised names.
  10.461 +During inspection, initialisers will have been recorded in terms of expression
  10.462 +results such as names, invocations, readily-identifiable instances, and
  10.463 +attribute accesses. With details of the whole program plus constant accesses
  10.464 +having been made available, such results may be definitively identified and
  10.465 +associated with initialised names.
  10.466  
  10.467 -Alias information may also be refined at this point by identifying attribute accesses that are used to initialise names.
  10.468 +Alias information may also be refined at this point by identifying attribute
  10.469 +accesses that are used to initialise names.
  10.470  
  10.471  === Finalising Program Details ===
  10.472  
  10.473 -With class contents and relationships identified, class attribute inheritance and instance attribute availability can be defined. Some classes may depend on external state for their initialisation, and so additional module dependencies may be defined at this stage. The `importer` module contains the functionality to conduct these whole-program activities.
  10.474 +With class contents and relationships identified, class attribute inheritance
  10.475 +and instance attribute availability can be defined. Some classes may depend on
  10.476 +external state for their initialisation, and so additional module dependencies
  10.477 +may be defined at this stage. The `importer` module contains the functionality
  10.478 +to conduct these whole-program activities.
  10.479  
  10.480  == Inspection Outcome ==
  10.481  
  10.482 -Once inspection is complete, the available knowledge concerns the whole program and not merely individual modules or parts of the program. However, only limited efforts have been made until this point, notably in the acquisition of statically-defined object details when referencing names or attributes, to integrate the different modules. The [[../Deduction|deduction]] process is concerned with such integration.
  10.483 +Once inspection is complete, the available knowledge concerns the whole
  10.484 +program and not merely individual modules or parts of the program. However,
  10.485 +only limited efforts have been made until this point, notably in the
  10.486 +acquisition of statically-defined object details when referencing names or
  10.487 +attributes, to integrate the different modules. The [[../Deduction|deduction]]
  10.488 +process is concerned with such integration.
    11.1 --- a/docs/wiki/Optimisation	Sun Jul 22 12:14:48 2018 +0200
    11.2 +++ b/docs/wiki/Optimisation	Thu Jan 17 14:23:38 2019 +0100
    11.3 @@ -1,14 +1,28 @@
    11.4  = Optimisation =
    11.5  
    11.6 -In the process of generating an output representation of a Lichen program, the process of optimisation is concerned with positioning members within program structures. Attributes are positioned in object structures, and '''object tables''' are defined so that attributes can be located in such objects. Similarly, parameters, whose positions are determined by their appearance in function and other callable signatures, have positioning information defined in '''parameter tables''' that can be used to position arguments in parameter arrays when their names are used in argument lists.
    11.7 +In the process of generating an output representation of a Lichen program, the
    11.8 +process of optimisation is concerned with positioning members within program
    11.9 +structures. Attributes are positioned in object structures, and '''object
   11.10 +tables''' are defined so that attributes can be located in such objects.
   11.11 +Similarly, parameters, whose positions are determined by their appearance in
   11.12 +function and other callable signatures, have positioning information defined
   11.13 +in '''parameter tables''' that can be used to position arguments in parameter
   11.14 +arrays when their names are used in argument lists.
   11.15  
   11.16 -Also performed during optimisation is the consolidation of constant information, initially discussed [[../Inspection#Literals_and_Constants|in the context of inspection]].
   11.17 +Also performed during optimisation is the consolidation of constant
   11.18 +information, initially discussed [[../Inspection#Literals_and_Constants|in the
   11.19 +context of inspection]].
   11.20  
   11.21  <<TableOfContents(2,3)>>
   11.22  
   11.23  == Populating Objects ==
   11.24  
   11.25 -A program will have objects that are classes, modules and instances, and each such object will provide a number of attributes. Each object's attributes are stored in an array. For simplicity, each attribute having a given name will always be positioned at the same place in any array provided by an object featuring an attribute with that name. Even different object types will use the same position.
   11.26 +A program will have objects that are classes, modules and instances, and each
   11.27 +such object will provide a number of attributes. Each object's attributes are
   11.28 +stored in an array. For simplicity, each attribute having a given name will
   11.29 +always be positioned at the same place in any array provided by an object
   11.30 +featuring an attribute with that name. Even different object types will use
   11.31 +the same position.
   11.32  
   11.33  Consider an attribute called `elephant` provided by a number of types:
   11.34  
   11.35 @@ -18,15 +32,30 @@
   11.36  || class E || `__class__` || `ant` || `dog` || `giraffe` || `horse` ||
   11.37  || module M || `__class__` || `rhino` || `hippo` ||<style="background-color: #ddd"> `elephant` || `zebra` ||
   11.38  
   11.39 -Where `elephant` is provided as an attribute, it will always appear in the same position - in this case as the fourth attribute - in any object attribute array.
   11.40 +Where `elephant` is provided as an attribute, it will always appear in the
   11.41 +same position - in this case as the fourth attribute - in any object attribute
   11.42 +array.
   11.43  
   11.44 -Consequently, any operation involving an object of unknown identity that employs `elephant` can employ the same position information to determine whether the attribute is supported and to retrieve or modify its value. Where an object does not support `elephant`, it may use the same attribute position for another attribute. Determining whether objects support attributes is done using tables, described below.
   11.45 +Consequently, any operation involving an object of unknown identity that
   11.46 +employs `elephant` can employ the same position information to determine
   11.47 +whether the attribute is supported and to retrieve or modify its value. Where
   11.48 +an object does not support `elephant`, it may use the same attribute position
   11.49 +for another attribute. Determining whether objects support attributes is done
   11.50 +using tables, described below.
   11.51  
   11.52 -It should be noted that the positions of attributes in object structures are the same as the positions of attribute identifiers in object tables. With attribute positions allocated, table definition is trivial.
   11.53 +It should be noted that the positions of attributes in object structures are
   11.54 +the same as the positions of attribute identifiers in object tables. With
   11.55 +attribute positions allocated, table definition is trivial.
   11.56  
   11.57  === Allocation Considerations ===
   11.58  
   11.59 -Such common positioning of attributes demands a degree of coordination between objects. If `elephant` is positioned in a given place in one object, then given the constraint of it only ever being positioned in a single location for all objects, it may not then be positioned in a different place in another object. Thus, where two attributes co-exist in an object, their positions must be different and will affect all objects supporting those attributes, regardless of whether they support them both. For example:
   11.60 +Such common positioning of attributes demands a degree of coordination between
   11.61 +objects. If `elephant` is positioned in a given place in one object, then
   11.62 +given the constraint of it only ever being positioned in a single location for
   11.63 +all objects, it may not then be positioned in a different place in another
   11.64 +object. Thus, where two attributes co-exist in an object, their positions must
   11.65 +be different and will affect all objects supporting those attributes,
   11.66 +regardless of whether they support them both. For example:
   11.67  
   11.68  || '''Type''' ||<-5> '''Attributes''' ||
   11.69  || class C || `__class__` ||<style="background-color: #ddd"> `ant` || `dog` ||<style="background-color: #ddd"> `elephant` || `cat` ||
   11.70 @@ -35,21 +64,48 @@
   11.71  || module M || `__class__` || `rhino` || `hippo` ||<style="background-color: #ddd"> `elephant` || `zebra` ||
   11.72  || module N || `__class__` || || ||<style="background-color: #ddd"> `elephant` || `zebra` ||
   11.73  
   11.74 -Here, module N still positions `elephant` in the fourth position. If `elephant` were moved to the second position, it would conflict with `ant` or `rhino` in those objects supporting those attributes. Such coordination therefore has an impact on allocation strategies. Care has to be taken to allocate attributes in such a way that small objects (having few attributes) do not have their attributes positioned far from the start of their attribute arrays, because failing to do so effectively makes those objects large, inefficiently-represented objects.
   11.75 +Here, module N still positions `elephant` in the fourth position. If
   11.76 +`elephant` were moved to the second position, it would conflict with `ant` or
   11.77 +`rhino` in those objects supporting those attributes. Such coordination
   11.78 +therefore has an impact on allocation strategies. Care has to be taken to
   11.79 +allocate attributes in such a way that small objects (having few attributes)
   11.80 +do not have their attributes positioned far from the start of their attribute
   11.81 +arrays, because failing to do so effectively makes those objects large,
   11.82 +inefficiently-represented objects.
   11.83  
   11.84 -A reasonable solution is to consider the following when allocating attribute positions:
   11.85 +A reasonable solution is to consider the following when allocating attribute
   11.86 +positions:
   11.87  
   11.88   * Attribute frequency (or ubiquity)
   11.89   * Object size
   11.90   * Object kind (whether the object is a class, module or instance)
   11.91  
   11.92 -More frequently-occurring (or ubiquitous) attributes may need to appear in both large and small objects and should probably be allocated in lower positions (`__class__` being an extreme example of needing to be allocated for all objects). Attributes featured in small objects should also be given priority for lower positions due to the desirability of keeping such objects small. Meanwhile, classes and modules only appear once in a program, whereas there may be countless instances allocated during the lifetime of a program's execution. Therefore, attributes featured in instances should have priority over other attributes for lower positions in object structures.
   11.93 +More frequently-occurring (or ubiquitous) attributes may need to appear in
   11.94 +both large and small objects and should probably be allocated in lower
   11.95 +positions (`__class__` being an extreme example of needing to be allocated for
   11.96 +all objects). Attributes featured in small objects should also be given
   11.97 +priority for lower positions due to the desirability of keeping such objects
   11.98 +small. Meanwhile, classes and modules only appear once in a program, whereas
   11.99 +there may be countless instances allocated during the lifetime of a program's
  11.100 +execution. Therefore, attributes featured in instances should have priority
  11.101 +over other attributes for lower positions in object structures.
  11.102  
  11.103  == Populating Signatures ==
  11.104  
  11.105 -The positions of parameters in callable signatures are determined by the definitions of those signatures in the source program. When callables are invoked using an argument list, arguments that are not specified using keywords are merely copied into the parameter array in the same position. However, keyword arguments will need to have their positions looked up in the appropriate parameter table for the callable involved.
  11.106 +The positions of parameters in callable signatures are determined by the
  11.107 +definitions of those signatures in the source program. When callables are
  11.108 +invoked using an argument list, arguments that are not specified using
  11.109 +keywords are merely copied into the parameter array in the same position.
  11.110 +However, keyword arguments will need to have their positions looked up in the
  11.111 +appropriate parameter table for the callable involved.
  11.112  
  11.113 -Consequently, no allocation needs to be performed on the parameters themselves: they have specific positions for each callable. However, just as attribute names must yield positions when accessing attributes on objects, a similar mechanism that takes parameter names and yields positions must be provided. It is instead the positions of parameter details that must be allocated in structures to be consulted as if parameter names were attribute names, with attributes providing parameter position details.
  11.114 +Consequently, no allocation needs to be performed on the parameters
  11.115 +themselves: they have specific positions for each callable. However, just as
  11.116 +attribute names must yield positions when accessing attributes on objects, a
  11.117 +similar mechanism that takes parameter names and yields positions must be
  11.118 +provided. It is instead the positions of parameter details that must be
  11.119 +allocated in structures to be consulted as if parameter names were attribute
  11.120 +names, with attributes providing parameter position details.
  11.121  
  11.122  Consider the following functions:
  11.123  
  11.124 @@ -64,28 +120,52 @@
  11.125      ...
  11.126  }}}
  11.127  
  11.128 -In order to find the position of each parameter using its name, the following table could be provided:
  11.129 +In order to find the position of each parameter using its name, the following
  11.130 +table could be provided:
  11.131  
  11.132  || '''Table''' ||<-4> '''Parameters''' ||
  11.133  || function f || `a` at 1 || `b` at 2 || `c` at 3 || `d` at 4 ||
  11.134  || function g || `q` at 2 ||<style="background-color: #ddd"> `p` at 1 || `r` at 3 || ||
  11.135  || function h || `v` at 2 ||<style="background-color: #ddd"> `p` at 2 || || `d` at 1 ||
  11.136  
  11.137 -Such a table behaves like an object structure (or an object table) with parameters acting like attributes in such a structure. Here, the attributes yield the positions of parameters within parameter arrays. Note how `p` always resides in the same location but may yield different positions depending on the actual callable involved (as is also observable with `d`).
  11.138 +Such a table behaves like an object structure (or an object table) with
  11.139 +parameters acting like attributes in such a structure. Here, the attributes
  11.140 +yield the positions of parameters within parameter arrays. Note how `p` always
  11.141 +resides in the same location but may yield different positions depending on
  11.142 +the actual callable involved (as is also observable with `d`).
  11.143  
  11.144  === Allocation Considerations ===
  11.145  
  11.146 -Parameter table allocation involves similar considerations to those influencing object table allocation. In order to keep parameter tables small, more frequently appearing parameters should be positioned earlier in arrays. A specific consideration of importance is that of the number of tables generated. There may be many callables with the same signature, and such callables therefore do not need their own parameter tables since they will merely be duplicates. An attempt is therefore made to identify distinct signatures and to generate parameter tables only for these signatures, instead of providing a one-to-one mapping between callables and tables.
  11.147 +Parameter table allocation involves similar considerations to those
  11.148 +influencing object table allocation. In order to keep parameter tables small,
  11.149 +more frequently appearing parameters should be positioned earlier in arrays. A
  11.150 +specific consideration of importance is that of the number of tables
  11.151 +generated. There may be many callables with the same signature, and such
  11.152 +callables therefore do not need their own parameter tables since they will
  11.153 +merely be duplicates. An attempt is therefore made to identify distinct
  11.154 +signatures and to generate parameter tables only for these signatures, instead
  11.155 +of providing a one-to-one mapping between callables and tables.
  11.156  
  11.157  == Populating Tables ==
  11.158  
  11.159 -With names allocated to positions in each kind of table, the straightforward task of generating such tables proceeds as follows.
  11.160 +With names allocated to positions in each kind of table, the straightforward
  11.161 +task of generating such tables proceeds as follows.
  11.162  
  11.163  === Object Tables and Attribute Codes ===
  11.164  
  11.165 -Object tables consist of locations directly corresponding to structure member locations. Each location in a table will correspond to a specific name, but since potentially many names may have the same position, a table must provide identifying details for the name that is actually supported.
  11.166 +Object tables consist of locations directly corresponding to structure member
  11.167 +locations. Each location in a table will correspond to a specific name, but
  11.168 +since potentially many names may have the same position, a table must provide
  11.169 +identifying details for the name that is actually supported.
  11.170  
  11.171 -In object tables, such identifying details take the form of '''attribute codes'''. Each attribute name is mapped to a distinct identifier, and upon consulting an object table, the lookup process must read the stored attribute code and compare it to the code for the attribute it intends to access. If these codes match, then it can be assumed that the object involved supports the named attribute. Otherwise, a different attribute (or even no attribute at all) resides at that location in the object structure, making the access inappropriate.
  11.172 +In object tables, such identifying details take the form of '''attribute
  11.173 +codes'''. Each attribute name is mapped to a distinct identifier, and upon
  11.174 +consulting an object table, the lookup process must read the stored attribute
  11.175 +code and compare it to the code for the attribute it intends to access. If
  11.176 +these codes match, then it can be assumed that the object involved supports
  11.177 +the named attribute. Otherwise, a different attribute (or even no attribute at
  11.178 +all) resides at that location in the object structure, making the access
  11.179 +inappropriate.
  11.180  
  11.181  A more comprehensive object table resembles the following:
  11.182  
  11.183 @@ -113,11 +193,22 @@
  11.184  
  11.185  === Parameter Tables and Parameter Codes ===
  11.186  
  11.187 -Parameter tables consist of locations yielding parameter position information. Each location in a table will correspond to a particular name, but since potentially many names may have the same position, a table must provide identifying details for the name that is actually supported.
  11.188 +Parameter tables consist of locations yielding parameter position information.
  11.189 +Each location in a table will correspond to a particular name, but since
  11.190 +potentially many names may have the same position, a table must provide
  11.191 +identifying details for the name that is actually supported.
  11.192  
  11.193 -Just as with object tables, in parameter tables, such identifying details take the form of '''parameter codes'''. Each parameter name is mapped to a distinct identifier, and upon consulting a parameter table, the lookup process must read the stored parameter code and compare it to the code for the parameter it is attempting to position in a parameter list. If these codes match, then it can be assumed that the signature supports the named parameter. Otherwise, a different parameter (or even no parameter at all) resides at that location in the parameter table, making any attempt to set such a parameter inappropriate.
  11.194 +Just as with object tables, in parameter tables, such identifying details take
  11.195 +the form of '''parameter codes'''. Each parameter name is mapped to a distinct
  11.196 +identifier, and upon consulting a parameter table, the lookup process must
  11.197 +read the stored parameter code and compare it to the code for the parameter it
  11.198 +is attempting to position in a parameter list. If these codes match, then it
  11.199 +can be assumed that the signature supports the named parameter. Otherwise, a
  11.200 +different parameter (or even no parameter at all) resides at that location in
  11.201 +the parameter table, making any attempt to set such a parameter inappropriate.
  11.202  
  11.203 -Since parameter tables provide both identifying information and parameter positions, a more comprehensive parameter table resembles the following:
  11.204 +Since parameter tables provide both identifying information and parameter
  11.205 +positions, a more comprehensive parameter table resembles the following:
  11.206  
  11.207  || '''Table''' ||<-4> '''Parameters (Codes)''' ||
  11.208  || function f || `a` at 1 (1) || `b` at 2 (2) || `c` at 3 (3) || `d` at 4 (4) ||
  11.209 @@ -138,9 +229,16 @@
  11.210  
  11.211  == Consolidating Constants ==
  11.212  
  11.213 -With knowledge about all constants used in a program, it becomes possible to prepare a catalogue of distinct constants and to assign each of them a unique name for convenient referencing in the generated program code. All constants are inspected in turn and a content digest prepared for each of them, establishing a mapping from values to such digests which act as global identifiers. Since local names are associated with constants, a mapping is also established from each local name to the corresponding global identifier.
  11.214 +With knowledge about all constants used in a program, it becomes possible to
  11.215 +prepare a catalogue of distinct constants and to assign each of them a unique
  11.216 +name for convenient referencing in the generated program code. All constants
  11.217 +are inspected in turn and a content digest prepared for each of them,
  11.218 +establishing a mapping from values to such digests which act as global
  11.219 +identifiers. Since local names are associated with constants, a mapping is
  11.220 +also established from each local name to the corresponding global identifier.
  11.221  
  11.222 -Considering the [[../Inspection#Literals_and_Constants|previous example]], the following mappings would be established, from values to global identifiers:
  11.223 +Considering the [[../Inspection#Literals_and_Constants|previous example]], the
  11.224 +following mappings would be established, from values to global identifiers:
  11.225  
  11.226  || '''Type''' || '''Encoding''' || '''Value''' || '''Global Identifier (Content Digest)''' ||
  11.227  || `__builtins__.str.string` || `iso-8859-15` || `'\xc6\xd8\xc5'` || `OeyoRfPI__XqwJcPgTDTItX9v__OM` ||
  11.228 @@ -154,4 +252,7 @@
  11.229  
  11.230  == Optimisation Products ==
  11.231  
  11.232 -The optimisation process should produce catalogues of attribute and parameter codes plus positioning information for object tables, object structures and parameter tables. It should also produce a catalogue of distinct constant identifiers.
  11.233 +The optimisation process should produce catalogues of attribute and parameter
  11.234 +codes plus positioning information for object tables, object structures and
  11.235 +parameter tables. It should also produce a catalogue of distinct constant
  11.236 +identifiers.
    12.1 --- a/docs/wiki/Prerequisites	Sun Jul 22 12:14:48 2018 +0200
    12.2 +++ b/docs/wiki/Prerequisites	Thu Jan 17 14:23:38 2019 +0100
    12.3 @@ -1,7 +1,17 @@
    12.4  = Prerequisites =
    12.5  
    12.6 -The software currently requires Python to operate. All recent development has been done using Python 2.7 which should be available for most operating system distributions. In future, Lichen will hopefully be compiled using its own toolchain and not require Python. See the Debian `python-minimal` or `python` packages or their equivalents.
    12.7 +The software currently requires Python to operate. All recent development has
    12.8 +been done using Python 2.7 which should be available for most operating system
    12.9 +distributions. In future, Lichen will hopefully be compiled using its own
   12.10 +toolchain and not require Python. See the Debian `python-minimal` or `python`
   12.11 +packages or their equivalents.
   12.12  
   12.13 -To produce executable programs, a C compiler and GNU make are required. The GNU C compiler, GCC, has been used for development and testing, and it should be available in various forms for most operating system distributions. Indeed, cross-compiling form of GCC has been used to test the toolchain output. See the Debian `gcc` and `make` packages or their equivalents.
   12.14 +To produce executable programs, a C compiler and GNU make are required. The GNU
   12.15 +C compiler, GCC, has been used for development and testing, and it should be
   12.16 +available in various forms for most operating system distributions. Indeed,
   12.17 +cross-compiling form of GCC has been used to test the toolchain output. See the
   12.18 +Debian `gcc` and `make` packages or their equivalents.
   12.19  
   12.20 -As Free Software, priority has been given to supporting Lichen on Free Software platforms, with Debian GNU/Linux being the development platform. Proprietary platforms are neither supported nor recommended.
   12.21 +As Free Software, priority has been given to supporting Lichen on Free Software
   12.22 +platforms, with Debian GNU/Linux being the development platform. Proprietary
   12.23 +platforms are neither supported nor recommended.
    13.1 --- a/docs/wiki/Representations	Sun Jul 22 12:14:48 2018 +0200
    13.2 +++ b/docs/wiki/Representations	Thu Jan 17 14:23:38 2019 +0100
    13.3 @@ -1,12 +1,16 @@
    13.4  = Representing Program Objects =
    13.5  
    13.6 -Certain representations have been chosen for program objects that attempt to support efficient access to those objects during the execution of a program.
    13.7 +Certain representations have been chosen for program objects that attempt to
    13.8 +support efficient access to those objects during the execution of a program.
    13.9  
   13.10  <<TableOfContents(2,2)>>
   13.11  
   13.12  == Attributes ==
   13.13  
   13.14 -The principal means of referring to an object in a program is by using an '''attribute''', having this name because it is the representation of an attribute in classes, instances and modules. Each attribute can hold a reference to an object, known as the '''value''', or other kinds of data:
   13.15 +The principal means of referring to an object in a program is by using an
   13.16 +'''attribute''', having this name because it is the representation of an
   13.17 +attribute in classes, instances and modules. Each attribute can hold a
   13.18 +reference to an object, known as the '''value''', or other kinds of data:
   13.19  
   13.20  {{{#!graphviz
   13.21  //format=svg
   13.22 @@ -25,19 +29,49 @@
   13.23  }
   13.24  }}}
   13.25  
   13.26 -Although a value indicates a specific object of interest for an attribute, if the object is callable then additional '''context''' information may be required to call the object. Such context information is not stored in an attribute record but is instead obtained from the object itself, if appropriate.
   13.27 +Although a value indicates a specific object of interest for an attribute, if
   13.28 +the object is callable then additional '''context''' information may be
   13.29 +required to call the object. Such context information is not stored in an
   13.30 +attribute record but is instead obtained from the object itself, if
   13.31 +appropriate.
   13.32  
   13.33  === Integer Representation ===
   13.34  
   13.35 -The `intvalue` member of the attribute structure is employed instead of the `value` member to store integer values. Since `value` normally holds a pointer, and since pointers are often aligned to certain address boundaries on many modern platforms (usually four-byte boundaries on 32-bit platforms, eight-byte boundaries on 64-bit platforms, two-byte boundaries on platforms with 16-bit addressing), the lowest significant bit (bit 0) will typically be zero for a valid pointer. Consequently, by setting bit 0 to 1, other data can be stored in the remaining bits and be distinguished from pointer information. Obviously, operations on attributes first need to test whether the `value` member or the `intvalue` member is in use by testing bit 0.
   13.36 +The `intvalue` member of the attribute structure is employed instead of the
   13.37 +`value` member to store integer values. Since `value` normally holds a
   13.38 +pointer, and since pointers are often aligned to certain address boundaries on
   13.39 +many modern platforms (usually four-byte boundaries on 32-bit platforms,
   13.40 +eight-byte boundaries on 64-bit platforms, two-byte boundaries on platforms
   13.41 +with 16-bit addressing), the lowest significant bit (bit 0) will typically be
   13.42 +zero for a valid pointer. Consequently, by setting bit 0 to 1, other data can
   13.43 +be stored in the remaining bits and be distinguished from pointer information.
   13.44 +Obviously, operations on attributes first need to test whether the `value`
   13.45 +member or the `intvalue` member is in use by testing bit 0.
   13.46  
   13.47 -Thus, integers are transformed and stored directly within attributes, and they are therefore passed around by value. When an attribute of an integer needs to be accessed, the operation usually providing the `value` member, thus obtaining an instance under normal circumstances, instead provides the address of a common integer instance. This instance may then provide instance attributes, whose values will be the same for all integers, or class attributes through a reference to the integer (`int`) class.
   13.48 +Thus, integers are transformed and stored directly within attributes, and they
   13.49 +are therefore passed around by value. When an attribute of an integer needs to
   13.50 +be accessed, the operation usually providing the `value` member, thus
   13.51 +obtaining an instance under normal circumstances, instead provides the address
   13.52 +of a common integer instance. This instance may then provide instance
   13.53 +attributes, whose values will be the same for all integers, or class
   13.54 +attributes through a reference to the integer (`int`) class.
   13.55  
   13.56 -Each method provided by `int`, when called, will be given the original attribute for which the method was obtained as its context. In such methods, operations on the context via `self` will either involve the propagation of the attribute to other functions or methods or attribute accesses on `self`, yielding common instance attributes or class attributes as already described. Only native functions will attempt to interpret the attribute in a different way, decoding the representation, performing arithmetic or logical operations, and encoding the result in a new attribute.
   13.57 +Each method provided by `int`, when called, will be given the original
   13.58 +attribute for which the method was obtained as its context. In such methods,
   13.59 +operations on the context via `self` will either involve the propagation of
   13.60 +the attribute to other functions or methods or attribute accesses on `self`,
   13.61 +yielding common instance attributes or class attributes as already described.
   13.62 +Only native functions will attempt to interpret the attribute in a different
   13.63 +way, decoding the representation, performing arithmetic or logical operations,
   13.64 +and encoding the result in a new attribute.
   13.65  
   13.66  == Contexts and Wrappers ==
   13.67  
   13.68 -The context of an object is of significance if that object is callable. For example, an instance may permit access to a method via an attribute. Since the method will be callable, and since the method is accessed via the instance, the context of that method will be the instance. In order to retain both context and value information, a '''wrapper''' may be created.
   13.69 +The context of an object is of significance if that object is callable. For
   13.70 +example, an instance may permit access to a method via an attribute. Since the
   13.71 +method will be callable, and since the method is accessed via the instance,
   13.72 +the context of that method will be the instance. In order to retain both
   13.73 +context and value information, a '''wrapper''' may be created.
   13.74  
   13.75  {{{#!graphviz
   13.76  //format=svg
   13.77 @@ -57,11 +91,14 @@
   13.78  }
   13.79  }}}
   13.80  
   13.81 -The context is not always the accessor of the object - in this case, the instance - because the object may already be a wrapper with its own context.
   13.82 +The context is not always the accessor of the object - in this case, the
   13.83 +instance - because the object may already be a wrapper with its own context.
   13.84  
   13.85  == Objects ==
   13.86  
   13.87 -Classes, instances and modules are objects, and all of these kinds of objects provide metadata describing the type of each object, together with a collection of attributes forming the contents of such objects.
   13.88 +Classes, instances and modules are objects, and all of these kinds of objects
   13.89 +provide metadata describing the type of each object, together with a
   13.90 +collection of attributes forming the contents of such objects.
   13.91  
   13.92  {{{#!graphviz
   13.93  //format=svg
   13.94 @@ -84,31 +121,52 @@
   13.95  }
   13.96  }}}
   13.97  
   13.98 -Classes are represented by structures whose members reference class attributes and class metadata (the class table), as well as incorporating invocation metadata (the `__args__` and `__fn__` special attributes).
   13.99 +Classes are represented by structures whose members reference class attributes
  13.100 +and class metadata (the class table), as well as incorporating invocation
  13.101 +metadata (the `__args__` and `__fn__` special attributes).
  13.102  
  13.103 -Instances are represented by structures whose members reference instance attributes (including `__class__` which indicates the class instantiated by a given instance) and instance metadata (the instance table), as well as incorporating invocation metadata (the `__args__` and `__fn__` special attributes).
  13.104 +Instances are represented by structures whose members reference instance
  13.105 +attributes (including `__class__` which indicates the class instantiated by a
  13.106 +given instance) and instance metadata (the instance table), as well as
  13.107 +incorporating invocation metadata (the `__args__` and `__fn__` special
  13.108 +attributes).
  13.109  
  13.110 -Functions are instances of a general function type that does not permit additional, general instance attributes. However, function instance structures may have extra members corresponding to default parameter values. Access to such extra members is performed using the minimum and maximum values provided via `__args__` and with knowledge of the number of declared instance attributes indicated by the instance table for the function type.
  13.111 +Functions are instances of a general function type that does not permit
  13.112 +additional, general instance attributes. However, function instance structures
  13.113 +may have extra members corresponding to default parameter values. Access to
  13.114 +such extra members is performed using the minimum and maximum values provided
  13.115 +via `__args__` and with knowledge of the number of declared instance
  13.116 +attributes indicated by the instance table for the function type.
  13.117  
  13.118 -Modules are represented by structures whose members reference module attributes and module metadata (the module table).
  13.119 +Modules are represented by structures whose members reference module
  13.120 +attributes and module metadata (the module table).
  13.121  
  13.122  == Special Members ==
  13.123  
  13.124 -All object representations support the following special members describing the invocation properties of each object:
  13.125 +All object representations support the following special members describing
  13.126 +the invocation properties of each object:
  13.127  
  13.128  {{{#!table
  13.129 -`__args__` || the minimum number of arguments supported in an invocation and a reference to the parameter table for the object
  13.130 +`__args__` || the minimum number of arguments supported in an invocation and a
  13.131 +           .. reference to the parameter table for the object
  13.132  ==
  13.133 -`__fn__` || a reference to a native function containing the actual code run when calling the object
  13.134 +`__fn__` || a reference to a native function containing the actual code run
  13.135 +         .. when calling the object
  13.136  }}}
  13.137  
  13.138 -Classes are invoked in order to create instances of each class ('''instantiation'''). Instances may support invocation by providing a `__call__` method. Functions are supported by instances of a general function class. Modules are generally not callable and will not actually provide these special members in practice.
  13.139 +Classes are invoked in order to create instances of each class
  13.140 +('''instantiation'''). Instances may support invocation by providing a
  13.141 +`__call__` method. Functions are supported by instances of a general function
  13.142 +class. Modules are generally not callable and will not actually provide these
  13.143 +special members in practice.
  13.144  
  13.145  All object representations support information about their type:
  13.146  
  13.147  {{{#!table
  13.148  `__class__`
  13.149 -|| the class of the object: instances refer to their classes, classes refer to the `type` class, functions are instances that refer to the `function` class, modules refer to the `module` class
  13.150 +|| the class of the object: instances refer to their classes, classes refer to
  13.151 +.. the `type` class, functions are instances that refer to the `function`
  13.152 +.. class, modules refer to the `module` class
  13.153  }}}
  13.154  
  13.155  Certain kinds of object support other descriptive attributes:
  13.156 @@ -119,31 +177,48 @@
  13.157  `__parent__` || the parent scope of a class or a function
  13.158  }}}
  13.159  
  13.160 -Objects supported by native, system-level functionality require a means of retaining information in special attributes:
  13.161 +Objects supported by native, system-level functionality require a means of
  13.162 +retaining information in special attributes:
  13.163  
  13.164  {{{#!table
  13.165  `__data__` || private data manipulated at the native level
  13.166  }}}
  13.167  
  13.168 -Strings support special annotation attributes that permit their use in dynamically resolving attributes:
  13.169 +Strings support special annotation attributes that permit their use in
  13.170 +dynamically resolving attributes:
  13.171  
  13.172  {{{#!table
  13.173 -`__key__` || the code and position of the attribute whose name is represented by the string
  13.174 +`__key__` || the code and position of the attribute whose name is represented
  13.175 +          .. by the string
  13.176  }}}
  13.177  
  13.178 -Such "key" attributes provide information that can be used to inspect an object table and to test for the presence of an attribute. With such information, the `getattr` and `hasattr` functions can be supported.
  13.179 +Such "key" attributes provide information that can be used to inspect an
  13.180 +object table and to test for the presence of an attribute. With such
  13.181 +information, the `getattr` and `hasattr` functions can be supported.
  13.182  
  13.183  == Attribute Tables ==
  13.184  
  13.185 -In order to provide information about the attributes supported by each object, the structure of each object will reference a table containing entries describing these supported attributes. The size of this table is declared within the table structure, and for each position in the table an entry corresponding to the same position within an actual object structure describes the nature of the attribute at that position.
  13.186 +In order to provide information about the attributes supported by each object,
  13.187 +the structure of each object will reference a table containing entries
  13.188 +describing these supported attributes. The size of this table is declared
  13.189 +within the table structure, and for each position in the table an entry
  13.190 +corresponding to the same position within an actual object structure describes
  13.191 +the nature of the attribute at that position.
  13.192  
  13.193  == Parameter Tables ==
  13.194  
  13.195 -In order to support argument validation and keyword arguments in invocations, a structure is referenced by `__args__` that indicates...
  13.196 +In order to support argument validation and keyword arguments in invocations,
  13.197 +a structure is referenced by `__args__` that indicates...
  13.198  
  13.199   * The minimum number of parameters supported by a callable
  13.200   * The maximum number of parameters supported by a callable
  13.201   * The size of the table describing the parameters
  13.202 - * A table of entries with each entry indicating the nature of a parameter (in effect, which name it uses, albeit as a generated code instead of a string) and the position of the parameter in any argument list prepared for an invocation
  13.203 + * A table of entries with each entry indicating the nature of a parameter (in
  13.204 +   effect, which name it uses, albeit as a generated code instead of a string)
  13.205 +   and the position of the parameter in any argument list prepared for an
  13.206 +   invocation
  13.207  
  13.208 -Parameter tables only need to be consulted at run-time if the nature of a callable is undetermined. By supporting a uniform interface, the arguments used in an invocation can be tested against the description provided by `__args__` and the table.
  13.209 +Parameter tables only need to be consulted at run-time if the nature of a
  13.210 +callable is undetermined. By supporting a uniform interface, the arguments
  13.211 +used in an invocation can be tested against the description provided by
  13.212 +`__args__` and the table.
    14.1 --- a/docs/wiki/Restarted	Sun Jul 22 12:14:48 2018 +0200
    14.2 +++ b/docs/wiki/Restarted	Thu Jan 17 14:23:38 2019 +0100
    14.3 @@ -1,8 +1,16 @@
    14.4  = Lichen Restarted =
    14.5  
    14.6 -Originally, lots of work was being put in to support various Python features that are arguably superfluous. The realisation was had that a lot of effort was being made for little practical benefit by trying to support things that are, [[../Design|in the larger picture]], not that important. Consequently, Lichen was refocused on a smaller set of more useful Python features.
    14.7 +Originally, lots of work was being put in to support various Python features
    14.8 +that are arguably superfluous. The realisation was had that a lot of effort
    14.9 +was being made for little practical benefit by trying to support things that
   14.10 +are, [[../Design|in the larger picture]], not that important. Consequently,
   14.11 +Lichen was refocused on a smaller set of more useful Python features.
   14.12  
   14.13 -This document is of historical interest only, with the [[../Design|design]] and other documents attempting to communicate the results of this restarting effort. Some obsolete information is therefore preserved below. For example, attributes hold context information in the diagrams, but context information is now held in wrappers or is maintained separately within programs.
   14.14 +This document is of historical interest only, with the [[../Design|design]]
   14.15 +and other documents attempting to communicate the results of this restarting
   14.16 +effort. Some obsolete information is therefore preserved below. For example,
   14.17 +attributes hold context information in the diagrams, but context information
   14.18 +is now held in wrappers or is maintained separately within programs.
   14.19  
   14.20  Objectives:
   14.21  
   14.22 @@ -10,7 +18,8 @@
   14.23   * No importing triggered during module inspection
   14.24   * All unresolved external references are set to `<depends>`
   14.25   * Hierarchical module namespaces are not exposed in programs
   14.26 - * Modules are independent: package hierarchies are not traversed when importing
   14.27 + * Modules are independent: package hierarchies are not traversed when
   14.28 +   importing
   14.29   * Nested scopes will be dropped
   14.30   * If expressions and comprehensions will be dropped
   14.31   * `self` is a reserved name and is optional in method parameter lists
   14.32 @@ -19,40 +28,51 @@
   14.33  
   14.34  == Names ==
   14.35  
   14.36 -Names are locals, globals or built-ins. (Special names exist internally to support certain operations.)
   14.37 +Names are locals, globals or built-ins. (Special names exist internally to
   14.38 +support certain operations.)
   14.39  
   14.40 -Locals inside functions are dynamic; locals outside functions are static, as are module globals. Built-ins are defined statically in the `__builtins__` package.
   14.41 +Locals inside functions are dynamic; locals outside functions are static, as
   14.42 +are module globals. Built-ins are defined statically in the `__builtins__`
   14.43 +package.
   14.44  
   14.45  == Imports ==
   14.46  
   14.47 -Imports provide access to external references. The "leaf" module in a module path is the module returned by the statement.
   14.48 +Imports provide access to external references. The "leaf" module in a module
   14.49 +path is the module returned by the statement.
   14.50  
   14.51  Indicate that module "compiler" is accessed via compiler...
   14.52 +
   14.53  {{{#!python numbers=disable
   14.54  import compiler
   14.55  }}}
   14.56  
   14.57  Indicate that module "compiler" is accessed via comp...
   14.58 +
   14.59  {{{#!python numbers=disable
   14.60  import compiler as comp
   14.61  }}}
   14.62  
   14.63 -Indicate that module "compiler.ast" is accessed via ast; module "compiler.transformer" is accessed via tr...
   14.64 +Indicate that module "compiler.ast" is accessed via ast; module
   14.65 +"compiler.transformer" is accessed via tr...
   14.66 +
   14.67  {{{#!python numbers=disable
   14.68  import compiler.ast as ast, compiler.transformer as tr
   14.69  }}}
   14.70  
   14.71  Import compiler.ast, access Function...
   14.72 +
   14.73  {{{#!python numbers=disable
   14.74  from compiler.ast import Function
   14.75  }}}
   14.76  
   14.77  Import compiler.ast, access Function as F...
   14.78 +
   14.79  {{{#!python numbers=disable
   14.80  from compiler.ast import Function as F
   14.81  }}}
   14.82  
   14.83 -This causes some semantic differences with Python, with the most significant one being the following:
   14.84 +This causes some semantic differences with Python, with the most significant
   14.85 +one being the following:
   14.86  
   14.87  {{{{#!table
   14.88  '''Python''' || '''Lichen'''
   14.89 @@ -108,10 +128,12 @@
   14.90  
   14.91  Some notes:
   14.92  
   14.93 - * Names not defined in a module and not declared in import statements are unresolved
   14.94 + * Names not defined in a module and not declared in import statements are
   14.95 +   unresolved
   14.96   * Modules are identified during inspection but are not loaded
   14.97   * Instead, modules are added to a list and are imported later
   14.98 - * Names imported from modules are set to `<depends>` (since the contents of modules will not generally be known)
   14.99 + * Names imported from modules are set to `<depends>` (since the contents of
  14.100 +   modules will not generally be known)
  14.101   * Names are resolved in a later activity
  14.102  
  14.103  == Self ==
  14.104 @@ -119,22 +141,37 @@
  14.105  In Python:
  14.106  
  14.107   * The `self` name provides access to the instance associated with a method
  14.108 - * The instance is supplied by a "context", initialised when a method is obtained from an instance (or through other attribute accesses)
  14.109 - * Upon invocation, any instance context must be assigned to the `self` parameter, provided the callable is a method
  14.110 - * Meanwhile, any non-instance context is not assigned to the `self` parameter, which should be provided explicitly for class-accessed methods
  14.111 - * Plain functions never expose `self` or have `self` initialised, even if they have been assigned to an instance
  14.112 + * The instance is supplied by a "context", initialised when a method is
  14.113 +   obtained from an instance (or through other attribute accesses)
  14.114 + * Upon invocation, any instance context must be assigned to the `self`
  14.115 +   parameter, provided the callable is a method
  14.116 + * Meanwhile, any non-instance context is not assigned to the `self`
  14.117 +   parameter, which should be provided explicitly for class-accessed methods
  14.118 + * Plain functions never expose `self` or have `self` initialised, even if
  14.119 +   they have been assigned to an instance
  14.120  
  14.121 -Apart from tests for the nature of the context and the callable, the argument list is effectively variable.
  14.122 +Apart from tests for the nature of the context and the callable, the argument
  14.123 +list is effectively variable.
  14.124  
  14.125  With `self` as a ubiquitous, hidden parameter:
  14.126  
  14.127 - * The `self` name still provides access to the instance associated with a method
  14.128 - * The instance is still supplied by a "context", initialised when a method is obtained from an instance (or through other attribute accesses)
  14.129 - * Upon invocation, `self` is included in the argument list regardless of the nature of the callable
  14.130 - * Class-accessed methods would have their class as context, following from the above, but `self` may not refer to a class: it must be an instance
  14.131 - * To combine class-accessed methods with instance contexts, a special function is employed
  14.132 + * The `self` name still provides access to the instance associated with a
  14.133 +   method
  14.134 + * The instance is still supplied by a "context", initialised when a method is
  14.135 +   obtained from an instance (or through other attribute accesses)
  14.136 + * Upon invocation, `self` is included in the argument list regardless of the
  14.137 +   nature of the callable
  14.138 + * Class-accessed methods would have their class as context, following from
  14.139 +   the above, but `self` may not refer to a class: it must be an instance
  14.140 + * To combine class-accessed methods with instance contexts, a special
  14.141 +   function is employed
  14.142  
  14.143 -The argument list for each callable thus remains static, at a cost of allocating an extra argument that may not be used. (Various calling conventions for certain processor architectures employ potentially unused registers, anyway.) Note that a callable may support defaults, however, and thus any argument list may need extending to include default values for parameters without corresponding arguments.
  14.144 +The argument list for each callable thus remains static, at a cost of
  14.145 +allocating an extra argument that may not be used. (Various calling
  14.146 +conventions for certain processor architectures employ potentially unused
  14.147 +registers, anyway.) Note that a callable may support defaults, however, and
  14.148 +thus any argument list may need extending to include default values for
  14.149 +parameters without corresponding arguments.
  14.150  
  14.151  {{{{#!table
  14.152  '''Python''' || '''Without self'''
  14.153 @@ -185,7 +222,13 @@
  14.154  
  14.155  }}}}
  14.156  
  14.157 -To avoid usage of `self` in undefined ways, only methods are able to use `self` and are not allowed to redefine it. Consequently, when invoking a callable, the context is set (where the callable is unknown until run-time; it is not set if compile-time knowledge indicates that it is not needed), and in situations where `self` is not permitted, the context is therefore safely ignored. Meanwhile, methods are always supplied with a context compatible with `self`.
  14.158 +To avoid usage of `self` in undefined ways, only methods are able to use
  14.159 +`self` and are not allowed to redefine it. Consequently, when invoking a
  14.160 +callable, the context is set (where the callable is unknown until run-time; it
  14.161 +is not set if compile-time knowledge indicates that it is not needed), and in
  14.162 +situations where `self` is not permitted, the context is therefore safely
  14.163 +ignored. Meanwhile, methods are always supplied with a context compatible with
  14.164 +`self`.
  14.165  
  14.166  || '''Callable''' || '''self''' || '''Remarks''' ||
  14.167  || Class || context || context discarded and replaced by allocated instance ||
  14.168 @@ -196,13 +239,24 @@
  14.169  || Method (via class) || class as context || method not called (see "unbound methods") ||
  14.170  || Method (via instance) || instance as context || `self` set to instance ||
  14.171  
  14.172 -Note that the treatment of functions stored on classes differs from Python. In Python, such functions would become unbound methods (see below) and would employ their first parameter as an effective `self` parameter (regardless of name).
  14.173 +Note that the treatment of functions stored on classes differs from Python. In
  14.174 +Python, such functions would become unbound methods (see below) and would
  14.175 +employ their first parameter as an effective `self` parameter (regardless of
  14.176 +name).
  14.177  
  14.178  == Unbound Methods ==
  14.179  
  14.180 -Since methods acquired directly from classes ("unbound methods" in Python) are meant to be combined with an instance as context (using the `get_using` function), they must be uncallable until combined with the appropriate context, yet the same methods when acquired via instances ("bound methods" in Python) must be immediately callable.
  14.181 +Since methods acquired directly from classes ("unbound methods" in Python) are
  14.182 +meant to be combined with an instance as context (using the `get_using`
  14.183 +function), they must be uncallable until combined with the appropriate
  14.184 +context, yet the same methods when acquired via instances ("bound methods" in
  14.185 +Python) must be immediately callable.
  14.186  
  14.187 -To support the two different states of methods, the principal structure of a class has attributes referencing uncallable versions of its methods. Meanwhile, such uncallable methods reference callable versions and when instances are employed to access the class attributes, it is these callable versions that are retrieved. For example:
  14.188 +To support the two different states of methods, the principal structure of a
  14.189 +class has attributes referencing uncallable versions of its methods.
  14.190 +Meanwhile, such uncallable methods reference callable versions and when
  14.191 +instances are employed to access the class attributes, it is these callable
  14.192 +versions that are retrieved. For example:
  14.193  
  14.194  {{{#!graphviz
  14.195  //format=svg
  14.196 @@ -253,7 +307,12 @@
  14.197  }
  14.198  }}}
  14.199  
  14.200 -Callable methods provide a reference to a callable routine in its special callable member, just as functions and classes do. Uncallable methods populate the callable member with a reference to an error routine. Thus, any attempt to call an uncallable method would cause the error routine to be invoked. In addition, uncallable methods reference the corresponding callable methods so that the callable methods can be found and referenced.
  14.201 +Callable methods provide a reference to a callable routine in its special
  14.202 +callable member, just as functions and classes do. Uncallable methods populate
  14.203 +the callable member with a reference to an error routine. Thus, any attempt to
  14.204 +call an uncallable method would cause the error routine to be invoked. In
  14.205 +addition, uncallable methods reference the corresponding callable methods so
  14.206 +that the callable methods can be found and referenced.
  14.207  
  14.208  || '''Accessor''' || '''Provider''' || '''Attribute''' || '''Context''' || '''Summary''' ||
  14.209  ||<|4> Instance ||<|4> Instance || Function || ''not used'' ||<|6> Preserve context ||
  14.210 @@ -269,14 +328,30 @@
  14.211  || Unbound method || Providing class ||
  14.212  || Other || Same as value ||
  14.213  
  14.214 -When obtaining an unbound method from an instance attribute, the context of the method attribute is provided. Indeed, the context is always preserved when accessing instance attributes.
  14.215 +When obtaining an unbound method from an instance attribute, the context of
  14.216 +the method attribute is provided. Indeed, the context is always preserved when
  14.217 +accessing instance attributes.
  14.218  
  14.219 -When obtaining an unbound method from a class attribute via an instance, the context of the method attribute is tested against the accessing instance. If compatible, an attribute is copied containing the instance as context and a callable method reference as value.
  14.220 +When obtaining an unbound method from a class attribute via an instance, the
  14.221 +context of the method attribute is tested against the accessing instance. If
  14.222 +compatible, an attribute is copied containing the instance as context and a
  14.223 +callable method reference as value.
  14.224  
  14.225 -When obtaining an unbound method from a class attribute, the context of the method attribute is provided. Indeed, the context is always preserved when accessing class attributes directly.
  14.226 +When obtaining an unbound method from a class attribute, the context of the
  14.227 +method attribute is provided. Indeed, the context is always preserved when
  14.228 +accessing class attributes directly.
  14.229  
  14.230 -When combining an unbound method obtained from a class with an instance using `get_using`, the context of the method attribute is tested against the supplied instance. If compatible, an attribute is copied containing the instance as context and a callable method reference as value.
  14.231 +When combining an unbound method obtained from a class with an instance using
  14.232 +`get_using`, the context of the method attribute is tested against the
  14.233 +supplied instance. If compatible, an attribute is copied containing the
  14.234 +instance as context and a callable method reference as value.
  14.235  
  14.236  === Functions as Unbound Methods ===
  14.237  
  14.238 -Functions not defined within classes could be treated as unbound methods if they were to employ `self` (thus indicating that they are intended as methods). Such functions would then be recorded as uncallable in the module namespace, needing to be explicitly bound to a class using a special function. However, there appears to be limited utility in defining functions in this way, instead of defining them directly as methods, or instead of merely using such generic functions from existing methods.
  14.239 +Functions not defined within classes could be treated as unbound methods if
  14.240 +they were to employ `self` (thus indicating that they are intended as
  14.241 +methods). Such functions would then be recorded as uncallable in the module
  14.242 +namespace, needing to be explicitly bound to a class using a special function.
  14.243 +However, there appears to be limited utility in defining functions in this
  14.244 +way, instead of defining them directly as methods, or instead of merely using
  14.245 +such generic functions from existing methods.
    15.1 --- a/docs/wiki/Structure	Sun Jul 22 12:14:48 2018 +0200
    15.2 +++ b/docs/wiki/Structure	Thu Jan 17 14:23:38 2019 +0100
    15.3 @@ -1,12 +1,28 @@
    15.4  = Program Structure =
    15.5  
    15.6 -A program consists of a number of '''modules''' with each module providing its own namespace. Modules can be organised within '''packages''' which define a hierarchical relationship between them. However, the relationship between modules is not automatically exposed within the program: it is more appropriate to consider modules as independent entities that have a particular naming scheme.
    15.7 +A program consists of a number of '''modules''' with each module providing its
    15.8 +own namespace. Modules can be organised within '''packages''' which define a
    15.9 +hierarchical relationship between them. However, the relationship between
   15.10 +modules is not automatically exposed within the program: it is more
   15.11 +appropriate to consider modules as independent entities that have a particular
   15.12 +naming scheme.
   15.13  
   15.14 -Within each module a hierarchy of namespaces is provided by '''classes''', with each class potentially containing other classes, and so on. Also appearing within modules and classes are '''functions''', with those appearing within classes being regarded as '''methods'''.
   15.15 +Within each module a hierarchy of namespaces is provided by '''classes''',
   15.16 +with each class potentially containing other classes, and so on. Also
   15.17 +appearing within modules and classes are '''functions''', with those appearing
   15.18 +within classes being regarded as '''methods'''.
   15.19  
   15.20 -Function namespaces are considered separately from module and class namespaces and are not considered part of the general namespace hierarchy, instead merely appearing as objects at the edges of that hierarchy. Functions may also be defined within functions, and such inner functions will be referenced within their parent functions, but no hierarchical relationship will exist between their namespaces.
   15.21 +Function namespaces are considered separately from module and class namespaces
   15.22 +and are not considered part of the general namespace hierarchy, instead merely
   15.23 +appearing as objects at the edges of that hierarchy. Functions may also be
   15.24 +defined within functions, and such inner functions will be referenced within
   15.25 +their parent functions, but no hierarchical relationship will exist between
   15.26 +their namespaces.
   15.27  
   15.28 -Thus, a program provides a static namespace hierarchy consisting of modules containing classes (containing other classes, and so on) plus functions. Objects residing within namespaces are accessed via '''attributes''' of those namespaces.
   15.29 +Thus, a program provides a static namespace hierarchy consisting of modules
   15.30 +containing classes (containing other classes, and so on) plus functions.
   15.31 +Objects residing within namespaces are accessed via '''attributes''' of those
   15.32 +namespaces.
   15.33  
   15.34  {{{{#!table
   15.35  {{{#!graphviz
   15.36 @@ -88,7 +104,9 @@
   15.37  
   15.38  == Referencing the Structure ==
   15.39  
   15.40 -Each part of the structure is catalogued using an '''object path''', indicating its location in the structure hierarchy, and a reference indicating its nature and origin. For example:
   15.41 +Each part of the structure is catalogued using an '''object path''',
   15.42 +indicating its location in the structure hierarchy, and a reference indicating
   15.43 +its nature and origin. For example:
   15.44  
   15.45  || '''Object Path''' || '''Reference''' || '''Explanation''' ||
   15.46  || `M.A` || `<class>:M.A` || The definition of class `A` in module `M` ||
   15.47 @@ -99,17 +117,36 @@
   15.48  || `__main__.M` || `<module>:M` || A reference to module `M` ||
   15.49  || `__main__.counter` || `<var>` || An undetermined object called `counter` in the `__main__` module ||
   15.50  
   15.51 -The reference therefore expresses the '''kind''' of object (class, function, instance, module or undetermined variable), possibly accompanied by an object type, and also possibly accompanied by an alias indicating where the reference was obtained.
   15.52 +The reference therefore expresses the '''kind''' of object (class, function,
   15.53 +instance, module or undetermined variable), possibly accompanied by an object
   15.54 +type, and also possibly accompanied by an alias indicating where the reference
   15.55 +was obtained.
   15.56  
   15.57  == Classes ==
   15.58  
   15.59 -Classes are regarded as statically-defined objects, meaning that they are only evaluated once and must not be defined within conditional sections or within functions. Base classes must be expressed using readily-identifiable class names, since any potential ambiguity or uncertainty with the identity of base classes could result in a class inheritance hierarchy that is effectively dynamic.
   15.60 +Classes are regarded as statically-defined objects, meaning that they are only
   15.61 +evaluated once and must not be defined within conditional sections or within
   15.62 +functions. Base classes must be expressed using readily-identifiable class
   15.63 +names, since any potential ambiguity or uncertainty with the identity of base
   15.64 +classes could result in a class inheritance hierarchy that is effectively
   15.65 +dynamic.
   15.66  
   15.67 -When '''instantiated''', classes provide '''instances''' that provide the attributes of each class's namespace plus any '''inherited''' attributes from base classes that would not be provided by the class itself. In addition, instances provide their own instance attributes.
   15.68 +When '''instantiated''', classes provide '''instances''' that provide the
   15.69 +attributes of each class's namespace plus any '''inherited''' attributes from
   15.70 +base classes that would not be provided by the class itself. In addition,
   15.71 +instances provide their own instance attributes.
   15.72  
   15.73  == Functions ==
   15.74  
   15.75 -Functions are regarded as statically-defined objects, but they may be defined either with names within other named functions or as '''lambdas''' (functions without names) within named functions or other lambdas. Thus, unlike classes, functions may be defined once but replicated many times, each time with different accompanying state information. Such state information needs to be provided via default parameters: it is not detected and propagated automatically. Closures are not supported: any state from enclosing scopes must be supplied at the point of definition of a function; it is not acquired from outer functions by name.
   15.76 +Functions are regarded as statically-defined objects, but they may be defined
   15.77 +either with names within other named functions or as '''lambdas''' (functions
   15.78 +without names) within named functions or other lambdas. Thus, unlike classes,
   15.79 +functions may be defined once but replicated many times, each time with
   15.80 +different accompanying state information. Such state information needs to be
   15.81 +provided via default parameters: it is not detected and propagated
   15.82 +automatically. Closures are not supported: any state from enclosing scopes
   15.83 +must be supplied at the point of definition of a function; it is not acquired
   15.84 +from outer functions by name.
   15.85  
   15.86  {{{#!python numbers=disable
   15.87  def outer(a):
   15.88 @@ -125,11 +162,17 @@
   15.89  
   15.90  === Lambdas ===
   15.91  
   15.92 -Lambdas are given special names for the purposes of identification within the program structure, being named relative to the scope in which they are defined. For example, the first lambda appearing within module `N` would have an object path of `N.$l0`, and a subsequent lambda within the same scope would have an object path of `N.$l1`. Lambdas may appear within classes and functions, including lambdas themselves. For example:
   15.93 +Lambdas are given special names for the purposes of identification within the
   15.94 +program structure, being named relative to the scope in which they are
   15.95 +defined. For example, the first lambda appearing within module `N` would have
   15.96 +an object path of `N.$l0`, and a subsequent lambda within the same scope would
   15.97 +have an object path of `N.$l1`. Lambdas may appear within classes and
   15.98 +functions, including lambdas themselves. For example:
   15.99  
  15.100  {{{#!python numbers=disable
  15.101  def f(x):
  15.102      return lambda y, x=x: lambda z, x=x, y=y: (x, y, z)
  15.103  }}}
  15.104  
  15.105 -Here, within a module `M`, the outer lambda would have the object path `M.f.$l0` whereas the inner lambda would have the object path `M.f.$l0.$l0`.
  15.106 +Here, within a module `M`, the outer lambda would have the object path
  15.107 +`M.f.$l0` whereas the inner lambda would have the object path `M.f.$l0.$l0`.
    16.1 --- a/docs/wiki/ToDo	Sun Jul 22 12:14:48 2018 +0200
    16.2 +++ b/docs/wiki/ToDo	Thu Jan 17 14:23:38 2019 +0100
    16.3 @@ -1,57 +1,102 @@
    16.4  = To Do =
    16.5  
    16.6 -As always with software, there are still many things that need to be done. Here is a list of a few of them.
    16.7 +As always with software, there are still many things that need to be done.
    16.8 +Here is a list of a few of them.
    16.9  
   16.10  <<TableOfContents(2,3)>>
   16.11  
   16.12  == Finish the Core Standard Library ==
   16.13  
   16.14 -Lichen provides its own core standard library featuring the [[../Builtins|built-ins]] and other essential modules, with only a small amount of native code supporting implementations written in the Lichen language.
   16.15 +Lichen provides its own core standard library featuring the
   16.16 +[[../Builtins|built-ins]] and other essential modules, with only a small
   16.17 +amount of native code supporting implementations written in the Lichen
   16.18 +language.
   16.19  
   16.20  === Numeric Types ===
   16.21  
   16.22 -Support all the numeric types. Currently, only `int` is supported, but `float` merely requires some native code handling operations and testing for exception conditions. A representation for `long` needs to be determined, and `complex` probably just needs some methods implementing.
   16.23 +Support all the numeric types. Currently, only `int` is supported, but `float`
   16.24 +merely requires some native code handling operations and testing for exception
   16.25 +conditions. A representation for `long` needs to be determined, and `complex`
   16.26 +probably just needs some methods implementing.
   16.27  
   16.28 -Support promotion between some of the numeric types. Currently, `int` values that overflow raise `OverflowError`, as was done in Python before automatic promotion to `long`.
   16.29 +Support promotion between some of the numeric types. Currently, `int` values
   16.30 +that overflow raise `OverflowError`, as was done in Python before automatic
   16.31 +promotion to `long`.
   16.32  
   16.33  === String Types ===
   16.34  
   16.35 -The remaining methods need defining for byte and Unicode strings. Some methods are more complicated than others, particularly where some interpretation of the content is required: identification of case, whitespace, punctuation, and so on.
   16.36 +The remaining methods need defining for byte and Unicode strings. Some methods
   16.37 +are more complicated than others, particularly where some interpretation of
   16.38 +the content is required: identification of case, whitespace, punctuation, and
   16.39 +so on.
   16.40  
   16.41  Some Unicode operations could be implemented in the Lichen language, not in C.
   16.42  
   16.43  === Sequence and Mapping Types ===
   16.44  
   16.45 -Various methods still need defining in the dictionary, list, tuple and set classes to provide parity with Python.
   16.46 +Various methods still need defining in the dictionary, list, tuple and set
   16.47 +classes to provide parity with Python.
   16.48  
   16.49  === Hashing ===
   16.50  
   16.51 -Unlike Python, the hashing approaches currently employed are not well-tuned. One important difference between Lichen and Python is that the former does not use hashtables as a general structure for objects, meaning that certain desirable criteria for Python hashtables (randomising certain aspects of their behaviour to prevent the exploitation of poorly performing cases) are less important for Lichen.
   16.52 +Unlike Python, the hashing approaches currently employed are not well-tuned.
   16.53 +One important difference between Lichen and Python is that the former does not
   16.54 +use hashtables as a general structure for objects, meaning that certain
   16.55 +desirable criteria for Python hashtables (randomising certain aspects of their
   16.56 +behaviour to prevent the exploitation of poorly performing cases) are less
   16.57 +important for Lichen.
   16.58  
   16.59  == Provide Peripheral Library Support ==
   16.60  
   16.61 -The Python standard library offers support for things like networking which require a degree of system-level integration. The Lichen libraries could be expanded to offer coverage of many of the same APIs.
   16.62 +The Python standard library offers support for things like networking which
   16.63 +require a degree of system-level integration. The Lichen libraries could be
   16.64 +expanded to offer coverage of many of the same APIs.
   16.65  
   16.66  === Versatile Native Libraries ===
   16.67  
   16.68 -Currently, the native functionality already exposes things like file descriptors, and a relatively simple versatile interface might be sufficient for many future libraries.
   16.69 +Currently, the native functionality already exposes things like file
   16.70 +descriptors, and a relatively simple versatile interface might be sufficient
   16.71 +for many future libraries.
   16.72  
   16.73  == Incremental Compilation ==
   16.74  
   16.75 -To some extent, the toolchain supports the caching of inspection results with selective regeneration upon changes to source files. However, deduction and subsequent activities are performed in their entirety every time.
   16.76 +To some extent, the toolchain supports the caching of inspection results with
   16.77 +selective regeneration upon changes to source files. However, deduction and
   16.78 +subsequent activities are performed in their entirety every time.
   16.79  
   16.80  === Preserving Structures ===
   16.81  
   16.82 -Where source files have changed, it may be possible that the structure details remain the same. Consequently, it is unnecessary to regenerate structures, and it is only necessary to determine the nature of modified program operations.
   16.83 +Where source files have changed, it may be possible that the structure details
   16.84 +remain the same. Consequently, it is unnecessary to regenerate structures, and
   16.85 +it is only necessary to determine the nature of modified program operations.
   16.86  
   16.87 -Where structures have changed, certain existing operations may need to be updated because such operations may no longer support certain types of object or may support additional types. Meanwhile, changes to structures may be compatible with existing structure layouts and not require such layouts to be recomputed.
   16.88 +Where structures have changed, certain existing operations may need to be
   16.89 +updated because such operations may no longer support certain types of object
   16.90 +or may support additional types. Meanwhile, changes to structures may be
   16.91 +compatible with existing structure layouts and not require such layouts to be
   16.92 +recomputed.
   16.93  
   16.94  == Exploit Deductions Further ==
   16.95  
   16.96 -So far, deductions have been used to inform the translation of certain operations and to identify situations where suitable operations cannot be generated. However, no attempt has been made to broaden the application of such information.
   16.97 +So far, deductions have been used to inform the translation of certain
   16.98 +operations and to identify situations where suitable operations cannot be
   16.99 +generated. However, no attempt has been made to broaden the application of
  16.100 +such information.
  16.101  
  16.102  === Parameter Type Checking ===
  16.103  
  16.104 -Elementary knowledge about function parameters is currently available when inspecting a program, but the deducer does not attempt to propagate such information to likely callers of such functions. Such propagation in its simplest form would merely use the list of potential targets for each invocation, obtain the parameter types from each target, compare these types to any readily-comparable argument (such as a name), and then determine which targets may still be invoked successfully, potentially imposing restrictions on the suitable targets as a result.
  16.105 +Elementary knowledge about function parameters is currently available when
  16.106 +inspecting a program, but the deducer does not attempt to propagate such
  16.107 +information to likely callers of such functions. Such propagation in its
  16.108 +simplest form would merely use the list of potential targets for each
  16.109 +invocation, obtain the parameter types from each target, compare these types
  16.110 +to any readily-comparable argument (such as a name), and then determine which
  16.111 +targets may still be invoked successfully, potentially imposing restrictions
  16.112 +on the suitable targets as a result.
  16.113  
  16.114 -Such propagation activities can be time-consuming because they can lead to iterative whole-program propagation occurring. Consider the restriction of an invocation target: this may indicate a restriction on a name that provides such a target, thus affecting the function parameter providing the name; such a restriction could then be propagated to callers of that function, initiating further refinement of other invocations, and so on.
  16.115 +Such propagation activities can be time-consuming because they can lead to
  16.116 +iterative whole-program propagation occurring. Consider the restriction of an
  16.117 +invocation target: this may indicate a restriction on a name that provides
  16.118 +such a target, thus affecting the function parameter providing the name; such
  16.119 +a restriction could then be propagated to callers of that function, initiating
  16.120 +further refinement of other invocations, and so on.
    17.1 --- a/docs/wiki/Toolchain	Sun Jul 22 12:14:48 2018 +0200
    17.2 +++ b/docs/wiki/Toolchain	Thu Jan 17 14:23:38 2019 +0100
    17.3 @@ -1,34 +1,48 @@
    17.4  = Toolchain =
    17.5  
    17.6 -The toolchain implements the process of analysing Lichen source files, compiling information about the structures and routines expressed in each program, and generating output for further processing that can produce an executable program.
    17.7 +The toolchain implements the process of analysing Lichen source files,
    17.8 +compiling information about the structures and routines expressed in each
    17.9 +program, and generating output for further processing that can produce an
   17.10 +executable program.
   17.11  
   17.12  <<TableOfContents(2,3)>>
   17.13  
   17.14  == Compiling Programs ==
   17.15  
   17.16 -The principal interface to the toolchain is the `lplc` command, run on source files as in the following example:
   17.17 +The principal interface to the toolchain is the `lplc` command, run on source
   17.18 +files as in the following example:
   17.19  
   17.20  {{{
   17.21  lplc tests/unicode.py
   17.22  }}}
   17.23  
   17.24 -There is no need to specify all the files that might be required by the complete program. Instead, the toolchain identifies files in the program by searching its module search path. This can be configured using the `LICHENPATH` environment variable and the `-E` option.
   17.25 +There is no need to specify all the files that might be required by the
   17.26 +complete program. Instead, the toolchain identifies files in the program by
   17.27 +searching its module search path. This can be configured using the
   17.28 +`LICHENPATH` environment variable and the `-E` option.
   17.29  
   17.30 -Various [[../Prerequisites|prerequisites]] are needed for the toolchain to work properly. By specifying the `-c` option, the specified program will be translated to a C programming language representation but not built, avoiding the need for some development tools to be installed if this is desirable.
   17.31 +Various [[../Prerequisites|prerequisites]] are needed for the toolchain to
   17.32 +work properly. By specifying the `-c` option, the specified program will be
   17.33 +translated to a C programming language representation but not built, avoiding
   17.34 +the need for some development tools to be installed if this is desirable.
   17.35  
   17.36 -The default output file from a successful compilation is a file called `_main`, but this can be overridden using the `-o` option. For example:
   17.37 +The default output file from a successful compilation is a file called
   17.38 +`_main`, but this can be overridden using the `-o` option. For example:
   17.39  
   17.40  {{{
   17.41  lplc -o unicode tests/unicode.py
   17.42  }}}
   17.43  
   17.44 -The complete set of options can be viewed by specifying the `--help` option, and a manual page is also provided in the `docs` directory of the source distribution:
   17.45 +The complete set of options can be viewed by specifying the `--help` option,
   17.46 +and a manual page is also provided in the `docs` directory of the source
   17.47 +distribution:
   17.48  
   17.49  {{{
   17.50  man -l docs/lplc.1
   17.51  }}}
   17.52  
   17.53 -This page may already be installed if the software was provided as a package as part of an operating system distribution:
   17.54 +This page may already be installed if the software was provided as a package
   17.55 +as part of an operating system distribution:
   17.56  
   17.57  {{{
   17.58  man lplc
   17.59 @@ -36,18 +50,68 @@
   17.60  
   17.61  == Toolchain Implementation ==
   17.62  
   17.63 -The toolchain itself is currently written in Python, but it is envisaged that it will eventually be written in the Lichen language, hopefully needing only minor modifications so that it may be able to accept its own source files as input and ultimately produce a representation of itself as an executable program. Since the Lichen language is based on Python, it is convenient to use existing Python implementations to access libraries that support the parsing of Python source files into useful representations.
   17.64 +The toolchain itself is currently written in Python, but it is envisaged that
   17.65 +it will eventually be written in the Lichen language, hopefully needing only
   17.66 +minor modifications so that it may be able to accept its own source files as
   17.67 +input and ultimately produce a representation of itself as an executable
   17.68 +program. Since the Lichen language is based on Python, it is convenient to use
   17.69 +existing Python implementations to access libraries that support the parsing
   17.70 +of Python source files into useful representations.
   17.71  
   17.72 -The Python standard library provides two particularly useful modules or packages of relevance: the `compiler` package and the `parser` module; `parser` is employed by `compiler` to decode source text, whereas `compiler` takes the concrete syntax tree representation from `parser` and produces an abstract syntax tree (AST) which is particularly helpful to software of the nature described here. (Contrary to impressions that [[http://eli.thegreenplace.net/2009/11/28/python-internals-working-with-python-asts/|some articles]] might give, the `ast` module available in Python 2.5 and later was not the first module to offer AST representations of Python programs in Python, nor was it even the first such module in the standard library.)
   17.73 +The Python standard library provides two particularly useful modules or
   17.74 +packages of relevance: the `compiler` package and the `parser` module;
   17.75 +`parser` is employed by `compiler` to decode source text, whereas `compiler`
   17.76 +takes the concrete syntax tree representation from `parser` and produces an
   17.77 +abstract syntax tree (AST) which is particularly helpful to software of the
   17.78 +nature described here. (Contrary to impressions that
   17.79 +[[http://eli.thegreenplace.net/2009/11/28/python-internals-working-with-python-asts/|some
   17.80 +articles]] might give, the `ast` module available in Python 2.5 and later was
   17.81 +not the first module to offer AST representations of Python programs in
   17.82 +Python, nor was it even the first such module in the standard library.)
   17.83  
   17.84 -However, it is not desirable to have a dependency on a Python implementation, which the `parser` module effectively is (as would the `ast` module also be if it were used here), with it typically being implemented as an extension module in a non-Python language (in C for CPython, in Java for Jython, and so on). Fortunately, the !PyPy project implemented their own parsing module, `pyparser`, that is intended to be used within the !PyPy environment together with their own `ast` equivalent, but it has been possible to rework `pyparser` to produce representations that are compatible with the `compiler` package, itself being modified in various ways to achieve compatibility (and also to provide various other conveniences).
   17.85 +However, it is not desirable to have a dependency on a Python implementation,
   17.86 +which the `parser` module effectively is (as would the `ast` module also be if
   17.87 +it were used here), with it typically being implemented as an extension module
   17.88 +in a non-Python language (in C for CPython, in Java for Jython, and so on).
   17.89 +Fortunately, the !PyPy project implemented their own parsing module,
   17.90 +`pyparser`, that is intended to be used within the !PyPy environment together
   17.91 +with their own `ast` equivalent, but it has been possible to rework `pyparser`
   17.92 +to produce representations that are compatible with the `compiler` package,
   17.93 +itself being modified in various ways to achieve compatibility (and also to
   17.94 +provide various other conveniences).
   17.95  
   17.96  == Program Analysis ==
   17.97  
   17.98 -With the means of inspecting source files available through a `compiler` package producing a usable representation of each file, it becomes possible to identify the different elements in each file and to collect information that may be put to use later. But before any files are inspected, it must be determined ''which'' files are to be inspected, these comprising the complete program to be analysed.
   17.99 +With the means of inspecting source files available through a `compiler`
  17.100 +package producing a usable representation of each file, it becomes possible to
  17.101 +identify the different elements in each file and to collect information that
  17.102 +may be put to use later. But before any files are inspected, it must be
  17.103 +determined ''which'' files are to be inspected, these comprising the complete
  17.104 +program to be analysed.
  17.105  
  17.106 -Both Lichen and Python support the notion of a main source file (sometimes called the "script" file or the main module or `__main__`) and of imported modules and packages. The complete set of modules employed in a program is defined as those imported by the main module, then those imported by those modules, and so on. Thus, the complete set is not known without inspecting part of the program, and this set must be built incrementally until no new modules are encountered.
  17.107 +Both Lichen and Python support the notion of a main source file (sometimes
  17.108 +called the "script" file or the main module or `__main__`) and of imported
  17.109 +modules and packages. The complete set of modules employed in a program is
  17.110 +defined as those imported by the main module, then those imported by those
  17.111 +modules, and so on. Thus, the complete set is not known without inspecting
  17.112 +part of the program, and this set must be built incrementally until no new
  17.113 +modules are encountered.
  17.114  
  17.115 -Where Lichen and Python differ is in the handling of [[../Imports|imports]] themselves. Python [[https://docs.python.org/3/reference/import.html|employs]] an intricate mechanism that searches for modules and packages, loading modules encountered when descending into packages to retrieve specific modules. In contrast, Lichen only imports the modules that are explicitly mentioned in programs. Thus, a Lichen program will not accumulate potentially large numbers of superfluous modules.
  17.116 +Where Lichen and Python differ is in the handling of [[../Imports|imports]]
  17.117 +themselves. Python [[https://docs.python.org/3/reference/import.html|employs]]
  17.118 +an intricate mechanism that searches for modules and packages, loading modules
  17.119 +encountered when descending into packages to retrieve specific modules. In
  17.120 +contrast, Lichen only imports the modules that are explicitly mentioned in
  17.121 +programs. Thus, a Lichen program will not accumulate potentially large numbers
  17.122 +of superfluous modules.
  17.123  
  17.124 -With a given module identified as being part of a program, the module will then be [[../Inspection|inspected]] for the purposes of gathering useful information. Since the primary objectives are to characterise the structure of the objects in a program and to determine how such objects are used, certain kinds of program constructs will be inspected more closely than others. Note that this initial inspection activity is not concerned with the translation of program operations to other forms: such [[../Translation|translation]] will occur later; this initial inspection is purely concerned with obtaining enough information to inform such later activities, with the original program being revisited to provide the necessary detail required to translate it.
  17.125 +With a given module identified as being part of a program, the module will
  17.126 +then be [[../Inspection|inspected]] for the purposes of gathering useful
  17.127 +information. Since the primary objectives are to characterise the structure of
  17.128 +the objects in a program and to determine how such objects are used, certain
  17.129 +kinds of program constructs will be inspected more closely than others. Note
  17.130 +that this initial inspection activity is not concerned with the translation of
  17.131 +program operations to other forms: such [[../Translation|translation]] will
  17.132 +occur later; this initial inspection is purely concerned with obtaining enough
  17.133 +information to inform such later activities, with the original program being
  17.134 +revisited to provide the necessary detail required to translate it.
    18.1 --- a/docs/wiki/Translation	Sun Jul 22 12:14:48 2018 +0200
    18.2 +++ b/docs/wiki/Translation	Thu Jan 17 14:23:38 2019 +0100
    18.3 @@ -1,32 +1,58 @@
    18.4  = Translating Programs =
    18.5  
    18.6 -The Lichen toolchain currently targets the C programming language, generating programs that are then compiled using existing C compilers. The process of translation involves the optimisation and generation of program structures, and the translation of Lichen language constructs into equivalent C language constructs.
    18.7 +The Lichen toolchain currently targets the C programming language, generating
    18.8 +programs that are then compiled using existing C compilers. The process of
    18.9 +translation involves the optimisation and generation of program structures,
   18.10 +and the translation of Lichen language constructs into equivalent C language
   18.11 +constructs.
   18.12  
   18.13  <<TableOfContents(2,3)>>
   18.14  
   18.15  == Attribute Access Plans ==
   18.16  
   18.17 -During [[../Inspection|inspection]], each attribute access is associated with a unique location and its details stored. During [[../Deduction|deduction]], such accesses are resolved and characterised. During [[../Optimisation|optimisation]], such accesses are encoded as '''instruction plans''' indicating the operations or instructions required to achieve the access. During translation, these instruction plans are generated as part of the final program.
   18.18 +During [[../Inspection|inspection]], each attribute access is associated with
   18.19 +a unique location and its details stored. During [[../Deduction|deduction]],
   18.20 +such accesses are resolved and characterised. During
   18.21 +[[../Optimisation|optimisation]], such accesses are encoded as '''instruction
   18.22 +plans''' indicating the operations or instructions required to achieve the
   18.23 +access. During translation, these instruction plans are generated as part of
   18.24 +the final program.
   18.25  
   18.26  == Structure Optimisation ==
   18.27  
   18.28 -All object structures need to support the attributes defined for them, and although it may be possible to determine the identity of attributes in advance of a program being run, there generally remains a need to provide mechanisms to determine the nature of an object at run-time and to access its attributes dynamically. To achieve this, an [[../Optimisation|optimisation]] process catalogues the presence of each attribute name on all program objects, then deciding upon a position for the structure member corresponding to that attribute name in all objects.
   18.29 +All object structures need to support the attributes defined for them, and
   18.30 +although it may be possible to determine the identity of attributes in advance
   18.31 +of a program being run, there generally remains a need to provide mechanisms
   18.32 +to determine the nature of an object at run-time and to access its attributes
   18.33 +dynamically. To achieve this, an [[../Optimisation|optimisation]] process
   18.34 +catalogues the presence of each attribute name on all program objects, then
   18.35 +deciding upon a position for the structure member corresponding to that
   18.36 +attribute name in all objects.
   18.37  
   18.38  == Structure Generation ==
   18.39  
   18.40 -Each program consists of a number of structures providing the program's objects and defining classes, functions, modules and any predefined instances.
   18.41 +Each program consists of a number of structures providing the program's
   18.42 +objects and defining classes, functions, modules and any predefined instances.
   18.43  
   18.44  === Attribute Representation ===
   18.45  
   18.46 -Attributes most typically provide references to objects within a certain optional context. Since attributes are employed to encode various other kinds of information, the data structure may be interpreted in other ways within suitably controlled environments. For example, the `__data__` attribute found on instances of certain built-in classes will encode references to other kinds of information that are used within native functions.
   18.47 +Attributes most typically provide references to objects within a certain
   18.48 +optional context. Since attributes are employed to encode various other kinds
   18.49 +of information, the data structure may be interpreted in other ways within
   18.50 +suitably controlled environments. For example, the `__data__` attribute found
   18.51 +on instances of certain built-in classes will encode references to other kinds
   18.52 +of information that are used within native functions.
   18.53  
   18.54  === Object Representation ===
   18.55  
   18.56 -Objects provide collections of attributes, and the object representation is meant to support efficient computed access to attributes whilst also allowing a reasonably efficient means of testing and accessing attributes at run-time.
   18.57 +Objects provide collections of attributes, and the object representation is
   18.58 +meant to support efficient computed access to attributes whilst also allowing
   18.59 +a reasonably efficient means of testing and accessing attributes at run-time.
   18.60  
   18.61  === Structure Constants ===
   18.62  
   18.63 -For clarity, several classes of symbolic constant are defined in the translated program to help define structures or to refer to structure members.
   18.64 +For clarity, several classes of symbolic constant are defined in the
   18.65 +translated program to help define structures or to refer to structure members.
   18.66  
   18.67  || '''Constant Class''' || '''Purpose''' || '''Example''' || '''Application''' ||
   18.68  || `csize` || Indicates the number of attributes in a class || `__csize___builtins___list_list` ||<|3> Structure definition ||
   18.69 @@ -39,11 +65,18 @@
   18.70  || `code` || Assigns a code to a particular attribute name || `__code_value` ||<|2> Attribute access ||
   18.71  || `pos` || Indicates the table position used by a particular attribute name || `__pos_value` ||
   18.72  
   18.73 -Such constants are encoded using helper functions, producing names resembling those above that are intended to be distinct and not to conflict with other defined names in the final program.
   18.74 +Such constants are encoded using helper functions, producing names resembling
   18.75 +those above that are intended to be distinct and not to conflict with other
   18.76 +defined names in the final program.
   18.77  
   18.78  === Instance Structures ===
   18.79  
   18.80 -Like all program objects, instances are defined in C according to a generic structure, meaning that all instances have the same generic members from the perspective of a C program. However, specific structures are defined for each class in order to conveniently describe the dimensions of instances of that class. For example, for the built-in list class, a declaration resembling the following is issued:
   18.81 +Like all program objects, instances are defined in C according to a generic
   18.82 +structure, meaning that all instances have the same generic members from the
   18.83 +perspective of a C program. However, specific structures are defined for each
   18.84 +class in order to conveniently describe the dimensions of instances of that
   18.85 +class. For example, for the built-in list class, a declaration resembling the
   18.86 +following is issued:
   18.87  
   18.88  {{{
   18.89  typedef struct {
   18.90 @@ -57,15 +90,24 @@
   18.91  
   18.92  === Useful C Language Features ===
   18.93  
   18.94 -The translated code relies on the availability of certain useful features of C, some of them supported only in modern compilers:
   18.95 +The translated code relies on the availability of certain useful features of
   18.96 +C, some of them supported only in modern compilers:
   18.97  
   18.98 - * Array and structure literals: these are used to define values within expressions that would otherwise require separate definition; invocation arguments are defined using inline expressions and the `__ARGS` macro (for unknown invocation targets and for keyword arguments)
   18.99 - * [[https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html#Variadic-Macros|Variadic macros]]: such macros permit variable numbers of arguments, thus making the `__ARGS` macro possible
  18.100 - * [[WikiPedia:Comma operator]] sequences: these are used to construct instruction sequences within expressions, providing a mechanism for formulating attribute accesses
  18.101 + * Array and structure literals: these are used to define values within
  18.102 +   expressions that would otherwise require separate definition; invocation
  18.103 +   arguments are defined using inline expressions and the `__ARGS` macro (for
  18.104 +   unknown invocation targets and for keyword arguments)
  18.105 + * [[https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html#Variadic-Macros|
  18.106 +   Variadic macros]]: such macros permit variable numbers of arguments, thus
  18.107 +   making the `__ARGS` macro possible
  18.108 + * [[WikiPedia:Comma operator]] sequences: these are used to construct
  18.109 +   instruction sequences within expressions, providing a mechanism for
  18.110 +   formulating attribute accesses
  18.111  
  18.112  === Function Naming ===
  18.113  
  18.114 -A naming scheme is applied to functions defined in the final C program to support different kinds of callables present in the original program.
  18.115 +A naming scheme is applied to functions defined in the final C program to
  18.116 +support different kinds of callables present in the original program.
  18.117  
  18.118  || '''Name Class''' || '''Purpose''' || '''Example''' ||
  18.119  || `fn` || Indicates a plain function || `__fn___builtins___str_str` ||
  18.120 @@ -75,9 +117,16 @@
  18.121  
  18.122  === Generic Program Functionality ===
  18.123  
  18.124 -Given the representation of the fundamental program objects, a suite of generic operations is provided to support the activities necessary to inspect and update program data. Such operations are largely concerned with generic object and attribute representations and are practically identical from program to program.
  18.125 +Given the representation of the fundamental program objects, a suite of
  18.126 +generic operations is provided to support the activities necessary to inspect
  18.127 +and update program data. Such operations are largely concerned with generic
  18.128 +object and attribute representations and are practically identical from
  18.129 +program to program.
  18.130  
  18.131 -For example, an attribute provided by an object can be accessed in several different ways depending on the information known about the object when the program is translated. Consequently, a suite of attribute access operations is provided to support each different scenario:
  18.132 +For example, an attribute provided by an object can be accessed in several
  18.133 +different ways depending on the information known about the object when the
  18.134 +program is translated. Consequently, a suite of attribute access operations is
  18.135 +provided to support each different scenario:
  18.136  
  18.137  ||<-2|2> ||<-2> '''Attribute provider''' ||
  18.138  || '''Class''' || '''Instance''' ||
  18.139 @@ -87,7 +136,12 @@
  18.140  
  18.141  === Specific Program Functionality ===
  18.142  
  18.143 -Since objects at the program level have their layouts [[../Optimisation|optimised]], and since these layouts may differ from program to program, a suite of specific operations is provided to support a range of activities within programs, such operations being configured with program-specific information in order to do the right thing within a given program.
  18.144 +Since objects at the program level have their layouts
  18.145 +[[../Optimisation|optimised]], and since these layouts may differ from program
  18.146 +to program, a suite of specific operations is provided to support a range of
  18.147 +activities within programs, such operations being configured with
  18.148 +program-specific information in order to do the right thing within a given
  18.149 +program.
  18.150  
  18.151  || '''Function''' || '''Purpose''' ||
  18.152  || `__invoke` || A generic invocation function that populates the argument array and obtains the appropriate target ||
  18.153 @@ -98,47 +152,113 @@
  18.154  
  18.155  === Module Files ===
  18.156  
  18.157 -Each program module is defined in its own file in the translated program. Since C has no notion of general module-level code, a special main function is generated for each module to perform the operations defined at the module level in the original program. These main functions are called in turn by the principal `main` function of the final C program, and the order of invocation is defined by the module initialisation ordering established when resolving inter-module dependencies.
  18.158 +Each program module is defined in its own file in the translated program.
  18.159 +Since C has no notion of general module-level code, a special main function is
  18.160 +generated for each module to perform the operations defined at the module
  18.161 +level in the original program. These main functions are called in turn by the
  18.162 +principal `main` function of the final C program, and the order of invocation
  18.163 +is defined by the module initialisation ordering established when resolving
  18.164 +inter-module dependencies.
  18.165  
  18.166  === Statement Translation ===
  18.167  
  18.168 -Statements in the original program are generally translated to statements in the final C program. During inspection and translation, certain constructs are transformed to produce the operations necessary to implement such constructs. For instance, operators need to be rewritten as function invocations, and thus the form of the original program changes before it is translated to C.
  18.169 +Statements in the original program are generally translated to statements in
  18.170 +the final C program. During inspection and translation, certain constructs are
  18.171 +transformed to produce the operations necessary to implement such constructs.
  18.172 +For instance, operators need to be rewritten as function invocations, and thus
  18.173 +the form of the original program changes before it is translated to C.
  18.174  
  18.175 -When translating certain operations that may appear within a statement, a sequence of instructions is generated, and such instructions must be performed in turn. Fortunately, C supports a wide variety of operations - such as assignments - within expressions, and it also provides the comma operator which permits sequences of operations to be performed in order and to yield a result that can then be used in the context of an expression. Such sequences of operations are employed extensively to realise attribute access instruction plans.
  18.176 +When translating certain operations that may appear within a statement, a
  18.177 +sequence of instructions is generated, and such instructions must be performed
  18.178 +in turn. Fortunately, C supports a wide variety of operations - such as
  18.179 +assignments - within expressions, and it also provides the comma operator
  18.180 +which permits sequences of operations to be performed in order and to yield a
  18.181 +result that can then be used in the context of an expression. Such sequences
  18.182 +of operations are employed extensively to realise attribute access instruction
  18.183 +plans.
  18.184  
  18.185  === Name References ===
  18.186  
  18.187 -Names can be translated in different ways for the final C program. Function locals and parameters correspond directly to locals in C functions. Module-level globals and class-level locals are translated to structure accesses. However, temporary names that are used in sequence assignment are translated to C function locals, either in functions corresponding to their equivalents in the original program, or in module initialisation functions corresponding to the module scope in the original program.
  18.188 +Names can be translated in different ways for the final C program. Function
  18.189 +locals and parameters correspond directly to locals in C functions.
  18.190 +Module-level globals and class-level locals are translated to structure
  18.191 +accesses. However, temporary names that are used in sequence assignment are
  18.192 +translated to C function locals, either in functions corresponding to their
  18.193 +equivalents in the original program, or in module initialisation functions
  18.194 +corresponding to the module scope in the original program.
  18.195  
  18.196 -Certain special names that are formulated during [[../Inspection|inspection]] are converted to constant or static references.
  18.197 +Certain special names that are formulated during [[../Inspection|inspection]]
  18.198 +are converted to constant or static references.
  18.199  
  18.200  === Attribute Accesses ===
  18.201  
  18.202 -Each attribute access is associated with a particular access location, corresponding to an operation occurring in the original program identified during inspection. This location can be used as a key to access a [[#Attribute_Access_Plans|plan]] that describes how the access may be realised in the final program. Since most of the work computing the nature of the access is done during the preceding deduction and optimisation activities, what remains is mostly the emission of the plan into the generated program text with some substitutions performed.
  18.203 +Each attribute access is associated with a particular access location,
  18.204 +corresponding to an operation occurring in the original program identified
  18.205 +during inspection. This location can be used as a key to access a
  18.206 +[[#Attribute_Access_Plans|plan]] that describes how the access may be realised
  18.207 +in the final program. Since most of the work computing the nature of the
  18.208 +access is done during the preceding deduction and optimisation activities,
  18.209 +what remains is mostly the emission of the plan into the generated program
  18.210 +text with some substitutions performed.
  18.211  
  18.212  === Invocations ===
  18.213  
  18.214 -Each invocation involves an object whose identity may or may not be known at compile-time, plus a collection of arguments. The translation process is concerned with obtaining a target to be invoked in the generated program, populating an argument array appropriately for the target, and providing the means of invocation.
  18.215 +Each invocation involves an object whose identity may or may not be known at
  18.216 +compile-time, plus a collection of arguments. The translation process is
  18.217 +concerned with obtaining a target to be invoked in the generated program,
  18.218 +populating an argument array appropriately for the target, and providing the
  18.219 +means of invocation.
  18.220  
  18.221 -Where the identity of the object is not known at all, the information provided is effectively passed on to the special `__invoke` function which is then required to do all the work at run-time. However, one of the goals of analysing and compiling the program is to avoid doing such work at run-time and to take advantage of those cases where the identity of the object can be determined.
  18.222 +Where the identity of the object is not known at all, the information provided
  18.223 +is effectively passed on to the special `__invoke` function which is then
  18.224 +required to do all the work at run-time. However, one of the goals of
  18.225 +analysing and compiling the program is to avoid doing such work at run-time
  18.226 +and to take advantage of those cases where the identity of the object can be
  18.227 +determined.
  18.228  
  18.229 -Thus, where the identity of the called object is known, details of the object's signature - its parameters and their positions - are used to populate the argument array and to determine whether this can be done successfully. Some callables require a context to be provided, and knowledge about the callable can be used to obtain or omit such a context from the arguments.
  18.230 +Thus, where the identity of the called object is known, details of the
  18.231 +object's signature - its parameters and their positions - are used to populate
  18.232 +the argument array and to determine whether this can be done successfully.
  18.233 +Some callables require a context to be provided, and knowledge about the
  18.234 +callable can be used to obtain or omit such a context from the arguments.
  18.235  
  18.236 -Finally, the invocation target must be obtained. Where some knowledge of the identity of the object is available, it may be sufficient to access the special `__fn__` member directly (potentially testing for its presence if this is not certain) and to call the appropriate function with the assembled argument array. Where a definite identity is available, the actual function itself may be named and thus called with the arguments. Such operations are mere fragments of the total work performed by the `__invoke` function in the least optimal case.
  18.237 +Finally, the invocation target must be obtained. Where some knowledge of the
  18.238 +identity of the object is available, it may be sufficient to access the
  18.239 +special `__fn__` member directly (potentially testing for its presence if this
  18.240 +is not certain) and to call the appropriate function with the assembled
  18.241 +argument array. Where a definite identity is available, the actual function
  18.242 +itself may be named and thus called with the arguments. Such operations are
  18.243 +mere fragments of the total work performed by the `__invoke` function in the
  18.244 +least optimal case.
  18.245  
  18.246  === Assignments ===
  18.247  
  18.248 -Assignments of objects to names occur for the following kinds of statements: assignments, definition statements (`class` and `def`), import statements (`from` and `import`).
  18.249 +Assignments of objects to names occur for the following kinds of statements:
  18.250 +assignments, definition statements (`class` and `def`), import statements
  18.251 +(`from` and `import`).
  18.252  
  18.253  === Guards ===
  18.254  
  18.255 -Guards are identified during [[../Deduction|deduction]] as being necessary to uphold deductions about the nature of objects referenced via names that may be undermined at run-time by potentially erroneous program behaviour.
  18.256 +Guards are identified during [[../Deduction|deduction]] as being necessary to
  18.257 +uphold deductions about the nature of objects referenced via names that may be
  18.258 +undermined at run-time by potentially erroneous program behaviour.
  18.259  
  18.260  === Exceptions ===
  18.261  
  18.262 -The C programming language does not support exceptions in a form supported by more modern programming languages. Consequently, lower-level mechanisms need to be employed to reproduce the semantics of the `try` statement, its `except` and `finally` clauses, and the `raise` statement, as well as considering the effect of exceptions on the `return` statement.
  18.263 +The C programming language does not support exceptions in a form supported by
  18.264 +more modern programming languages. Consequently, lower-level mechanisms need
  18.265 +to be employed to reproduce the semantics of the `try` statement, its `except`
  18.266 +and `finally` clauses, and the `raise` statement, as well as considering the
  18.267 +effect of exceptions on the `return` statement.
  18.268  
  18.269 -Fortunately, some consideration has been given to this topic before. The [[http://www.nicemice.net/cexcept/|cexcept]] library - really a collection of convenience macros - provides a way of expressing exception constructs in a natural way in C while translating those constructs to usage of the `setjmp` and `longjmp` functions. Although direct usage of such functions could be generated during program translation, for convenience and to focus on other implementation concerns, it was decided to employ the cexcept macros and to build any additional functionality in terms of their usage.
  18.270 +Fortunately, some consideration has been given to this topic before. The
  18.271 +[[http://www.nicemice.net/cexcept/|cexcept]] library - really a collection of
  18.272 +convenience macros - provides a way of expressing exception constructs in a
  18.273 +natural way in C while translating those constructs to usage of the `setjmp`
  18.274 +and `longjmp` functions. Although direct usage of such functions could be
  18.275 +generated during program translation, for convenience and to focus on other
  18.276 +implementation concerns, it was decided to employ the cexcept macros and to
  18.277 +build any additional functionality in terms of their usage.
  18.278  
  18.279  A structure is defined to hold exception-related information:
  18.280  
  18.281 @@ -152,7 +272,8 @@
  18.282  } __exc;
  18.283  }}}
  18.284  
  18.285 -Here, the `arg` member holds a raised exception object reference. Meanwhile, the remaining members interact with language constructs as follows:
  18.286 +Here, the `arg` member holds a raised exception object reference. Meanwhile,
  18.287 +the remaining members interact with language constructs as follows:
  18.288  
  18.289  || '''Member''' || '''Description''' || '''Output Program Operation''' ||
  18.290  || `raising` || Indicates an exception that has been raised and not yet handled || `__Raise` ||
  18.291 @@ -161,4 +282,6 @@
  18.292  
  18.293  === Memory Allocation and Garbage Collection ===
  18.294  
  18.295 -To avoid having to write a garbage collector, the [[http://www.hboehm.info/gc/|Boehm-Demers-Weiser garbage collector]] is employed by programs to allocate and free memory needed for its objects.
  18.296 +To avoid having to write a garbage collector, the
  18.297 +[[http://www.hboehm.info/gc/|Boehm-Demers-Weiser garbage collector]] is
  18.298 +employed by programs to allocate and free memory needed for its objects.