Lichen

Annotated docs/wiki/Restarted

856:263b43304dca
2018-07-16 Paul Boddie Added some comments identifying the nature of the tested operations.
paul@810 1
= Lichen Restarted =
paul@810 2
paul@810 3
Originally, lots of work was being put in to support various Python features that are arguably superfluous. The realisation was had that a lot of effort was being made for little practical benefit by trying to support things that are, [[../Design|in the larger picture]], not that important. Consequently, Lichen was refocused on a smaller set of more useful Python features.
paul@810 4
paul@810 5
This document is of historical interest only, with the [[../Design|design]] and other documents attempting to communicate the results of this restarting effort. Some obsolete information is therefore preserved below. For example, attributes hold context information in the diagrams, but context information is now held in wrappers or is maintained separately within programs.
paul@810 6
paul@810 7
Objectives:
paul@810 8
paul@810 9
 * Individual module inspection
paul@810 10
 * No importing triggered during module inspection
paul@810 11
 * All unresolved external references are set to `<depends>`
paul@810 12
 * Hierarchical module namespaces are not exposed in programs
paul@810 13
 * Modules are independent: package hierarchies are not traversed when importing
paul@810 14
 * Nested scopes will be dropped
paul@810 15
 * If expressions and comprehensions will be dropped
paul@810 16
 * `self` is a reserved name and is optional in method parameter lists
paul@810 17
 * Unbound methods must be bound using a special function taking an instance
paul@810 18
 * Functions assigned to classes do not become unbound methods
paul@810 19
paul@810 20
== Names ==
paul@810 21
paul@810 22
Names are locals, globals or built-ins. (Special names exist internally to support certain operations.)
paul@810 23
paul@810 24
Locals inside functions are dynamic; locals outside functions are static, as are module globals. Built-ins are defined statically in the `__builtins__` package.
paul@810 25
paul@810 26
== Imports ==
paul@810 27
paul@810 28
Imports provide access to external references. The "leaf" module in a module path is the module returned by the statement.
paul@810 29
paul@810 30
Indicate that module "compiler" is accessed via compiler...
paul@810 31
{{{#!python numbers=disable
paul@810 32
import compiler
paul@810 33
}}}
paul@810 34
paul@810 35
Indicate that module "compiler" is accessed via comp...
paul@810 36
{{{#!python numbers=disable
paul@810 37
import compiler as comp
paul@810 38
}}}
paul@810 39
paul@810 40
Indicate that module "compiler.ast" is accessed via ast; module "compiler.transformer" is accessed via tr...
paul@810 41
{{{#!python numbers=disable
paul@810 42
import compiler.ast as ast, compiler.transformer as tr
paul@810 43
}}}
paul@810 44
paul@810 45
Import compiler.ast, access Function...
paul@810 46
{{{#!python numbers=disable
paul@810 47
from compiler.ast import Function
paul@810 48
}}}
paul@810 49
paul@810 50
Import compiler.ast, access Function as F...
paul@810 51
{{{#!python numbers=disable
paul@810 52
from compiler.ast import Function as F
paul@810 53
}}}
paul@810 54
paul@810 55
This causes some semantic differences with Python, with the most significant one being the following:
paul@810 56
paul@810 57
{{{{#!table
paul@810 58
'''Python''' || '''Lichen'''
paul@810 59
==
paul@810 60
<style="vertical-align:top">
paul@810 61
paul@810 62
Import compiler, import compiler.ast, set ast on compiler...
paul@810 63
{{{#!python numbers=disable
paul@810 64
import compiler.ast
paul@810 65
}}}
paul@810 66
...returning compiler
paul@810 67
paul@810 68
||
paul@810 69
<style="vertical-align:top">
paul@810 70
paul@810 71
Import compiler.ast...
paul@810 72
{{{#!python numbers=disable
paul@810 73
import compiler.ast
paul@810 74
}}}
paul@810 75
...returning compiler.ast as ast
paul@810 76
paul@810 77
}}}}
paul@810 78
paul@810 79
Some statements can be rewritten to achieve the same effect:
paul@810 80
paul@810 81
{{{{#!table
paul@810 82
'''Python''' || '''Lichen'''
paul@810 83
==
paul@810 84
<style="vertical-align:top">
paul@810 85
paul@810 86
Import compiler, access ast as submodule...
paul@810 87
{{{#!python numbers=disable
paul@810 88
from compiler import ast
paul@810 89
}}}
paul@810 90
paul@810 91
||
paul@810 92
<style="vertical-align:top">
paul@810 93
paul@810 94
Import compiler.ast...
paul@810 95
{{{#!python numbers=disable
paul@810 96
import compiler.ast
paul@810 97
}}}
paul@810 98
...returning compiler.ast as ast
paul@810 99
paul@810 100
}}}}
paul@810 101
paul@810 102
Other kinds of import are not directly possible with Lichen. For example:
paul@810 103
paul@810 104
Import all names from compiler.ast...
paul@810 105
{{{#!python numbers=disable
paul@810 106
from compiler.ast import *
paul@810 107
}}}
paul@810 108
paul@810 109
Some notes:
paul@810 110
paul@810 111
 * Names not defined in a module and not declared in import statements are unresolved
paul@810 112
 * Modules are identified during inspection but are not loaded
paul@810 113
 * Instead, modules are added to a list and are imported later
paul@810 114
 * Names imported from modules are set to `<depends>` (since the contents of modules will not generally be known)
paul@810 115
 * Names are resolved in a later activity
paul@810 116
paul@810 117
== Self ==
paul@810 118
paul@810 119
In Python:
paul@810 120
paul@810 121
 * The `self` name provides access to the instance associated with a method
paul@810 122
 * The instance is supplied by a "context", initialised when a method is obtained from an instance (or through other attribute accesses)
paul@810 123
 * Upon invocation, any instance context must be assigned to the `self` parameter, provided the callable is a method
paul@810 124
 * Meanwhile, any non-instance context is not assigned to the `self` parameter, which should be provided explicitly for class-accessed methods
paul@810 125
 * Plain functions never expose `self` or have `self` initialised, even if they have been assigned to an instance
paul@810 126
paul@810 127
Apart from tests for the nature of the context and the callable, the argument list is effectively variable.
paul@810 128
paul@810 129
With `self` as a ubiquitous, hidden parameter:
paul@810 130
paul@810 131
 * The `self` name still provides access to the instance associated with a method
paul@810 132
 * The instance is still supplied by a "context", initialised when a method is obtained from an instance (or through other attribute accesses)
paul@810 133
 * Upon invocation, `self` is included in the argument list regardless of the nature of the callable
paul@810 134
 * Class-accessed methods would have their class as context, following from the above, but `self` may not refer to a class: it must be an instance
paul@810 135
 * To combine class-accessed methods with instance contexts, a special function is employed
paul@810 136
paul@810 137
The argument list for each callable thus remains static, at a cost of allocating an extra argument that may not be used. (Various calling conventions for certain processor architectures employ potentially unused registers, anyway.) Note that a callable may support defaults, however, and thus any argument list may need extending to include default values for parameters without corresponding arguments.
paul@810 138
paul@810 139
{{{{#!table
paul@810 140
'''Python''' || '''Without self'''
paul@810 141
==
paul@810 142
<style="vertical-align:top">
paul@810 143
paul@810 144
{{{#!python numbers=disable
paul@810 145
inst.method(x, y, z)
paul@810 146
# -> inst.method(inst, x, y, z)
paul@810 147
paul@810 148
def method(self, a, b, c):
paul@810 149
    # self = inst; a = x; b = y; c = z
paul@810 150
}}}
paul@810 151
paul@810 152
||
paul@810 153
<style="vertical-align:top">
paul@810 154
paul@810 155
{{{#!python numbers=disable
paul@810 156
inst.method(x, y, z)
paul@810 157
# -> inst.method(inst, x, y, z)
paul@810 158
paul@810 159
def method(a, b, c):
paul@810 160
    # self = inst; a = x; b = y; c = z
paul@810 161
}}}
paul@810 162
paul@810 163
==
paul@810 164
<style="vertical-align:top">
paul@810 165
paul@810 166
{{{#!python numbers=disable
paul@810 167
cls.method(self, x, y, z)
paul@810 168
paul@810 169
def method(self, a, b, c):
paul@810 170
    # parameters = arguments
paul@810 171
}}}
paul@810 172
paul@810 173
||
paul@810 174
<style="vertical-align:top">
paul@810 175
paul@810 176
{{{#!python numbers=disable
paul@810 177
f = get_using(cls.method, self)
paul@810 178
# context of f = self; value of f = cls.method
paul@810 179
f(x, y, z)
paul@810 180
# -> f(context of f, x, y, z)
paul@810 181
paul@810 182
def method(a, b, c):
paul@810 183
    # self = context of f = self; a = x; b = y; c = z
paul@810 184
}}}
paul@810 185
paul@810 186
}}}}
paul@810 187
paul@810 188
To avoid usage of `self` in undefined ways, only methods are able to use `self` and are not allowed to redefine it. Consequently, when invoking a callable, the context is set (where the callable is unknown until run-time; it is not set if compile-time knowledge indicates that it is not needed), and in situations where `self` is not permitted, the context is therefore safely ignored. Meanwhile, methods are always supplied with a context compatible with `self`.
paul@810 189
paul@810 190
|| '''Callable''' || '''self''' || '''Remarks''' ||
paul@810 191
|| Class || context || context discarded and replaced by allocated instance ||
paul@810 192
|| Function || null || `self` not permitted, context ignored ||
paul@810 193
|| Function (stored on class) || class as context || `self` not permitted, context ignored ||
paul@810 194
|| Function (stored on instance) || instance as context || `self` not permitted, context ignored ||
paul@810 195
|| Instance || instance as context || `self` set to instance ||
paul@810 196
|| Method (via class) || class as context || method not called (see "unbound methods") ||
paul@810 197
|| Method (via instance) || instance as context || `self` set to instance ||
paul@810 198
paul@810 199
Note that the treatment of functions stored on classes differs from Python. In Python, such functions would become unbound methods (see below) and would employ their first parameter as an effective `self` parameter (regardless of name).
paul@810 200
paul@810 201
== Unbound Methods ==
paul@810 202
paul@810 203
Since methods acquired directly from classes ("unbound methods" in Python) are meant to be combined with an instance as context (using the `get_using` function), they must be uncallable until combined with the appropriate context, yet the same methods when acquired via instances ("bound methods" in Python) must be immediately callable.
paul@810 204
paul@810 205
To support the two different states of methods, the principal structure of a class has attributes referencing uncallable versions of its methods. Meanwhile, such uncallable methods reference callable versions and when instances are employed to access the class attributes, it is these callable versions that are retrieved. For example:
paul@810 206
paul@810 207
{{{#!graphviz
paul@810 208
//format=svg
paul@810 209
//transform=notugly
paul@810 210
digraph structures {
paul@810 211
  node [shape=box,fontsize="13.0",fontname="Helvetica",tooltip="Instance and class structures"];
paul@810 212
  edge [fontsize="13.0",fontname="Helvetica",tooltip="Instance and class structures"];
paul@810 213
  rankdir=TB;
paul@810 214
paul@810 215
  instanceC [label="<main> instance of C |{ context of a | value of a }|{context of b | value of b }",shape=record];
paul@810 216
  classC [label="<main> class C |{ context of m | <m> value of m }|{ context of n | <n> value of n}",shape=record];
paul@810 217
  callables [label="<m> m\ncallable |<n> n\ncallable",shape=record];
paul@810 218
  uncallables [label="<m> m\nuncallable |<n> n\nuncallable",shape=record];
paul@810 219
paul@810 220
  instanceC:main -> classC:main;
paul@810 221
  classC:m -> uncallables:m [label="C.m",style=dashed];
paul@810 222
  classC:n -> uncallables:n [label="C.n",style=dashed];
paul@810 223
  uncallables:m -> callables:m [label="get_using(C.m, instance)",style=dashed];
paul@810 224
  uncallables:n -> callables:n [label="get_using(C.n, instance)",style=dashed];
paul@810 225
}
paul@810 226
}}}
paul@810 227
paul@810 228
The precise structure usage is as follows:
paul@810 229
paul@810 230
{{{#!graphviz
paul@810 231
//format=svg
paul@810 232
//transform=notugly
paul@810 233
digraph methods {
paul@810 234
  node [shape=box,fontsize="13.0",fontname="Helvetica",tooltip="Method structures"];
paul@810 235
  edge [fontsize="13.0",fontname="Helvetica",tooltip="Method structures"];
paul@810 236
  rankdir=TB;
paul@810 237
paul@810 238
  classC [label="<main> class C | { context of m | <mvalue> uncallable for m } | ...",shape=record];
paul@810 239
  uncallableattr [label="attr | { <context> C | <value> uncallable for m }",shape=record];
paul@810 240
  callableattr [label="attr | { <context> instance | <value> callable for m }",shape=record];
paul@810 241
  uncallable [label="<main> uncallable for m |{ __fn__ | <b> bound method reference | <fn> unbound method routine }|{ __args__ | minimum #parameters | <ptable> parameter table reference }",shape=record];
paul@810 242
  callable [label="<main> callable for m |{ __fn__ | 0 | <fn> bound method routine }|{ __args__ | minimum #parameters | <ptable> parameter table reference }",shape=record];
paul@810 243
  ptable [label="<main> parameter table for m | ...",shape=record];
paul@810 244
paul@810 245
  classC:mvalue -> uncallableattr [label="C.m",style=dashed];
paul@810 246
  classC:mvalue -> uncallable:main;
paul@810 247
  uncallableattr:value -> uncallable:main;
paul@810 248
  uncallableattr -> callableattr [label="get_using(C.m, instance)",style=dashed];
paul@810 249
  uncallable:b -> callable:main;
paul@810 250
  callableattr:value -> callable:main;
paul@810 251
  uncallable:ptable -> ptable:main;
paul@810 252
  callable:ptable -> ptable:main;
paul@810 253
}
paul@810 254
}}}
paul@810 255
paul@810 256
Callable methods provide a reference to a callable routine in its special callable member, just as functions and classes do. Uncallable methods populate the callable member with a reference to an error routine. Thus, any attempt to call an uncallable method would cause the error routine to be invoked. In addition, uncallable methods reference the corresponding callable methods so that the callable methods can be found and referenced.
paul@810 257
paul@810 258
|| '''Accessor''' || '''Provider''' || '''Attribute''' || '''Context''' || '''Summary''' ||
paul@810 259
||<|4> Instance ||<|4> Instance || Function || ''not used'' ||<|6> Preserve context ||
paul@810 260
|| Bound method || Original instance ||
paul@810 261
|| Unbound method || Providing class ||
paul@810 262
|| Other || Same as value ||
paul@810 263
||<|4> Instance ||<|4> Class || Function || ''not used'' ||
paul@810 264
|| Bound method || Original instance ||
paul@810 265
|| Unbound method || Accessing instance, if compatible || Test and replace context ||
paul@810 266
|| Other || Same as value ||<|5> Preserve context ||
paul@810 267
||<|4> Class ||<|4> Class || Function || ''not used'' ||
paul@810 268
|| Bound method || Original instance ||
paul@810 269
|| Unbound method || Providing class ||
paul@810 270
|| Other || Same as value ||
paul@810 271
paul@810 272
When obtaining an unbound method from an instance attribute, the context of the method attribute is provided. Indeed, the context is always preserved when accessing instance attributes.
paul@810 273
paul@810 274
When obtaining an unbound method from a class attribute via an instance, the context of the method attribute is tested against the accessing instance. If compatible, an attribute is copied containing the instance as context and a callable method reference as value.
paul@810 275
paul@810 276
When obtaining an unbound method from a class attribute, the context of the method attribute is provided. Indeed, the context is always preserved when accessing class attributes directly.
paul@810 277
paul@810 278
When combining an unbound method obtained from a class with an instance using `get_using`, the context of the method attribute is tested against the supplied instance. If compatible, an attribute is copied containing the instance as context and a callable method reference as value.
paul@810 279
paul@810 280
=== Functions as Unbound Methods ===
paul@810 281
paul@810 282
Functions not defined within classes could be treated as unbound methods if they were to employ `self` (thus indicating that they are intended as methods). Such functions would then be recorded as uncallable in the module namespace, needing to be explicitly bound to a class using a special function. However, there appears to be limited utility in defining functions in this way, instead of defining them directly as methods, or instead of merely using such generic functions from existing methods.