paul@766 | 1 | Low-level Implementation Details
|
paul@766 | 2 | ================================
|
paul@766 | 3 |
|
paul@766 | 4 | Although micropython delegates the generation of low-level program code and
|
paul@766 | 5 | data to syspython, various considerations of how an eventual program might be
|
paul@766 | 6 | structured have been used to inform the way micropython represents the details
|
paul@766 | 7 | of a program. This document describes these considerations and indicates how
|
paul@766 | 8 | syspython or other technologies might represent a working program.
|
paul@766 | 9 |
|
paul@766 | 10 | Objects and Structures
|
paul@766 | 11 | ======================
|
paul@766 | 12 |
|
paul@766 | 13 | As well as references, micropython needs to have actual objects to refer to.
|
paul@766 | 14 | Since classes, functions and instances are all objects, it is desirable that
|
paul@766 | 15 | certain common features and operations are supported in the same way for all
|
paul@766 | 16 | of these things. To permit this, a common data structure format is used.
|
paul@766 | 17 |
|
paul@766 | 18 | Header.................................................... Attributes.................
|
paul@766 | 19 |
|
paul@766 | 20 | Identifier Identifier Address Identifier Size Object ...
|
paul@766 | 21 |
|
paul@766 | 22 | 0 1 2 3 4 5 6
|
paul@766 | 23 | classcode attrcode/ invocation funccode size attribute ...
|
paul@766 | 24 | instance reference reference
|
paul@766 | 25 | status
|
paul@766 | 26 |
|
paul@766 | 27 | Classcode
|
paul@766 | 28 | ---------
|
paul@766 | 29 |
|
paul@766 | 30 | Used in attribute lookup.
|
paul@766 | 31 |
|
paul@766 | 32 | Here, the classcode refers to the attribute lookup table for the object (as
|
paul@766 | 33 | described in concepts.txt). Classes and instances share the same classcode,
|
paul@766 | 34 | and their structures reflect this. Functions all belong to the same type and
|
paul@766 | 35 | thus employ the classcode for the function built-in type, whereas modules have
|
paul@766 | 36 | distinct types since they must support different sets of attributes.
|
paul@766 | 37 |
|
paul@766 | 38 | Attrcode
|
paul@766 | 39 | --------
|
paul@766 | 40 |
|
paul@766 | 41 | Used to test instances for membership of classes (or descendants of classes).
|
paul@766 | 42 |
|
paul@766 | 43 | Since, in traditional Python, classes are only ever instances of some generic
|
paul@766 | 44 | built-in type, support for testing such a relationship directly has been
|
paul@766 | 45 | removed and the attrcode is not specified for classes: the presence of an
|
paul@766 | 46 | attrcode indicates that a given object is an instance. In addition, support
|
paul@766 | 47 | has also been removed for testing modules in the same way, meaning that the
|
paul@766 | 48 | attrcode is also not specified for modules.
|
paul@766 | 49 |
|
paul@766 | 50 | See the "Testing Instance Compatibility with Classes (Attrcode)" section below
|
paul@766 | 51 | for details of attrcodes.
|
paul@766 | 52 |
|
paul@766 | 53 | Invocation Reference
|
paul@766 | 54 | --------------------
|
paul@766 | 55 |
|
paul@766 | 56 | Used when an object is called.
|
paul@766 | 57 |
|
paul@766 | 58 | This is the address of the code to be executed when an invocation is performed
|
paul@766 | 59 | on the object.
|
paul@766 | 60 |
|
paul@766 | 61 | Funccode
|
paul@766 | 62 | --------
|
paul@766 | 63 |
|
paul@766 | 64 | Used to look up argument positions by name.
|
paul@766 | 65 |
|
paul@766 | 66 | The strategy with keyword arguments in micropython is to attempt to position
|
paul@766 | 67 | such arguments in the invocation frame as it is being constructed.
|
paul@766 | 68 |
|
paul@766 | 69 | See the "Parameters and Lookups" section for more information.
|
paul@766 | 70 |
|
paul@766 | 71 | Size
|
paul@766 | 72 | ----
|
paul@766 | 73 |
|
paul@766 | 74 | Used to indicate the size of an object including attributes.
|
paul@766 | 75 |
|
paul@766 | 76 | Attributes
|
paul@766 | 77 | ----------
|
paul@766 | 78 |
|
paul@766 | 79 | For classes, modules and instances, the attributes in the structure correspond
|
paul@766 | 80 | to the attributes of each kind of object. For functions, however, the
|
paul@766 | 81 | attributes in the structure correspond to the default arguments for each
|
paul@766 | 82 | function, if any.
|
paul@766 | 83 |
|
paul@766 | 84 | Structure Types
|
paul@766 | 85 | ---------------
|
paul@766 | 86 |
|
paul@766 | 87 | Class C:
|
paul@766 | 88 |
|
paul@766 | 89 | 0 1 2 3 4 5 6
|
paul@766 | 90 | classcode (unused) __new__ funccode size attribute ...
|
paul@766 | 91 | for C reference for reference
|
paul@766 | 92 | instantiator
|
paul@766 | 93 |
|
paul@766 | 94 | Instance of C:
|
paul@766 | 95 |
|
paul@766 | 96 | 0 1 2 3 4 5 6
|
paul@766 | 97 | classcode attrcode C.__call__ funccode size attribute ...
|
paul@766 | 98 | for C for C reference for reference
|
paul@766 | 99 | (if exists) C.__call__
|
paul@766 | 100 |
|
paul@766 | 101 | Function f:
|
paul@766 | 102 |
|
paul@766 | 103 | 0 1 2 3 4 5 6
|
paul@766 | 104 | classcode attrcode code funccode size attribute ...
|
paul@766 | 105 | for for reference (default)
|
paul@766 | 106 | function function reference
|
paul@766 | 107 |
|
paul@766 | 108 | Module m:
|
paul@766 | 109 |
|
paul@766 | 110 | 0 1 2 3 4 5 6
|
paul@766 | 111 | classcode attrcode (unused) (unused) (unused) attribute ...
|
paul@766 | 112 | for m for m (global)
|
paul@766 | 113 | reference
|
paul@766 | 114 |
|
paul@766 | 115 | The __class__ Attribute
|
paul@766 | 116 | -----------------------
|
paul@766 | 117 |
|
paul@766 | 118 | All objects should support the __class__ attribute, and in most cases this is
|
paul@766 | 119 | done using the object table, yielding a common address for all instances of a
|
paul@766 | 120 | given class.
|
paul@766 | 121 |
|
paul@766 | 122 | Function: refers to the function class
|
paul@766 | 123 | Instance: refers to the class instantiated to make the object
|
paul@766 | 124 |
|
paul@766 | 125 | The object table cannot support two definitions simultaneously for both
|
paul@766 | 126 | instances and their classes. Consequently, __class__ access on classes must be
|
paul@766 | 127 | tested for and a special result returned.
|
paul@766 | 128 |
|
paul@766 | 129 | Class: refers to the type class (type.__class__ also refers to the type class)
|
paul@766 | 130 |
|
paul@766 | 131 | For convenience, the first attribute of a class will be the common __class__
|
paul@766 | 132 | attribute for all its instances. As noted above, direct access to this
|
paul@766 | 133 | attribute will not be possible for classes, and a constant result will be
|
paul@766 | 134 | returned instead.
|
paul@766 | 135 |
|
paul@766 | 136 | Lists and Tuples
|
paul@766 | 137 | ----------------
|
paul@766 | 138 |
|
paul@766 | 139 | The built-in list and tuple sequences employ variable length structures using
|
paul@766 | 140 | the attribute locations to store their elements, where each element is a
|
paul@766 | 141 | reference to a separately stored object.
|
paul@766 | 142 |
|
paul@766 | 143 | Testing Instance Compatibility with Classes (Attrcode)
|
paul@766 | 144 | ------------------------------------------------------
|
paul@766 | 145 |
|
paul@766 | 146 | Although it would be possible to have a data structure mapping classes to
|
paul@766 | 147 | compatible classes, such as a matrix indicating the subclasses (or
|
paul@766 | 148 | superclasses) of each class, the need to retain the key to such a data
|
paul@766 | 149 | structure for each class might introduce a noticeable overhead.
|
paul@766 | 150 |
|
paul@766 | 151 | Instead of having a separate structure, descendant classes of each class are
|
paul@766 | 152 | inserted as special attributes into the object table. This requires an extra
|
paul@766 | 153 | key to be retained, since each class must provide its own attribute code such
|
paul@766 | 154 | that upon an instance/class compatibility test, the code may be obtained and
|
paul@766 | 155 | used in the object table.
|
paul@766 | 156 |
|
paul@766 | 157 | Invocation and Code References
|
paul@766 | 158 | ------------------------------
|
paul@766 | 159 |
|
paul@766 | 160 | Modules: there is no meaningful invocation reference since modules cannot be
|
paul@766 | 161 | explicitly called.
|
paul@766 | 162 |
|
paul@766 | 163 | Functions: a simple code reference is employed pointing to code implementing
|
paul@766 | 164 | the function. Note that the function locals are completely distinct from this
|
paul@766 | 165 | structure and are not comparable to attributes. Instead, attributes are
|
paul@766 | 166 | reserved for default parameter values, although they do not appear in the
|
paul@766 | 167 | object table described above, appearing instead in a separate parameter table
|
paul@766 | 168 | described in concepts.txt.
|
paul@766 | 169 |
|
paul@766 | 170 | Classes: given that classes must be invoked in order to create instances, a
|
paul@766 | 171 | reference must be provided in class structures. However, this reference does
|
paul@766 | 172 | not point directly at the __init__ method of the class. Instead, the
|
paul@766 | 173 | referenced code belongs to a special initialiser function, __new__, consisting
|
paul@766 | 174 | of the following instructions:
|
paul@766 | 175 |
|
paul@766 | 176 | create instance for C
|
paul@766 | 177 | call C.__init__(instance, ...)
|
paul@766 | 178 | return instance
|
paul@766 | 179 |
|
paul@766 | 180 | Instances: each instance employs a reference to any __call__ method defined in
|
paul@766 | 181 | the class hierarchy for the instance, thus maintaining its callable nature.
|
paul@766 | 182 |
|
paul@766 | 183 | Both classes and modules may contain code in their definitions - the former in
|
paul@766 | 184 | the "body" of the class, potentially defining attributes, and the latter as
|
paul@766 | 185 | the "top-level" code in the module, potentially defining attributes/globals -
|
paul@766 | 186 | but this code is not associated with any invocation target. It is thus
|
paul@766 | 187 | generated in order of appearance and is not referenced externally.
|
paul@766 | 188 |
|
paul@766 | 189 | Invocation Operation
|
paul@766 | 190 | --------------------
|
paul@766 | 191 |
|
paul@766 | 192 | Consequently, regardless of the object an invocation is always done as
|
paul@766 | 193 | follows:
|
paul@766 | 194 |
|
paul@766 | 195 | get invocation reference from the header
|
paul@766 | 196 | jump to reference
|
paul@766 | 197 |
|
paul@766 | 198 | Additional preparation is necessary before the above code: positional
|
paul@766 | 199 | arguments must be saved in the invocation frame, and keyword arguments must be
|
paul@766 | 200 | resolved and saved to the appropriate position in the invocation frame.
|
paul@766 | 201 |
|
paul@766 | 202 | See invocation.txt for details.
|
paul@766 | 203 |
|
paul@766 | 204 | Instantiation
|
paul@766 | 205 | =============
|
paul@766 | 206 |
|
paul@766 | 207 | When instantiating classes, memory must be reserved for the header of the
|
paul@766 | 208 | resulting instance, along with locations for the attributes of the instance.
|
paul@766 | 209 | Since the instance header contains data common to all instances of a class, a
|
paul@766 | 210 | template header is copied to the start of the newly reserved memory region.
|
paul@766 | 211 |
|
paul@766 | 212 | List and Tuple Representations
|
paul@766 | 213 | ==============================
|
paul@766 | 214 |
|
paul@766 | 215 | Since tuples have a fixed size, the representation of a tuple instance is
|
paul@766 | 216 | merely a header describing the size of the entire object, together with a
|
paul@766 | 217 | sequence of references to the object "stored" at each position in the
|
paul@766 | 218 | structure. Such references consist of the usual context and reference pair.
|
paul@766 | 219 |
|
paul@766 | 220 | Lists, however, have a variable size and must be accessible via an unchanging
|
paul@766 | 221 | location even as more memory is allocated elsewhere to accommodate the
|
paul@766 | 222 | contents of the list. Consequently, the representation must resemble the
|
paul@766 | 223 | following:
|
paul@766 | 224 |
|
paul@766 | 225 | Structure header for list (size == header plus special attribute)
|
paul@766 | 226 | Special attribute referencing the underlying sequence
|
paul@766 | 227 |
|
paul@766 | 228 | The underlying sequence has a fixed size, like a tuple, but may contain fewer
|
paul@766 | 229 | elements than the size of the sequence permits:
|
paul@766 | 230 |
|
paul@766 | 231 | Special header indicating the current size and allocated size
|
paul@766 | 232 | Element
|
paul@766 | 233 | ... <-- current size
|
paul@766 | 234 | (Unused space)
|
paul@766 | 235 | ... <-- allocated size
|
paul@766 | 236 |
|
paul@766 | 237 | This representation permits the allocation of a new sequence when space is
|
paul@766 | 238 | exhausted in an existing sequence, with the new sequence address stored in the
|
paul@766 | 239 | main list structure. Since access to the contents of the list must go through
|
paul@766 | 240 | the main list structure, underlying allocation activities may take place
|
paul@766 | 241 | without the users of a list having to be aware of such activities.
|