paul@199 | 1 | Concepts
|
paul@199 | 2 | ========
|
paul@199 | 3 |
|
paul@201 | 4 | This document describes the underlying concepts employed in micropython.
|
paul@201 | 5 |
|
paul@201 | 6 | * Namespaces and attribute definition
|
paul@199 | 7 | * Contexts and values
|
paul@200 | 8 | * Tables, attributes and lookups
|
paul@199 | 9 | * Objects and structures
|
paul@200 | 10 | * Parameters and lookups
|
paul@200 | 11 | * Instantiation
|
paul@222 | 12 | * Register usage
|
paul@199 | 13 |
|
paul@201 | 14 | Namespaces and Attribute Definition
|
paul@201 | 15 | ===================================
|
paul@201 | 16 |
|
paul@201 | 17 | Namespaces are any objects which can retain attributes.
|
paul@201 | 18 |
|
paul@201 | 19 | * Module attributes are defined either at the module level or by global
|
paul@201 | 20 | statements.
|
paul@201 | 21 | * Class attributes are defined only within class statements.
|
paul@201 | 22 | * Instance attributes are defined only by assignments to attributes of self
|
paul@201 | 23 | within __init__ methods.
|
paul@201 | 24 |
|
paul@201 | 25 | These restrictions apply because such attributes are thus explicitly declared,
|
paul@201 | 26 | permitting the use of tables (described below). Module and class attributes
|
paul@201 | 27 | can also be finalised in this way in order to permit certain optimisations.
|
paul@201 | 28 |
|
paul@201 | 29 | See rejected.txt for complicating mechanisms which could be applied to
|
paul@201 | 30 | mitigate the effects of these restrictions on optimisations.
|
paul@201 | 31 |
|
paul@199 | 32 | Contexts and Values
|
paul@199 | 33 | ===================
|
paul@199 | 34 |
|
paul@199 | 35 | Values are used as the common reference representation in micropython: as
|
paul@199 | 36 | stored representations of attributes (of classes, instances, modules, and
|
paul@199 | 37 | other objects supporting attribute-like entities) as well as the stored values
|
paul@199 | 38 | associated with names in functions and methods.
|
paul@199 | 39 |
|
paul@199 | 40 | Unlike other implementations, micropython does not create things like bound
|
paul@199 | 41 | method objects for individual instances. Instead, all objects are referenced
|
paul@199 | 42 | using a context, reference pair:
|
paul@199 | 43 |
|
paul@199 | 44 | Value Layout
|
paul@199 | 45 | ------------
|
paul@199 | 46 |
|
paul@199 | 47 | 0 1
|
paul@199 | 48 | context object
|
paul@199 | 49 | reference reference
|
paul@199 | 50 |
|
paul@199 | 51 | Specific implementations might reverse this ordering for optimisation
|
paul@199 | 52 | purposes.
|
paul@199 | 53 |
|
paul@199 | 54 | Rationale
|
paul@199 | 55 | ---------
|
paul@199 | 56 |
|
paul@199 | 57 | To reduce the number of created objects whilst retaining the ability to
|
paul@199 | 58 | support bound method invocations. The context indicates the context in which
|
paul@199 | 59 | an invocation is performed, typically the owner of the method.
|
paul@199 | 60 |
|
paul@199 | 61 | Usage
|
paul@199 | 62 | -----
|
paul@199 | 63 |
|
paul@199 | 64 | The context may be inserted as the first argument when a value is involved in
|
paul@199 | 65 | an invocation. This argument may then be omitted from the invocation if its
|
paul@199 | 66 | usage is not appropriate.
|
paul@199 | 67 |
|
paul@199 | 68 | See invocation.txt for details.
|
paul@199 | 69 |
|
paul@199 | 70 | Contexts in Acquired Values
|
paul@199 | 71 | ---------------------------
|
paul@199 | 72 |
|
paul@223 | 73 | There are three classes of instructions which provide values:
|
paul@199 | 74 |
|
paul@199 | 75 | Instruction Purpose Context Operations
|
paul@199 | 76 | ----------- ------- ------------------
|
paul@199 | 77 |
|
paul@223 | 78 | 1) LoadConst Load class, module, Use loaded object with itself
|
paul@223 | 79 | constant as context
|
paul@199 | 80 |
|
paul@223 | 81 | 2) LoadFunction Load function Combine null context with
|
paul@223 | 82 | loaded object
|
paul@223 | 83 |
|
paul@223 | 84 | 3) LoadAddress* Load attribute from Preserve or override stored
|
paul@201 | 85 | LoadAttr* class, module, context (as described in
|
paul@201 | 86 | instance assignment.txt)
|
paul@199 | 87 |
|
paul@199 | 88 | In order to comply with traditional Python behaviour, contexts may or may not
|
paul@199 | 89 | represent the object from which an attribute has been acquired.
|
paul@199 | 90 |
|
paul@199 | 91 | See assignment.txt for details.
|
paul@199 | 92 |
|
paul@199 | 93 | Contexts in Stored Values
|
paul@199 | 94 | -------------------------
|
paul@199 | 95 |
|
paul@223 | 96 | There are two classes of instruction for storing values:
|
paul@199 | 97 |
|
paul@223 | 98 | Instruction Purpose Context Operations
|
paul@223 | 99 | ----------- ------- ------------------
|
paul@199 | 100 |
|
paul@223 | 101 | 1) StoreAddress Store attribute in a Preserve context; note that no
|
paul@223 | 102 | known object test for class attribute
|
paul@223 | 103 | assignment should be necessary
|
paul@223 | 104 | since this instruction should only
|
paul@223 | 105 | be generated for module globals
|
paul@199 | 106 |
|
paul@223 | 107 | StoreAttr Store attribute in an Preserve context; note that no
|
paul@223 | 108 | instance test for class attribute
|
paul@223 | 109 | assignment should be necessary
|
paul@223 | 110 | since this instruction should only
|
paul@223 | 111 | be generated for self accesses
|
paul@199 | 112 |
|
paul@223 | 113 | StoreAttrIndex Store attribute in an Preserve context; since the index
|
paul@223 | 114 | unknown object lookup could yield a class
|
paul@223 | 115 | attribute, a test of the nature of
|
paul@223 | 116 | the nature of the structure is
|
paul@223 | 117 | necessary in order to prevent
|
paul@223 | 118 | assignments to classes
|
paul@199 | 119 |
|
paul@223 | 120 | 2) StoreAddressContext Store attribute in a Override context if appropriate;
|
paul@223 | 121 | known object if the value has a null context,
|
paul@223 | 122 | permit the target to take
|
paul@223 | 123 | ownership of the value
|
paul@199 | 124 |
|
paul@199 | 125 | See assignment.txt for details.
|
paul@199 | 126 |
|
paul@200 | 127 | Tables, Attributes and Lookups
|
paul@200 | 128 | ==============================
|
paul@199 | 129 |
|
paul@199 | 130 | Attribute lookups, where the exact location of an object attribute is deduced,
|
paul@199 | 131 | are performed differently in micropython than in other implementations.
|
paul@199 | 132 | Instead of providing attribute dictionaries, in which attributes are found,
|
paul@199 | 133 | attributes are located at fixed places in object structures (described below)
|
paul@199 | 134 | and their locations are stored using a special representation known as a
|
paul@199 | 135 | table.
|
paul@199 | 136 |
|
paul@199 | 137 | For a given program, a table can be considered as being like a matrix mapping
|
paul@199 | 138 | classes to attribute names. For example:
|
paul@199 | 139 |
|
paul@199 | 140 | class A:
|
paul@200 | 141 | # instances have attributes x, y
|
paul@199 | 142 |
|
paul@199 | 143 | class B(A):
|
paul@200 | 144 | # introduces attribute z for instances
|
paul@199 | 145 |
|
paul@199 | 146 | class C:
|
paul@200 | 147 | # instances have attributes a, b, z
|
paul@199 | 148 |
|
paul@200 | 149 | This would provide the following table, referred to as an object table in the
|
paul@200 | 150 | context of classes and instances:
|
paul@199 | 151 |
|
paul@199 | 152 | Class/attr a b x y z
|
paul@199 | 153 |
|
paul@199 | 154 | A 1 2
|
paul@199 | 155 | B 1 2 3
|
paul@199 | 156 | C 1 2 3
|
paul@199 | 157 |
|
paul@199 | 158 | A limitation of this representation is that instance attributes may not shadow
|
paul@199 | 159 | class attributes: if an attribute with a given name is not defined on an
|
paul@199 | 160 | instance, an attribute with the same name cannot be provided by the class of
|
paul@199 | 161 | the instance or any superclass of the instance's class.
|
paul@199 | 162 |
|
paul@199 | 163 | The table can be compacted using a representation known as a displacement
|
paul@200 | 164 | list (referred to as an object list in this context):
|
paul@199 | 165 |
|
paul@199 | 166 | Classes with attribute offsets
|
paul@199 | 167 |
|
paul@199 | 168 | classcode A
|
paul@199 | 169 | attrcode a b x y z
|
paul@199 | 170 |
|
paul@199 | 171 | B
|
paul@199 | 172 | a b x y z
|
paul@199 | 173 |
|
paul@199 | 174 | C
|
paul@199 | 175 | a b x y z
|
paul@199 | 176 |
|
paul@199 | 177 | List . . 1 2 1 2 3 1 2 . . 3
|
paul@199 | 178 |
|
paul@199 | 179 | Here, the classcode refers to the offset in the list at which a class's
|
paul@199 | 180 | attributes are defined, whereas the attrcode defines the offset within a
|
paul@199 | 181 | region of attributes corresponding to a single attribute of a given name.
|
paul@199 | 182 |
|
paul@200 | 183 | Attribute Locations
|
paul@200 | 184 | -------------------
|
paul@200 | 185 |
|
paul@200 | 186 | The locations stored in table/list elements are for instance attributes
|
paul@200 | 187 | relative to the location of the instance, whereas those for class attributes
|
paul@200 | 188 | and modules are absolute addresses (although these could also be changed to
|
paul@200 | 189 | object-relative locations).
|
paul@200 | 190 |
|
paul@199 | 191 | Objects and Structures
|
paul@199 | 192 | ======================
|
paul@199 | 193 |
|
paul@199 | 194 | As well as references, micropython needs to have actual objects to refer to.
|
paul@199 | 195 | Since classes, functions and instances are all objects, it is desirable that
|
paul@199 | 196 | certain common features and operations are supported in the same way for all
|
paul@199 | 197 | of these things. To permit this, a common data structure format is used.
|
paul@199 | 198 |
|
paul@215 | 199 | Header.................................................... Attributes.................
|
paul@200 | 200 |
|
paul@215 | 201 | Identifier Identifier Address Identifier Size Object Object ...
|
paul@199 | 202 |
|
paul@215 | 203 | 0 1 2 3 4 5 6 7
|
paul@215 | 204 | classcode attrcode/ invocation funccode size __class__ attribute ...
|
paul@215 | 205 | instance reference reference reference
|
paul@215 | 206 | status
|
paul@199 | 207 |
|
paul@206 | 208 | Classcode
|
paul@206 | 209 | ---------
|
paul@206 | 210 |
|
paul@206 | 211 | Used in attribute lookup.
|
paul@206 | 212 |
|
paul@199 | 213 | Here, the classcode refers to the attribute lookup table for the object (as
|
paul@200 | 214 | described above). Classes and instances share the same classcode, and their
|
paul@200 | 215 | structures reflect this. Functions all belong to the same type and thus employ
|
paul@200 | 216 | the classcode for the function built-in type, whereas modules have distinct
|
paul@200 | 217 | types since they must support different sets of attributes.
|
paul@199 | 218 |
|
paul@206 | 219 | Attrcode
|
paul@206 | 220 | --------
|
paul@206 | 221 |
|
paul@206 | 222 | Used to test instances for membership of classes (or descendants of classes).
|
paul@206 | 223 |
|
paul@206 | 224 | Since, in traditional Python, classes are only ever instances of the "type"
|
paul@207 | 225 | built-in class, support for testing such a relationship directly has been
|
paul@207 | 226 | removed and the attrcode is not specified for classes: the presence of an
|
paul@207 | 227 | attrcode indicates that a given object is an instance.
|
paul@206 | 228 |
|
paul@215 | 229 | See the "Testing Instance Compatibility with Classes (Attrcode)" section below
|
paul@215 | 230 | for details of attrcodes.
|
paul@214 | 231 |
|
paul@213 | 232 | Invocation Reference
|
paul@213 | 233 | --------------------
|
paul@213 | 234 |
|
paul@213 | 235 | Used when an object is called.
|
paul@213 | 236 |
|
paul@213 | 237 | This is the address of the code to be executed when an invocation is performed
|
paul@213 | 238 | on the object.
|
paul@213 | 239 |
|
paul@215 | 240 | Funccode
|
paul@215 | 241 | --------
|
paul@213 | 242 |
|
paul@215 | 243 | Used to look up argument positions by name.
|
paul@213 | 244 |
|
paul@215 | 245 | The strategy with keyword arguments in micropython is to attempt to position
|
paul@215 | 246 | such arguments in the invocation frame as it is being constructed.
|
paul@215 | 247 |
|
paul@215 | 248 | See the "Parameters and Lookups" section for more information.
|
paul@215 | 249 |
|
paul@215 | 250 | Size
|
paul@215 | 251 | ----
|
paul@215 | 252 |
|
paul@219 | 253 | Used to indicate the size of an object including attributes.
|
paul@213 | 254 |
|
paul@209 | 255 | Attributes
|
paul@209 | 256 | ----------
|
paul@209 | 257 |
|
paul@209 | 258 | For classes, modules and instances, the attributes in the structure correspond
|
paul@209 | 259 | to the attributes of each kind of object. For functions, however, the
|
paul@209 | 260 | attributes in the structure correspond to the default arguments for each
|
paul@209 | 261 | function, if any.
|
paul@209 | 262 |
|
paul@206 | 263 | Structure Types
|
paul@206 | 264 | ---------------
|
paul@206 | 265 |
|
paul@199 | 266 | Class C:
|
paul@199 | 267 |
|
paul@215 | 268 | 0 1 2 3 4 5 6 7
|
paul@215 | 269 | classcode (unused) __new__ funccode size class type attribute ...
|
paul@215 | 270 | for C reference for reference reference
|
paul@215 | 271 | instantiator
|
paul@199 | 272 |
|
paul@199 | 273 | Instance of C:
|
paul@199 | 274 |
|
paul@215 | 275 | 0 1 2 3 4 5 6 7
|
paul@215 | 276 | classcode attrcode C.__call__ funccode size class C attribute ...
|
paul@215 | 277 | for C for C reference for reference reference
|
paul@215 | 278 | (if exists) C.__call__
|
paul@199 | 279 |
|
paul@200 | 280 | Function f:
|
paul@199 | 281 |
|
paul@215 | 282 | 0 1 2 3 4 5 6 7
|
paul@215 | 283 | classcode attrcode code funccode size class attribute ...
|
paul@215 | 284 | for for reference function (default)
|
paul@215 | 285 | function function reference reference
|
paul@200 | 286 |
|
paul@200 | 287 | Module m:
|
paul@200 | 288 |
|
paul@215 | 289 | 0 1 2 3 4 5 6 7
|
paul@219 | 290 | classcode attrcode (unused) (unused) (unused) module type attribute ...
|
paul@215 | 291 | for m for m reference (global)
|
paul@215 | 292 | reference
|
paul@200 | 293 |
|
paul@200 | 294 | The __class__ Attribute
|
paul@200 | 295 | -----------------------
|
paul@200 | 296 |
|
paul@200 | 297 | All objects support the __class__ attribute and this is illustrated above with
|
paul@200 | 298 | the first attribute.
|
paul@200 | 299 |
|
paul@200 | 300 | Class: refers to the type class (type.__class__ also refers to the type class)
|
paul@200 | 301 | Function: refers to the function class
|
paul@200 | 302 | Instance: refers to the class instantiated to make the object
|
paul@200 | 303 |
|
paul@203 | 304 | Lists and Tuples
|
paul@203 | 305 | ----------------
|
paul@203 | 306 |
|
paul@203 | 307 | The built-in list and tuple sequences employ variable length structures using
|
paul@203 | 308 | the attribute locations to store their elements, where each element is a
|
paul@203 | 309 | reference to a separately stored object.
|
paul@203 | 310 |
|
paul@214 | 311 | Testing Instance Compatibility with Classes (Attrcode)
|
paul@200 | 312 | ------------------------------------------------------
|
paul@200 | 313 |
|
paul@200 | 314 | Although it would be possible to have a data structure mapping classes to
|
paul@200 | 315 | compatible classes, such as a matrix indicating the subclasses (or
|
paul@200 | 316 | superclasses) of each class, the need to retain the key to such a data
|
paul@200 | 317 | structure for each class might introduce a noticeable overhead.
|
paul@200 | 318 |
|
paul@200 | 319 | Instead of having a separate structure, descendant classes of each class are
|
paul@200 | 320 | inserted as special attributes into the object table. This requires an extra
|
paul@200 | 321 | key to be retained, since each class must provide its own attribute code such
|
paul@200 | 322 | that upon an instance/class compatibility test, the code may be obtained and
|
paul@200 | 323 | used in the object table.
|
paul@200 | 324 |
|
paul@200 | 325 | Invocation and Code References
|
paul@200 | 326 | ------------------------------
|
paul@200 | 327 |
|
paul@200 | 328 | Modules: there is no meaningful invocation reference since modules cannot be
|
paul@200 | 329 | explicitly called.
|
paul@200 | 330 |
|
paul@200 | 331 | Functions: a simple code reference is employed pointing to code implementing
|
paul@200 | 332 | the function. Note that the function locals are completely distinct from this
|
paul@200 | 333 | structure and are not comparable to attributes. Instead, attributes are
|
paul@200 | 334 | reserved for default parameter values, although they do not appear in the
|
paul@200 | 335 | object table described above, appearing instead in a separate parameter table
|
paul@200 | 336 | described below.
|
paul@200 | 337 |
|
paul@200 | 338 | Classes: given that classes must be invoked in order to create instances, a
|
paul@200 | 339 | reference must be provided in class structures. However, this reference does
|
paul@200 | 340 | not point directly at the __init__ method of the class. Instead, the
|
paul@200 | 341 | referenced code belongs to a special initialiser function, __new__, consisting
|
paul@200 | 342 | of the following instructions:
|
paul@200 | 343 |
|
paul@200 | 344 | create instance for C
|
paul@200 | 345 | call C.__init__(instance, ...)
|
paul@200 | 346 | return instance
|
paul@200 | 347 |
|
paul@200 | 348 | Instances: each instance employs a reference to any __call__ method defined in
|
paul@200 | 349 | the class hierarchy for the instance, thus maintaining its callable nature.
|
paul@200 | 350 |
|
paul@200 | 351 | Both classes and modules may contain code in their definitions - the former in
|
paul@200 | 352 | the "body" of the class, potentially defining attributes, and the latter as
|
paul@200 | 353 | the "top-level" code in the module, potentially defining attributes/globals -
|
paul@200 | 354 | but this code is not associated with any invocation target. It is thus
|
paul@200 | 355 | generated in order of appearance and is not referenced externally.
|
paul@200 | 356 |
|
paul@200 | 357 | Invocation Operation
|
paul@200 | 358 | --------------------
|
paul@200 | 359 |
|
paul@200 | 360 | Consequently, regardless of the object an invocation is always done as
|
paul@200 | 361 | follows:
|
paul@200 | 362 |
|
paul@200 | 363 | get invocation reference from the header
|
paul@200 | 364 | jump to reference
|
paul@200 | 365 |
|
paul@200 | 366 | Additional preparation is necessary before the above code: positional
|
paul@200 | 367 | arguments must be saved in the invocation frame, and keyword arguments must be
|
paul@200 | 368 | resolved and saved to the appropriate position in the invocation frame.
|
paul@200 | 369 |
|
paul@200 | 370 | See invocation.txt for details.
|
paul@200 | 371 |
|
paul@200 | 372 | Parameters and Lookups
|
paul@200 | 373 | ======================
|
paul@200 | 374 |
|
paul@200 | 375 | Since Python supports keyword arguments when making invocations, it becomes
|
paul@200 | 376 | necessary to record the parameter names associated with each function or
|
paul@200 | 377 | method. Just as object tables record attributes positions on classes and
|
paul@200 | 378 | instances, parameter tables record parameter positions in function or method
|
paul@200 | 379 | parameter lists.
|
paul@200 | 380 |
|
paul@200 | 381 | For a given program, a parameter table can be considered as being like a
|
paul@200 | 382 | matrix mapping functions/methods to parameter names. For example:
|
paul@200 | 383 |
|
paul@200 | 384 | def f(x, y, z):
|
paul@200 | 385 | pass
|
paul@200 | 386 |
|
paul@200 | 387 | def g(a, b, c):
|
paul@200 | 388 | pass
|
paul@200 | 389 |
|
paul@200 | 390 | def h(a, x):
|
paul@200 | 391 | pass
|
paul@200 | 392 |
|
paul@200 | 393 | This would provide the following table, referred to as a parameter table in
|
paul@200 | 394 | the context of functions and methods:
|
paul@200 | 395 |
|
paul@200 | 396 | Function/param a b c x y z
|
paul@200 | 397 |
|
paul@200 | 398 | f 1 2 3
|
paul@200 | 399 | g 1 2 3
|
paul@200 | 400 | h 1 2
|
paul@200 | 401 |
|
paul@233 | 402 | Confusion can occur when functions are adopted as methods, since the context
|
paul@233 | 403 | then occupies the first slot in the invocation frame:
|
paul@233 | 404 |
|
paul@233 | 405 | def f(x, y, z):
|
paul@233 | 406 | pass
|
paul@233 | 407 |
|
paul@233 | 408 | f(x=1, y=2, z=3) -> f(<context>, 1, 2, 3)
|
paul@233 | 409 | -> f(1, 2, 3)
|
paul@233 | 410 |
|
paul@233 | 411 | class C:
|
paul@233 | 412 | f = f
|
paul@233 | 413 |
|
paul@233 | 414 | def g(x, y, z):
|
paul@233 | 415 | pass
|
paul@233 | 416 |
|
paul@233 | 417 | c = C()
|
paul@233 | 418 |
|
paul@233 | 419 | c.f(y=2, z=3) -> f(<context>, 2, 3)
|
paul@233 | 420 | c.g(y=2, z=3) -> C.g(<context>, 2, 3)
|
paul@233 | 421 |
|
paul@200 | 422 | Just as with parameter tables, a displacement list can be prepared from a
|
paul@200 | 423 | parameter table:
|
paul@200 | 424 |
|
paul@200 | 425 | Functions with parameter (attribute) offsets
|
paul@200 | 426 |
|
paul@200 | 427 | funccode f
|
paul@200 | 428 | attrcode a b c x y z
|
paul@200 | 429 |
|
paul@200 | 430 | g
|
paul@200 | 431 | a b c x y z
|
paul@200 | 432 |
|
paul@200 | 433 | h
|
paul@200 | 434 | a b c x y z
|
paul@200 | 435 |
|
paul@200 | 436 | List . . . 1 2 3 1 2 3 1 . . 2 . .
|
paul@200 | 437 |
|
paul@200 | 438 | Here, the funccode refers to the offset in the list at which a function's
|
paul@200 | 439 | parameters are defined, whereas the attrcode defines the offset within a
|
paul@200 | 440 | region of attributes corresponding to a single parameter of a given name.
|
paul@200 | 441 |
|
paul@200 | 442 | Instantiation
|
paul@200 | 443 | =============
|
paul@200 | 444 |
|
paul@200 | 445 | When instantiating classes, memory must be reserved for the header of the
|
paul@200 | 446 | resulting instance, along with locations for the attributes of the instance.
|
paul@200 | 447 | Since the instance header contains data common to all instances of a class, a
|
paul@200 | 448 | template header is copied to the start of the newly reserved memory region.
|
paul@222 | 449 |
|
paul@222 | 450 | Register Usage
|
paul@222 | 451 | ==============
|
paul@222 | 452 |
|
paul@222 | 453 | During code generation, much of the evaluation produces results which are
|
paul@222 | 454 | implicitly recorded in the "active value" register, and various instructions
|
paul@222 | 455 | will consume the active value. In addition, some instructions will consume a
|
paul@222 | 456 | separate "active source value" from a register, typically those which are
|
paul@222 | 457 | assigning the result of an expression to an assignment target.
|
paul@222 | 458 |
|
paul@222 | 459 | Since values often need to be retained for later use, a set of temporary
|
paul@222 | 460 | storage locations are typically employed. However, optimisations may reduce
|
paul@222 | 461 | the need to use such temporary storage where instructions which provide the
|
paul@222 | 462 | "active value" can be re-executed and will produce the same result.
|