Lichen

Annotated docs/wiki/Design

933:520036942e1e
2021-06-28 Paul Boddie Removed "WikiWord" prevention syntax since MoinLight ignores it.
paul@810 1
= Design Decisions =
paul@810 2
paul@861 3
The Lichen language design involves some different choices to those taken in
paul@861 4
Python's design. Many of these choices are motivated by the following
paul@861 5
criteria:
paul@861 6
paul@861 7
 * To simplify the language and to make what programs do easier to understand
paul@861 8
   and to predict
paul@861 9
 * To make analysis of programs easier, particularly
paul@861 10
   [[../Deduction|deductions]] about the nature of the code
paul@861 11
 * To simplify and otherwise reduce the [[../Representations|representations]]
paul@861 12
   employed and the operations performed at run-time
paul@810 13
paul@861 14
Lichen is in many ways a restricted form of Python. In particular,
paul@861 15
restrictions on the attribute names supported by each object help to clearly
paul@861 16
define the object types in a program, allowing us to identify those objects
paul@861 17
when they are used. Consequently, optimisations that can be employed in a
paul@861 18
Lichen program become possible in situations where they would have been
paul@861 19
difficult or demanding to employ in a Python program.
paul@810 20
paul@861 21
Some design choices evoke memories of earlier forms of Python. Removing nested
paul@861 22
scopes simplifies the [[../Inspection|inspection]] of programs and run-time
paul@861 23
[[../Representations|representations]] and mechanisms. Other choices seek to
paul@861 24
remedy difficult or defective aspects of Python, notably the behaviour of
paul@861 25
Python's [[../Imports|import]] system.
paul@810 26
paul@810 27
<<TableOfContents(2,3)>>
paul@810 28
paul@810 29
== Attributes ==
paul@810 30
paul@810 31
{{{#!table
paul@810 32
'''Lichen''' || '''Python''' || '''Rationale'''
paul@810 33
==
paul@810 34
Objects have a fixed set of attribute names
paul@810 35
|| Objects can gain and lose attributes at run-time
paul@810 36
|| Having fixed sets of attributes helps identify object types
paul@810 37
==
paul@810 38
Instance attributes may not shadow class attributes
paul@810 39
|| Instance attributes may shadow class attributes
paul@810 40
|| Forbidding shadowing simplifies access operations
paul@810 41
==
paul@810 42
Attributes are simple members of object structures
paul@810 43
|| Dynamic handling and computation of attributes is supported
paul@810 44
|| Forbidding dynamic attributes simplifies access operations
paul@810 45
}}}
paul@810 46
paul@810 47
=== Fixed Attribute Names ===
paul@810 48
paul@861 49
Attribute names are bound for classes through assignment in the class
paul@861 50
namespace, for modules in the module namespace, and for instances in methods
paul@861 51
through assignment to `self`. Class and instance attributes are propagated to
paul@861 52
descendant classes and instances of descendant classes respectively. Once
paul@861 53
bound, attributes can be modified, but new attributes cannot be bound by other
paul@861 54
means, such as the assignment of an attribute to an arbitrary object that
paul@861 55
would not already support such an attribute.
paul@810 56
paul@810 57
{{{#!python numbers=disable
paul@810 58
class C:
paul@810 59
    a = 123
paul@810 60
    def __init__(self):
paul@810 61
        self.x = 234
paul@810 62
paul@810 63
C.b = 456 # not allowed (b not bound in C)
paul@810 64
C().y = 567 # not allowed (y not bound for C instances)
paul@810 65
}}}
paul@810 66
paul@861 67
Permitting the addition of attributes to objects would then require that such
paul@861 68
addition attempts be associated with particular objects, leading to a
paul@861 69
potentially iterative process involving object type deduction and
paul@861 70
modification, also causing imprecise results.
paul@810 71
paul@810 72
=== No Shadowing ===
paul@810 73
paul@810 74
Instances may not define attributes that are provided by classes.
paul@810 75
paul@810 76
{{{#!python numbers=disable
paul@810 77
class C:
paul@810 78
    a = 123
paul@810 79
    def shadow(self):
paul@810 80
        self.a = 234 # not allowed (attribute shadows class attribute)
paul@810 81
}}}
paul@810 82
paul@861 83
Permitting this would oblige instances to support attributes that, when
paul@861 84
missing, are provided by consulting their classes but, when not missing, may
paul@861 85
also be provided directly by the instances themselves.
paul@810 86
paul@810 87
=== No Dynamic Attributes ===
paul@810 88
paul@861 89
Instance attributes cannot be provided dynamically, such that any missing
paul@861 90
attribute would be supplied by a special method call to determine the
paul@861 91
attribute's presence and to retrieve its value.
paul@810 92
paul@810 93
{{{#!python numbers=disable
paul@810 94
class C:
paul@810 95
    def __getattr__(self, name): # not supported
paul@810 96
        if name == "missing":
paul@810 97
            return 123
paul@810 98
}}}
paul@810 99
paul@861 100
Permitting this would require object types to potentially support any
paul@861 101
attribute, undermining attempts to use attributes to identify objects.
paul@810 102
paul@810 103
== Naming ==
paul@810 104
paul@810 105
{{{#!table
paul@810 106
'''Lichen''' || '''Python''' || '''Rationale'''
paul@810 107
==
paul@861 108
Names may be local, global or built-in: nested namespaces must be initialised
paul@861 109
explicitly
paul@810 110
|| Names may also be non-local, permitting closures
paul@810 111
|| Limited name scoping simplifies program inspection and run-time mechanisms
paul@810 112
==
paul@810 113
`self` is a reserved name and is optional in method parameter lists
paul@861 114
|| `self` is a naming convention, but the first method parameter must always
paul@861 115
.. refer to the accessed object
paul@861 116
|| Reserving `self` assists deduction; making it optional is a consequence of
paul@861 117
.. the method binding behaviour
paul@822 118
==
paul@822 119
Instance attributes can be initialised using `.name` parameter notation
paul@861 120
|| [[https://stackoverflow.com/questions/1389180/automatically-initialize-instance-variables|Workarounds]]
paul@861 121
.. involving decorators and introspection are required for similar brevity
paul@822 122
|| Initialiser notation eliminates duplication in program code and is convenient
paul@810 123
}}}
paul@810 124
paul@810 125
=== Traditional Local, Global and Built-In Scopes Only ===
paul@810 126
paul@861 127
Namespaces reside within a hierarchy within modules: classes containing
paul@861 128
classes or functions; functions containing other functions. Built-in names are
paul@861 129
exposed in all namespaces, global names are defined at the module level and
paul@861 130
are exposed in all namespaces within the module, locals are confined to the
paul@861 131
namespace in which they are defined.
paul@810 132
paul@861 133
However, locals are not inherited by namespaces from surrounding or enclosing
paul@861 134
namespaces.
paul@810 135
paul@810 136
{{{#!python numbers=disable
paul@810 137
def f(x):
paul@810 138
    def g(y):
paul@810 139
        return x + y # not permitted: x is not inherited from f in Lichen (it is in Python)
paul@810 140
    return g
paul@810 141
paul@810 142
def h(x):
paul@810 143
    def i(y, x=x): # x is initialised but held in the namespace of i
paul@810 144
        return x + y # succeeds: x is defined
paul@810 145
    return i
paul@810 146
}}}
paul@810 147
paul@861 148
Needing to access outer namespaces in order to access any referenced names
paul@861 149
complicates the way in which such dynamic namespaces would need to be managed.
paul@861 150
Although the default initialisation technique demonstrated above could be
paul@861 151
automated, explicit initialisation makes programs easier to follow and avoids
paul@861 152
mistakes involving globals having the same name.
paul@810 153
paul@810 154
=== Reserved Self ===
paul@810 155
paul@861 156
The `self` name can be omitted in method signatures, but in methods it is
paul@861 157
always initialised to the instance on which the method is operating.
paul@810 158
paul@810 159
{{{#!python numbers=disable
paul@810 160
class C:
paul@810 161
    def f(y): # y is not the instance
paul@810 162
        self.x = y # self is the instance
paul@810 163
}}}
paul@810 164
paul@861 165
The assumption in methods is that `self` must always be referring to an
paul@861 166
instance of the containing class or of a descendant class. This means that
paul@861 167
`self` cannot be initialised to another kind of value, which Python permits
paul@861 168
through the explicit invocation of a method with the inclusion of the affected
paul@861 169
instance as the first argument. Consequently, `self` becomes optional in the
paul@861 170
signature because it is not assigned in the same way as the other parameters.
paul@810 171
paul@822 172
=== Instance Attribute Initialisers ===
paul@822 173
paul@861 174
In parameter lists, a special notation can be used to indicate that the given
paul@861 175
name is an instance attribute that will be assigned the argument value
paul@861 176
corresponding to the parameter concerned.
paul@822 177
paul@822 178
{{{#!python numbers=disable
paul@822 179
class C:
paul@822 180
    def f(self, .a, .b, c): # .a and .b indicate instance attributes
paul@822 181
        self.c = c # a traditional assignment using a parameter
paul@822 182
}}}
paul@822 183
paul@861 184
To use the notation, such dot-qualified parameters must appear only in the
paul@861 185
parameter lists of methods, not plain functions. The qualified parameters are
paul@861 186
represented as locals having the same name, and assignments to the
paul@861 187
corresponding instance attributes are inserted into the generated code.
paul@822 188
paul@822 189
{{{#!python numbers=disable
paul@822 190
class C:
paul@822 191
    def f1(self, .a, .b): # equivalent to f2, below
paul@822 192
        pass
paul@822 193
paul@822 194
    def f2(self, a, b):
paul@822 195
        self.a = a
paul@822 196
        self.b = b
paul@822 197
paul@822 198
    def g(self, .a, .b, a): # not permitted: a appears twice
paul@822 199
        pass
paul@822 200
}}}
paul@822 201
paul@861 202
Naturally, `self`, being a reserved name in methods, can also be omitted from
paul@861 203
such parameter lists. Moreover, such initialising parameters can have default
paul@861 204
values.
paul@823 205
paul@823 206
{{{#!python numbers=disable
paul@823 207
class C:
paul@823 208
    def __init__(.a=1, .b=2):
paul@823 209
        pass
paul@823 210
paul@823 211
c1 = C()
paul@823 212
c2 = C(3, 4)
paul@823 213
print c1.a, c1.b # 1 2
paul@823 214
print c2.a, c2.b # 3 4
paul@823 215
}}}
paul@822 216
paul@810 217
== Inheritance and Binding ==
paul@810 218
paul@810 219
{{{#!table
paul@810 220
'''Lichen''' || '''Python''' || '''Rationale'''
paul@810 221
==
paul@861 222
Class attributes are propagated to class hierarchy members during
paul@861 223
initialisation: rebinding class attributes does not affect descendant class
paul@861 224
attributes
paul@861 225
|| Class attributes are propagated live to class hierarchy members and must be
paul@861 226
.. looked up by the run-time system if not provided by a given class
paul@861 227
|| Initialisation-time propagation simplifies access operations and attribute
paul@861 228
.. table storage
paul@810 229
==
paul@810 230
Unbound methods must be bound using a special function taking an instance
paul@810 231
|| Unbound methods may be called using an instance as first argument
paul@810 232
|| Forbidding instances as first arguments simplifies the invocation mechanism
paul@810 233
==
paul@810 234
Functions assigned to class attributes do not become unbound methods
paul@810 235
|| Functions assigned to class attributes become unbound methods
paul@861 236
|| Removing method assignment simplifies deduction: methods are always defined
paul@861 237
.. in place
paul@810 238
==
paul@810 239
Base classes must be well-defined
paul@810 240
|| Base classes may be expressions
paul@861 241
|| Well-defined base classes are required to establish a well-defined
paul@861 242
.. hierarchy of types
paul@810 243
==
paul@810 244
Classes may not be defined in functions
paul@810 245
|| Classes may be defined in any kind of namespace
paul@861 246
|| Forbidding classes in functions prevents the definition of countless class
paul@861 247
.. variants that are awkward to analyse
paul@810 248
}}}
paul@810 249
paul@810 250
=== Inherited Class Attributes ===
paul@810 251
paul@861 252
Class attributes that are changed for a class do not change for that class's
paul@861 253
descendants.
paul@810 254
paul@810 255
{{{#!python numbers=disable
paul@810 256
class C:
paul@810 257
    a = 123
paul@810 258
paul@810 259
class D(C):
paul@810 260
    pass
paul@810 261
paul@810 262
C.a = 456
paul@810 263
print D.a # remains 123 in Lichen, becomes 456 in Python
paul@810 264
}}}
paul@810 265
paul@861 266
Permitting this requires indirection for all class attributes, requiring them
paul@861 267
to be treated differently from other kinds of attributes. Meanwhile, class
paul@861 268
attribute rebinding and the accessing of inherited attributes changed in this
paul@861 269
way is relatively rare.
paul@810 270
paul@810 271
=== Unbound Methods ===
paul@810 272
paul@861 273
Methods are defined on classes but are only available via instances: they are
paul@861 274
instance methods. Consequently, acquiring a method directly from a class and
paul@861 275
then invoking it should fail because the method will be unbound: the "context"
paul@861 276
of the method is not an instance. Furthermore, the Python technique of
paul@861 277
supplying an instance as the first argument in an invocation to bind the
paul@861 278
method to an instance, thus setting the context of the method, is not
paul@861 279
supported. See [[#Reserved Self|"Reserved Self"]] for more information.
paul@810 280
paul@810 281
{{{#!python numbers=disable
paul@810 282
class C:
paul@810 283
    def f(self, x):
paul@810 284
        self.x = x
paul@810 285
    def g(self):
paul@810 286
        C.f(123) # not permitted: C is not an instance
paul@810 287
        C.f(self, 123) # not permitted: self cannot be specified in the argument list
paul@810 288
        get_using(C.f, self)(123) # binds C.f to self, then the result is called
paul@810 289
}}}
paul@810 290
paul@861 291
Binding methods to instances occurs when acquiring methods via instances or
paul@861 292
explicitly using the `get_using` built-in. The built-in checks the
paul@861 293
compatibility of the supplied method and instance. If compatible, it provides
paul@861 294
the bound method as its result.
paul@810 295
paul@861 296
Normal functions are callable without any further preparation, whereas unbound
paul@861 297
methods need the binding step to be performed and are not immediately
paul@861 298
callable. Were functions to become unbound methods upon assignment to a class
paul@861 299
attribute, they would need to be invalidated by having the preparation
paul@861 300
mechanism enabled on them. However, this invalidation would only be relevant
paul@861 301
to the specific case of assigning functions to classes and this would need to
paul@861 302
be tested for. Given the added complications, such functionality is arguably
paul@861 303
not worth supporting.
paul@810 304
paul@810 305
=== Assigning Functions to Class Attributes ===
paul@810 306
paul@861 307
Functions can be assigned to class attributes but do not become unbound
paul@861 308
methods as a result.
paul@810 309
paul@810 310
{{{#!python numbers=disable
paul@810 311
class C:
paul@810 312
    def f(self): # will be replaced
paul@810 313
        return 234
paul@810 314
paul@810 315
def f(self):
paul@810 316
    return self
paul@810 317
paul@810 318
C.f = f # makes C.f a function, not a method
paul@810 319
C().f() # not permitted: f requires an explicit argument
paul@810 320
C().f(123) # permitted: f has merely been exposed via C.f
paul@810 321
}}}
paul@810 322
paul@861 323
Methods are identified as such by their definition location, they contribute
paul@861 324
information about attributes to the class hierarchy, and they employ certain
paul@861 325
structure details at run-time to permit the binding of methods. Since
paul@861 326
functions can defined in arbitrary locations, no class hierarchy information
paul@861 327
is available, and a function could combine `self` with a range of attributes
paul@861 328
that are not compatible with any class to which the function might be
paul@861 329
assigned.
paul@810 330
paul@810 331
=== Well-Defined Base Classes ===
paul@810 332
paul@861 333
Base classes must be clearly identifiable as well-defined classes. This
paul@861 334
facilitates the cataloguing of program objects and further analysis on them.
paul@810 335
paul@810 336
{{{#!python numbers=disable
paul@810 337
class C:
paul@810 338
    x = 123
paul@810 339
paul@810 340
def f():
paul@810 341
    return C
paul@810 342
paul@810 343
class D(f()): # not permitted: f could return anything
paul@810 344
    pass
paul@810 345
}}}
paul@810 346
paul@861 347
If base class identification could only be done reliably at run-time, class
paul@861 348
relationship information would be very limited without running the program or
paul@861 349
performing costly and potentially unreliable analysis. Indeed, programs
paul@861 350
employing such dynamic base classes are arguably resistant to analysis, which
paul@861 351
is contrary to the goals of a language like Lichen.
paul@810 352
paul@810 353
=== Class Definitions and Functions ===
paul@810 354
paul@861 355
Classes may not be defined in functions because functions provide dynamic
paul@861 356
namespaces, but Lichen relies on a static namespace hierarchy in order to
paul@861 357
clearly identify the principal objects in a program. If classes could be
paul@861 358
defined in functions, despite seemingly providing the same class over and over
paul@861 359
again on every invocation, a family of classes would, in fact, be defined.
paul@810 360
paul@810 361
{{{#!python numbers=disable
paul@810 362
def f(x):
paul@810 363
    class C: # not permitted: this describes one of potentially many classes
paul@810 364
        y = x
paul@810 365
    return f
paul@810 366
}}}
paul@810 367
paul@861 368
Moreover, issues of namespace nesting also arise, since the motivation for
paul@861 369
defining classes in functions would surely be to take advantage of local state
paul@861 370
to parameterise such classes.
paul@810 371
paul@810 372
== Modules and Packages ==
paul@810 373
paul@810 374
{{{#!table
paul@810 375
'''Lichen''' || '''Python''' || '''Rationale'''
paul@810 376
==
paul@810 377
Modules are independent: package hierarchies are not traversed when importing
paul@861 378
|| Modules exist in hierarchical namespaces: package roots must be imported
paul@861 379
.. before importing specific submodules
paul@861 380
|| Eliminating module traversal permits more precise imports and reduces
paul@861 381
.. superfluous code
paul@810 382
==
paul@861 383
Only specific names can be imported from a module or package using the `from`
paul@861 384
statement
paul@810 385
|| Importing "all" from a package or module is permitted
paul@861 386
|| Eliminating "all" imports simplifies the task of determining where names in
paul@861 387
.. use have come from
paul@810 388
==
paul@810 389
Modules must be specified using absolute names
paul@810 390
|| Imports can be absolute or relative
paul@810 391
|| Using only absolute names simplifies the import mechanism
paul@810 392
==
paul@861 393
Modules are imported independently and their dependencies subsequently
paul@861 394
resolved
paul@810 395
|| Modules are imported as import statements are encountered
paul@861 396
|| Statically-initialised objects can be used declaratively, although an
paul@861 397
.. initialisation order may still need establishing
paul@810 398
}}}
paul@810 399
paul@810 400
=== Independent Modules ===
paul@810 401
paul@861 402
The inclusion of modules in a program affects only explicitly-named modules:
paul@861 403
they do not have relationships implied by their naming that would cause such
paul@861 404
related modules to be included in a program.
paul@810 405
paul@810 406
{{{#!python numbers=disable
paul@810 407
from compiler import consts # defines consts
paul@810 408
import compiler.ast # defines ast, not compiler
paul@810 409
paul@810 410
ast # is defined
paul@810 411
compiler # is not defined
paul@810 412
consts # is defined
paul@810 413
}}}
paul@810 414
paul@861 415
Where modules should have relationships, they should be explicitly defined
paul@861 416
using `from` and `import` statements which target the exact modules required.
paul@861 417
In the above example, `compiler` is not routinely imported because modules
paul@861 418
within the `compiler` package have been requested.
paul@810 419
paul@810 420
=== Specific Name Imports Only ===
paul@810 421
paul@861 422
Lichen, unlike Python, also does not support the special `__all__` module
paul@861 423
attribute.
paul@810 424
paul@810 425
{{{#!python numbers=disable
paul@810 426
from compiler import * # not permitted
paul@810 427
from compiler import ast, consts # permitted
paul@810 428
paul@810 429
interpreter # undefined in compiler (yet it might be thought to reside there) and in this module
paul@810 430
}}}
paul@810 431
paul@861 432
The `__all__` attribute supports `from ... import *` statements in Python, but
paul@861 433
without identifying the module or package involved and then consulting
paul@861 434
`__all__` in that module or package to discover which names might be involved
paul@861 435
(which might require the inspection of yet other modules or packages), the
paul@861 436
names imported cannot be known. Consequently, some names used elsewhere in the
paul@861 437
module performing the import might be assumed to be imported names when, in
paul@861 438
fact, they are unknown in both the importing and imported modules. Such
paul@861 439
uncertainty hinders the inspection of individual modules.
paul@810 440
paul@810 441
=== Modules Imported Independently ===
paul@810 442
paul@861 443
When indicating an import using the `from` and `import` statements, the
paul@861 444
[[../Toolchain|toolchain]] does not attempt to immediately import other
paul@861 445
modules. Instead, the imports act as declarations of such other modules or
paul@861 446
names from other modules, resolved at a later stage. This permits mutual
paul@861 447
imports to a greater extent than in Python.
paul@810 448
paul@810 449
{{{#!python numbers=disable
paul@810 450
# Module M
paul@810 451
from N import C # in Python: fails attempting to re-enter N
paul@810 452
paul@810 453
class D(C):
paul@810 454
    y = 456
paul@810 455
paul@810 456
# Module N
paul@810 457
from M import D # in Python: causes M to be entered, fails when re-entered from N
paul@810 458
paul@810 459
class C:
paul@810 460
    x = 123
paul@810 461
paul@810 462
class E(D):
paul@810 463
    z = 789
paul@810 464
paul@810 465
# Main program
paul@810 466
import N
paul@810 467
}}}
paul@810 468
paul@861 469
Such flexibility is not usually needed, and circular importing usually
paul@861 470
indicates issues with program organisation. However, declarative imports can
paul@861 471
help to decouple modules and avoid combining import declaration and module
paul@861 472
initialisation order concerns.
paul@810 473
paul@810 474
== Syntax and Control-Flow ==
paul@810 475
paul@810 476
{{{#!table
paul@810 477
'''Lichen''' || '''Python''' || '''Rationale'''
paul@810 478
==
paul@810 479
If expressions and comprehensions are not supported
paul@810 480
|| If expressions and comprehensions are supported
paul@861 481
|| Omitting such syntactic features simplifies program inspection and
paul@861 482
.. translation
paul@810 483
==
paul@810 484
The `with` statement is not supported
paul@861 485
|| The `with` statement offers a mechanism for resource allocation and
paul@861 486
.. deallocation using context managers
paul@861 487
|| This syntactic feature can be satisfactorily emulated using existing
paul@861 488
.. constructs
paul@810 489
==
paul@810 490
Generators are not supported
paul@810 491
|| Generators are supported
paul@810 492
|| Omitting generator support simplifies run-time mechanisms
paul@810 493
==
paul@810 494
Only positional and keyword arguments are supported
paul@810 495
|| Argument unpacking (using `*` and `**`) is supported
paul@810 496
|| Omitting unpacking simplifies generic invocation handling
paul@810 497
==
paul@810 498
All parameters must be specified
paul@810 499
|| Catch-all parameters (`*` and `**`) are supported
paul@861 500
|| Omitting catch-all parameter population simplifies generic invocation
paul@861 501
.. handling
paul@810 502
}}}
paul@810 503
paul@810 504
=== No If Expressions or Comprehensions ===
paul@810 505
paul@861 506
In order to support the classic [[WikiPedia:?:|ternary operator]], a construct
paul@861 507
was [[https://www.python.org/dev/peps/pep-0308/|added]] to the Python syntax
paul@861 508
that needed to avoid problems with the existing grammar and notation.
paul@861 509
Unfortunately, it reorders the components from the traditional form:
paul@810 510
paul@810 511
{{{#!python numbers=disable
paul@810 512
# Not valid in Lichen, only in Python.
paul@810 513
paul@810 514
# In C: condition ? true_result : false_result
paul@810 515
true_result if condition else false_result
paul@810 516
paul@810 517
# In C: (condition ? inner_true_result : inner_false_result) ? true_result : false_result
paul@810 518
true_result if (inner_true_result if condition else inner_false_result) else false_result
paul@810 519
}}}
paul@810 520
paul@861 521
Since if expressions may participate within expressions, they cannot be
paul@861 522
rewritten as if statements. Nor can they be rewritten as logical operator
paul@861 523
chains in general.
paul@810 524
paul@810 525
{{{#!python numbers=disable
paul@810 526
# Not valid in Lichen, only in Python.
paul@810 527
paul@810 528
a = 0 if x else 1 # x being true yields 0
paul@810 529
paul@810 530
# Here, x being true causes (x and 0) to complete, yielding 0.
paul@810 531
# But this causes ((x and 0) or 1) to complete, yielding 1.
paul@810 532
paul@810 533
a = x and 0 or 1 # not valid
paul@810 534
}}}
paul@810 535
paul@861 536
But in any case, it would be more of a motivation to support the functionality
paul@861 537
if a better syntax could be adopted instead. However, if expressions are not
paul@861 538
particularly important in Python, and despite enhancement requests over many
paul@861 539
years, everybody managed to live without them.
paul@810 540
paul@861 541
List and generator comprehensions are more complicated but share some
paul@861 542
characteristics of if expressions: their syntax contradicts the typical
paul@861 543
conventions established by the rest of the Python language; they create
paul@861 544
implicit state that is perhaps most appropriately modelled by a separate
paul@861 545
function or similar object. Since Lichen does not support generators at all,
paul@861 546
it will obviously not support generator expressions.
paul@810 547
paul@810 548
Meanwhile, list comprehensions quickly encourage barely-readable programs:
paul@810 549
paul@810 550
{{{#!python numbers=disable
paul@810 551
# Not valid in Lichen, only in Python.
paul@810 552
paul@810 553
x = [0, [1, 2, 0], 0, 0, [0, 3, 4]]
paul@810 554
a = [z for y in x if y for z in y if z]
paul@810 555
}}}
paul@810 556
paul@861 557
Supporting the creation of temporary functions to produce list comprehensions,
paul@861 558
while also hiding temporary names from the enclosing scope, adds complexity to
paul@861 559
the toolchain for situations where programmers would arguably be better
paul@861 560
creating their own functions and thus writing more readable programs.
paul@810 561
paul@810 562
=== No With Statement ===
paul@810 563
paul@861 564
The
paul@861 565
[[https://docs.python.org/2.7/reference/compound_stmts.html#the-with-statement|with
paul@861 566
statement]] introduced the concept of
paul@861 567
[[https://docs.python.org/2.7/reference/datamodel.html#context-managers|context
paul@861 568
managers]] in Python 2.5, with such objects supporting a
paul@861 569
[[https://docs.python.org/2.7/library/stdtypes.html#typecontextmanager|programming
paul@861 570
interface]] that aims to formalise certain conventions around resource
paul@861 571
management. For example:
paul@810 572
paul@810 573
{{{#!python numbers=disable
paul@810 574
# Not valid in Lichen, only in Python.
paul@810 575
paul@810 576
with connection = db.connect(connection_args):
paul@810 577
    with cursor = connection.cursor():
paul@810 578
        cursor.execute(query, args)
paul@810 579
}}}
paul@810 580
paul@861 581
Although this makes for readable code, it must be supported by objects which
paul@861 582
define the `__enter__` and `__exit__` special methods. Here, the `connect`
paul@861 583
method invoked in the first `with` statement must return such an object;
paul@861 584
similarly, the `cursor` method must also provide an object with such
paul@861 585
characteristics.
paul@810 586
paul@810 587
However, the "pre-with" solution is as follows:
paul@810 588
paul@810 589
{{{#!python numbers=disable
paul@810 590
connection = db.connect(connection_args)
paul@810 591
try:
paul@810 592
    cursor = connection.cursor()
paul@810 593
    try:
paul@810 594
        cursor.execute(query, args)
paul@810 595
    finally:
paul@810 596
        cursor.close()
paul@810 597
finally:
paul@810 598
    connection.close()
paul@810 599
}}}
paul@810 600
paul@861 601
Although this seems less readable, its behaviour is more obvious because magic
paul@861 602
methods are not being called implicitly. Moreover, any parameterisation of the
paul@861 603
acts of resource deallocation or closure can be done in the `finally` clauses
paul@861 604
where such parameterisation would seem natural, rather than being specified
paul@861 605
through some kind of context manager initialisation arguments that must then
paul@861 606
be propagated to the magic methods so that they may take into consideration
paul@861 607
contextual information that is readily available in the place where the actual
paul@861 608
resource operations are being performed.
paul@810 609
paul@810 610
=== No Generators ===
paul@810 611
paul@861 612
[[https://www.python.org/dev/peps/pep-0255/|Generators]] were
paul@861 613
[[https://docs.python.org/release/2.3/whatsnew/section-generators.html|added]]
paul@861 614
to Python in the 2.2 release and became fully part of the language in the 2.3
paul@861 615
release. They offer a convenient way of writing iterator-like objects,
paul@861 616
capturing execution state instead of obliging the programmer to manage such
paul@861 617
state explicitly.
paul@810 618
paul@810 619
{{{#!python numbers=disable
paul@810 620
# Not valid in Lichen, only in Python.
paul@810 621
paul@810 622
def fib():
paul@810 623
    a, b = 0, 1
paul@810 624
    while 1:
paul@810 625
        yield b
paul@810 626
        a, b = b, a+b
paul@810 627
paul@810 628
# Alternative form valid in Lichen.
paul@810 629
paul@810 630
class fib:
paul@810 631
    def __init__(self):
paul@810 632
        self.a, self.b = 0, 1
paul@810 633
paul@810 634
    def next(self):
paul@810 635
        result = self.b
paul@810 636
        self.a, self.b = self.b, self.a + self.b
paul@810 637
        return result
paul@810 638
paul@810 639
# Main program.
paul@810 640
paul@810 641
seq = fib()
paul@810 642
i = 0
paul@810 643
while i < 10:
paul@810 644
    print seq.next()
paul@810 645
    i += 1
paul@810 646
}}}
paul@810 647
paul@861 648
However, generators make additional demands on the mechanisms provided to
paul@861 649
support program execution. The encapsulation of the above example generator in
paul@861 650
a separate class illustrates the need for state that persists outside the
paul@861 651
execution of the routine providing the generator's results. Generators may
paul@861 652
look like functions, but they do not necessarily behave like them, leading to
paul@861 653
potential misunderstandings about their operation even if the code is
paul@861 654
superficially tidy and concise.
paul@810 655
paul@810 656
=== Positional and Keyword Arguments Only ===
paul@810 657
paul@861 658
When invoking callables, only positional arguments and keyword arguments can
paul@861 659
be used. Python also supports `*` and `**` arguments which respectively unpack
paul@861 660
sequences and mappings into the argument list, filling the list with sequence
paul@861 661
items (using `*`) and keywords (using `**`).
paul@810 662
paul@810 663
{{{#!python numbers=disable
paul@810 664
def f(a, b, c, d):
paul@810 665
    return a + b + c + d
paul@810 666
paul@810 667
l = range(0, 4)
paul@810 668
f(*l) # not permitted
paul@810 669
paul@810 670
m = {"c" : 10, "d" : 20}
paul@810 671
f(2, 4, **m) # not permitted
paul@810 672
}}}
paul@810 673
paul@861 674
While convenient, such "unpacking" arguments obscure the communication between
paul@861 675
callables and undermine the safety provided by function and method signatures.
paul@861 676
They also require run-time support for the unpacking operations.
paul@810 677
paul@810 678
=== Positional Parameters Only ===
paul@810 679
paul@861 680
Similarly, signatures may only contain named parameters that correspond to
paul@861 681
arguments. Python supports `*` and `**` in parameter lists, too, which
paul@861 682
respectively accumulate superfluous positional and keyword arguments.
paul@810 683
paul@810 684
{{{#!python numbers=disable
paul@810 685
def f(a, b, *args, **kw): # not permitted
paul@810 686
    return a + b + sum(args) + kw.get("c", 0) + kw.get("d", 0)
paul@810 687
paul@810 688
f(1, 2, 3, 4)
paul@810 689
f(1, 2, c=3, d=4)
paul@810 690
}}}
paul@810 691
paul@861 692
Such accumulation parameters can be useful for collecting arbitrary data and
paul@861 693
applying some of it within a callable. However, they can easily proliferate
paul@861 694
throughout a system and allow erroneous data to propagate far from its origin
paul@861 695
because such parameters permit the deferral of validation until the data needs
paul@861 696
to be accessed. Again, run-time support is required to marshal arguments into
paul@861 697
the appropriate parameter of this nature, but programmers could just write
paul@861 698
functions and methods that employ general sequence and mapping parameters
paul@861 699
explicitly instead.