OT 2003 Conference [Cambridge, UK]

  Invited Presentor, Plenary Talk — April 1, 2003

Title: Issues in Just In Time Software Composition and their Fundamental Relationship with Scripting Languages

Speaker: David Simmons, SmallScript Corp

Abstract: David Simmons, the designer of the S# language [a optionally typed superset of the Smalltalk-98 language] and its implementations for both .NET and AOS, will discuss driving forces behind the software system composition features within the S# language.

The talk will touch upon some of the basic aspects of S#’s unique selector-namespace facilities for providing dynamic modularized subview constraints on the visibility [within a unit of behavior – a method] of behavioral changes and substructure extensions as well as their implementation technique within S#’s trampoline architecture for caching discrete solutions to dynamic binding predicate and marshalling equations.

Overview:
Dynamic languages [and scripting languages as a subset] are fundamentally about lazy deferral of the implementation of the executable aspects of compositional interdependencies. This approach/philosophy of dynamic language system design has led to specific characteristics in the evolution of their just in time architectures and related virtual machine technology. These dynamic language technologies or their implementation patterns have been increasingly recognized and adopted within mainstream platforms to support management of change and composition within complex software systems and their deployment.
One of the underlying principles for dynamic [as opposed to static] languages and their virtual machines [execution technology] is the presumption that source languages are principally for the human expression of program semantics and corresponding compositional/relational contracts. Such language source is compiled into one or more intermediate semantic language [ISL/IL] forms designed for binary distribution and just-in-time machine analysis, composition and compilation [not evaluation/interpretation].

Dynamic language developers [esp. Smalltalk] tend to be encouraged by their language environments to focus on contracts, behavioral collaboration, encapsulation and minimization of exposure to structural representation within source language expressions.

As a result, dynamic languages will often avoid classic issues within change management because execution/implementation-level dependencies are deferred until just-in-time composition occurs. Within the implementation of such languages, this is achieved through IL and object model designs that retain system collaboration semantics and relations.
Dynamic languages offering this [sometimes deferred verification] approach to software design and integration often conflict with more traditional mainstream system and software design views regarding safety and the use of statically typed and composed software construction.

Within a running application system [represented by virtual machines/runtimes and their object systems] the IL is dynamically re-optimized and compiled based on various environmental characteristic changes and heuristic observations. Dynamic language virtual machines will often include mechanisms for persistent caching of optimized IL or native-machine code translations and related heuristics for fast execution in subsequently restarted virtual machines. The structural representation forms and their composition as well as optimization of dependent behavioral aspects are deferred until just-in-time [allowing up to the last instruction support for all such changes] within a running system.

The result is that the typical compilation phases/processes within classic static language architectures are transformed from development/deployment/packaging time whole program analysis and dependency resolution, into recursive incremental stages that are temporally disjoint. The analysis and dependency resolution and specifically the related optimizations for data representation and machine code implementation of algorithms is transformed from deployment time to just-in-execution-time feedback driven compilation and representation models.

How are these dynamic language characteristics relevant to the challenges encountered within software engineering designs today?
The answer can perhaps be found in recognizing parallels between the patterns by which software systems [esp. distributed] are built and designed to interact, and the basic patterns that can be found within dynamic language architectures.
Within such systems we can note a variety of common characteristics.

• First, such systems are generally designed/intended to run and evolve continuously [24x7] in the face of architectural or other changes without requiring shutdown-save/restore of the software or termination/relaunch of its execution.

• Second, such systems are typically designed based on modularized units of software elements composed together through design constraint principles of contracts and collaborative relationships.

• Third, such systems are often designed or used to support requirements for replication, distributed execution, and transactional behavior. This third characteristic introduces a very different and crucial perspective [from mainstream static language design patterns] with regard to issues of robustness, safety, and security.

If we distill out some basic elements from the above three characteristics we find that we are talking about the ability to not only manage continuous compositional change, but minimize the constraints on the structural or behavioral nature of software changes in the face of any unanticipated/unforeseen or unplanned design constraints/elements in the pre-existing software of such systems. This is one reason why concepts of sealed and final attributes should be considered exclusively as design characteristics for collaboration and composition, and not runtime technology requirements.

While it is surely a debated and oft written about topic -- the desire is to have a unified development process that includes smooth integration of the capture and evolution of collaboration relationships and contracts within and between modularized systems of classes [or equivalent behavioral units]. This issue is a matter of designer perspective.

As one is working within a given modularized subsystem, the perspectually interesting view of the system shifts depending on the context one is looking from as filtered by the items one can view from that context within a given subsystem. From our experience exploring and designing software systems we can recognize that the level of effort required to extract these viewpoints determines our ability to understand or design the collaboration and contracts of such a system. During software design [incl. code-writing] we observe a recursive pattern of change evolution and handle it accordingly based on capabilities of the source language(s), toolset(s), skills, and utilized design practices.

What we also observe is that the human pattern of adaptability to temporal change is reflected in parallel within the nature/form of any resultant software module changes. This suggests that the same information a human needs to effect a change to a system and make it work, should be available within a unit of executable code to enable adaptive compositional changes to both the runtime execution system state and the corresponding operations incorporated within the IL or the executable code unit.
What [static] software languages and systems often fail to support is the ability to either adequately express those “rules” of collaboration and relationships and if they do, they almost always compile away [discard] at design/packaging time any such semantic information thus preventing it from being re-applied to adjust system parameters for just-in-time composition within a running system that is being subjected to change.

If the collaboration view of the visible/accessible system as a whole can be expected to change, then language features for composition and interaction need to both reflect this reality and support processes for expressing design and specialization that minimize redundant expression.

Underlying dynamic language execution technology is explicit architecture and semantics for supporting change within a modularized unit of classes and behavior, to minimize the impact of such changes on collaboration between external modularized units of classes and behavior.

• Deployment of software [design] changes in the form of first class modular executable units

• Modularly adding or removing behavior to any aspect of a whole system

• Modularly adding or removing structure (state containers) to any aspect of a whole system.

• Desire to support unlimited [simultaneous] subviews of fine grained constraint over the visibility/access of such changes.

• Current approaches handle this with moderate success through versioning models.

• S# offers versioning and selector namespaces [a multiple-inheritance namespace meta-model]. Enabling dynamic extension and constraint of a whole system, composed from modularized units of behavior [behavior] and structure [constructors], as viewed from within the perspective of a given modular subsystem. Compositional semantics within S#:

• Desire to dynamically [think rules] compose changes based on control flow and data flow throughout an entire program system

• Within most software systems this is a challenging task which requires carefully planned enforcement of encapsulation and bottlenecking of communication conduits.

• S# operates on a pure unified behavioral model down to the per/object instance level

  • All actions/operations are semantically modeled as messages/behavior

  • All execution elements are semantically modeled as objects
    
    
• Threads, contexts, classes/behavior, methods, closures, variables/fields, etc
  
  
  • S#’s managed behavior services provide declarative and imperative operations for dynamic intercession of arbitrary collections of [multiple-inheritance concrete reusable mixin] behavior at any control point. This provides a simplified extensible model for the development of distributed object services, persistence mapping systems, and encompasses the realm of aspect oriented programming mechanisms.

Compositional Semantics within S#:
One of the driving forces behind the design of S#, as an evolution from Mr. Simmons years of work on commercial Smalltalk language system architectures, was to design a unified model for expressing compositional design at the language level and within the underlying runtime system [object model]. A model that would simultaneously support multiple subsystem views enabling the elimination of behavioral and structural characteristic clashes/constraints found in traditional language semantics and execution models.

Within SmallScript Corp’s Agents Object System [AOS] architecture and related VM technology that supports S#, is the concept of “selector namespaces”; perhaps better described as a comprehensive dynamic form of units of compositional/collaborative privilege and permission access often seen in static languages in the more limited and sometimes awkward forms of public, protected, and private.

Selector namespaces provide constraints on the visibility [within a unit of behavior – a method] of behavioral changes and substructure extensions. Selector namespaces offer the ability to dynamically modularize subview constraints within software systems. Selector namespace semantics provide the foundation for S#’s zero-cost security model that supports simultaneous coexistence of multiple dynamically composed sandboxes and their related provisions for capability based security.

The idea and creation of selector namespaces evolved from David Simmons research work from 1994 through 1999 to address scalability in modularly composed systems. The design of S# is a direct outgrowth of this research, and similar ideas can now be seen in some of the 2002 ECMA Standard proposed characteristics for JavaScript 2.0.

Bio: David Simmons has been designing and developing language systems and virtual machines for since the early 1980’s. He was the principal designer and architect for commercial toolset within QKS Smalltalk-91 and its multi-language, multi-threaded execution engine. His most recent work has been the design and development of S# within the SmallScript Language System, a modular multi-threaded platform for dynamic languages. His design work has focused heavily on complexity management, portability, modularity, performance, object models, and meta-object protocol capabilities for supporting a superset of today’s popular programming language features.