ExpressionParser and the ESP scripting language
The class ExpressionParser
is an instance of
ObjectParser
that was originally written to allow strings
that contain expressions to be parsed. These strings might appear
in text fields or as part of a YAML file. ExpressionParser
also provides the implementation for the ESP (Expression Sequence Parser)
scripting language. For ease of learning, the syntax is similar to
that used by ECMA script, although with fewer constructs and some
minor additions and changes, most adopted from Java. The following
sections provide a description of the ESP scripting language
and how it is configured:
- Overview.
- Script Engines.
- Constructors.
- ESP Syntax:
- Iteration.
- Threading.
- Bindings.
- The ESP API.
- Helper Classes.
- Examples.
- Hints.
- Security.
- Implementation Notes.
In the simplest use case, ESP provides a reasonable level of
security. The
constructor for this case is
ExpressionParser(Class...)
,
and for each of this constructor's arguments, the public
static methods whose return type is a double, int, long, boolean,
or void, and whose arguments types are int, long, boolean, Number,
Integer, Long, or Boolean, are made accessible as functions. A
series of built-in classes can also be used as return values or
arguments by default. These are classes defined in the packages
java.util.stream and java.util.function and are listed explicitly
below. Each string that is parsed is either
- an expression, prefaced by an =
character. This use of an = for an expression
that is to be validated is common in spreadsheets, but also
simplifies the use of
ExpressionParser
with YAML by making it easy to distinguish expressions from strings or numbers. - a variable definition. These start with the keyword var, followed by Java identifier, followed by an =.
- a function definition. These start with the keyword function, followed by a Java identifier providing a function name, followed by a parameter list, a { a series of semicolon-separated statements, and finally a }. A statement in a function definition can be either an expression, an assignment statement, or a variable definition, with the variable definition defining a local variable. In a function definition, a semicolon immediately before the final closing brace is ignored (when switching between writing code in Java and ESP, it is easy to type a semicolon at the end of every statement, including the last, as this is always done in Java).
ExpressionParser
's constructor are Math
,
MathOps
, and Functions
.
ESP also provides objects and arrays/lists, using the same notation
as that used by ECMAScript. For more fine-grained control over how
classes are handled, the constructor
ExpressionParser(Class[],Class[],Class[],Class[],Class[])
can be used.
An instance of ExpressionParser can be configured by setting up to five modes:
-
ExpressionParser.setScriptingMode()
. An input string can contain multiple statements, separated by semicolons, and an input string must not contain an initial "=" unlessExpressionParser.setPrefixMode()
is also called. In this mode, if the input contains an initial line (whether terminated with a new-line sequence or not) that starts with the sequence "#!", that line is treated as a comment and is ignored so that an ESP script can be used as an executable script by following "#!" with the path name of an ESP interpreter (a Unix/Linux convention, but possibly used in other systems as well). -
ExpressionParser.setPrefixMode()
. This mode is meaningful when scripting mode is enabled. Prefix mode requires that each top-level statement start with either an '=' or the keywords "var", "function", or "synchronized" followed by "function". The classObjectNamerLauncher
enables this mode. -
ExpressionParser.setImportMode()
. When import mode is set, classes can be added after scripts have been run, provided that two scripts have not been running concurrently. To add a class, use the methodExpressionParser.addClasses(Class...)
. Unless script import mode is set, the methods that are accessible are those whose value types and argument types are already known. WhenExpressionParser.addClasses(Class...)
is called, all the classes provided as arguments are added to the value-type and argument-type tables before determining which methods are accessible. In this case, a class and its superclasses and/or interfaces have to be provided as arguments explicitly. -
ExpressionParser.setScriptImportMode()
. When script import mode is set, the parser will recognize an "import" statement that will allow scripts to specify classes to add using a syntax described below. Import mode must also be set. script import mode looses the restrictions on methods. When script import mode is set andExpressionParser.addClasses(Class...)
is called, superclasses and interfaces are also added to the value-type and argument-type tables, as are the return types and parameter types of methods. The effect is that all of these methods become accessible. -
ExpressionParser.setGlobalMode()
. When global mode is set, the parser will have a predefined variable named "global" that provides various methods described below. When global mode is set, the global object provides access to aReader
and AWriter
, one writer for normal output and one for error output. These readers and writers can be set by using the following methods: The variants whose names end in TL set thread-specific readers and writers. The remainder set thread-independent values. When both are present, the thread-specific values are preferred.
Finally, an instance of ExpressionParser
can manipulate
expression-parser variables and evaluate expressions or statements
by using the following methods:
-
ExpressionParser.exists(String)
— determine if a variable exists. -
ExpressionParser.get(String)
— get the value of a variable. If the variable's value is a "typed null", the constant null is returned. -
ExpressionParser.set(String,Object)
— set the value of a variable -
ExpressionParser.parse(String)
— process a string and return a value.
ExpressionParser
documentation for additional methods,
including ones used to help generate documentation for the classes
that a given instance has available.
Overview
Programmers familiar with Java and ECMAScript can learn how to use ESP with very little effort, and those familiar with ECMAScript and Java can learn to read code written in ESP in a couple minutes: ESP's syntax is borrowed from elements of both languages, with only a few additions: the ?= operator (which assigns a value to a variable if it does not already exist), the ??= operator (which assigns a value to a variable if it does not exist or has the value null), and the sequence var . IDENTIFIER, which tests if the identifier IDENTIFIER exists, and the ### delimiter, which results in previous statements being immediately executed.
The following provides an overview of the differences.
- Variables must be declared or defined before they are use. A declaration or definition starts with the keyword var. For a definition, the keyword var is followed by an identifier, which is followed by either =, ?=, or ??=, which is followed by an expression. For a declaration, the keyword var is followed by a comma separated list of identifiers. A declaration indicates that a variable was defined externally or as a side effect of some other operation.
- A variable definition can use the ?=
operator to provide an expression giving its value for the
case where the variable is not already defined. If the
variable is already defined, the expression is not evaluated.
This allows default values to be readily provided: the ESP
statement
is equivalent to the ECMAScript statementsvar out ?= global.getWriter();
(The scrunner command will run a script with a predefined variable named scripting.)if (typeof(out) == "undefined") { out = scripting.getWriter(); }
- A variable definition can use the ??= operator to provide an expression giving its value for the case where either the variable is not already defined or the variable has the value of null. Aside from an additional test, this operator behave the same as the ?= operator.
- The operator <=> may appear between two variable names and swaps the values.
- Java method references may be used as method or function arguments, and may be assigned to variables.
- A class name, followed by ::;, followed by null; denotes the value null but with an associated type. This value may be assigned to variables or used as function, method, or constructor arguments. It in effect casts the value null to a particular type and that type is used in looking up the appropriate Java method or constructor. Null values returned by Java methods are handled in this way as well.
- ESP functions that provided by Java are static methods of Java classes. The name of such a method can be used by itself when unambiguous; otherwise the last components of the class name must appear before the method name, separated from the method name by a period. Only enough of the class name has to be provided for the class to be unambiguous.
- The Java instanceof operator may be used.
The value null, if typed, can result in the
instanceof operator recognizes typed null
values but not untyped null values:
- null instanceof String returns false.
- String::null instanceof String returns true.
- Number::null instanceof String returns false.
- x instanceof String returns true when x has the value String::null and return false when x has the value Number::null and return
- Objects use ECMAScript syntax with some semantic differences. When a method is defined by providing a method name followed by a parentheses-delimited argument list, the reserved identifier this is an implicit parameter for a method and allows access to the method's properties. When the value of a property is a lambda expression (e.g., an unnamed function), there is not an implicit parameter named this.
- Functions and methods may be synchronized by using the
synchronized keyword before a function definition,
a method definition, or a lambda expression. For example,
will define a function namedvar x = 0; synchronized function increment() { x = x + 1 }
increment
that can be used concurrently in multiple threads to increment the variablex
. The function will be synchronized on the expression parser itself. Similarly
will define a synchronized method that will modify the propertyvar obj = { x: 0, synchronized increment() { this.x = this.x + 1 } }
x
belonging to the objectobj
. - The symbol `, followed by opening curly bracket, a series of statements, and a closing curly bracket represents a lambda expression with no arguments that is immediately called. If the backquote immediately follows the operators && or ||, the expression always returns true.
- The expression var.IDENTIFIER tests if a variable named IDENTIFIER exists.
- The keyword void can be used as the final expression in a function's body. This is equivalent to null but indicates that the function may not be called to provide an argument for a function or method.
- At the top level of a script, the token ### indicates that portion of the script preceding this token is to be processed before continuing. Otherwise ESP will read the whole script before actually executing anything. Using ### is convenient when variables are defined as a side effect of some operation: if that operation has not yet been performed, an undefined-variable error can occur.
- There is an explicit syntax for importing variables (ECMAScript leaves that to each implementation).
- There is a global object named global, but in ESP, this object merely provides some standard methods.
- Operator precedence and associativity is the same in ESP as in Java, but there are fewer operators. Unlike ECMAScript, there is no === operator. The ESP operator == works like the equal method in Java (or == for primitive types).
- A class name followed by .class denotes a
class object (i.e., an instance of
Class
). Arrays are not allowed (for example, double[].class), but the primitive classes double.class, long.class, int.class and boolean.class are allowed. - A class name followed by .class, followed by a sequence of one or more pairs of opening and closing brackets denotes a multidimensional array, with the number of pairs indicating the number of dimensions. For example, double.class[]) denotes a one-dimensional array of double, and String.class[][] denotes a two-dimensional array of String.
- ESP objects are not much more than dictionaries. Unlike ECMAScript, there is not a prototype chain, explicit constructors, inheritance, etc. ECMAScript keywords such as class or let are not supported by ESP. ESP objects can, however, be used to implement Java interfaces.
- ESP does not provide any control-flow constructs other than a
ternary operator (? followed by :),
?= assignment, and ??= assignment. For
iteration, ESP makes use of the
java.util.stream
package.
The Script Engine
The BZDev class library contains an script engine that implements ESP. This script engine has the following properties:- the script engine name is "BZDEV ESP Engine".
- the recognized file name extension is ".esp".
- the scripting-language name is "ESP".
- the media type for an ESP file is "text/esp".
- besides "ESP", the name "esp" is also recognized as an alias.
- the script engine automatically recognizes the following classes as return types for both methods and constructors): java.io.Reader, java.io.PrintWriter, java.io.Writer, java.nio.CharBuffer, java.util.Set, java.lang.Object.
- the script engine automatically recognizes the following classes as argument types: java.lang.Object, java.lang.Class, org.bzdev.ScriptingContext, org.bzdev.ExtendedScriptingContext, java.io.Reader, java.io.PrintWriter, java.nio.CharBuffer.
Constructors
ExpressionParser
has two constructors:
-
ExpressionParser(Class...)
creates anExpressionParser
configured for parsing expressions with predefined functions. The functions are static methods of the argument classes. The type of the value returned by a method and used as arguments are strings, booleans, numbers (doubles, integers, and long integers) and streams-related classes. -
ExpressionParser(Class[],Class[],Class[],Class[],Class[])
creates anExpressionParser
with explicit values for- the classes of objects that can be returned by functions and methods, and the types of the objects that are created by explicitly called constructors.
- the classes of objects that can be used as arguments to functions, methods, or constructors.
- The classes whose static methods can be used to as functions.
- The classes whose instance methods can be used.
- The classes whose public, static, and final fields may be used. The values of these fields must be an enumeration constant, a string, or the primitive type boolean, int, double, or long.
The method ExpressionParser.addClasses(Class...)
will
behave as if each class in its argument list was added to each
class arrays in the
ExpressionParser(Class[],Class[],Class[],Class[],Class[])
constructor. unless ExpressionParser.setImportMode()
was
called, calls to to ExpressionParser.addClasses(Class...)
must be made before a script is run.
After the constructor is called, more classes can be added by using
the method ExpressionParser.addClasses(Class...)
. Typically
one should use a single call to this method rather than multiple calls:
In determining which methods are accessible, the allowed return types
and argument type are extended to include these classes, and that is
done before tables for methods and constructors are extended.
ExpressionParser.addClasses(Class...)
must not be called after
a script has executed an expression, however, unless
ExpressionParser.setImportMode()
has been called before any
script has been executed.
ESP Syntax
ESP's syntax is described in the following subsections:
- Comments.
- Reserved Words
- Primitive Values
- Constants
- Operators
- Statements
- Expressions
- Variable Definitions
- Variable Declarations
- Function Definitions
- Assignment Statement
- Import Statement
- Objects
- Arrays
- The global object
- Tracing Options
Comments
Two types of comments are recognized in scripting mode. One starts with the token '//' (unless within a string), and continues until the end of a line or the end of the input. The other starts with the sequence '/*' and ends with the sequence '*/'. This is the same convention used by Java and ECMAScript.
If scripting mode is not in effect, a comment must not appear before
- an '=' at the start of a top-level expression.
- the first '=' in a variable definition.
- the starting '{' in a function definition.
Reserved Words
ESP has a small number of reserved words:
- function. This keyword is used for lambda expressions and function definitions. It may be used in fully qualified names but should not be used as an identifier.
- null. This is a constant listed below, but may appear in some import statements.
- this. This keyword is used in methods.
- var. This keyword indicates that a new variable is being defined. When followed by a period and a variable name, it tests if that variable has been defined.
Primitive Values
ESP recognizes several types of primitive values:
- Integers. These use Java syntax but with no support for octal or hexadecimal numbers.
- Long Integers.These use Java syntax but with no support for octal or hexadecimal numbers. If an integer is within the range allowed for an int, it will normally be interpreted as an 32-bit integer. Adding an 'L' to the end of the number, with no intervening whitespace, will result in a 64-bit integer.
- Double-precision numbers.
- Booleans. The allowed values are true or false
- Strings. These are double-quoted strings
with '\' indicating the start of an escape
sequence. A '\' followed by a character is replaced
with that character with the following exceptions:
- \b is replaced with a backspace.
- \f is replaced with with a formfeed.
- \n is replaced with with a new-line.
- \r is replaced with with a carriage-return.
- \t is replaced with a tab.
- \u followed by 4 hexadecimal digits is replaced with the corresponding UNICODE character.
- Method references. A method reference has the same syntax and meaning as in Java and can be used as an argument for Java functions that accept method references. The syntax is shown below. There are some constraints (please see Expressions).
- null. This is just the value null as used by Java, ECMAScript, and various other languages.
- Casted nulls. A class name, followed by ::, followed by null is treated the same as null but with the value tagged with the type given by the class name. This is useful in cases where one would otherwise have to cast a null value to select the desired method or constructor.
- void. This value is treated like null, but indicates that a value is not appropriate as the argument to a function of method. It will typically be used as the last statement in a function or method that does not return a useful value. If the value returned by a 'parse' method is void, that value is converted to null.
Constants
Constants in ESP include numbers and strings as one would expect,
but also enumeration constants and the fields of Java objects that
are declared to be public, static, and final, and whose type has
been either imported or configured using one of
ExpressionParser
's constructors.
The full form for an enumeration constant or field is a fully qualified class name, followed by a '.' and the constant itself. The leading components of the class name (that is, anything before and including a period) can be dropped as long as the resulting name is not ambiguous.
Variable names may shadow a constant. For example, the following script
will generate the following output:import (java.lang.Thread.State); global.getWriter().println(BLOCKED); var BLOCKED = 10; global.getWriter().println(BLOCKED); global.getWriter().println(State.BLOCKED);
because the variableBLOCKED 10 BLOCKED
BLOCKED
shadows the constant
BLOCKED
, but the variable is defined after the
first output line was printed. Meanwhile, the use of the constant
State.BLOCKED
resolves the ambiguity.
Operators
Operators are a subset of the operators provided by Java. Unary operators are '-', '~' and '!' and 'throw' (with a String as an argument). These operators provide sign inversion, bitwise complement, logical-not, and throwing an exception respectively. These have the highest precedence. For the sign inversion operator, if the following token is 9223372036854775808 or 9223372036854775808L, the sign inversion will be ignored and the number will be replaced with the long integer -9223372036854775808 (the minimum value for a long integer). Similarly, if the sign inversion operator is followed by the token 2147483648 (but not 2147483648L), the sign inversion operator will be ignored and the number will be replaced with the integer -2147483648.
The binary operators, grouped by precedence are shown in the following table:
Operator(s) | Description |
---|---|
* / % %% | multiplication, division, mod, and mod as used in mathematics respectively |
+ - | addition, subtraction respectively, with '+' overloaded to handle string concatenation, in which case at least one argument must be a string. |
<< >> >>> | left-shift, right-shift, or unsigned right-shift respectively. |
> < >= <= instanceof <=> | greater-than, less-than, greater-than- or-equal-to, equal-to instance-of, or swap respectively |
==, != | equals and not equal respectively |
& | bitwise AND |
^ | bitwise exclusive OR |
| | bitwise OR |
&& | logical AND |
|| | logical OR |
The swap operator <=> is specific to ESP and exchanges the values of its arguments, which must be two identifiers representing variables (e.g., identifiers defined using the var statement or by the argument list for a function or lambda expression). The value returned by the swap operator is always true. For addition, subtraction, multiplication, and division, the type of the result of binary operation depends on the type of the operands. The normal behavior is shown in the following table:
Left Operand | Right Operand | Result |
---|---|---|
int | int | int |
int | long | long |
int | double | double |
long | int | long |
long | long | long |
long | double | double |
double | int | double |
double | long | double |
double | double | double |
If the result should be an int, but is out of range, it is promoted to a long, and if the value cannot be represented as an integer, it will be replaced with a double. For example, the value of 2000000000 + 1000000000 in ESP is the long integer 3000000000, whereas in Java, it is -1294967296 (because of Java's use of 32-bit two's complement arithmetic).
There is one ternary operator: ? with :. This is a conditional construct as in Java and has a precedence below that of the binary operators. For this ternary operator, the expression before the question mark is always evaluated. If that value is 'true', the expression after the question mark, and terminating just before the colon, is the value, with the expression after the colon skipped. If that value is 'false', the expression between the question mark and the colon is skipped and the expression after the colon is the value. This ternary operator can be nested without the use of parentheses: which question mark is paired with which colon can be unambiguously determined by incrementing a 'depth' when a question mark is seen and decrementing it when a colon is seen. This operator can be used as a replacement for the traditional if-then-else construct by using lambda expressions. For example,
This can be shortened tovar a = ... ; var case = 0; var result = 0; (a < 10)? function() { case = 1; result = a * a; }(): (a < 20) function() { case = 2; result = 2*a; }(): function() { case = 3; result = a + 1; }();
orvar a = ... ; var case = 0; var result = 0; (a < 10)? `{ case = 1; result = a * a; }: (a < 20)? `{ case = 2; result = 2*a; }: `{ case = 3; result = a + 1; };
var a = ... ; var case = 0; var result = (a < 10)? `{ case = 1; a * a; }: (a < 20)? `{ case = 2; 2*a; }: `{ case = 3; a + 1; };
As with the ternary operator, the operators "&&" and "||" skip expressions that do not have to be evaluated to produce the result. Because these operators are left associative,
- the expression x && y will not evaluate y if the value of x is 'false'.
- the expression x || y will not evaluate y if the value of x is 'true'.
var x = 10; var y = 20; (x < y) || function() {x = x * y; true}();
The lambda expression has to return true because the preceding logical operator expects a boolean value, but this can be shortened to
var x = 10; var y = 20; (x < y) || `{x = x * y;};
because `{x = x * y;}
evaluates to
true when it immediately follows ||
or &&.
Statements
Statements are described by the following grammar:
statements: statement | statements ';' statement | statements '###' statement | statements ';' '###' statement statement: expression | var-definition | function-definition | assignment-statement | import-statement
The assignment statement can be used when scripting mode is set, and the import-statement can be used when script-import mode is set. Each type of statement is described in a subsequent section. If a series of statements ends with a semicolon, that final semicolon is ignored (it is easy to type a final semicolon, when alternating between Java, ECMAScript, and ESP).
The token '###' is allowed only in scripting mode and cannot appear within an expression, object, or function definition. It causes the preceding statements to be evaluated before parsing continues. It is provided to handle cases in which variables are defined via side effects: for example, the following script
will generate an error because, while theimport(org.bzdev.anim2d, [Animation2D, AnimationLayer2DFactory, AnimationPath2DFactory]); var out = global.getWriter(); var a2d = new Animation2D(scripting); a2d.createFactories("org.bzdev.anim2d", { alf: "AnimationLayer2DFactory", apf: "AnimationPath2DFactory" }); out.println("list factories created"); out.println(alf);
createFactories
method will define two
variables alf
and apf
, but those
variables are defined when the method is actually run.
ExpressionParser
will notice that there is no previously
defined or declared variable named alf
(because
createFactories
has not yet been called) and generate
an error. The script
will function as desired because the '###' line causes the previous statements to be evaluated, which will result in a variable named 'alf' being defined. While not a requirement, as a stylistic hint, the '###' token should be placed on its own line (to improve readability). One of the use cases for ESP involves configuring simulations (seeimport(org.bzdev.anim2d, [Animation2D, AnimationLayer2DFactory, AnimationPath2DFactory]); var out = global.getWriter(); var a2d = new Animation2D(scripting); a2d.createFactories("org.bzdev.anim2d", { alf: "AnimationLayer2DFactory", apf: "AnimationPath2DFactory" }); ### out.println("list factories created"); out.println(alf);
Simulation
and its subclasses). The run time for simulations can be lengthy,
so parsing a script to check for syntax errors is useful as errors
can be detected faster than if the previous statements were actually
run. If one does not want to use the '###' syntax, an alternative
is to declare some variable in advance. For example,
The disadvantage in using variable declarations in this example is that the declarations and the createFactory call have to be kept in sync. That is trivial when one or two factories are being created, but awkward if there are a sizable number.import(org.bzdev.anim2d, [Animation2D, AnimationLayer2DFactory, AnimationPath2DFactory]); var out = global.getWriter(); var a2d = new Animation2D(scripting); var alf, apf; a2d.createFactories("org.bzdev.anim2d", { alf: "AnimationLayer2DFactory", apf: "AnimationPath2DFactory" }); out.println("list factories created"); out.println(alf);
Expressions
Expressions in general the same meaning as in Java or ECMAScript, although both languages have more operators that ESP does. There are a few exceptions:
- the notation "var . IDENTIFIER" has the value true when IDENTIFIER is the name of a variable that exists. The value is false if the variable does not exist.
- the keyword this is a reference to the current object and is valid only for methods defined for an object. this . IDENTIFIER refers to an object's property with the property name given by the identifier.
- Method references have the same meaning as in Java, but are
represented in ESP by objects. As a result, they can be stored
in variables or returned as the value of a functions. They
can also be called directly, whether from ESP or Java,
by using their "invoke" method. Typically a method reference is
used as an argument that is declared to have type that implements
a functional interface.
In searching for a method, the class of the
value the method returns and the classes of the arguments must be
classes that ESP recognized. These classes are configured by
ESP either via an ExpressionParser constructor, the
ExpressionParser.addClasses(Class...)
method, or an import statement. Because of this, and because ESP is typeless, the choice of an overloaded method may not match the one that Java uses. ESP method references implement the interfaceExpressionParser.ESPMethodReference
, and can be used by Java if desired.
Expressions use the following syntax:
expression: term | '!' expression | 'throw' expression | expression '+' term | expression '-' term | expression '<<' term | expression '>>' term | expression '>>>' term | expression '<' expression | expression '>' expression | expression '>=' expression | expression '<=' expression | expression 'instanceof' qualified-name | expression '==' expression | expression '!=' expression | expression '^' expression | expression '|' expression | expression '&' expression | expression '^' expression | expression '||' expression | expression '&&' expression | expression '?' expression ':' expression term: value | term '*' value | term '/' value | term '&' value | term '%' value | '-' term value: number | true | false | null | void | string | variable | var '.' identifier | qualified-name | qualified-name '.' 'class' | function-call | lambda-expression | lambda-expression '(' arguments ')' | '`' '{' statements '}' | method-call | constructor | object | array | '(' expression ')' | value '[' value ']' | 'this' | 'this' '.' identifier | typed-null | method-reference | class function-call: function-name '(' arguments ')' function-name: identifier | qualified-name '.' identifier arguments: expression | arguments ',' expression lambda-expression: 'function' '(' parameters ')' '{' statements '}' parameters: identifier | parameters ',' identifier method-call: value '.' method-name '(' arguments ')' method-name: identifier constructor: 'new' qualified-name '(' arguments ')' object: [described below] array: [described below] qualified-name: identifier | qualified-name '.' identifier method-reference: qualified-name '::' identifier class: class-name '.' 'class' | class '[&pos; ']&pos; class-name: qualified-name typed-null: qualified-name '::' 'null'
Variable Definitions
Variable definitions always start with the keyword var, and assignments to variables that are not already defined (or being defined) is not allowed. Such declarations can appear at the top level of a script or within functions or lambda expressions. The syntax is
A variable definition defines a variable whose name is given by the identifier following 'var'. The value of the variable for the '=' case is the value of the expression, and an exception will be thrown if the variable has already been defined. For the ?= case, the value of the variable is the value of the expression when the variable does not exist, and is the variable's existing value when the variable already exists. For the ??= case, the value of the variable is the value of the expression when the variable either does not exist or has a null value, and is the variable's existing value otherwise. The ?= and ?= cases are provided so that variables can have default values. For the ?= case, if the variable already exists, the expression will not be evaluated. For the ?= case, if the variable already exists and has a non-null value, the expression will not be evaluated. In either case, the value of the statement is the value assigned to the variable, whether a new value or an existing value. While not necessary, the ??= operator was introduced to handle a common programming convention: setting a value to null to request a default. It makes some scripts easier to read.var-definition : 'var' identifier '=' expression | 'var' identifier ?= expression | 'var' identifier ??= expression
When lambda expressions are nested whether within another lambda expression or within a top-level function, the previously defined variables in all enclosing lambda expressions and any top-level function, are visible. For example
will return the value 2x+20 as both a and b are visible in the inner lambda expression. On the other handvar a = 10; function f(x) { var b = x + a; b + function(y) {y+b+a}(x) }
will result in an exception being thrown because b is not visible in the lambda expression function(y){y+b}. By contrastvar a = 10; function f(x) { b + function(y) {y+b}(x) var b = x + a; }
will work as expected.var a = 10; function f(x) { var b = x + a; b + function(y) {y+a}(x) }
When a var statement occurs inside a function, any variable with the same name is shadowed (i.e., not directly accessible). For example
will return the value x+30 instead of x + 10.var c = 10; function g(x) { var c = 30; x+c }
Variable Declarations
Variable declarations indicate that an identifier represents a variable but does not provide a value for it. The syntax is
A variable that is declared may or may not exist, but existence of a variable can be tested. For example,var-declaration: 'var' identifier | var-declaration '%apos; identifier
will print "foo exists = false" unlessvar foo; println(" foo exists = " + var.foo);
foo
is externally
defined, in which case the output will be "foo exists = true".
These declarations can be used when variables are created as a side
effect of a function or method (an alternative is to use the
###
syntax), or when a variable may be defined
externally (for example, via the scrunner command line).
Function Definitions
Function definitions uses a syntax based on ECMAScript: the keyword function, followed a function name, followed by a parameter list delimited by parentheses, and finally a sequence of statements delimited by curly braces. There is not a return statement. Instead, the last file is the value of the final statement.
The syntax for a function definition is as follows:
function-definition: 'function' identifier '(' parameter-list ')' '{' function-statements '}' | 'synchronized' 'function' identifier '(' parameter-list ')' '{' function-statements '}' function-statements: function-statement | function-statements ';' function-statement function-statement: expression | var-definition | assignment-statement
When the keyword synchronized is present, the function call will be synchronized on the expression parser itself. In a multi-threaded application, this will allow code such as
to work correctly: the variablevar count = 0; synchronized function incr() { count = count + 1; }
count
will record
the number of times incr()
was called. If
incr()
had not been synchronized, two simultaneous
calls to incr()
could read the same value of count,
and set count to one more than that value.
Variable definitions in functions and lambda expressions are lexically scoped. For example, for the script
calling f() will return 10, and calling h() will return 30. Also, unlike variables, a function call can appear before the function is defined, provided that the function call is lexically within a function or method definition or within a lambda expression. Once a function is called outside of a function, lambda expression, or method body, all functions that are called as a result must have been defined.var value = 10; function f() { var value = 20; g() } function g() {value} function h() { var value = 30; value }
Assignment Statement
Assign statements allow the values of previously defined variables (that is, ones defined used a var statement) to be modified. These statements are valid only when scripting mode has been set. In all cases, the assignment operator is "=". The value to the left of the equal sign for a simple assignment is a variable name or the keyword this followed by a period and an identifier that names a field in an object, with the restriction that this type of assignment can be used only within a method defined for some object. If a value is followed by an expression delimited by square brackets, the value must be an ESP array or object. When it is an array, the expression must evaluate to an integer, and when it is an object, the expression must evaluate to a string.
The syntax for an assignment statement is as follows:
assignment-statement: variable '=' expression | 'this' '.' identifier '=' expression | value '[' expression ']' '=' expression
Import Statement
Import statements are recognized when import mode and script
import mode have been set. The syntax is similar to that used by
the methods ExtendedScriptingContext.importClass(String,String)
and
ExtendedScriptingContext.importClasses(String,Object)
,
but where, as a convenience, the package names and class names can
be qualified names instead of strings. If a package name is an
empty string or the token null, that denotes the
unnamed package. The variants with an array are provided for
cases where multiple classes are imported from the same
package. If the methods that import classes that are defined in
ExtendedScriptingContext
are used, a
line containing ### should be inserted at the top
level of the script before the classes defined are used: in a few
cases, the ESP parser will check a type during the
lexical-analysis phase. This is not required when
the import statements described below are used.
The syntax is as follows:
import-statement: 'import' '(' package-name ',' class-name ')' | 'import' '(' package-name ',' '[' class-names ']' ')' | 'import' '(' class-name ')' | 'import '(' '[' class-names ']' ')' | 'import' package-name ',' class-name | 'import' package-name ',' '[' class-names ']' | 'import' class-name | 'import '[' class-names ']' package-name: qualified-name | '"' '"' | 'null' | '"' 'null' '"' | '"' qualified-name '"' class-names: import-class-name | class-names ',' import-class-name import-class-name: class-name | '"' class-name '"'
Whether parentheses are used in an import statement is a matter of style. When there are multiple import statements in a row (that is, without any intervening statement), all are imported at once - the classes will be added to lists of allowed values and allowed arguments, and those lists determine which methods are visible. Public, static methods will be treated as functions in the ESP scripting language. Also, method arguments and return values are added to the lists of allowed values and allowed arguments. When a class is imported, the return types and argument types must match those that are in the current sequence of import statements or a previous statements, but once the methods for a class are imported, those methods cannot be augmented by importing the class a second time.
As a general rule, a class, or a superclass declaring a method must be imported before that method is used. In a few cases, such an import is done implicitly. For example, when the following script
is passed toimport(org.bzdev.anim2d, [Animation2D, AnimationLayer2DFactory, AnimationPath2DFactory]); var out = global.getWriter(); var a2d = new Animation2D(scripting); var alf, apf; a2d.createFactories("org.bzdev.anim2d", { alf: "AnimationLayer2DFactory", apf: "AnimationPath2DFactory" });
scrunner
, the call to
createFactories will create two new variables
name alf and apf by processing
the object
with the help of a private script. This private script contains the statement{ alf: "AnimationLayer2DFactory", apf: "AnimationPath2DFactory" }
which allows methods defined byimport(org.bzdev.obnaming, [ObjectNamerOps, NamedObjectFactory]);
ObjectNamerOps<TO extends NamedObjectOps>
to be used, including
those that return an instance of
NamedObjectFactory<F extends NamedObjectFactory<F,NMR,NMD,OBJ>,NMR extends ObjectNamerOps<NMD>,NMD extends NamedObjectOps,OBJ extends NMD>
. If the class
NamedObjectFactory<F extends NamedObjectFactory<F,NMR,NMD,OBJ>,NMR extends ObjectNamerOps<NMD>,NMD extends NamedObjectOps,OBJ extends NMD>
is not included, a
method that is needed will not be accessible. Other classes automatically
imported by scrunner
are
FileAccessor
and
DirectoryAccessor
as objects with these classes can
be created via the command line an added to scripts.
Another constraint involves threading. When import statements are
processed, multiple threads cannot be processing scripts. This
constraint is imposed for performance reasons: internally
ExpressionParser
maintains tables used to look up Java methods,
and those tables are not thread safe when the tables are being modified.
The tables are modified, however, only while import statements and
methods such as ExpressionParser.addClasses(Class...)
are
being executed. This is not a significant limitation in practice:
typically a top-level script will set everything up, with import
statements occurring first.
Objects
ESP objects are basically dictionaries that map properties to values, and that provide methods to access and modify the values associated with a property. ESP objects also provide methods, both predefined and user defined. User-defined methods are essentially ESP functions but with a hidden argument named this. These methods are defined by providing an identifier, a parentheses delimited parameter list, and a series of statements delimited by curly braces. Properties can be defined by providing either an identifier or a string, followed by a colon that in turn is followed by a value.
The syntax is as follows:
As with functions, the use of the keyword synchronized will cause the method call to be synchronized on the expression parser.object: '{' object-elements '}' object-elements: | object-elements ',' object-element object-element: identifier ':' value | string ':' value | identifier '(' parameter-list ')' '{' function-statements '}' | 'synchronized' identifier '(' parameter-list ')' '{' function-statements '}'
ESP Objects have several predefined methods:
- hasProperty(name). This method returns true if this object has the property name.
- get(name). This method returns the value for this object's property name or null if there is no property with that name.
- propertyNames(). This method returns a
Set
of property names, where each name is represented by a string. - properties(). This method returns a
Map.Entry
, each element of which is a pair whose key is a property name and whose value is the corresponding property value. - put(name,value). This method sets the value of a property whose name is name to value and returns the previous value; null if there is none.
will return the string "property names", whereasvar obj = { propertyNames() {"property names"} }; obj["propertyNames"]();
will return a set of property names.var obj = { propertyNames() {"property names"} }; obj.propertyNames();
On the other hand, if the ESP object (obj) is passed to a Java method as an argument whose type is an interface declared by the Java code
then the user-defined method will be used to implement the Java interface.public interface Foo { String propertyNames(); }
Because ESP methods are implemented as ESP functions with a hidden argument, the following code
will define an ESP function named f and calling f() repeatedly will generate a series of increasing integer values (the classvar obj = { counter: 0, increment() {this.counter = asInt(this.counter + 1)} } var f = obj["increment"];
MathOps
has to be
imported for the function asInt to be accessible.)
Arrays
ESP arrays are similar to ECMAScript arrays and Java objects whose
type is ArrayList
. The syntax is essentially the
same as that used by ECMAScript and some other scripting languages:
An array element can be dereferenced by following the array (include ones provided by expressions) with a [, followed by an expression that evaluates to an index, followed by ]. An array's indices start at 0. This form can be used as the target for an assignment as well. As objects, arrays have the following methods:array: '[' array-elements ']' array-elements: | value | array-elements ',' value
- get(index). This method returns the array element whose index is index.
- set(index,object). This method sets the array element at index to object and returns the previous value.
- add(object). Add a new element to the end of the array and return true.
- size(). return the size of an array.
- stream(). Get a
stream
with this array as its source. - toStream(type). Get
a
stream
of a specified type with this array as its source. The argument type can be int.class, long.class, or double.class, and the classes of the objects returned areIntStream
,LongStream
, andDoubleStream
respectively. - parallelStream(). Get a parallel
stream
with this array as its source. - forEach(action). Apply an action to reach element of the array. The action can be an ESP function that takes a single argument, or a method reference that implements a function with a single argument.
In addition there are several methods that can convert an ESP array to a Java array and vice versa. For ESP arrays, these methods are the following:
- toArray(). Convert an ESP array to a Java array
whose component type is
Object
. - toArray(class). Convert an ESP array to an array whose component type is class (the ESP syntax for the argument is the class name followed by ".class"). Besides non-primitive classes, the primitive classes double, long, int, and boolean are recognized.
- toMatrix(). Convert a nested ESP array
to an n by m Java array whose component type is
Object
. - toMatrix(class). Convert a nested ESP array to an n by m Java array whose component type the type specified by class (the ESP syntax for the argument is the class name followed by ".class"). Besides non-primitive classes, the primitive classes double, long, int, and boolean are recognized.
- toESPArray. Convert a Java array to an ESP array.
- toESPMatrix. Convert an n by m Java array to an ESP array. The Java array must be declared as TYPE[][].
The global object
ECMAScript assumes the existence of a global object, but the name of this object was not specified initially, resulting in implementations using "self", "window", or "global" as typical choices. Finally an the name globalThis was picked, with (as of 2021) some variation in implementation status. ESP uses the name "global", but the ESP global object, unlike the corresponding object in ECMAScript, provides methods but not properties. Furthermore, ESP treats "global" as a reserved word: a statement such as "global = 10.0" will fail. This name was chosen because, given the history, it is not likely to collide with the names of variables that a user would typically choose, simplifying the porting of simple scripts written in ECMAScript to ESP.
While global cannot be used as an explicitly
declared identifier, whether it is present in any particular
instance of ExpressionParser
depends on its configuration:
the method ExpressionParser.setGlobalMode()
must have been
called before a script is run to make the variable
global accessible.
The methods associated with global are
- blockConstructor(clasz,argumentClasses...). For class clasz, this method prevents ESP scripts from using a constructor whose argument classes are given by the remaining arguments. Classes in ESP (as mentioned above) are denoted by a Java class name followed by the string .class. If a constructor takes a variable number of arguments, the argument should be an array type as at runtime Java treats the additional arguments as elements of an array. The method typeForArrayOf described below will create an array type for the specified component type, as will following .class with [] (multiple times for a multi-dimensional array).
- blockImports(). Once this method has been executed, the use of an import statement will result in an error. Because blockImports is a method, it will not be executed until the current file has been parsed or until a line starting with the token ### has been parsed. As a result, an import statement that occurs after a call to blockImports, but before the ### token or the end of the current file, will still be processed.
- blockMethod(clasz,name,argumentClasses...). For class clasz, this method prevents ESP scripts from using a method whose name is the string name and whose argument classes are given by the remaining arguments. Classes in ESP (as mentioned above) are denoted by a Java class name followed by the string .class. If a method takes a variable number of arguments, the argument should be an array type as Java treats the additional arguments as elements of an array. The method typeForArrayOf described below will create an array type for the specified component type, as will following .class with [] (multiple times for a multi-dimensional array).
- ESPObjectType(). This method returns the Java class for expression-parser objects.
- ESPArrayType(). This method returns the Java class for expression-parser arrays.
- exists(objectName). This method returns true if the global namespace contains an object with the name given by this method's argument; false otherwise.
- finishImport(). A sequence of classes to be imported has ended. This should be called after a series of calls to importClasses.
- generateDocs(w,list).
This method uses a
PrintWriter
or anOutputStream
(which will be replaced with a print writer) w as the output for an HTML document describing the Java classes, interfaces, enums, methods, and fields, that ESP recognizes given the current set of explicit and implicit import statements. The argument list is an ESP list of strings, each providing URLs or file names for the corresponding javadoc-generated API documentation, with the URLs or file names using the syntax described in the documentation forURLPathParser
. This method is provided primarily for debugging, but may have other uses. The value it returns, ESP's value void), should be ignored. - get(objectName). This method returns the object with the name specified by this method's argument and stored in the global namespace; null if there is no such object.
- get(objectName,property). This method returns a property value for the object whose variable name is given by by this method's first argument and whose property name is given by this method's second argument.
- getReader(). This method returns a reader; null
if one is not configured. The reader is configured by calling
ExpressionParser.setReader(java.io.Reader)
orExpressionParser.setReaderTL(java.io.Reader)
. - getWriter(). This method returns a writer; null
if one is not configured. The writer is configured by calling
ExpressionParser.setWriter(java.io.PrintWriter)
orExpressionParser.setWriterTL(java.io.PrintWriter)
. - getErrorWriter(). This method returns a
writer for error output; null if one is not configured. The
writer is configured by calling
ExpressionParser.setErrorWriter(java.io.PrintWriter)
orExpressionParser.setErrorWriterTL(java.io.PrintWriter)
. - globals(). This method returns an ESP array
containing the names (or keys) of objects stored as global bindings.
The global bindings are represented by a map that can configured by
calling
ExpressionParser.setGlobalBindings(java.util.Map)
, which supports the API provided in the packagejavax.script
. - getGlobal(key). This method returns an object stored in the global bindings with a name given by key, null if there is no entry in the global table.
- getGlobal(key,defaultObject). This method returns an object stored in the global bindings with a name given by key, defaultObject if there is no entry in the global table.
- importClasses(pkg,spec). This method imports classes from package specified by the argument pkg, which must be a string containing a fully-qualified package name. The classes from this package that are imported are the ones specified by the second argument spec, which is either a string containing the qualified class name of a class within the package, or an ESP array whose elements are strings containing the qualified class names of classes within the package. Multiple calls to this method may occur in a row, and must be followed by a call to finishImport().
- isArray(object). This method returns true if its argument is an expression-parser array or a Java array; false otherwise.
- isESPArray(object). This method returns true if its argument is an expression-parser array; false otherwise.
- isJavaArray(object). This method returns true if its argument is a Java array; false otherwise.
- isObject(object). This method returns true if its argument is an expression-parser object; false otherwise.
- newJavaArray(clasz,dimension...). This method creates a new Java array, possibly multidimensional, whose component type is provided by the argument clasz and whose dimensions are given by the remaining arguments (there must be at least one dimension provided). A maximum of 255 dimensions may be provided (this limit is set by Java).
- typeForArrayOf(clasz). This method returns the class of a Java array whose element type is clasz.
- typeof(object). This method returns the Java class of the object provided by its argument; null if the argument is null
- set(objectName,value). This method sets the global value corresponding to this method's first argument to the value specified by this method's second argument, and returns either the previous value or null if there is no previous value.
- set(objectName,property,value). This method sets the property specified by this method's second argument to the value specified by this method's third argument for the object specified by this method's first argument
- size(object). This method returns the size of the object provided as this method's first argument, provided that the object is an expression-parser object, an expression-parser array, or a Java array. An exception will be thrown otherwise.
Directives for Tracing
ESP supports 4 directives that configure tracing. These can appear anywhere that a token can appear, but are ignored if inside comments or strings. The directives are
- #+T — this directive turns token-tracing on. A description of each token will appear on standard error when token tracing is enabled. Tokens are shown as they appear when processed. During parsing, some tokens are combined: for example, function names are treated as the value for a function keyword, and the bodies of functions are not shown when a function is defined. This directive is thread-specific, so it will not apply to ESP statements that appear in a different thread.
- #-T — this directive turns token-tracing off in the current thread.
- #+S — this directive turns stack-tracing on. Tokens and values pushed and popped from the operator stack and value stack respectively are printed to standard error. This directive is thread-specific, so it will not apply to ESP statements that appear in a different thread.
- #-S — this directive turns stack-tracing off. In the typical case, ESP will push a token onto the operator stack but not evaluate it until the next token going onto the operator stack is available. This delay allows operator precedence to be handled. As a result, one should be careful about where stack tracing is disabled. A useful trick is to place a This directive is thread-specific, so it will not apply to ESP statements that appear in a different thread. ### token before a #-S directive in order to force all previously pushed operators to be processed. Note that ### token can appear only at the top level of a script.
Iteration
ESP does not provide any explicit iteration constructs, other than
recursion. Java, however, provides the packages java.util.stream
and java.util.function
, and ESP allows lambda expressions or
functions to be passed as an argument to Java methods when the type
of the argument is expected to be a functional interface.
ESP is preconfigured so it can use classes and interfaces from the
packages java.util.stream
and java.util.function
.
The classes and interfaces from the java.util.stream
package
are the following:
java.util.function
package
are BiConsumer
,
BinaryOperator
,
Consumer
,
DoubleBinaryOperator
,
DoubleConsumer
,
DoublePredicate
,
DoubleSupplier
,
DoubleUnaryOperator
,
Function
,
IntBinaryOperator
,
IntConsumer
,
IntPredicate
,
IntSupplier
,
IntUnaryOperator
,
LongBinaryOperator
,
LongConsumer
,
LongPredicate
,
LongSupplier
,
LongUnaryOperator
,
Predicate
,
Supplier
,
ToDoubleBiFunction
,
ToDoubleFunction
,
ToIntBiFunction
,
ToIntFunction
,
ToLongBiFunction
,
ToLongFunction
,
and UnaryOperator
.
As an example, the following script will add the numbers between 1 and 1000000.
The function asLong (which is defined inimport(org.bzdev.lang, MathOps); LongStream.rangeClosed(1, 1000000) .reduce(0, function(x,y) {x + y})
MathOps
,
may be needed to force a value to have a specific type.
If a class has methods with the same name that accept either int or long
arguments, one can preface an integer constant with the letter 'L',
which will force the value to be interpreted as a long. For example,
The use of the 'L' notation is mostly for clarity: ESP will convert 32-bit integer values to long values when needed for method or function calls. An explicit choice might be needed if methods are overloaded.import(org.bzdev.lang, MathOps); LongStream.rangeClosed(1, 1000000) .reduce(0L, function(x,y) {20L})
As a second example,
will produce a sum of the absolute values of the numbers inimport (java.lang, Math); var list = [10, -20, 30, 40, -50, 60]; list.toStream(int.class).map(Math::abs).reduce(0, function(x, y) {x + y});
list
. The use of toStream(int.class)
is
necessary so that the type of map
's argument will be
a functional interface that the method reference can implement. The
script
will result in an error: whileimport (java.lang, Math); var list = [10, -20, 30, 40, -50, 60]; list.stream().map(Math::abs).reduce(0, function(x, y) {x + y});
list.toStream(int.class)
provides an instance
of IntStream
, list.stream
provides an
instance of Stream<Object>
, and while each have
a map
method, those methods have different signatures.
Threading
The bindings ExpressionParser
uses
by default are thread-safe—they use synchronized
maps—but this is not sufficient to ensure that a multi
threaded applications work correctly. If exclusive access is
needed while changing multiple global variables or while reading
one and then replacing it, use synchronized functions or methods
as described above.
When using Simulation
or any of its
subclasses (for example, @link org.bzdev.anim2d.Animation2D or
DramaSimulation
), synchronization is not
needed: while multithreaded for convenience, the simulation
classes allow a single simulation thread to run at a time. Since
these threads can pause, using synchronized functions or methods
creates a risk of a deadlock.
If a Java class is not thread safe, one should consider accessing
it only within a synchronized function or method if the application
is multithreaded. Again, this is not an issue for subclasses
of Simulation
.
Bindings
Bindings are provided primarily to support the Java Scripting API.
For ExpressionParser, a binding is simply a map whose keys are
instances of String
and whose values are
instances of Object
. For multithreaded applications,
these maps must be synchronized maps.
The method ExpressionParser.parse(String,Map)
allows a
script to be evaluated with a temporary set of bindings. Multiple
threads may call this method concurrently: thread local variables
are used so that bindings used in different threads do not
interfere with each other. By contrast, bindings set with the
method ExpressionParser.setBindings(Map)
will apply to
all scripts being run.
When a function or method is executed, the bindings it uses are the bindings in effect when the function or method was defined. As an example, consider the variable and function
defined above, and suppose this code was run when the bindings werevar count = 0; synchronized function incr() { count = count + 1; }
bindings1
. If the function incr
is
copied or moved to a new set of bindings, bindings2
,
then calling incr()
when the current bindings is set
to bindings2
will modify the value of count
in bindings1
.
The ESP API
The data structures that ESP uses to represent functions, objects, and arrays can be used directly in Java programs.The ESPArray class
The ExpressionParser.ESPArray
class is a
subclass of JSArray
and provides no
additional public methods. An ESPArray and a JSArray are, however,
treated differently internally.
The ESPFunction class
The class ExpressionParser.ESPFunction
provides
provides a method
ExpressionParser.ESPFunction.invoke(Object...)
that will call the corresponding ESP function. For example, the
following script defines a function that computes the square
root of the sum of the square of two values:
Iffunction foo(x, y) {Math.sqrt(x*x + y*y)}
foo
is a java variable containing
foo
's ESPFunction, then
will return the value 5.0.foo.invoke(30.0, 40.0)
This class also provide method
ExpressionParser.ESPFunction.convert(Class)
that will convert this function to an implementation of a functional
interface.
The ESPMethodReference Interface
ESP method references are represented by implementations of the
interface ExpressionParser.ESPMethodReference
. This
interface provides two methods:
-
ExpressionParser.ESPMethodReference.invoke(Object...)
. This method executes the method or constructor that a method reference implements. -
ExpressionParser.ESPMethodReference.convert(Class)
. This method converts a functional interface into an implementation of a functional interface whose type is the class provided as an argument.
The ESPObject class
The class ExpressionParser.ESPObject
is a
subclass of JSObject
.
JSObject
is similar to
Map
, with put, get, and
remove methods to insert retrieve, or remove entries.
The method JSObject.toKeyMap()
is provided so
that template processing can be used to print a representation of
this object. There is one method specific to ESP:
ExpressionParser.ESPObject.convert(Class)
.
This method converts the object into an implementation of an
interface defining a series of methods.
Helper classes
There are a few classes in the BZDev class library that are sometimes needed when writing ESP scripts and that are not obviously related to applications.
-
MathOps
has methods that will convert numbers to numbers of a particular type. For example,import (org.bzdev.lang.MathOps); var start = asLong(10); a2d.scheduleFrames(start, maxFrames);
-
BasicStrokeBuilder
can be used to create an instance ofBasicStroke
. The constructors for that class use floats as arguments, and ESP does not support floats, only doubles. For example,import (java.awt.BasicStroke); import (org.bzdev.obnaming.misc.BasicStrokeBuilder); var stroke = (new BasicStrokeBuilder()) .setWidth(2.0) .createStroke();
-
Colors.getColorBysRGB(double,double,double)
andColors.getColorBysRGB(double,double,double,double)
can be used to represent sRBG colors, by converting double values to the corresponding float values used byColor
constructors.
Examples
The following examples illustrate how to use ESP.
Iteration Example
Iteration in ESP, as mentioned above, is based the Java package
java.util.stream
. The following example uses two
package from the BZDev class library: org.bzdev.math.rv
for random variables and org.bzdev.math.stats
for a
class that computes the mean and standard deviation of a sequence of
numbers. The implementation is straightforward:
The first step is to import two classes:import(org.bzdev.math.stats, BasicStats.Sample); import(org.bzdev.math.rv, GaussianRV); var out = global.getWriter(); var rv = new GaussianRV(10.0, 1.0); var stats = new BasicStats.Sample(); rv.stream(1000000).forEach(function (x) {stats.add(x)}); out.format("mean = %g, sdev=%g\n", stats.getMean(), stats.getSDev());
BasicStats.Sample
, which computes
the mean and standard deviation for a representative sample of a data
set (in this case, an infinite sequence of random value), and
GaussianRV
, which provides an infinite
sequence of random numbers with a Gaussian distribution. Then a
Gaussian-distributed random variable with a mean value of 10.0 and
a standard deviation of 1.0 is created, and an object named
stats
that computes means and standard deviations.
The script uses a stream with one million entries to compute the
statistics, and these values are printed at the end.
Because the lambda expression merely calls a method with the same arguments, a much more efficient implementation is possible:
The use of a method reference instead of a lambda expression will improve execution time by roughly a factor of 10 in this case.import(org.bzdev.math.stats.BasicStats.Sample); import(org.bzdev.math.rv, GaussianRV); var out = global.getWriter(); var rv = new GaussianRV(10.0, 1.0); var stats = new BasicStats.Sample(); rv.stream(1000000).forEach(stats::add); out.format("mean = %g, sdev=%g\n", stats.getMean(), stats.getSDev());
Typed-Null example
Suppose there is a Java class defined as follows
The ESP statementpublic class NullTest { int value; public NullTest(Number n) {value = 10;} public NullTest(String s) {value = 20;} public int getValue() {return value;} public void set(Number n) {value = 10;} public void set (String s) {value = 20;} public static Number getNullNumber() {return null;} public static String getNullString() {return null;} }
is ambiguous as there are two constructors that could be used. A similar issue exists when thevar nt = new NullTest(null);
set
method is used.
With a null argument, each of the following will create a NullTest
object with a value of 10 or set an existing NullTest object's value
to 10:
Similarly, with a null argument, each of the following will create a NullTest object with a value of 20 or set an existing NullTest object's value to 20:var nt = NullTest(Number::null); nt.set(Number::null); nt.set(NullTest.getNullNumber());
Providing a type for null does not affect equality:var nt = NullTest(String::null); nt.set(String::null); nt.set(NullTest.getNullString());
(Number::null == String::null)
has the value
true
.
System of Linear Equations
A system of linear equations can be written a Ax = y where A is a matrix and x and y are vectors. One can create the vectors and matrices as follows:
import (org.bzdev.math.LUDecomp); var y = [5.0, 10.0, 20.0].toArray(double.class); var A = [[10.0, 70.0, 3.0], [ 7.0, 20.0, 4.0], [ 3.0, 4.0, 30.0]].toMatrix(double.class); var lud = new LUDecomp(A); var x = lud.solve(y);
The methods toArray
and toMatrix
are
needed because the constructor and method of the class
LUDecomp
used in the example take Java arrays
as their arguments, both arrays of double
.
To print a vector, one can use the following function:
Thefunction printArray(array) { var out = global.getWriter(); var len = array.size(); IntStream.range(0, len).forEachOrdered(function (index) { (index == 0)? out.print("[" + array.get(0)): out.print(", " + array.get(index)); }); out.print("]"); out.flush(); void }
printArray
function can be used when its argument
is either an ESP array or a Java array.
Use of the <=> Operator
If the following example is placed in an executable file
named ecirc
on a Linux system, the program
scrunner
will be used as an interpreter for
ESP. The program will expect two arguments, both double
precision numbers, and will set the variable a
to the first argument and the variable b
to the
second argument. The program will also print the last ESP
statement executed and will then exit immediately instead
of waiting for threads to terminate. The command line
arguments are the lengths of an ellipse's semi major
and semi minor axes, and the program will print the
ellipse's circumference.
#!/usr/bin/scrunner -sD:a,D:b,E:true,P:true import (java.lang.Math); import (org.bzdev.math.Functions); (a < b) && a <=> b; var ε = Math.sqrt(1.0 - (b*b)/(a*a)); 4.0 * a * eE(ε);
The swap operator was used so that a
will contain
the length of the semi major axis and b will contain the length
of the semi minor axis. The function eE
is the
complete elliptic integral of the second kind as is implemented
by the Java class org.bzdev.math.Functions
.
For illustrative purposes, the variable containing the
eccentricity of the ellipse is represented by a Greek letter:
ESP allows the same characters to appear in identifiers as
Java does.
Drama-Simulation Example
The file TestActor.java, which defines an actor, contains the following:
import org.bzdev.devqsim.*; import org.bzdev.drama.*; import org.bzdev.lang.*; import org.bzdev.lang.annotations.*; import org.bzdev.drama.common.ConditionMode; @DMethodContext(helper="org.bzdev.drama.DoReceive", localHelper = "TestActorDoReceive") public class TestActor extends Actor { static { TestActorDoReceive.register(); } public TestActor(DramaSimulation sim, String name, boolean intern) { super(sim, name, intern); } public void sendNewMsg(Actor dest, long delay) { Object msg = new TestMessage("hello"); send(msg, dest, delay); } @DMethodImpl("org.bzdev.drama.DoReceive") protected void doReceiveImpl(TestMessage msg, Actor source, boolean wereQueued) { System.out.println(" " + getName() + " received msg \"" + msg.getString() + "\" from " + source.getName() + " at time " + getSimulation().currentTicks()); } } }
The annotations allow dynamic methods to be used for processing
messages that an actor receives. The use of these avoids the need
to create a series of if statements that check if a
message has a particular type. See org.bzdev.lang
for
details, particularly the package description link.
An ESP script that can run this script (assume the file name is test.esp) is:
The variablevar err ?= global.getErrorWriter(); var out ?= global.getWriter(); out.println("started script"); import (org.bzdev.drama, [DramaSimulation, DoubleCondition, IntegerCondition DoubleConditionFactory, IntegerConditionFactory]); import ([TestActor, TestActorFactory]); var sim = new DramaSimulation(scripting); var taf = sim.createFactory(TestActorFactory.class); sim.createFactories("org.bzdev.drama", { dcf: "DoubleConditionFactory", icf: "IntegerConditionFactory" } ### var dc = dcf.createObject("doubleCondition", { initialValue: 1.0 }); var ic = icf.createObject("integerCondition", { initialValue: 3 }); out.println(dc.getName() + ": value = " + dc.getValue()); out.println(ic.getName() + ": value = " + ic.getValue()); var a1 = taf.createObject("a1"); var a2 = taf.createObject("a2"); a1.sendNewMsg(a2, 10); var adapter = sim.createAdapter({ simulationStart: function(s) {out.println("simulation started")}, simulationStop: function(s) {out.println("simulation stopped")}, messageReceiveStart: function(s,f,t,msg) { out.println("starting msg " + msg)}, messageReceiveEnd: function(s,f,t,msg) { out.println("ending msg " + msg)}}); sim.addSimulationListener(adapter); a2.addSimulationListener(adapter); sim.run();
scripting
is defined automatically by the
scrunner command. By passing it to the constructor for the simulation,
the simulation can use the scripting environment to perform tasks such
as creating factories. The ### token after the call to
createFactories
forces the previously parsed statements
to be evaluated. This is needed because createFactories
programmatically creates the variables dcf
and
icf
, and these must either exist or be explicitly declared
before they are referenced later in the script. The calls to
createObject
creates two conditions named "integerCondition"
and "doubleCondition", and two actors named "a1" and "a2". The factories
icf
and dcf
have a parameter named
initialCondition
and the script objects provide values for
that parameter. The statement a1.sendNewMsg(a2, 10)
makes actor a1 send a message to actor a2, with that message received
after 10 simulation ticks. The statement sim.run(),
starts
the simulation. Since there is only a single event, the simulation will
halt almost immediately.
The object created by createAdapter
allows a script object
to implement a simulation listener, which is added to the simulation
and one of the actors to instrument the simulation. Such adapters are
useful for debugging.
The following commands will compile the Java code:
mkdir -p classes tmpsrc javac -d classes -classpath /usr/share/bzdev/libbzdev.jar -s tmpsrc *.java
The following command will run the script using the scrunner command:
scrunner --classpathCodebase classes/ test.esp
Animation Example
The following example creates a simple animation.
The list (and nested objects for the "background" object are not shown due to length, but is available in the 'examples' directory's 'layer' subdirectory. The code functions as follows:import(org.bzdev.anim2d, [Animation2D, AnimationPath2DFactory, GraphViewFactory, AnimationLayer2DFactory]); import(org.bzdev.util.units.MKS); import(org.bzdev.io, [FileAccessor, DirectoryAccessor]); var err ?= global.getErrorWriter(); var out ?= global.getWriter(); var tmpdir ?= throw "tmpdir undefined (need scrunner -d tmpdir:TMP option)"; var a2d = new Animation2D(scripting, 1920, 1080, 10000.0, 400); var pathf = a2d.createFactory("org.bzdev.anim2d.AnimationPath2DFactory"); var gvf = a2d.createFactory("org.bzdev.anim2d.GraphViewFactory"); var alf = a2d.createFactory("org.bzdev.anim2d.AnimationLayer2DFactory"); var scaleFactor = 1080.0/MKS.feet(60.0); a2d.setRanges(0.0, 0.0, 0.0, 0.5, scaleFactor, scaleFactor); var GREEN = {green: 255, red: 0, blue: 0}; var BLUE = {red: 0, green: 0, blue: 255}; var DARKGRAY = {red: 96, green: 96, blue: 96}; var LIGHTGRAY = {red: 160, green: 160, blue: 160}; var WHITE = {red: 255, green: 255, blue: 255}; var SQUARE6 = {gcsMode: true, cap: "SQUARE", width: MKS.inches(6.0)}; var vpath = pathf.createObject("vpath", [ {zorder: 10, visible: false}, {withPrefix: "color", config: BLUE}, {withPrefix: "stroke", config: {width: 2.0, gcsMode: false}}, {withPrefix: "cpoint", withIndex: [ {type: "MOVE_TO", x: 1.0, y: 0.0}, {type: "SEG_END", x: 70.0, y: 0.0}]}]); var gv = gvf.createObject("view", [ {initialX: 1.0, initialY: 0.0}, {xFrameFraction: 0.0, yFrameFraction: 0.5}, {scaleX: scaleFactor, scaleY: scaleFactor}, {withPrefix: "timeline", withIndex: [ {time: 0.0, path: vpath, u0: 0.0, velocity: 0, acceleration: MKS.mphPerSec(5)}, {time: 3.0, acceleration: 0.0}, {time: 7.7, acceleration: -MKS.mphPerSec(7.5)}, {time: 9.7, velocity: 0, acceleration: 0}]}]); alf.createObject("background", [...]); tmpdir.listFileAccessors().forEach(function(fa){fa.delete()}); var maxFrames = a2d.estimateFrameCount(10.0); out.println("maxFrames = " + maxFrames); a2d.initFrames(maxFrames, "col-", "png", tmpdir); a2d.scheduleFrames(0, maxFrames); a2d.run();
- The import statements make various classes accessible.
- The statements containing "?=" either provide default values or throw an exception of a needed variable was not defined by scrunner.
- The constructor for the animation is given the scripting environment as its first argument, followed by the number of 'points' for each image in the horizontal and vertical directions respectively, followed by the number of simulation ticks per second, followed by the number of simulation ticks per frame. The ratio of these gives the number of frames per second (25).
- The statements defining the variables
pathf
,gvf
, andalf
create the factories that the animation needs. - The
scaleFactor
definition indicates that a user-space dimension of 1080 points corresponds to a physical distance of 60 feet. - The call to
setRanges
places the origin at the left edge of the frame, half-way up, and the twoscaleFactor
arguments configure the X and Y axes to have the same scale. - The definitions for the variables
GREEN
,BLUE
,DARKGRAY
,LIGHTGRAY
andWHITE
are provided to avoid having to retype the corresponding value repeatedly. - The createObjects method takes two arguments: a string giving a name to the object and a specification of how that object should be configured. In some cases, the values are the results of function calls.
- The statement
removes all the files from the temporary directory to avoid inadvertently mixing images from different runs.tmpdir.listFileAccessors().forEach(function(fa){fa.delete()});
- The sequence of statements
computes the number of frames created by the animation, provides a pattern for names of the files used to store these frames, provides an object (tmpdir) that provides access to a temporary directory, schedules the frames, and finally runs the animation to create the frames.var maxFrames = a2d.estimateFrameCount(10.0); out.println("maxFrames = " + maxFrames); a2d.initFrames(maxFrames, "col-", "png", tmpdir); a2d.scheduleFrames(0, maxFrames); a2d.run();
Hints
ESP is actually rather strict about what types it recognizes. As a result, when importing classes, it is usually necessary to import the classes for the return-value and the arguments of whatever methods (or constructors) are used. This should be an exact match. Furthermore, once a sequence of import statements in complete all of those classes are processed at concurrently. and only methods that match known types are visible. Repeating a class in a later import statement will not change the methods for that class that are recognized. For animations and simulations, importing one should import not only the classes of factory methods, but also the classes of the objects those methods create. To see exactly which methods are available, after the imports one can call the method generateDocs provided by the global object global at the appropriate point in a script.
When scrunner is used a variable named scripting is defined. That variable can also be used to import classes (various ECMAScript engines have different procedures for importing classes, and methods were provided for scripting in order to hide these engine-specific procedures). These methods are
-
ExtendedScriptingContext.importClass(String)
, -
ExtendedScriptingContext.importClass(String,String)
, -
ExtendedScriptingContext.importClasses(String,Object)
, -
ExtendedScriptingContext.finishImport()
.
finishImport
has an empty implementation for
some ECMAScript implementations (Nashorn, GraalVM, etc.) but performs
a series of operations for ESP. If there are problems with importing
Java classes in this way, check that a call to finishImport
is not missing. Also make sure that a sequence of such import
statements is followed by a line containing "###".
ESP also makes little effort at casting values to an appropriate type.
When this is an issue, consider methods from the class
MathOps
. Arithmetical operations in ESP tend
to produce double-precision floating-point numbers. For simulations
and animations there are methods such as
Simulation.run(long)
whose arguments are
long integers. When the value is a constant, an 'L' immediately
after the number will create an integer with the desired type, but
this is typically not necessary as ESP will convert integers to longs
when a long integer is expected.
Finally, one can use the directives #+T, #-T, #+S, and #-S, which are described in the section on tracing, to turn token tracing and stack tracing on and off. This can be useful in determining how ESP is parsing and processing a script. Because these directives can produce significant amounts of output, one would typically use them for small portions of a script.
Security
Originally, ESP was written under the assumption that the Java security manager could be used to prevent scripts from overwriting files that should not be modified, whether accidentally or maliciously. One could use the scrunner command to define file and directory accessors on the command line that would then give users access to specific files or directories, while the security manager would prevent scripts from opening arbitrary files. Then, in the middle of 2021, JEP 411 decreed that the security manager, which had been available since the early 1990s, would be removed from Java. As a result, one should
- Place import statements at or near the start of a script.
- Call the global object's methods blockConstructor and blockMethod to eliminate the use of constructors and methods that might pose a security risk (e.g., using a PrintWriter constructor that opens a file).
- Call the global object's method blockImports (this must follow the calls to blockConstructor and blockMethod) in order to prevent additional classes from being imported (this is useful for restricting the imports that have to be checked manually to a specific file or portion of a script).
- Place the sequence described above in its own file or end the sequence with a line containing the token ###.
ESP automatically imports the class java.io.PrintWriter which contains constructors that can open a file. One would want to block several constructors:
If java.nio.charset.Charset has been imported, one will also needglobal.blockConstructor(PrintWriter.class, String.class); global.blockConstructor(PrintWriter.class, String.class, String.class); global.blockConstructor(PrintWriter.class, String.class, Charset.class);
global.blockConstructor(PrintWriter.class, String.class, Charset.class);
If org.bzdev.anim2d.Animation has been imported, one should call
to prevent scripts from determining where image files will be stored in the file system.global.blockMethod(Animation2D.class, "initFrames", String.class, String.class);
Finally, the global object's method generateDocs can be used, after the import statements, to get a list of all the classes, constructors, and methods that can be used. Since the output is an HTML file containing links to API documentation, it is relatively easy to check methods and constructors. Beyond that, one may have to use containers or other operating-system facilities to enforce a security policy.
Unless the option --unsetScripting is used, The
utility scrunner will provide scripts with a
variable named scripting whose value is an
instance of ExtendedScriptingContext
,
and that class contains methods that can import classes and run
scripts. Unless needed (for example, subclasses of
Simulation
have constructors that accept
a 'parent' and that parent may be a scripting context), it is
safer to use the --unsetScripting options:
computing a script to be executed is one way to obscure that code's
intent. Regardless, the constraints set by calling
global.blockConstructor,
global.blockMethod, and
global.blockImports will apply to the
scripting methods that import classes.
Implementation Notes
ESP was designed to be a rudimentary scripting language, primarily
intended for tasks such as setting up simulations, animations, or other
software that might be based on the org.bzdev.obnaming
package.
Based on experience using ECMAScript (i.e., the Rhino and Nashorn
script engines), scripts tended to be mostly declarative. As an
example, for an animation, one would create the animation, configure
its graph, create animation objects by passing scripting-language
objects/arrays to factories, configure the number of frames and some
output-related data, and then run the animation. In addition there
is typically some simple arithmetic or function calls to determine
various time points based on constraints such as covering a
specified distance with an initial velocity v1 and a final velocity v2.
There were, however, some issues with the ECMAScript
implementations. First, the Rhino and Nashorn implementations had
some annoying differences. For example, a 'print' function in one
added a new line while the other did not. There was also no
standard way of importing Java classes and each of these used a
different procedure. Finally, the BZDev class library's
simulation packages allow one to schedule multiple threads. While
just a single thread is active at any time, an object's methods
(typically methods for "adapter" class) may be called from
different threads at different times. This worked with both Rhino
and Nashorn, but now Nashorn, which replaced Rhino, is being
deprecated. The likely replacement, GraalVM, includes an
ECMAScript implementation, but that implementation seems to have
no way (or at least no portable way) of allowing multiple threads
to use the same bindings. The result is that you cannot create a
script object to use as a Runnable
and schedule
that to run as a thread. ESP does not have any of these issues:
there is a specified syntax for handling imports, the bindings are
thread safe, and objects can be used in multiple threads. A function
defined in one binding can be passed arguments defined in another
binding (a capability Rhino and Nashorn has, but that seems to be
missing from GraalVM).
ESP is a suitable replacement for ECMAScript in cases where
- ESP's language features are sufficient for easily coding an application.
- Scripts will not account for a significant fraction of the application's execution time.
- Stability is important - i.e., one needs to save a script for a long period of time due to it being the input for an application and has to just work without being "maintained" (The intention, at least of ESP's designer, is to resist the natural impulse to add new features and inadvertently modify old ones).
Because of its limited scope, ESP in the BZDev class library has a simple implementation. There is some code, based on the Java reflection API, for mapping java methods to ESP functions and methods, and a parser that is little more than a lexical analyzer. The parser works in two steps. The first turns a script into a series of tokens that are annotated with a "level" used to determine operator precedence. This number replaces a parse tree, which is implicit. For functions or methods, a object (of type ESPFunction) stores a list of tokens representing the body of the function or method, and some additional data such as the names of parameters. When a function is called, its token list is scanned after setting up tables to represent its arguments. Because the only operators that do any sort of branching are the ternary operator (?:), and the logical operators (|| and &&), the flow of control simply move forwards along a queue, possibly skipping subsequences of tokens.
ESP Objects are a subclass of JSObject
and ESP arrays are
a subclass of JSArray
. Neither subclass adds any public
methods. Non-abstract subclasses of
NamedObjectFactory<F extends NamedObjectFactory<F,
can handle script
objects that are instances of JSArray
or JSObject
directly, a capability that was added in order to use a JSON
object or array to configure a factory without needing a script
engine.
Performance
In some cases, ESP is significantly faster than the Nashorn ECMAScript implementation. For example, consider the scripts layer.js and layer.esp in the examples/layer directory. These scripts are set up to be run using the scrunner command. The last lines of these scripts are
var maxFrames = a2d.estimateFrameCount(10.0); out.println("maxFrames = " + maxFrames); a2d.initFrames(maxFrames, "col-", "png", tmpdir); a2d.scheduleFrames(0, maxFrames);If these lines are commented out, the scripts will stop just after an animation and the animation objects it needs are defined and configured. For ECMAScript using the Nashorn script engine, the Linux "time" command reports
when the command "scrunner -d:tmpdir:latmp layer.js" is run. For ESP, the Linux "time" command reports10.74user 0.26system 0:02.81elapsed 390%CPU (0avgtext+0avgdata 297176maxresident)k 2624inputs+128outputs (0major+89038minor)pagefaults 0swaps
when the command "scrunner -d:tmpdir:latmp layer.esp" is run.2.17user 0.12system 0:00.77elapsed 296%CPU (0avgtext+0avgdata 115192maxresident)k 144inputs+128outputs (0major+33513minor)pagefaults 0swaps
There are several factors that contribute to this rather surprising difference in performance:
- ESP's codebase is roughly 5 times smaller than Nashorn's codebase, which allows ESP to be initialized faster than Nashorn.
- The scripts listed above for the most part merely defines objects and arrays, and ESP's implementation of these is significantly simpler than Nashorn, primarily because ESP does not include features such as prototypes.
- ESP's objects and arrays are subclasses of
JSObject
andJSArray
, which were written to support JSON and YAML parsers.NamedObjectFactory<F extends NamedObjectFactory<F,
supports these two classes directly so that JSON and YAML data sets can be used to configure named-object factories when a script engine is not used. As a result, unlike ECMAScript, a "helper script" is not needed to configure a factory.NMR, NMD, OBJ>, NMR extends ObjectNamerOps<NMD>, NMD extends NamedObjectOps, OBJ extends NMD> - The algorithm Nashorn uses for parsing a script are far more complex than ESP's algorithm, which is not much more complex than lexical analysis. This gives ESP an advantage for scripts whose execution time is mostly due to parsing.
- When reading a script ESP's default behavior is to parse the script in its entirety before executing anything. If there is a syntax error in a script, the error will be detected significantly faster than with Nashorn, not only because ESP initializes faster, but also because Nashorn will run the script as it is being read.
One would expect Nashorn (or GraalVM) to have significant
performance advantages over ESP, given its current implementation,
for complex scripts with more substantial running times. A higher
performance implementation could be created by modifying
ExpressionParser.ESPFunction
so that its token queue is
compiled into some data structure that allows more efficient
execution.