Version 3 External Reference Spec

=Overview= Context Free 3 is the next major release of Context Free. Version 3 is motivated by these new features:
 * Parameterized rules and paths. In a nutshell, when you specify a shape replacement in a rule you can pass information to the replacement rule that affects how it looks. There has always been information passed from rule to replacement rule (affine transform, color, etc), but this information has no bearing on how the replacement rule is expanded. It affects how the shapes appear in the bitmap but is outside of the context-free grammar. The new shape parameters are capable of changing how a rule is expanded. This moves Context Free away from context-free grammars in the Chomsky grammar hierarchy to an indexed grammar.
 * The parameters can be any of four types: numeric, natural, transform, and shape.
 * Numeric parameters are just real numbers, or vectors of real numbers. They can be inserted into any expression where a real number is allowed.
 * Natural parameters are non-negative integer numeric parameters.
 * Adjustment parameters are shape adjustments (affine transform, color, etc.). They can be inserted into any shape adjustment or loop adjustment. A new expression-like syntax for shape adjustments has been defined for specifying transform parameters.
 * Shape parameters encapsulate the name of a shape and any parameters that the shape may require. The shape parameter can be directly invoked as a replacement.
 * Random numbers that evaluate to different values each time a rule is expanded.
 * Extensions to expressions:
 * The comma token, used in version 2 to separate function argument, has been elevated to a tuple operator. Multiple numeric expressions can be grouped together with commas to produce tuples. Functions and shape adjustments that take more than one numeric argument (e.g. atan2 or size) can take tuples for the argument.
 * Two new range operators were added that are functionally identical to the new random number function: x .. y is equivalent to rand(x, y), and x +/- y is equivalent to rand(x - y, x + y). The Unicode character ± can also be used, but it must be UTF8 encoded.
 * Natural numbers cannot be subtracted because the result may be negative. A new proper subtraction operator, --, is defined that returns zero if the second operand is greater than the first operand.
 * A full suite of relational and boolean operators is defined: &lt;, &gt;, &lt;=, &gt;=, ==, &lt;&gt; and &&, ||, ^^ (the latter is boolean exclusive-OR). Unicode characters &le;, &ge;, and &ne; can also be used.
 * Definitions: syntax elements that let you bind an expression to a variable. The expression is evaluated and bound to the variable for the scope of the block containing the definition. Definitions outside of rules are "global", they have scope from their position forward, and into any included files. Global definitions inside included files have scope in the including file. If the expression is constant then it is only evaluated once. Global definitions must be constant.
 * Loop variables: Acts like a definition, binding the loop index to a variable. The loop index is bound to a named variable that has scope within the loop body (but not the loop modification). We shall refer to version 2 loops as anonymous loops, as they have no accessible loop variable.
 * Loop finally block: A group of replacements (or path elements) that follow a loop. The replacements/path-elements are executed as a last iteration of the loop. The loop variable has scope within the finally block of the loop.
 * Shape declarations: A syntax for declaring a shape and its parameters. The shape declaration is followed by a shape path or one or more shape rules.
 * Name spaces: including a cfdg file can cause problems if the included file has shapes or definitions with names that conflict with those in the including file. It is now possible to include a cfdg file into its own name space, so that its shapes and definitions remain distinct.
 * Background, Tile and Size directives are going away in version 3. They will be replaced with configuration definitions CF::Background, CF::Tile, and CF::Size.

But indexed grammars can implement context-sensitive grammars. You can't call this Context Free
The intent of rule and path parameters is to increase the expressive power of cfdg designers, not to violate the context-free purity of Context Free. Rule parameters will be subject to limits to prevent context-sensitive grammars. If no limitations are placed on how parameters are used then the feature practically begs to be used for context-sensitive grammars. The rule-of-thumb that we use is that if the results of a new feature (i.e, the string of non-terminals) can be generated without the new feature then the new feature is context-free. When loops were added with version 2 it was still possible to manually unroll the loop and get the same result. The new loop variable and loop finally features are more of the same. The random number feature can be laboriously emulated with rules that random walk. The trick is figure out what restriction on parameters allow emulation with a finite-sized family of rules that don't have parameters:
 * Numeric parameters can be any expression that has no dependency on the incoming parameters.
 * Numeric parameters can depend on an incoming parameter if the incoming parameter is not modified.
 * Natural parameters can be an expression with dependencies on incoming natural parameters. But the expression must meet the requirement for primitive recursive functions.
 * Adjustment and shape parameters are like numeric parameters: they can either be an incoming parameter, unmodified, or they must have no dependence on any incoming parameter.

=Using Parameters= You declare that a rule has parameters by putting a parameter declaration right after the rule name: The keywords number, natural, adjustment, and shape declare the type for the parameter. In addition, there are type keywords vector1, vector2, ... vector8 that indicate a numeric n-tuple of the specified order is to be bound to that parameter.

Numeric parameters
Numeric parameters are floating point numbers. They can be used in any expression within a rule.

Natural Parameters
Natural parameters are numeric parameters that are constrained to be natural numbers (non-negative integers).

Adjustment parameters
There are two ways to use adjustment parameters: they can be inserted into a shape adjustment list using the new transform adjustment (which has the abbreviated form tr). Or the parameter can be directly inserted an expression that has adjustment type.

Shape parameters
Shape parameters are used in place of the shape name in a replacement. In the above example there is no shape named someShape. The someShape parameter is bound to some rule or path name, which is the rule or path used for the shape replacement.

Specifying shape parameters
Shape parameters use a function-like syntax for shapes that take parameters to bind parameters to the shape. Parameters must be bound to the shape in order to invoke it in a replacement and also when creating a rule parameter. If there is a shape beeTemplate that takes two numeric parameters, an adjustment parameter, and a shape parameter then you would invoke it in a replacement like so: The CIRCLE shape does not take parameters, so the parentheses are optional. If you wanted to pass an instance of beeTemplate as a parameter to rule foo then you would do this:

Restrictions on Numeric and Natural Parameters
When specifying numeric or natural parameters in a rule replacement there are certain restrictions designed to prevent context-sensitive grammars. Numeric parameters must not depend on any incoming numeric or natural parameters unless the incoming parameter is used unchanged: A natural parameter specification may depend on incoming natural parameters and may even modify them. But it must be an expression that only contains natural numbers and they may only be combined in certain ways:
 * Incoming natural parameters are natural number of course
 * Non-negative integer constants are natural numbers
 * a + b and a * b are both natural if a and b are natural
 * a -- b is proper subtraction under the naturals, it is the same as a - b if a>b, but is zero if b>a
 * randint(a, b), mod(a, b), and abs(a, b) are natural if a and b are natural
 * factorial(a), sg(a), and divides(a, b) issue an error if a or b is not natural
 * div(a, b) returns the integer part of a/b and issues an error if a or b is not natural
 * min(a, b, &hellip;) and max(a, b, &hellip;) are natural if all the arguments are natural
 * ab, a<=b, a>=b, a==b, a<>b, a&&b, a||b, and a^^b are natural if a and b are natural
 * select(i, a, b, &hellip;) if natural if all a, b, &hellip; are natural, i does not need to be natural

Parameters for path commands and operations
The parameter strings that determine how FILL, STROKE, ARCTO/REL, and CLOSEPOLY are really inelegant as is the use of the shape adjustment syntax for setting the parameters for path operations. Note that transform expressions cannot express parameters or stroke widths, this is by design. Instead the path commands and operations will use parameter syntax: The path operations completely dispense with the shape adjustment syntax while path command retain the shape adjustment, but only use it for actual shape adjustments. Flags are a fourth expression type that are analogous to C/C++ enumerations. The only legal operations on flags is to add them together. Flags are all constant values, there is no option for the user to define more:
 * MOVETO {x xval y yval} would become MOVETO(xval, yval)
 * LINETO {x xval y yval} would become LINETO(xval, yval)
 * ARCTO {x xval y yval xrad xradval yrad yradval r rotval p param} becomes ARCTO(xval, yval, xradval, yradval, rotval, flags)
 * ARCTO {x xval y yval r radval p param} becomes ARCTO(xval, yval, radval, flags)
 * CURVETO {x xval y yval x1 x1val y1 y1val} becomes CURVETO(xval, yval, x1val, y1val, flags)
 * CURVETO {x xval y yval x1 x1val y1 y1val x2 x2val y2 y2val} becomes CURVETO(xval, yval, x1val, y1val, x2val, y2val, flags)
 * CURVETO {x xval y yval} becomes CURVETO(xval, yval, CF_CONTINUOUS)
 * CURVETO {x xval y yval x2 x2val y2 y2val} becomes CURVETO(xval, yval, x2val, y2val, CF_CONTINUOUS)
 * CLOSEPOLY {p param} becomes CLOSEPOLY(flags)
 * FILL {shape_adjustments p param) becomes FILL(flags)[shape_adjustments]
 * STROKE {shape_adjustments p param width stroke_width) becomes STROKE(stroke_width, flags)[shape_adjustments]
 * CF&#58;&#58;None
 * Identity flag. Indicates that no flags are set. This is not really needed in path operations or commands because if there are no flags then the flags parameter can be omitted. But if we decide to allow flag expressions to be shape parameters then it will be needed.


 * CF&#58;&#58;ArcCW
 * Indicates that an ARCTO/ARCREL is to use a clock-wise arc direction.


 * CF&#58;&#58;ArcLarge
 * Indicates that and ARCTO/ARCREL is to use the arc segment that is greater than 180°.


 * CF&#58;&#58;Continuous
 * Indicates that a CURVETO/REL is the continuous form and the (x1, y1) control point is to be computed. The CF&#58;&#58;Continuous flag can be omitted for continuous quadratic bezier curves, Context Free can figure this out from the number of parameters.


 * CF&#58;&#58;Align
 * Indicates that a CLOSEPOLY aligns its end to its beginning.


 * CF&#58;&#58;EvenOdd
 * Indicates that a FILL uses the even-odd filling rule, instead of the non-zero filling rule


 * CF&#58;&#58;IsoWidth
 * Indicates that a transformed stroke is transformed before stroking, instead of afterward. This makes the stroke width the same for all orientations.


 * CF&#58;&#58;MiterJoin, CF&#58;&#58;RoundJoin, CF&#58;&#58;BevelJoin
 * Indicates the segment join type for STROKE.


 * CF&#58;&#58;ButtCap, CF&#58;&#58;RoundCap, CF&#58;&#58;SquareCap
 * Indicates the path end shape for STROKE.

For the STROKE command both parameters are optional, it is OK to just the stroke width or just flags. For all other commands the flags parameter is optional.

Reusing Parameters
If a rule invokes itself or another rule or path with the same parameters then there is a shorthand for reusing the same parameters: use (=) for the parameter list. Not only is this easier than explicitly relisting the parameters, it also speeds up Context Free. Ordinarily shape curve must be invoked with two parameters, but curve(=) indicates that the incoming parameters will be reused. Paths can also reuse parameters when invoking subpaths.

=Random Numbers= The rand and randint functions has the same argument signature as the rand_static function, but it evaluates to a different random value each time it is executed. The rand function cannot be used in the global scope, use rand_static instead. When rand is used in the body of a loop it evaluates differently for each iteration of the loop. If it is used for the loop index parameters or the loop modification then it is evaluated once at the beginning of the loop. randint is exactly equivalent to floor(rand) and is provided to help keep things simple. Two new operator have been defined that are syntactic sugar for rand. The output range rand, randint, and rand_static are all half-open intervals. rand(a, b) has an output range of [a, b) if a&lt;b and (b, a] if a&gt;b
 * x .. y is equivalent to rand(x, y)
 * x +/- y and x ± y are equivalent to rand(x-y, x+y)

=Expression Extensions= In addition to the two random operator mentioned above a new comma operator (,) has been added in version 3. The comma operator acts as a cons operator, binding together a list of numeric or transform expressions into a numeric or transform tuple. Various shape adjustments and functions have been redefined in term of tuples:
 * atan2 and mod take a 2-tuple or pair
 * rand and rand_static take a 0-tuple, 1-tuple, or 2-tuple
 * x and size take a 1-tuple, 2-tuple, or 3-tuple
 * skew takes a 2-tuple
 * transform takes a 1-tuple, 2-tuple, 4-tuple, or 6-tuple for numeric tuples and a 1-tuple for transform tuples

Rule Expressions
A rule expression specifies a shape (a rule or a path) along with any parameters that the shape might take. If the shape take parameters then a function-like syntax is used. The expression is the shape's name followed by the parameter list in parentheses; e.g., beeTemplate(0.75, 2, [size 0.95 x 1.5], CIRCLE). If the shape does not take parameters then the parentheses are empty or they can be omitted: CIRCLE or CIRCLE. Rule expressions in parameter specification or definitions must have parentheses, even if the shape takes no parameters. In an actual shape replacement empty parentheses can be omitted.

=Adjustment Syntax Changes= In version 2 rules and paths, adjustments have two forms: basic adjustments, in {}; and ordered adjustments, in []. In version 3 shapes, square brackets are used for all adjustments. Single pairs of square brackets [] enclose basic adjustments (reordered to canonical form), and double pairs of square brackets  enclose ordered adjustments. is equivalent to

=Definitions= Definitions are used to bind a variable name to an expression either locally within a rule or path, or globally within a cfdg file. Local definition expressions can be dynamic, they can depend on shape parameters or the random function/operator. Global definitions must be constant. The syntax of a global definition is: There is no type specifier as there are in parameter definitions. The type is inferred from the expression. The scope of each variable is from the end of its expression to the end of the file. In the syntax example expression2 can reference variable1 but expression1 cannot. The syntax for a local definition is the same as global definitions, but the scope is to the end of the block containing the definition.

The use case for definitions
The use case for global definitions is pretty straight-forward. It allows designers to parameterize an entire cfdg file. It is also a handy way to bind a name to a constant that is used multiple times and requires a tedious expression to evaluate: The use case for local definitions is to bind a name to a 'local' expression that is used multiple places within a rule or path. An expression could be considered local if it depends on shape parameters, if it depends on the rand function, or if it is constant but is only needed within a particular rule. If an expression is random then the same random value will be used each time the variable is referenced in its scope. If different random values are needed within the define body then there must be a separate instance of rand for each unique random value.

=Named Loops= Named loops extend version 2 anonymous loops in two ways: the loop index is bound to a variable name, and the loop count can be an expression instead of just a numeric constant. The syntax for named loops is: The loop transform is a transform expression in square brackets. The loop body can either be a single replacement (or path element within a path) or it can be a group of replacement/path-elements contained in curly brackets.

Loop expression details
The loop expression can be a 1-tuple, 2-tuple, or 3-tuple. If the expression is a 3-tuple then they are of the form initial_value, final_value, increment_value. The loop index starts with initial_value, adds increment_value with each loop iteration, and the loop ends when the loop index is greater than or equal to the final_value (or less than or equal if final_value < initial_value). If the expression is a 2-tuple then they are of the form initial_value, final_value and the increment_value is 1. If the expression is a 1-tuple then it is the final_value, and initial_value is 0 and increment_value is 1.

Anonymous Loops
The version 2 anonymous loop syntax cannot be used inside of shapes, but not all loops need an accessible loop index variable. So a new anonymous loop syntax is also defined:

=Loop Finally Blocks= A loop finally block is a block of replacements or path elements that executes after the end of a loop but in the context of the loop. The body of the finally block is executed as if it is one last iteration of the preceding loop. The loop index variable is still in scope and has the terminal value for the loop. The format for the finally block is: The loop body and the finally body can be a single replacement or path element (simple form) or a group surrounded by curly brackets (compound form).

=Shape Declaration= A shape declaration is a syntax for declaring the name and parameter signature for a shape. The rule(s) or path of a shape follow immediately. Most of the new features in version 3 are only available inside of shape blocks (local definitions, named loops, loop finally blocks, rule/path parameters). A version-2-style rule or path (in which the shape name is part of the rule/path header) cannot have parameters and it cannot contain a replacement that has parameters. The following two are equivalent: and If the shape has parameters then they parameter declaration is in the shape block header and not in the shape rule headers.

Shape Singletons
If a shape has exactly one rule then the rule keyword can be omitted.

Forward Declaration of Shapes
A shape that takes parameters must be declared before it can be used in replacements. In general, shapes must be ordered so that dependent shapes come after the shapes that they depend on. If two or more shapes are mutually dependent then there is no possible order that satisfies this rule. But, it is possible to forward declare the parameters of a shape and declare the actual rules or path of the shape later. In this ridiculous example shape coat has two shape declarations. The first is a forward declaration that defines coat's parameters, but does not introduce its rule. The second does not list the parameters, but does introduce the rule for coats. Shape egg has a more conventional shape declaration that declares its parameters and introduces its rule.

Shapes that do not have parameters can be used in replacements before the shape declaration, just as with version 2 rules/paths.

Shapes or paths with parameters can be used before the parameters are declared. When this occurs Context Free performs a second compiler pass after gathering all of the shape parameter declarations. Forward declarations of parameters are still permitted, but they are not required.

=Conditional Code= Context Free can really use something like if-then-else statements and something like the ternary (test ? expression1 : expression2) operator found in C. I would like to avoid creating new expression syntax and also avoid the ambiguity of the if-then-else statement (in nested IFs it can be unclear which IF statement an ELSE belongs to). Instead, I propose a select for conditional expressions, and a select {} syntax for conditional code.

Conditional Expressions
The select function returns one of its arguments. It has the form: select(numeric expression, expression_0, expression_1, expression_2, ...). The first argument must be a numeric expression. It is evaluated and used to select which of the remaining arguments is returned as the result of the function, the remaining arguments can be numeric, transform, or rule expressions, but they must all be of the same type. If the first expression is < 1, then expression_0. If it is in the range [1, 2), then expression_1 is returned, etc.

The if function is a special form of the select function. if(condition_expression, expression_true, expression_false) return expression_true if condition_expression evaluates true and expression_false if it evaluates false.

If Statements
The syntax for the if statement is:

if (conditional_expression) then_code or if (conditional_expression) then_code else else_code

The then_code or else_code can be either a single statement (a replacement, path operation, or path command) or it can be collection of statements enclosed in curly brackets.

Switch Statement
The switch statement in cfdg is very much like the switch statement in the C language. A switch expression is evaluated and matched against a list of case expressions. The case expressions must be constant numeric expressions. Although all expressions in Context Free are floating point, the expression values are converted to integers before matching. If there is no match and there is an else block then the else block is executed. As with the if statement, the code following the colon can be either a single statement (a replacement, path operation, or path command) or it can be collection of statements enclosed in curly brackets.

Conditional Operators
These conditional code features require new conditional operators to work. I propose that <, >, <=, >=, ==, <> be used and that &&, ||, and ^^ be used for combining conditional expressions. ! would be used for logical not. In the spirit of allowing non-ASCII Unicode characters for ± I could allow ≠, ≤, and ≥ as well. ^^ is an exclusive-OR logical operator, it is true if either the left or right operand is true, but false if both are true or both are false. I rarely use exclusive-OR in a program, but when I do I always wish that languages had an exclusive-OR operator.

=Shape Adjustment Keywords= All of the shape adjustments are parsed by the lexer as tokens that are distinct from identifier strings. This means that you can't have shapes or variables with names like a or b or size because these get converted into modtype tokens regardless of where they are in the file. However, in the new syntax it is easy to disambiguate shape adjustments from variables and shapes and it isn't too hard in the old syntax either. We should probably lex the shape adjustment strings as identifiers and have the Builder class figure it out.

=Subpaths within Paths= Paths can now include other paths. In addition to path operations and path commands a path can now reference other paths. The referred path, called a subpath, is included directly into the referring path. Subpaths can have parameters and can be all path operation, all path command, or a mixture of both. The only restriction is that the subpath must be defined earlier in the cfdg file (thus, a path cannot include itself). If a subpath is a mixture of path operations and path commands then the subpath must be "self-contained", just as in mixed loops in paths (see below). The syntax for a subpath is the path token followed by the subpath specifier, which looks like a replacement specifier in a rule.

=Transform Blocks= A transform block is a way of applying an adjustment to a group of replacements in a rule or path elements in a path: The body of the transform block is executed in the context of the transform adjustment. This doesn't add any new functionality to Context Free. It is only intended to allow a designer to express their intent more clearly. If the transform body contains both path operations and path commands then it should be 'self-contained' (see below).

=Miscellaneous Changes=

Mixed Loops inside Paths
In version 2, loops inside of paths could either be all path operations or all path commands. This restriction is lifted in version 3. But if a loop body contains a mix of path operations and path commands then is must be "self-contained", otherwise the behavior will be unexpected. By self-contained I mean that path commands inside the loop body must only paint path operations that are also inside the loop body and that path operations inside the loop body must be painted by path commands inside the loop body, not by commands following the loop. This same requirement applies to transform bodies inside paths and to subpath bodies.

Percentage Constants
As syntactic sugar, numeric constants can be expressed as percentages. {b 5%} would be exactly equivalent to {b 0.05}.

In Rule Weights
When percentages are used for rule weights then the meaning is a little different. If a rule has a weight of 5% then its probability is 5% regardless of the weights of the other rules. If all of a shape's rule weights are percentages then they must total to 100%. If only some of the rule weights are percentages then they must total to less than 100% and the leftover weight will be divided among the remaining rules in proportion to their non-percentage rule weight. Both version 2 rules and version 3 shapes can use percentage rule weights. If none of the weights are percentages then the version 2 weighting behavior remains in effect.

Color Targets Deprecated
The color target adjustments (|hue x, sat x|, etc) are deprecated. They still work in version 2-style rules and paths, but not in version 3 shapes. They are replaced by two-valued color adjustments: hue x y, alpha x y, etc. These color adjustments shift the color component x% toward the target y. The color targets are no longer pieces of hidden state, you must specify the target each time you want to make a targeted color change. Version 2 rules and paths can also use two-value color adjustments, but mixing the two type of targeted color adjustments will have strange results, just use one or the other.

Color Space Control
The pixel type used in the canvas can be changed from the default value using the global variables CF::ColorDepth, CF::Color, and CF::Alpha. Setting the color depth to 16 changes the pixel type from 8-bit channels to 16-bit channels (gray16, rgb48, or rgba64 instead of gray8, rgb24, or rgba32). Setting CF::Color to 0 forces the canvas to grayscale and setting it to 1 forces the canvas to color. Setting CF::Alpha to 0 forces the canvas to be opaque and setting it to 1 forces the canvas to have an alpha channel.