The Myra language

Introduction

The Myra language is a stack based, higher order language, that can be compiled into Propeller (TM) assembler code. Propeller is a trademark of the Parallax company, that has developed and manufactures the Propeller processors. As a language it is fairly universal, but some of its features make use of features in the Propeller assembler language. Above all, the Propeller computer is a parallell processing computer with 8 separate processes, called cogs, communicating via a common memory.

The language is named after the Swedish word 'myra', which is 'ant' in English, as it is a stack based language. (Ants build stacks in Swedish and heaps in English, but these words are used more or less interchangeably in computer science)

Compilation

A Myra program called 'example.myr' can be compiled using

java Myra example   (no need to write '.myr')
This generates a number of .spin files, one for each cog used in the system. The filenames are derived from a system name, given on a system-line in the Myra program. The files are then named
systemname1.spin
systemname2.spin
systemname3.spin
etc.
They are assembled and loaded from the Propeller environment, by opening the first of these files. The Propellent environment can also be used.

The generated code is slower than manually made assembler code, as there is some overhead for managing the stack, but the code is much faster than spin code.

Stack based programs

A stack is a heap or stack of values stacked on top of each other. You can only enter data on the top of the stack, and only the value currently on the top of the stack is visible. Normally when you use a value, it is simultatneously lifted off the stack, making the value under it visible. The normal operations are

The functions can be normal operations like + or *, or they can be any function with a name like sin. This is the elegance with stack based programs: operations and functions look the same. This also means that there is a standardized way of supplying arguments to functions, and retrieve the result. Unlike most programming languages, a function is free to deliver more than one value as a result.

The language has four special stack operations Stack machines inherently work with Łukaszevicz notation, also called reversed Polish notation, or postfix notation. This means that all arithmetic expressions can be written without parentheses, and these expressions can be treated without parsing. This simplifies the compiler. The presence of a parenthesis in normal code, implies that there is a need for a temporary variable. In a stack based system you can avoid temporary variables, and use the stack instead. You can develop it into an art to write stack based programs with as few variables as possible, but beyond some limit, this can be harmfull for the readability of the code.

Example program

As an example to look back to, when reading the following paragraph, here's a piece of code:

system colors global x y exec mix an process mix red = 3 green = 6 blue = 5 black = 7 colormask = 7 tred tgreen tblue trg ttot = 40960 begin colormask setdir :loop y ttot * 12 >> ->tblue ttot tblue - ->trg x trg * 12 >> ->tred trg tred - ->tgreen red tred [show] green tgreen [show] blue tblue [show] black 0 [show] go(loop) function show tshow begin ->tshow colormask outpins tshow wait return \ process an chx = 0 chy = 4 begin init_analog :loop chx analog 4 >> ->x chy analog 4 >> ->y go(loop) \ The program sends out different colors on an RGB-led. The led is connected to pins 0,1 and 2 on the Propeller, and it is connected so that, a diode is on, when the corresponding Propeller output is zero. Hence 7 means that all three diodes are off, so the color is "black". The color is controlled by two potentiometers, Their values are read in the second process. The light is controlled in the function show which sets up a "pure" color, and then lets that be on for a specified time. The times are set in the process mix, which proportionates a total time of 40960 tics to each of the colors, depending on the potentiometer settings.

General program structure

A Myra program has the following separate parts:

Structure of a process

A process starts with a process keyword, followed by the process name. This name should appear also in the exec-part of the program as mentioned above. (if it does not, this process will never be executed, and it will not even be compiled.)

After this, variable declarations follow, each on one line with a variable name optionally followed by an '=' sign and a value. These variables are of type long and are stored in the cog's local memory. There is also provisions for declaring arrays. We will come back to that later. Values are written with the same standard as in Propeller assembler, i.e. a '$' sign means hexacecimal notation, '%' means binary notation, and "a" means a character value, i.e. the ASCII-code for the character 'a' is stored. Symbols like '|<20', which means a 1 in the 20th position, can also be used. Also, expressions involving contstants, like 80000000/9600, can be used.

After the begin keyword follows the executable code. This code can call functions, and it can contain macros. These can be declared locally in the process itself, or externally on separate files. In the former case the functions follow the executable code of the process. The scope of these declarations is limited to within the process. In the latter case, the functions have to be referred to with a 'load from'-section.

A local function declaration starts with a function keyword, followed by the function name. The rest of the function declaration is similar to the process declaration, except that the function must finish with a return statement, while a process must not contain a return statement. (Processes are either infinite loops, or just terminate.)

A 'load from'-section starts with a load from keyword followed by a filename within parentheses. The functions to be loaded are then listed, each on one line starting with an asterisk (*). If functions are loaded from several files, each file requires its 'load from'-section.

It is recommended that these external functions are named in object oriented style with a dot, like display.write. As Propeller assembler is intolerant to these dots, they are replaced with underscores in the assembler code.

A process declaration ends with a \ (backslash). The backslash of the last process ends the whole program. These backslash signs are crucial for correct compilation.

Empty lines are ignored. Lines starting with '--' are comments, and are ignored by the compiler. If a line contains '--', the rest of the line after that is ignored as a comment.

External function files

External functions are written on external function files, preferably with a '.myo'-extention. These files don't contain any processes, and are thus not executable. They merely contain declarations of functions.

It is recommended that functions are collected into .myo-files in such a way, that each file represents some kind of an object, like a sensor, a display, something simulated etc.

Apart from functions (or methods in object oriented terminology) such an object file can also contain variables, that represent the state of the object. Such variables are called fields. They are declared in the beginning of the file, between a fields- line and a endfields-line. These variables can be assigned intial values like local variables in functions. They are stored in cog memory, and are reachable from all functions of an object. But they are not reachable from one process to another.Fields should not be referenced from outside the object functions. Instead they should be reached through so called 'put-' and 'get-' functions. The natural way for these functions to communicate with the outside world, is to use the stack.

The functions on a myo-file may further refer to other functions on other external files, or on their own file. If this is done, the function body should be followed by one or more 'load from'-section, as in the main program.

It is recommended that an object oriented style dot notation is used for the functions of an object. An object representing a display could have funtions called display.init, display.writetext etc. The dots in these names will be replaced with underscores in the assembler code.

Instructions

The instructions are found in the executable part of processes and functions. Instructions are separated with white spaces, or with line feeds if instructions are written on several lines. Except for what is mentioned about conditional statements later, the programmer is free to divide his code into lines as he wishes.

Instructions are interpreted in the following way by the compiler;

The "unpop" instruction maybe deserves the following comment. A write instruction
->x, saves the value on the top of the stack, but removes it from the stack. The combination ->x ' can then be seen as a modified write instruction, that maintains the stored value on the stack for future use.

Macros

A macro differs from a function, in that it is never called, i.e. there are no jumps to a macro. Instead, the macro code is substituted for the call of the macro directly in the code before compilation. This gives faster code, as the jumps to and back from the macro are avoided. But there is a penalty in memory usage, if the same macro is used more than once. If the macro is used n times, the macro code will be appear in the compiled program n times.

A macro is defined in a macro definition. On its first line is the keyword macro, followed by the macro name. On the second line is the macro code. Hence the macro code can not be longer than that it fits into one line. Macros are supposed to be small.

The 'call' of a macro (which isn't a call anyway) looks like the function call, i.e. it is the macro name placed between brackets. The compiler will substitute the macro name and the two brackets with the macro code.

Macro definitions can be placed in the main program file (the '.myr-file'), or in object files ('.myo-files). There, they can be used either by the main program or by the object-functions.However, the main program can use the macros only if the object file is referred to at all, through a loading of some object funtion. For macros no 'load from'-operation is necessary; the macros will be found, once the system has had reason to open the object file.

There is also a special macro file, for universally usefull macros, called macro.myo. The macros on this file are always available.

Name uniqueness is as urgent for macros as it is for functions. Hence it is recommended that macros on objectfiles are named with a 'dot'-notation like the functions.

A macro can use a macro, but currently this is limited to two levels, i.e. a macro can us a macro, but that macro cannot use a macro.

Technically macros don't add anything to the system functionality. There is no difference between using the macro concept, and substituting the macro code yourself. Macros are there to enhance the readability of the code. You substitute a piece of technical code with a name, which reflects what the code is good for.

Conditional statements

Instructions inscribed between curly brackets {...} are executed only if the condition preceding the left bracket is satisfied. The condition is based on the value on the top of the stack, at the time when the last ?-instruction was executed.

The available conditions are as follows:

If the condition is not satisfied, the instructions are executed as nop-statements, as this is the way Propeller assembler works.

A line can only contain one curly-bracket-pair, but that pair may contain as many instructions as one wishes. If there isn't space for all the instructions on one line one can continue on the next line with a new curly-bracket-pair, but the condition has to be repeated.

The principle for conditional statements here mimics what happens in Assembler and in the computer itself, but it is also in a way quite elegant. You can write something that works as if-then-else constructs, without letting the compiler construct lots of jumps. But there is a very important pitfall. Instructions inside curly brackets can change the status registers. The ?-instruction does that, but you probably learn pretty soon to avoid ?-instructions inside curly brackets. But the problem is with functions and assembler functions. Remember that arithmetic instructions are executed as assembler functions. Most of them don't change the status registers, but multiplication and division do. And many other assembler functions do.

If the status registers are changed anywhere between a ?-instruction and any conditions that is supposed to use it, things don't work the way they should. If it happens inside a curly bracket, the rest of the instructions in there will not be executed. If there is a curly bracket pair on the next line with the opposite condition (an 'else branch') then the code in there may be executed, even though it shouldn't.

As a remedy to this, there is a version of ? called ?s, which stores away the stack content that set the status registers, in a fixed place. We call this the status variable. Then, you can at any time restore the status register, which is done with a ! instruction. Place this instruction after any instruction that may have changed the status register. Here's an example: x ?s ={a b c * ! + ->z} if x is zero z is computed as a+bc. '!' protects for the multiplication, which might have changed the status registers.

All this i OK, unless a function that you call also uses the ?s-instruction. That will destroy the status variable. As a remedy to this, we have two other instructions, called S and R. S saves the status variable, and R restores it. As a matter of fact, they push and pop the values into a small stack. It only has a height of two now, but that should be sufficient.

Now the rule is: If a function uses a ?s instruction, it should start with an S instruction and end with an R instruction. Assembler functions give no problems; they don't use the ?s-instruction.

The codes for ?s, !, S and R are quite small. In fact ?s is no bigger than the normal ?.

(It would of course be tempting to use a more consistent stack concept for all this. The problem is that there can be several !-instructions for each ?s-instruction, and it is difficult for the compiler to know which ?s- and !-instructions belong together).

Booleans

The mechanism with conditional statements, as described above, is both elegant and powerful. But it makes it difficult to combine several conditions with boolean operators. To help with this, a notion of a boolean variables is introduced. Boolean variables are either true or false. These values ar represented as the integer 1 (a 1 only in the least significant bit), and 0 (all bits zero). With this representation, the operators &, or and xor, act on booleans, as one would expect. There are two instructions that generate boolean values on the stack:

  1. Z, which replaces the stack top with true, if the current value on the stack was exactly zero
  2. G, which replaces the stack top with true if the current value on the stack was greater than zero
Note that Z also serves as a complement function, which replaces true with false, and false with true.

Arrays

An array is defined simply by writing several values separated by commas after the equals sign:

M = 1,2,3,1,2,3,1,2,3
declares an array M with 9 elements. A string after the equals sign, like
message = "Hello world"
is interpreted as
message = "H","e","l","l","o"," ","w","o","r","l","d",0
"H" means the ASCII-code for the character H. The final 0 can be seen as the ASCII-code for the null-character. Hence we represent a so called null-terminated string.

Arrays can also be declared with a bracket-notation.
area[256]
reserves 256 long words, i.e. 1024 bytes for the array area.

Arrays can be adressed in two ways:
i M[]
will push the i:th element of the array M on the stack. The enumeration starts with zero. Hence
4 message[]
will push the ASCII-code for 'o' on the stack.

The other alternative is used to adress an arbitrary array. It uses the function @.
adress i @
loads the i:th element of the array starting at adress adress on the stack. If we want to load the first character in message, we do the following:
#message 0 @
This loads the "H" character on the stack.

Serial execution

The following code after the exec keyword:

exec proc1 proc2 sema/proc3a/proc3b proc4 makes the process proc3a and proc3b alternate in one and the same cog. They are controlled by the variable sema, which should also be declared as a global variable. When sema switches to the value 0, which it always does initially, proc3a is executed in the cog nr 3. If sema switches to 1, the cog is instead loaded with proc3b. The variable sema can then switch back to 0 or to higher values, and then other processes are executed, if they are mentioned after more "/":es.

When sema switches values, the current process is interupted abruptly, so it may be an advantage to let each process interrupt itself in a controlled way. Thus, it would be safer only to allow the processes controlled by sema to control sema.

After change of process the exiting process is completely wiped out, so the only way it can communicate with the world into the future, is by writing to global variables, or output pins.

The motivation for this whole concept is, that the limited size of the cog memories (512 long words) maybe is the strongest limitation to what you can do with a Propeller. As long as you can divide your computations into independent chunks, this concept allows you to do as big computations as you like, upto the limitation of the size of the global memory (which is 8k long words). Naturally the reloading of a process into a cog takes some time. A natural use, is when a process requires much code for initialization. Then you let one process (init) initialize, and another process (run) execute. Then you write exec ... s/init/run ... When init has done its work, it sets s to 1.

This is the case when the processes actually execute in series, but the concept allows you to let a process tree branch out, depending on the results of the computations.

A high level construct

Macros are treated by a preprocessor, which substitutes the macro name with the macro code. The same preprocessing can be used to handle high level constructs. I have made one, which mimics a standard for loop. It is made in stack processing style. The idea is, that if you load two numbers, k and n, on the stack, we can let that represent the interval between k and n, i.e. all the integers between k and n. Then we have a function [all:i] which produces all the integers between k and n. These values are produced consecutively in time in the variable i. These values are used in a number of statements enclosed in standard parentheses (()). This means that the code within () is repeated for each value of i. Here's an example:

1 ->nfact 1 n [all:i] (nfact i * ->nfact ) With this code, nfact is the factorial of n (called n!). Here's another example: 1 ->pn 1 n [all:i] (pn p * ->pn ) With this code, pn is the n:th power of p. Note that the variable i is not at all mentioned between ( and ).

As the system is now, the loop variable i has to be declared separately.

Assembler functions

A backbone in the system is the assembler resource file Assembler.spin. This file contains a number of usefull functions, that can be directly interfaced from Myra. How to add functions to this file is described later. Here's what it contains now.

A number of functions are handling the stack, and implement simple arithmetic operations. Multiplication is worth mentioning specially, as Propeller assembler doesn't have any multiplication instruction. The same is true for the division instruction, but it is mentioned further down. Non arithmetic instructions are logical and (&), logical or (or), exclusive or (xor), right shift (>>), and left shift (<<). The right shift is arithmetic, i.e. it preserves the sign of the number.

Adding assembler functions

The user can write his own Myra functions, as he likes. To speed up the programs, he can also add his own assembler functions to the file Assembler.spin.

These functions should have the following properties:

  1. They should be correct assembler code, of course. This is validated when assembling the complete program with the Propeller environment. The whole file Assembler.spin file is written, so that it could be assembled on its own, but it is now too big for that; it contains more than 512 instructions. The assembler functions are there to be called, and hence they need a properly labeled return statement. If the function is called fun, then the return statement should be labeled fun_ret.
  2. Each assembler function must have a header. From the assembler language point of view, this is a comment with the following format:
    '> name aliasname
    The name should be the same as the label on the first line of the actual assembler function (the "name" of the assembler function). The aliasname, can be the same as the name, or something shorter, down to a simple operator symbol like '+'.

    The system uses these headers to load the used assembler functions into the code. It does so by matching the call in the Myra code with the aliasname.
  3. The assembler functions should have a stack type interface with the rest of the system, i.e. arguments should be popped from the stack, and the results should be pushed on the stack.
  4. Functions should not unnecessarily have side effects, i.e. their only effect should be the result they deliver. However, when we use assembler functions to send things out to the output pins, or to control timing (by waiting for instance), these are of course unavoidable side effects
The stack handling is made using self modifying code, i.e. using the movs (move source) and movd (move destination) instructions. Globally there is an array, whose first element is called stack, and there is a stack pointer called sa ("stack adress"). Then the following code loads the item on the top of the stack (which is at the adress sa) to a local variable arg1: movs instr1,sa nop instr1 mov arg1,stack sub sa,#1 ...| arg1 long 0 The instruction instr1 is modified, so that data are fetched at the adress sa. This overwrites the adress 'stack', so we could write whatever we like there, but 'stack' is a litte bit informative. The final instruction moves the stack pointer down, so that the loaded value is no longer reachable. (The value is "popped" from the stack). The nop instruction is necessary, because the Propeller uses pipeling. Without it, the instruction instr1 would be loaded before it were modified.

You could look inte the file Assembler.spin to find examples to learn from.

Variable and symbol scopes

The scope of the global variables is of course global, i.e. the variable names can be used throughout the system.

Processes can not be "called" by each other; they can only be called on the line after the exec keyword.

The scope of the rest of the symbols (variables, labels, function names) is the containing process. For some tastes, this might seem a bit too wide. For instance one function in a process could use the local variable of another. This is due to, that the assembler language doesn't have much of a notion of scope. Nevertheless a check against this missuse, could be made at compile time, but this hasn't been implemented so far. A consequence of this is that variables in functions must have unique names. Otherwise the Propeller assembler will complain ("symbol already defined"). I think it would seem acceptable, to let the variables defined in the process be accessible also to the functions, but the programmer should avoid to borrow variables between the functions.

Note that if the same function shall be used by more than one process, the function has to be repeated in each of the processes.

Suffixing

As a remedy to the scope problem (that the scope of variables and labels is too wide), there is a mechanism of suffixing. If the function statement is followed by at least one space, and then the tag "-≺tag≻", all local variables, and all labels (goals for jumps) are suffixed with _≺tag≻. Hence if one writes -cos, then the variable x will be renamed to x_cos, and the label loop will be renamed to loop_cos. These suffixed names will appear in the assembler code, but in the Myra code, the names will of course remain unsuffixed. If all functions are given unique tags, then there is no problem with variable scopes, unless of course one deliberately creates names like x_cos somewhere.

It may happen that the unsiffixed name of a local variable coincides with the name of a variable in the process or a field variable in an object file. In that case, the compiler prefers to interpret the variable as a local variable, i.e. it gets a suffix. (For those who study the compiler source code, Myra.java, this is the reason for the notion of a PVariable (a public variable), so there is a function 'recognizeableAsPVariable(...)').

Recursive calls

The Propeller doesn't have any subroutine call stack, and there is no attempt to construct any call stack in Myra. Hence recursive calls are not possible. If an assembler or Myra function tries to call itself, it will get lost.

Motivations for the language

The Propeller computers are programmed with the Propeller assembler language, and the Spin language. Assembler is very fast, but not always easy to write and to read. Spin is an interpreted language, and thus relatively slow. One can see this for the Spin instruction

wait(581+cnt)
This instruction works. It will wait untill 581 clock cycles have elapsed from now. But if we wrote 580 instead, more than 580 clock cycles would have elapsed, before the actual waiting started, so then waiting would go on till the clock had completed a full cycle through spilling. But 581 clock cycles is pretty much.

In that situation, one would like to mix spin and assembler code, and the ideal way would be to write fast assembler routines, and call them from spin.

But the spin code doesn't really call assembler code; it loads assembler code into cogs, and lets it run there. Execution doesn't return from the assembler code, unless the assembler code halts the whole cog. Also the interface to the assembler code is quite thin; it is only a single long variables, which preferably would contain an adress, through which the assembler code can interface.

In Myra, everything is assembler, (except a few coginit- statements and some code for handling serial execution). This opens up for using pre-written assembler routines directly and uniformly. The stack-based architecture gives a standardized interface to these routines. Likewise one can use pre-written Myra routines, and they are fast, as they are compiled into assembler code. As compared to standard languages like C and Forth, a specialized language makes it easy to use features of the Propellers, like reading and controlling time, reading and writing to i/o pins, and the language/instruction feature, that each instruction can be run conditionally.

Myra also has a nice concept for making object oriented code. Except for object-code files, the whole system is kept together in a single file, that completely defines the system.

As for all stack based languages, one has to get used to the stack architecture, and till then, it may seem like "write only programs", but once one is used to it, programs are actually quite elegant.