Token is the smallest element of a mikroBasic PRO for ARM program, meaningful to the compiler. The parser separates tokens from the input stream by creating the longest token possible using the input characters in a left–to–right scan.

mikroBasic PRO for ARM recognizes the following kinds of tokens:

• keywords
• identifiers
• constants
• operators
• punctuators (also known as separators)

Token Extraction Example

Here is an example of token extraction. Take a look at the following example code sequence:

end_flag = 0

First, note that end_flag would be parsed as a single identifier, rather than as the keyword end followed by the identifier _flag.

The compiler would parse it as the following four tokens:

end_flag  ' variable identifier
=         ' assignment operator
0         ' literal
newline   ' statement terminator

Want more examples and libraries? 
Find them on