Token is the smallest element of a C program that compiler can recognize. The parser separates tokens from the input stream by creating the longest token possible using the input characters in a left–to–right scan.

The mikroC PRO for PIC32 recognizes the following kinds of tokens:

• keywords
• identifiers
• constants
• operators
• punctuators (also known as separators)

Tokens can be concatenated (pasted) by means of the preprocessor operator ##. See the Preprocessor Operators for details.

Token Extraction Example

Here is an example of token extraction. Take a look at the following example code sequence:

inter =  a+++b;

First, note that inter would be parsed as a single identifier, rather than as the keyword int followed by the identifier er.

The programmer who has written the code might have intended to write inter = a + (++b), but it wouldn’t work that way. The compiler would parse it into the seven following tokens:

inter    // variable identifier
=        // assignment operator
a        // variable identifier
++       // postincrement operator
+        // addition operator
b        // variable identifier
;        // statement terminator

Note that +++ parses as ++ (the longest token possible) followed by +.

According to the operator precedence rules, our code sequence is actually:

inter (a++)+b;

Want more examples and libraries? 
Find them on