BaseTokenizer
Common methods and variables of all tokenizers.
Gets the current column within the source code.
Gets the current character.
Gets or sets the insertion point.
Gets the status of reading the source code, are we beyond the stream?
Gets the current line within the source code.
Gets the current position within the source code.
Advances one character in the source code.
Advances n characters in the source code.
Moves back one character in the source code.
Moves back n characters in the source code.
Looks if the current character / next characters match a certain string.
Disposes all disposable objects.
Gets the current position.
Gets the next character (by advancing and returning the current character).
Gets the previous character (by rewinding and returning the current character).
Resets the insertion point to the end of the buffer.
Advances to the next non-space character.