BaseTokenizer abstract class BaseTokenizer : SourceManager Common methods and variables of all tokenizers. Documentation Code protected StringBuilder _stringBuffer public event EventHandler<ParseErrorEventArgs> ErrorOccurred The event will be fired once an error has been detected. public BaseTokenizer(ITextSource source) protected void RaiseErrorOccurred(ErrorCode code) Fires an error occurred event.