TokenizerExtensions
Extensions to access the underying tokenizer.
public static IEnumerable<HtmlToken> Tokenize(this TextSource source, IEntityProvider provider = null, EventHandler<HtmlErrorEvent> errorHandler = null)
Performs the tokenization on the given text source.
public static IEnumerable<HtmlToken> Tokenize(this TextSource source, HtmlTokenizerOptions? options = default, IEntityProvider provider = null, EventHandler<HtmlErrorEvent> errorHandler = null)
Performs the tokenization on the given text source.