Analyser
- Namespace
- Rowles.LeanCorpus.Analysis.Analysers
- Assembly
- Rowles.LeanCorpus.dll
Composable analyser that runs a tokeniser followed by a chain of span filters.
public sealed class Analyser : IAnalyser- Inheritance
-
Analyser
- Implements
Analyser(ISpanTokeniser, params ISpanTokenFilter[])
Initialises a new Analyser with the specified span tokeniser and optional filter chain.
Analyse(ReadOnlySpan<char>, ISpanTokenSink)
Analyses the input text and emits tokens synchronously into sink.
Callers that require a materialised List<T> should use
CountingTokenSink as the sink argument.