WhitespaceAnalyser
- Namespace
- Rowles.LeanCorpus.Analysis.Analysers
- Assembly
- Rowles.LeanCorpus.dll
Analyser that splits text only on whitespace and applies no token filters.
public sealed class WhitespaceAnalyser : IAnalyser- Inheritance
-
WhitespaceAnalyser
- Implements
WhitespaceAnalyser(int)
Initialises a new WhitespaceAnalyser.
Analyse(ReadOnlySpan<char>, ISpanTokenSink)
Analyses the input text and emits tokens synchronously into sink.
Callers that require a materialised List<T> should use
CountingTokenSink as the sink argument.