StemmerAnalyser
- Namespace
- Rowles.LeanCorpus.Analysis.Analysers
- Assembly
- Rowles.LeanCorpus.dll
Generic analyser that runs tokenise → lowercase → stop-word removal → stem using the supplied ISpanStemmer.
public sealed class StemmerAnalyser : IAnalyser- Inheritance
-
StemmerAnalyser
- Implements
Remarks
Thread-safety: the inner Analyser pipeline maintains per-instance buffers. Create one instance per thread, or use the indexer's built-in per-thread analyser cloning.
Convenience factory methods are provided for common stemmer configurations.
StemmerAnalyser(ISpanStemmer)
Initialises a new StemmerAnalyser with the given stemmer.
Analyse(ReadOnlySpan<char>, ISpanTokenSink)
Analyses the input text and emits tokens synchronously into sink.
Callers that require a materialised List<T> should use
CountingTokenSink as the sink argument.
FromStemmer(ISpanStemmer)
Creates a StemmerAnalyser that uses a named stemmer. Resolves the stemmer from the provided callback, useful with DI.
Hunspell(HunspellDictionary)
Creates a StemmerAnalyser using a Hunspell dictionary.
KStem(IKStemLexicon)
Creates a StemmerAnalyser using the Krovetz stemmer with an in-memory lexicon.
KStem(string)
Creates a StemmerAnalyser using the Krovetz stemmer with a file-based lexicon.
LightEnglish()
Creates a StemmerAnalyser using the lightweight English stemmer.
Porter()
Creates a StemmerAnalyser using the Porter stemmer.