StemTokenFilter
- Namespace
- Rowles.LeanCorpus.Analysis.Filters
- Assembly
- Rowles.LeanCorpus.dll
Applies an ISpanStemmer to each token in the list. Useful as a drop-in filter in the composable Analyser pipeline.
public sealed class StemTokenFilter : ISpanTokenFilter- Inheritance
-
StemTokenFilter
- Implements
StemTokenFilter(ISpanStemmer)
Initialises a new StemTokenFilter that stems all tokens.
StemTokenFilter(ISpanStemmer, KeywordMarkerFilter?)
Initialises a new StemTokenFilter with an optional keyword marker.
Tokens recognised by keywordMarker are passed through unchanged.
Apply(ReadOnlySpan<char>, int, int, string, int, byte[]?, ISpanTokenSink)
Applies the filter to a token and emits the transformed token into sink.
Clone()
Creates an independent copy of this filter with the same configuration
but fresh state. The default returns this, which is safe for
stateless filters. Stateful filters must override to return a new instance.