SynonymGraphFilter

Namespace

Rowles.LeanCorpus.Analysis.Filters

Assembly

Rowles.LeanCorpus.dll

Token filter that supports multi-token synonym expansion using a trie-based SynonymMap. Uses longest-match lookahead for multi-word synonyms and inserts replacement tokens at the same position offsets.

public sealed class SynonymGraphFilter : ISpanTokenFilter

Inheritance

object

SynonymGraphFilter

Implements

ISpanTokenFilter

Remarks

Buffers tokens during application and performs trie-based longest-match synonym expansion in Finish(ISpanTokenSink).

SynonymGraphFilter(SynonymMap)

Initialises a new SynonymGraphFilter with the specified synonym map.

Apply(ReadOnlySpan<char>, int, int, string, int, byte[]?, ISpanTokenSink)

Applies the filter to a token and emits the transformed token into sink.

Apply(ReadOnlySpan<char>, int, int, string, int, int, byte[]?, ISpanTokenSink)

Applies the filter to a token edge with an explicit position length.

Clone()

Creates an independent copy of this filter with the same configuration but fresh state. The default returns this, which is safe for stateless filters. Stateful filters must override to return a new instance.

Finish(ISpanTokenSink)

Called after all tokens have been processed, allowing stateful filters to flush buffered tokens into the pipeline. The default implementation is a no-op.