TruncateTokenFilter

Namespace

Rowles.LeanCorpus.Analysis.Filters

Assembly

Rowles.LeanCorpus.dll

Truncates token text to a maximum character length.

public sealed class TruncateTokenFilter : ISpanTokenFilter

Inheritance

object

TruncateTokenFilter

Implements

ISpanTokenFilter

TruncateTokenFilter(int)

Initialises a new TruncateTokenFilter with the specified maximum length.

Apply(ReadOnlySpan<char>, int, int, string, int, byte[]?, ISpanTokenSink)

Applies the filter to a token and emits the transformed token into sink.