AccentFoldingFilter
- Namespace
- Rowles.LeanCorpus.Analysis.Filters
- Assembly
- Rowles.LeanCorpus.dll
Normalises accented/diacritic characters to their ASCII base form (e.g., é→e, ñ→n, ü→u) for language-neutral matching. Uses Unicode canonical decomposition followed by stripping combining marks.
public sealed class AccentFoldingFilter : ISpanTokenFilter- Inheritance
-
AccentFoldingFilter
- Implements
Apply(ReadOnlySpan<char>, int, int, string, int, byte[]?, ISpanTokenSink)
Applies the filter to a token and emits the transformed token into sink.