SplitOnWhitespace

Namespace

Rowles.LeanCorpus.Analysis.Tokenisers

Assembly

Rowles.LeanCorpus.dll

Properties

SplitOnWhitespace

Gets whether the tokeniser splits on whitespace before applying n-grams. When true, no gram spans a word boundary.