OpenSearch/docs/reference/analysis/tokenizers
Andy Bristol 48696ab544 expose simple pattern tokenizers (#25159)
Expose the experimental simplepattern and 
simplepatternsplit tokenizers in the common 
analysis plugin. They provide tokenization based 
on regular expressions, using Lucene's 
deterministic regex implementation that is usually 
faster than Java's and has protections against 
creating too-deep stacks during matching.

Both have a not-very-useful default pattern of the 
empty string because all tokenizer factories must 
be able to be instantiated at index creation time. 
They should always be configured by the user 
in practice.
2017-06-13 12:46:59 -07:00
..
classic-tokenizer.asciidoc Remove wait_for_status=yellow from the docs 2016-07-15 16:02:07 -04:00
edgengram-tokenizer.asciidoc Remove wait_for_status=yellow from the docs 2016-07-15 16:02:07 -04:00
keyword-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
letter-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
lowercase-tokenizer.asciidoc Update lowercase-tokenizer.asciidoc (#21896) 2016-12-02 10:49:51 -05:00
ngram-tokenizer.asciidoc Remove wait_for_status=yellow from the docs 2016-07-15 16:02:07 -04:00
pathhierarchy-tokenizer.asciidoc Remove wait_for_status=yellow from the docs 2016-07-15 16:02:07 -04:00
pattern-tokenizer.asciidoc Added warning messages about the dangers of pathological regexes to: 2016-09-09 09:53:07 +02:00
simplepattern-tokenizer.asciidoc expose simple pattern tokenizers (#25159) 2017-06-13 12:46:59 -07:00
simplepatternsplit-tokenizer.asciidoc expose simple pattern tokenizers (#25159) 2017-06-13 12:46:59 -07:00
standard-tokenizer.asciidoc Remove wait_for_status=yellow from the docs 2016-07-15 16:02:07 -04:00
thai-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
uaxurlemail-tokenizer.asciidoc Remove wait_for_status=yellow from the docs 2016-07-15 16:02:07 -04:00
whitespace-tokenizer.asciidoc Fix incorrect heading for Whitespace Tokenizer (#22883) 2017-01-31 12:51:37 +01:00