OpenSearch/docs/reference/analysis/tokenizers
Clinton Gormley 5da9e5dcbc Docs: Improved tokenizer docs (#18356)
* Docs: Improved tokenizer docs

Added descriptions and runnable examples

* Addressed Nik's comments

* Added TESTRESPONSEs for all tokenizer examples

* Added TESTRESPONSEs for all analyzer examples too

* Added docs, examples, and TESTRESPONSES for character filters

* Skipping two tests:

One interprets "$1" as a stack variable - same problem exists with the REST tests

The other because the "took" value is always different

* Fixed tests with "took"

* Fixed failing tests and removed preserve_original from fingerprint analyzer
2016-05-19 19:42:23 +02:00
..
classic-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
edgengram-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
keyword-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
letter-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
lowercase-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
ngram-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
pathhierarchy-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
pattern-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
standard-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
thai-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
uaxurlemail-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00
whitespace-tokenizer.asciidoc Docs: Improved tokenizer docs (#18356) 2016-05-19 19:42:23 +02:00