iSharkFly-Docs
/
opensearch-docs-cn
mirror of https://github.com/iSharkFly-Docs/opensearch-docs-cn
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Update index.md (#6223) 

Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
This commit is contained in:
kolchfa-aws 2024-01-18 16:29:20 -05:00 committed by GitHub
parent 49767585c4
commit 05e44c0730
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
_analyzers/tokenizers/index.md
View File

@ -14,7 +14,7 @@ The output of a tokenizer is a stream of tokens. Tokenizers also maintain the fo
- The **order** or **position** of each token: This information is used for word and phrase proximity queries. - The **order** or **position** of each token: This information is used for word and phrase proximity queries.
- The starting and ending positions (**offsets**) of the tokens in the text: This information is used for highlighting search terms. - The starting and ending positions (**offsets**) of the tokens in the text: This information is used for highlighting search terms.
- The token **type**: Some tokenizers (for example, `standard`) classify tokens by type, for example, <ALPHANUM> or <NUM>. Simpler tokenizers (for example, `letter`) only classify tokens as type `word`. - The token **type**: Some tokenizers (for example, `standard`) classify tokens by type, for example, `<ALPHANUM>` or `<NUM>`. Simpler tokenizers (for example, `letter`) only classify tokens as type `word`.
You can use tokenizers to define custom analyzers. You can use tokenizers to define custom analyzers.