33 lines
879 B
Plaintext
33 lines
879 B
Plaintext
|
[[analysis-pathhierarchy-tokenizer]]
|
||
|
|
=== Path Hierarchy Tokenizer
|
||
|
|
|
||
|
|
The `path_hierarchy` tokenizer takes something like this:
|
||
|
|
|
||
|
|
-------------------------
|
||
|
|
/something/something/else
|
||
|
|
-------------------------
|
||
|
|
|
||
|
|
And produces tokens:
|
||
|
|
|
||
|
|
-------------------------
|
||
|
|
/something
|
||
|
|
/something/something
|
||
|
|
/something/something/else
|
||
|
|
-------------------------
|
||
|
|
|
||
|
|
[cols="<,<",options="header",]
|
||
|
|
|=======================================================================
|
||
|
|
|Setting |Description
|
||
|
|
|`delimiter` |The character delimiter to use, defaults to `/`.
|
||
|
|
|
||
|
|
|`replacement` |An optional replacement character to use. Defaults to
|
||
|
|
the `delimiter`.
|
||
|
|
|
||
|
|
|`buffer_size` |The buffer size to use, defaults to `1024`.
|
||
|
|
|
||
|
|
|`reverse` |Generates tokens in reverse order, defaults to `false`.
|
||
|
|
|
||
|
|
|`skip` |Controls initial tokens to skip, defaults to `0`.
|
||
|
|
|=======================================================================
|
||
|
|
|