[DOCS] Add token filter reference docs template (#52290)
Creates a reusable template for token filter reference documentation. Contributors can make a copy of this template and customize it when documenting new token filters.
This commit is contained in:
James Rodewig
parent
35c195b224
commit
d5a609a2e5
|
|
@ -0,0 +1,233 @@
|
|||
////
|
||||
This is a template for token filter reference documentation.
|
||||
|
||||
To document a new token filter, copy this file, remove comments like this, and
|
||||
replace "sample" with the appropriate filter name.
|
||||
|
||||
Ensure the new filter docs are linked and included in
|
||||
docs/reference/analysis/tokefilters.asciidoc
|
||||
////
|
||||
|
||||
[[sample-tokenfilter]]
|
||||
=== Sample token filter
|
||||
++++
|
||||
<titleabbrev>Sample</titleabbrev>
|
||||
++++
|
||||
|
||||
////
|
||||
INTRO
|
||||
Include a brief, 1-2 sentence description.
|
||||
If based on a Lucene token filter, link to the Lucene documentation.
|
||||
////
|
||||
|
||||
Does a cool thing. For example, the `sample` filter changes `x` to `y`.
|
||||
|
||||
The filter uses Lucene's
|
||||
{lucene-analysis-docs}/sampleFilter.html[SampleFilter].
|
||||
|
||||
[[analysis-sample-tokenfilter-analyze-ex]]
|
||||
==== Example
|
||||
////
|
||||
Basic example of the filter's input and output token streams.
|
||||
|
||||
Guidelines
|
||||
***************************************
|
||||
* The _analyze API response should be included but commented out.
|
||||
* Ensure // TEST[skip:...] comments are removed.
|
||||
***************************************
|
||||
////
|
||||
|
||||
The following <<indices-analyze,analyze API>> request uses the `sample`
|
||||
filter to do a cool thing to `the quick fox jumps the lazy dog`:
|
||||
|
||||
[source,console]
|
||||
----
|
||||
GET /_analyze
|
||||
{
|
||||
"tokenizer" : "standard",
|
||||
"filter" : ["sample"],
|
||||
"text" : "the quick fox jumps the lazy dog"
|
||||
}
|
||||
----
|
||||
// TEST[skip: REMOVE THIS COMMENT.]
|
||||
|
||||
The filter produces the following tokens:
|
||||
|
||||
[source,text]
|
||||
----
|
||||
[ the, quick, fox, jumps, the, lazy, dog ]
|
||||
----
|
||||
|
||||
////
|
||||
[source,console-result]
|
||||
----
|
||||
{
|
||||
"tokens" : [
|
||||
{
|
||||
"token" : "the",
|
||||
"start_offset" : 0,
|
||||
"end_offset" : 3,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 0
|
||||
},
|
||||
{
|
||||
"token" : "quick",
|
||||
"start_offset" : 4,
|
||||
"end_offset" : 9,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 1
|
||||
},
|
||||
{
|
||||
"token" : "fox",
|
||||
"start_offset" : 10,
|
||||
"end_offset" : 13,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 2
|
||||
},
|
||||
{
|
||||
"token" : "jumps",
|
||||
"start_offset" : 14,
|
||||
"end_offset" : 19,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 3
|
||||
},
|
||||
{
|
||||
"token" : "over",
|
||||
"start_offset" : 20,
|
||||
"end_offset" : 24,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 4
|
||||
},
|
||||
{
|
||||
"token" : "the",
|
||||
"start_offset" : 25,
|
||||
"end_offset" : 28,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 5
|
||||
},
|
||||
{
|
||||
"token" : "lazy",
|
||||
"start_offset" : 29,
|
||||
"end_offset" : 33,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 6
|
||||
},
|
||||
{
|
||||
"token" : "dog",
|
||||
"start_offset" : 34,
|
||||
"end_offset" : 37,
|
||||
"type" : "<ALPHANUM>",
|
||||
"position" : 7
|
||||
}
|
||||
]
|
||||
}
|
||||
----
|
||||
// TEST[skip: REMOVE THIS COMMENT.]
|
||||
////
|
||||
|
||||
[[analysis-sample-tokenfilter-analyzer-ex]]
|
||||
==== Add to an analyzer
|
||||
////
|
||||
Example of how to add a pre-configured token filter to an analyzer.
|
||||
If the filter requires arguments, skip this section.
|
||||
|
||||
Guidelines
|
||||
***************************************
|
||||
* If needed, change the tokenizer so the example fits the filter.
|
||||
* Ensure // TEST[skip:...] comments are removed.
|
||||
***************************************
|
||||
////
|
||||
|
||||
The following <<indices-create-index,create index API>> request uses the
|
||||
`sample` filter to configure a new <<analysis-custom-analyzer,custom analyzer>>.
|
||||
|
||||
[source,console]
|
||||
----
|
||||
PUT sample_example
|
||||
{
|
||||
"settings" : {
|
||||
"analysis" : {
|
||||
"analyzer" : {
|
||||
"my_sample_analyzer" : {
|
||||
"tokenizer" : "standard",
|
||||
"filter" : ["sample"]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
----
|
||||
// TEST[skip: REMOVE THIS COMMENT.]
|
||||
|
||||
|
||||
[[analysis-sample-tokenfilter-configure-parms]]
|
||||
==== Configurable parameters
|
||||
////
|
||||
Documents each parameter for the token filter.
|
||||
If the filter does not have any configurable parameters, skip this section.
|
||||
|
||||
Guidelines
|
||||
***************************************
|
||||
* Use a definition list.
|
||||
* End each definition with a period.
|
||||
* Include whether the parameter is Optional or Required and the data type.
|
||||
* Include default values as the last sentence of the first paragraph.
|
||||
* Include a range of valid values, if applicable.
|
||||
* If the parameter requires a specific delimiter for multiple values, say so.
|
||||
* If the parameter supports wildcards, ditto.
|
||||
* For large or nested objects, consider linking to a separate definition list.
|
||||
***************************************
|
||||
////
|
||||
|
||||
`foo`::
|
||||
(Optional, boolean)
|
||||
If `true`, do a cool thing.
|
||||
Defaults to `false`.
|
||||
|
||||
`baz`::
|
||||
(Optional, string)
|
||||
Path to another cool thing.
|
||||
|
||||
[[analysis-sample-tokenfilter-customize]]
|
||||
==== Customize
|
||||
////
|
||||
Example of a custom token filter with configurable parameters.
|
||||
If the filter does not have any configurable parameters, skip this section.
|
||||
|
||||
Guidelines
|
||||
***************************************
|
||||
* If able, use a different tokenizer than used in "Add to an analyzer."
|
||||
* Ensure // TEST[skip:...] comments are removed.
|
||||
***************************************
|
||||
////
|
||||
|
||||
To customize the `sample` filter, duplicate it to create the basis
|
||||
for a new custom token filter. You can modify the filter using its configurable
|
||||
parameters.
|
||||
|
||||
For example, the following request creates a custom `sample` filter with
|
||||
`foo` set to `true`:
|
||||
|
||||
[source,console]
|
||||
----
|
||||
PUT sample_example
|
||||
{
|
||||
"settings" : {
|
||||
"analysis" : {
|
||||
"analyzer" : {
|
||||
"my_custom_analyzer" : {
|
||||
"tokenizer" : "whitespace",
|
||||
"filter" : ["my_custom_sample_token_filter"]
|
||||
}
|
||||
},
|
||||
"filter" : {
|
||||
"my_custom_sample_token_filter" : {
|
||||
"type" : "sample",
|
||||
"foo" : true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
----
|
||||
// TEST[skip: REMOVE THIS COMMENT.]
|
||||
Loading…
Reference in New Issue