honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

CONSOLEify some more docs 

And expand on the `stemmer_override` examples, including the
file on disk and an example of specifying the rules inline.

Relates to #18160
This commit is contained in:
Nik Everett 2017-03-22 17:56:53 -04:00
parent f927a2708d
commit a783c6c85c
5 changed files with 69 additions and 21 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/build.gradle
View File

@ -73,9 +73,6 @@ buildRestTests.expectedUnconvertedCandidates = [
'reference/analysis/tokenfilters/keyword-repeat-tokenfilter.asciidoc',
'reference/analysis/tokenfilters/limit-token-count-tokenfilter.asciidoc',
'reference/analysis/tokenfilters/lowercase-tokenfilter.asciidoc',
'reference/analysis/tokenfilters/pattern-capture-tokenfilter.asciidoc',
'reference/analysis/tokenfilters/snowball-tokenfilter.asciidoc',
'reference/analysis/tokenfilters/stemmer-override-tokenfilter.asciidoc',
'reference/cat/snapshots.asciidoc',
'reference/cat/templates.asciidoc',
'reference/cat/thread_pool.asciidoc',
@ -140,6 +137,7 @@ integTestCluster {
configFile 'scripts/my_combine_script.painless'
configFile 'scripts/my_reduce_script.painless'
configFile 'analysis/synonym.txt'
configFile 'analysis/stemmer_override.txt'
configFile 'userdict_ja.txt'
configFile 'KeywordTokenizer.rbbi'
// Whitelist reindexing from the local node so we can test it.

12
docs/reference/analysis/tokenfilters/pattern-capture-tokenfilter.asciidoc
View File

@ -23,14 +23,14 @@ Read more about http://www.regular-expressions.info/catastrophic.html[pathologic
For instance a pattern like :
[source,js]
[source,text]
--------------------------------------------------
"(([a-z]+)(\d*))"
--------------------------------------------------
when matched against:
[source,js]
[source,text]
--------------------------------------------------
"abc123def456"
--------------------------------------------------
@ -74,7 +74,7 @@ PUT test
When used to analyze the text
[source,js]
[source,java]
--------------------------------------------------
import static org.apache.commons.lang.StringEscapeUtils.escapeHtml
--------------------------------------------------
@ -117,7 +117,7 @@ PUT test
When the above analyzer is used on an email address like:
[source,js]
[source,text]
--------------------------------------------------
john-smith_123@foo-bar.com
--------------------------------------------------
@ -136,14 +136,14 @@ original token will be highlighted, not just the matching subset. For
instance, querying the above email address for `"smith"` would
highlight:
[source,js]
[source,html]
--------------------------------------------------
<em>john-smith_123@foo-bar.com</em>
--------------------------------------------------
not:
[source,js]
[source,html]
--------------------------------------------------
john-<em>smith</em>_123@foo-bar.com
--------------------------------------------------

4
docs/reference/analysis/tokenfilters/snowball-tokenfilter.asciidoc
View File

@ -12,8 +12,9 @@ For example:
[source,js]
--------------------------------------------------
PUT /my_index
{
"index" : {
"settings": {
"analysis" : {
"analyzer" : {
"my_analyzer" : {
@ -31,3 +32,4 @@ For example:
}
}
--------------------------------------------------
// CONSOLE

67
docs/reference/analysis/tokenfilters/stemmer-override-tokenfilter.asciidoc
View File

@ -20,15 +20,60 @@ Here is an example:
[source,js]
--------------------------------------------------
index :
analysis :
analyzer :
myAnalyzer :
type : custom
tokenizer : standard
filter : [lowercase, custom_stems, porter_stem]
filter:
custom_stems:
type: stemmer_override
rules_path : analysis/custom_stems.txt
PUT /my_index
{
"settings": {
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"tokenizer" : "standard",
"filter" : ["lowercase", "custom_stems", "porter_stem"]
}
},
"filter" : {
"custom_stems" : {
"type" : "stemmer_override",
"rules_path" : "analysis/stemmer_override.txt"
}
}
}
}
}
--------------------------------------------------
// CONSOLE
Where the file looks like:
[source,stemmer_override]
--------------------------------------------------
include::{docdir}/../src/test/cluster/config/analysis/stemmer_override.txt[]
--------------------------------------------------
You can also define the overrides rules inline:
[source,js]
--------------------------------------------------
PUT /my_index
{
"settings": {
"analysis" : {
"analyzer" : {
"my_analyzer" : {
"tokenizer" : "standard",
"filter" : ["lowercase", "custom_stems", "porter_stem"]
}
},
"filter" : {
"custom_stems" : {
"type" : "stemmer_override",
"rules" : [
"running => run",
"stemmer => stemmer"
]
}
}
}
}
}
--------------------------------------------------
// CONSOLE

3
docs/src/test/cluster/config/analysis/stemmer_override.txt Normal file
View File

@ -0,0 +1,3 @@
running => run
stemmer => stemmer