mirror of https://github.com/apache/lucene.git
LUCENE-1817: Warn that file formats for smartcn might change in the future, and do not encourage custom dictionary use in the meantime
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@808570 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
parent
26c5af3a33
commit
5d05bbfd59
|
@ -50,27 +50,9 @@ import org.apache.lucene.analysis.cn.smart.WordTokenFilter;
|
||||||
* The included dictionary data is from <a href="http://www.ictclas.org">ICTCLAS1.0</a>.
|
* The included dictionary data is from <a href="http://www.ictclas.org">ICTCLAS1.0</a>.
|
||||||
* Thanks to ICTCLAS for their hard work, and for contributing the data under the Apache 2 License!
|
* Thanks to ICTCLAS for their hard work, and for contributing the data under the Apache 2 License!
|
||||||
* </p>
|
* </p>
|
||||||
* <p>
|
|
||||||
* In special circumstances a user may wish to configure SmartChineseAnalyzer with a custom data directory location, containing bigramdict.dct and coredict.dct
|
|
||||||
* </p>
|
|
||||||
* The following order is used to determine the location of the data directory:
|
|
||||||
*
|
|
||||||
* <ol>
|
|
||||||
* <li>System property: -Danalysis.data.dir=/path/to/analysis-data</li>
|
|
||||||
* <li>Relative path: analysis-data</li>
|
|
||||||
* <li>Relative path: lib/analysis-data</li>
|
|
||||||
* <li>Property file: analysis.data.dir property from relative path analysis.properties</li>
|
|
||||||
* <li>Property file: analysis.data.dir property from relative path lib/analysis.properties</li>
|
|
||||||
* </ol>
|
|
||||||
*
|
|
||||||
* Example property file:
|
|
||||||
*
|
|
||||||
* <pre>
|
|
||||||
* analysis.data.dir=D:/path/to/analysis-data/
|
|
||||||
* </pre>
|
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -26,7 +26,7 @@ Analyzer for Simplified Chinese, which indexes words.
|
||||||
<div>
|
<div>
|
||||||
<font color="#FF0000">
|
<font color="#FF0000">
|
||||||
WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental. The APIs
|
WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental. The APIs
|
||||||
introduced here might change in the future and will not be supported anymore
|
and file formats introduced here might change in the future and will not be supported anymore
|
||||||
in such a case.
|
in such a case.
|
||||||
</font>
|
</font>
|
||||||
</div>
|
</div>
|
||||||
|
|
|
@ -27,28 +27,9 @@ import java.util.Properties;
|
||||||
* <p>
|
* <p>
|
||||||
* SmartChineseAnalyzer has a built-in dictionary and stopword list out-of-box.
|
* SmartChineseAnalyzer has a built-in dictionary and stopword list out-of-box.
|
||||||
* </p>
|
* </p>
|
||||||
* <p>
|
|
||||||
* In special circumstances a user may wish to configure SmartChineseAnalyzer with a custom data directory location.
|
|
||||||
* </p>
|
|
||||||
* AnalyzerProfile is used to determine the location of the data directory containing bigramdict.dct and coredict.dct.
|
|
||||||
* The following order is used to determine the location of the data directory:
|
|
||||||
*
|
|
||||||
* <ol>
|
|
||||||
* <li>System property: -Danalysis.data.dir=/path/to/analysis-data</li>
|
|
||||||
* <li>Relative path: analysis-data</li>
|
|
||||||
* <li>Relative path: lib/analysis-data</li>
|
|
||||||
* <li>Property file: analysis.data.dir property from relative path analysis.properties</li>
|
|
||||||
* <li>Property file: analysis.data.dir property from relative path lib/analysis.properties</li>
|
|
||||||
* </ol>
|
|
||||||
*
|
|
||||||
* Example property file:
|
|
||||||
*
|
|
||||||
* <pre>
|
|
||||||
* analysis.data.dir=D:/path/to/analysis-data/
|
|
||||||
* </pre>
|
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*
|
*
|
||||||
|
|
|
@ -21,7 +21,7 @@ package org.apache.lucene.analysis.cn.smart;
|
||||||
* Internal SmartChineseAnalyzer character type constants.
|
* Internal SmartChineseAnalyzer character type constants.
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -33,7 +33,7 @@ import org.apache.lucene.util.AttributeSource;
|
||||||
* </p>
|
* </p>
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -23,7 +23,7 @@ import org.apache.lucene.analysis.cn.smart.hhmm.SegTokenFilter; // for javadoc
|
||||||
* SmartChineseAnalyzer utility constants and methods
|
* SmartChineseAnalyzer utility constants and methods
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -28,7 +28,7 @@ import org.apache.lucene.analysis.cn.smart.hhmm.SegTokenFilter;
|
||||||
* Segment a sentence of Chinese text into words.
|
* Segment a sentence of Chinese text into words.
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -32,7 +32,7 @@ import org.apache.lucene.analysis.tokenattributes.TypeAttribute;
|
||||||
* A {@link TokenFilter} that breaks sentences into words.
|
* A {@link TokenFilter} that breaks sentences into words.
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -21,7 +21,7 @@ package org.apache.lucene.analysis.cn.smart;
|
||||||
* Internal SmartChineseAnalyzer token type constants
|
* Internal SmartChineseAnalyzer token type constants
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -28,7 +28,7 @@ import java.io.UnsupportedEncodingException;
|
||||||
* </p>
|
* </p>
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -33,7 +33,7 @@ import org.apache.lucene.analysis.cn.smart.Utility;
|
||||||
* </p>
|
* </p>
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -36,7 +36,7 @@ import org.apache.lucene.analysis.cn.smart.AnalyzerProfile;
|
||||||
* SmartChineseAnalyzer Bigram dictionary.
|
* SmartChineseAnalyzer Bigram dictionary.
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -28,7 +28,7 @@ import org.apache.lucene.analysis.cn.smart.hhmm.PathNode;//javadoc @link
|
||||||
* Finds the optimal segmentation of a sentence into Chinese words
|
* Finds the optimal segmentation of a sentence into Chinese words
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -24,7 +24,7 @@ package org.apache.lucene.analysis.cn.smart.hhmm;
|
||||||
* </p>
|
* </p>
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -30,7 +30,7 @@ import java.util.Map;
|
||||||
* </p>
|
* </p>
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -25,7 +25,7 @@ import org.apache.lucene.analysis.cn.smart.WordType; // for javadocs
|
||||||
* SmartChineseAnalyzer internal token
|
* SmartChineseAnalyzer internal token
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -27,7 +27,7 @@ import org.apache.lucene.analysis.cn.smart.WordType;
|
||||||
* </p>
|
* </p>
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -23,7 +23,7 @@ import java.util.Arrays;
|
||||||
* A pair of tokens in {@link SegGraph}
|
* A pair of tokens in {@link SegGraph}
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -38,7 +38,7 @@ import org.apache.lucene.analysis.cn.smart.Utility;
|
||||||
*
|
*
|
||||||
* <p><font color="#FF0000">
|
* <p><font color="#FF0000">
|
||||||
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
* WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental.
|
||||||
* The APIs introduced here might change in the future and will not be
|
* The APIs and file formats introduced here might change in the future and will not be
|
||||||
* supported anymore in such a case.</font>
|
* supported anymore in such a case.</font>
|
||||||
* </p>
|
* </p>
|
||||||
*/
|
*/
|
||||||
|
|
|
@ -23,7 +23,7 @@ SmartChineseAnalyzer Hidden Markov Model package
|
||||||
<div>
|
<div>
|
||||||
<font color="#FF0000">
|
<font color="#FF0000">
|
||||||
WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental. The APIs
|
WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental. The APIs
|
||||||
introduced here might change in the future and will not be supported anymore
|
and file formats introduced here might change in the future and will not be supported anymore
|
||||||
in such a case.
|
in such a case.
|
||||||
</font>
|
</font>
|
||||||
</div>
|
</div>
|
||||||
|
|
|
@ -23,7 +23,7 @@ SmartChineseAnalyzer Tokenizers and TokenFilters
|
||||||
<div>
|
<div>
|
||||||
<font color="#FF0000">
|
<font color="#FF0000">
|
||||||
WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental. The APIs
|
WARNING: The status of the analyzers/smartcn <b>analysis.cn</b> package is experimental. The APIs
|
||||||
introduced here might change in the future and will not be supported anymore
|
and file formats introduced here might change in the future and will not be supported anymore
|
||||||
in such a case.
|
in such a case.
|
||||||
</font>
|
</font>
|
||||||
</div>
|
</div>
|
||||||
|
|
Loading…
Reference in New Issue