18 Commits

Author SHA1 Message Date
Robert Muir
1ebbe2abd1 LUCENE-1793: Deprecate custom encoding support in Greek and Russian analyzers
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@806886 13f79535-47bb-0310-9956-ffa450edef68
2009-08-22 20:36:06 +00:00
Robert Muir
1d9a96c2fc LUCENE-1813: Add option to ReverseStringFilter to mark reversed tokens
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@805769 13f79535-47bb-0310-9956-ffa450edef68
2009-08-19 12:07:15 +00:00
Robert Muir
58cd4a04d7 LUCENE-1794: Ensure analyzer options are applied immediately when using reusable token streams
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@805766 13f79535-47bb-0310-9956-ffa450edef68
2009-08-19 11:56:31 +00:00
Robert Muir
3887cf9419 LUCENE-1692: Additional tests and javadocs for contrib/analyzers
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@805400 13f79535-47bb-0310-9956-ffa450edef68
2009-08-18 12:55:26 +00:00
Robert Muir
d2af6ef0bd LUCENE-1794: Implement TokenStream reuse for contrib Analyzers
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@804680 13f79535-47bb-0310-9956-ffa450edef68
2009-08-16 12:37:05 +00:00
Uwe Schindler
b16e0aa31b LUCENE-1801: All Tokenizers/TokenStreams that are source of tokens call AttributeSource.clearAttributes() first. Made Token.clear() consistent to AttributeImpl (clear everything)
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@804392 13f79535-47bb-0310-9956-ffa450edef68
2009-08-14 22:01:42 +00:00
Robert Muir
43a5bd6c19 LUCENE-1628: Add Persian Analyzer
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802955 13f79535-47bb-0310-9956-ffa450edef68
2009-08-10 23:29:27 +00:00
Michael McCandless
35ea5c1350 LUCENE-1786: make the patternsFileContent static, so we only load it once, not 4 times, when running this test
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802767 13f79535-47bb-0310-9956-ffa450edef68
2009-08-10 12:47:52 +00:00
Uwe Schindler
911df49bcb LUCENE-1607: Change some more String.intern() in contrib
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802095 13f79535-47bb-0310-9956-ffa450edef68
2009-08-07 17:19:53 +00:00
Robert Muir
820620f3a7 LUCENE-1758: Update ArabicAnalyzer to light10 stemming, stopwords improvements, lowercase non-arabic text
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@801348 13f79535-47bb-0310-9956-ffa450edef68
2009-08-05 18:22:22 +00:00
Grant Ingersoll
ab276a5ab9 Javadoc updates
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@801219 13f79535-47bb-0310-9956-ffa450edef68
2009-08-05 13:17:20 +00:00
Grant Ingersoll
c0d86a4e30 Javadoc updates
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@801218 13f79535-47bb-0310-9956-ffa450edef68
2009-08-05 13:17:11 +00:00
Michael Busch
c91651e4f2 LUCENE-1775: Change contrib tee/sink filters to use new TokenStream API.
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@800606 13f79535-47bb-0310-9956-ffa450edef68
2009-08-03 22:45:27 +00:00
Michael Busch
457c29d31e LUCENE-1775: Change remaining contrib TokenFilters (shingle, prefix-suffix) to use the new TokenStream API.
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@800195 13f79535-47bb-0310-9956-ffa450edef68
2009-08-03 04:33:10 +00:00
Michael Busch
b91f993a0e LUCENE-1460: Additional cleanup in two contrib junit tests.
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@799973 13f79535-47bb-0310-9956-ffa450edef68
2009-08-02 02:57:30 +00:00
Michael Busch
537aeb24e0 LUCENE-1759: Set final offset correctly in contrib TokenStreams.
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@799968 13f79535-47bb-0310-9956-ffa450edef68
2009-08-02 02:10:46 +00:00
Michael Busch
1743081b07 LUCENE-1460: Changed TokenStreams/TokenFilters in contrib to use the new TokenStream API.
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@799953 13f79535-47bb-0310-9956-ffa450edef68
2009-08-01 22:52:32 +00:00
Simon Willnauer
999f6157c7 LUCENE-1728: Splitted contrib/analyzers into common and smartcn. Smartcn depends on a large dictionary that causes the analyzers jar to grow up to 3MB compressed size.
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@797150 13f79535-47bb-0310-9956-ffa450edef68
2009-07-23 17:11:22 +00:00