Apache/lucene
1
0
Fork 0
mirror of https://github.com/apache/lucene.git synced 2025-02-25 20:12:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,776 Commits 31 Branches 469 Tags
Commit Graph

108 Commits

Author SHA1 Message Date
Robert Muir
8e4e7e17d0 LUCENE-1834: Remove unused code in SmartChineseAnalyzer hmm pkg 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@806891 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-22 21:03:08 +00:00
Robert Muir
1ebbe2abd1 LUCENE-1793: Deprecate custom encoding support in Greek and Russian analyzers 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@806886 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-22 20:36:06 +00:00
Robert Muir
1d9a96c2fc LUCENE-1813: Add option to ReverseStringFilter to mark reversed tokens 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@805769 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-19 12:07:15 +00:00
Robert Muir
58cd4a04d7 LUCENE-1794: Ensure analyzer options are applied immediately when using reusable token streams 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@805766 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-19 11:56:31 +00:00
Robert Muir
3887cf9419 LUCENE-1692: Additional tests and javadocs for contrib/analyzers 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@805400 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-18 12:55:26 +00:00
Robert Muir
d2af6ef0bd LUCENE-1794: Implement TokenStream reuse for contrib Analyzers 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@804680 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-16 12:37:05 +00:00
Uwe Schindler
b16e0aa31b LUCENE-1801: All Tokenizers/TokenStreams that are source of tokens call AttributeSource.clearAttributes() first. Made Token.clear() consistent to AttributeImpl (clear everything) 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@804392 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-14 22:01:42 +00:00
Robert Muir
43a5bd6c19 LUCENE-1628: Add Persian Analyzer 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802955 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-10 23:29:27 +00:00
Michael McCandless
35ea5c1350 LUCENE-1786: make the patternsFileContent static, so we only load it once, not 4 times, when running this test 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802767 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-10 12:47:52 +00:00
Uwe Schindler
911df49bcb LUCENE-1607: Change some more String.intern() in contrib 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802095 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-07 17:19:53 +00:00
Michael McCandless
ab0de2448a fix smartcn to be JDK 1.4 only 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@802085 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-07 16:48:09 +00:00
Robert Muir
820620f3a7 LUCENE-1758: Update ArabicAnalyzer to light10 stemming, stopwords improvements, lowercase non-arabic text 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@801348 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-05 18:22:22 +00:00
Grant Ingersoll
ab276a5ab9 Javadoc updates 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@801219 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-05 13:17:20 +00:00
Grant Ingersoll
c0d86a4e30 Javadoc updates 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@801218 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-05 13:17:11 +00:00
Michael Busch
c91651e4f2 LUCENE-1775: Change contrib tee/sink filters to use new TokenStream API. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@800606 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-03 22:45:27 +00:00
Michael Busch
457c29d31e LUCENE-1775: Change remaining contrib TokenFilters (shingle, prefix-suffix) to use the new TokenStream API. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@800195 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-03 04:33:10 +00:00
Michael Busch
b91f993a0e LUCENE-1460: Additional cleanup in two contrib junit tests. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@799973 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-02 02:57:30 +00:00
Michael Busch
537aeb24e0 LUCENE-1759: Set final offset correctly in contrib TokenStreams. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@799968 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-02 02:10:46 +00:00
Michael Busch
1743081b07 LUCENE-1460: Changed TokenStreams/TokenFilters in contrib to use the new TokenStream API. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@799953 13f79535-47bb-0310-9956-ffa450edef68
 2009-08-01 22:52:32 +00:00
Simon Willnauer
999f6157c7 LUCENE-1728: Splitted contrib/analyzers into common and smartcn. Smartcn depends on a large dictionary that causes the analyzers jar to grow up to 3MB compressed size. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@797150 13f79535-47bb-0310-9956-ffa450edef68
 2009-07-23 17:11:22 +00:00
Mark Robert Miller
ea7e4ad344 LUCENE-1688: Deprecate static final String stop word array in and StopAnalzyer and replace it with an immutable implementation of CharArraySet. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@794078 13f79535-47bb-0310-9956-ffa450edef68
 2009-07-14 21:39:22 +00:00
Otis Gospodnetic
b393e4d0af LUCENE-1491 - EdgeNGramTokenFilter no longer stops on tokens shorter than minimum n-gram size. 
- line, and those below, will be ignored--

M    CHANGES.txt
M    analyzers/src/test/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilterTest.java
M    analyzers/src/test/org/apache/lucene/analysis/ngram/NGramTokenFilterTest.java
M    analyzers/src/java/org/apache/lucene/analysis/ngram/EdgeNGramTokenFilter.java
M    analyzers/src/java/org/apache/lucene/analysis/ngram/NGramTokenFilter.java


git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@794034 13f79535-47bb-0310-9956-ffa450edef68
 2009-07-14 19:44:52 +00:00
Uwe Schindler
705f099238 Convert and cleanup the test files to UTF-8. What is still broken is the incorrect usage of KOI8 and CP1251 encodings. Added svn:eol-style=native to all files again. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@791483 13f79535-47bb-0310-9956-ffa450edef68
 2009-07-06 13:50:17 +00:00
Simon Willnauer
5265dc1bb2 LUCENE-1722: SmartChineseAnalyzer JavaDoc improvements - Replacing Chinese JavaDoc with English version. Robert Muir via Simon Willnauer 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@790102 13f79535-47bb-0310-9956-ffa450edef68
 2009-07-01 10:32:23 +00:00
Michael McCandless
87de0c9688 LUCENE-1466: added chainable CharFilter stage before Tokenizer to allow mapping of characters before tokenization 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@787795 13f79535-47bb-0310-9956-ffa450edef68
 2009-06-23 19:15:31 +00:00
Michael McCandless
ec8088654d bulk fix svn:eol-style to native for text files 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@787424 13f79535-47bb-0310-9956-ffa450edef68
 2009-06-22 22:18:56 +00:00
Michael McCandless
19234f12bd LUCENE-1692: add new contrib analyzer tests 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@786606 13f79535-47bb-0310-9956-ffa450edef68
 2009-06-19 18:02:12 +00:00
Michael McCandless
2f2cd20828 LUCENE-1692: add tests for Thai & SmartChinese analyzers; fix wrong endOffset bug in ThaiWordFilter; use stop words by default with SmartChineseAnalyzer 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@786560 13f79535-47bb-0310-9956-ffa450edef68
 2009-06-19 15:52:36 +00:00
Michael McCandless
835c405be0 LUCENE-973: add test case for CJKAnalyzer; fix trailing empty string bug 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@785287 13f79535-47bb-0310-9956-ffa450edef68
 2009-06-16 16:38:39 +00:00
Grant Ingersoll
1511ec5e31 LUCENE-1676: in-stream payload support 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@784297 13f79535-47bb-0310-9956-ffa450edef68
 2009-06-12 22:26:01 +00:00
Michael McCandless
af550281cb LUCENE-1629: remove unnecessary source files 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@775468 13f79535-47bb-0310-9956-ffa450edef68
 2009-05-16 14:08:38 +00:00
Michael McCandless
f81f6796a2 LUCENE-1629: correct ASF source headers 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@775444 13f79535-47bb-0310-9956-ffa450edef68
 2009-05-16 09:55:34 +00:00
Michael McCandless
be0a47b7e3 LUCENE-1629: move CHANGES entry to contrib; add TestArabicAnalyzer 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@774727 13f79535-47bb-0310-9956-ffa450edef68
 2009-05-14 10:50:52 +00:00
Michael McCandless
e01aad89fe LUCENE-1629: adding new contrib analyzer SmartChineseAnalyzer 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@774718 13f79535-47bb-0310-9956-ffa450edef68
 2009-05-14 10:09:22 +00:00
Michael McCandless
c73712d1bb LUCENE-1576: fix BrazilianAnalyzer to downcase before filtering stop words 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@759307 13f79535-47bb-0310-9956-ffa450edef68
 2009-03-27 19:04:25 +00:00
Michael McCandless
0f17904f1e remove slow download host; only download zip file once for all tests 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@759061 13f79535-47bb-0310-9956-ffa450edef68
 2009-03-27 08:44:33 +00:00
Michael McCandless
96863198a5 LUCENE-1490: fix latin1 conversion of HALFWIDTH_AND_FULLWIDTH_FORMS characters to only apply to the correct subset 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@755666 13f79535-47bb-0310-9956-ffa450edef68
 2009-03-18 17:28:53 +00:00
Yonik Seeley
6c176eb016 LUCENE-1398: Add ReverseStringFilter to contrib/analyzers 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@747915 13f79535-47bb-0310-9956-ffa450edef68
 2009-02-25 20:44:05 +00:00
Karl-Johan Wettin
d7376608b2 LUCENE-1514 
ShingleMatrixFilter#next(Token) easily throws a StackOverflowException due to recursive invocation. (Karl Wettin)


git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@733064 13f79535-47bb-0310-9956-ffa450edef68
 2009-01-09 15:34:52 +00:00
Grant Ingersoll
2225462178 LUCENE-1380: Add PositionFilter 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@725691 13f79535-47bb-0310-9956-ffa450edef68
 2008-12-11 14:17:44 +00:00
Grant Ingersoll
702ea32da7 make constructors public 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@724059 13f79535-47bb-0310-9956-ffa450edef68
 2008-12-07 00:39:35 +00:00
Yonik Seeley
8e8e8ddec4 set svn:eol-style to native on java files 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@712922 13f79535-47bb-0310-9956-ffa450edef68
 2008-11-11 02:35:46 +00:00
Grant Ingersoll
8dfe073760 LUCENE-1406. Added Arabic stemming and normalization. Also added new method to WordListLoader to allow for comments in word lists. 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@706342 13f79535-47bb-0310-9956-ffa450edef68
 2008-10-20 17:19:29 +00:00
Michael McCandless
3f27b17a89 fix non-1.4-compatible throws clause 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@701827 13f79535-47bb-0310-9956-ffa450edef68
 2008-10-05 16:40:59 +00:00
Otis Gospodnetic
0195fcd03d LUCENE-1378 - Removed the remaining 199 @author references 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@695514 13f79535-47bb-0310-9956-ffa450edef68
 2008-09-15 15:42:11 +00:00
Karl-Johan Wettin
71f2d8199b LUCENE-1320 
ShingleMatrixFilter JDK downgrade 1.5 -> 1.4 
Grant Ingersoll via Karl Wettin

git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@694393 13f79535-47bb-0310-9956-ffa450edef68
 2008-09-11 18:23:18 +00:00
Michael McCandless
4218996230 LUCENE-1366: rename Field.Index.* options 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@694004 13f79535-47bb-0310-9956-ffa450edef68
 2008-09-10 21:38:52 +00:00
Michael McCandless
d5a40278bc LUCENE-1369: switch from Hashtable to HashMap and from Vector to List, when possible 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@692921 13f79535-47bb-0310-9956-ffa450edef68
 2008-09-07 19:22:40 +00:00
Karl-Johan Wettin
bf238a5743 Javadocs fix 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@690779 13f79535-47bb-0310-9956-ffa450edef68
 2008-08-31 20:46:47 +00:00
Michael McCandless
003a853cc8 LUCENE-1333: don't use LuceneTestCase in contrib until we can fix the build dependency 
git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@687539 13f79535-47bb-0310-9956-ffa450edef68
 2008-08-21 02:45:37 +00:00