f75d40dad5
LUCENE-3894: try toning down for this tokenizer (it builds lots of tokens from the input treated as a path)
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1303276 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 04:30:11 +00:00
1156de050f
LUCENE-3894: add large docs tests for more tokenizers
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1303273 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 03:59:14 +00:00
dd7bfc78d9
LUCENE-3894: for tokenizers, add some tests for larger documents
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1303258 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 02:54:07 +00:00
3d73a3014e
LUCENE-3896: beef up TestDuelingAnalyzers for larger documents
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1303253 13f79535-47bb-0310-9956-ffa450edef68
2012-03-21 01:52:22 +00:00
c20242721f
LUCENE-3894: some tokenizers weren't reading all input chars
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1303193 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 23:02:37 +00:00
b7a7e5a625
LUCENE-3889: remove unnecessary/unused base class
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1303026 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 17:28:26 +00:00
5648222e86
SOLR-2764: Fix testcase for minimal stemmer
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1302872 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 13:12:39 +00:00
54d48eb98b
SOLR-2764: Create a NorwegianLightStemmer and NorwegianMinimalStemmer
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1302833 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 10:57:50 +00:00
790323780f
basic javadocs improvements, mostly simple descriptions where the class had nothing before
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1302752 13f79535-47bb-0310-9956-ffa450edef68
2012-03-20 02:09:25 +00:00
4a2b1d974a
javadocs: add missing package.htmls
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1302713 13f79535-47bb-0310-9956-ffa450edef68
2012-03-19 23:20:25 +00:00
c4f72f61ac
LUCENE-3880: UAX29URLEmailTokenizer now recognizes emails when the mailto: scheme is prepended.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1302265 13f79535-47bb-0310-9956-ffa450edef68
2012-03-19 03:13:52 +00:00
3d2d144f92
LUCENE-3848: don't produce tokenstreams that start with posinc=0
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1301478 13f79535-47bb-0310-9956-ffa450edef68
2012-03-16 13:06:30 +00:00
3d8b22ffd0
LUCENE-3850: Fix rawtypes warnings for Java 7 compiler ( #2 )
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1297162 13f79535-47bb-0310-9956-ffa450edef68
2012-03-05 18:48:04 +00:00
989530e17e
LUCENE-3850: Fix rawtypes warnings for Java 7 compiler
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1297048 13f79535-47bb-0310-9956-ffa450edef68
2012-03-05 13:34:40 +00:00
430365f7cc
Kuromoji now produces both compound words and the segmentation of those words in search mode (LUCENE-3767)
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1296805 13f79535-47bb-0310-9956-ffa450edef68
2012-03-04 13:34:13 +00:00
8c2e3cef8f
LUCENE-3820: limiting the amount of input for pattern matching to go past exponential time patterns, even if they happen. A nice catch from Mike too -- un-ignore testNastyPattern and look at processing time go wild with each additional input character...
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1294797 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 19:26:05 +00:00
f3cc65733b
Sysout of the randomized pattern.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1294518 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 08:15:38 +00:00
4d401ca87d
Test thread's name reflects the current seed.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1294514 13f79535-47bb-0310-9956-ffa450edef68
2012-02-28 08:04:42 +00:00
493bd8b42f
LUCENE-3820: optimistic limit on running time for the randomized pattern test. This doesn't eliminate the possibility of hitting an exponential time pattern, but I re-run a few times and it seems to be pretty stbale.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1294322 13f79535-47bb-0310-9956-ffa450edef68
2012-02-27 20:50:24 +00:00
7be5533989
LUCENE-3820: Wrong trailing index calculation in PatternReplaceCharFilter.
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1294141 13f79535-47bb-0310-9956-ffa450edef68
2012-02-27 13:13:10 +00:00
482c0610fd
[LUCENE-3731] - refactored analyzeText method to initializeIterator and made it abstract inside BaseUIMATokenizer
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1293614 13f79535-47bb-0310-9956-ffa450edef68
2012-02-25 14:14:00 +00:00
930816cc5b
LUCENE-3731 - AEProviderFactory getAEProvider logic cleaned
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1292585 13f79535-47bb-0310-9956-ffa450edef68
2012-02-22 23:39:51 +00:00
e51795be39
LUCENE-3731: remove unnecessary code
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1244714 13f79535-47bb-0310-9956-ffa450edef68
2012-02-15 20:53:53 +00:00
c97e3edbb9
LUCENE-3731: performance improvements and thread safety fixes to UIMA tokenizers
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1244688 13f79535-47bb-0310-9956-ffa450edef68
2012-02-15 20:29:20 +00:00
c454ae6a66
[LUCENE-3731] - creating and using simple wst and pos tagger implementations for analyzers' random string testing
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1244474 13f79535-47bb-0310-9956-ffa450edef68
2012-02-15 13:17:57 +00:00
cea3acb111
LUCENE-3731: fix javadoc warnings, add uima to eclipse project
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1244350 13f79535-47bb-0310-9956-ffa450edef68
2012-02-15 04:41:32 +00:00
8d9bfe9245
LUCENE-3731: adding missing overview.html
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1244340 13f79535-47bb-0310-9956-ffa450edef68
2012-02-15 04:01:57 +00:00
d66d97790b
[LUCENE-3731] - Creating the analysis-uima module for UIMA based tokenizers/analyzers
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1244236 13f79535-47bb-0310-9956-ffa450edef68
2012-02-14 22:13:34 +00:00
087f1e3126
LUCENE-3774: Optimized and streamlined license and notice file validation
...
by refactoring the build task into an ANT task and modifying build scripts
to perform top-level checks. (Dawid Weiss, Steve Rowe, Robert Muir)
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1243527 13f79535-47bb-0310-9956-ffa450edef68
2012-02-13 14:12:59 +00:00
6a07201844
don't fail test due to jre bugs in String.toLowerCase
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1243415 13f79535-47bb-0310-9956-ffa450edef68
2012-02-13 04:50:12 +00:00
590741dcfe
LUCENE-3766: Remove Tokenizer's default ctor
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1242890 13f79535-47bb-0310-9956-ffa450edef68
2012-02-10 19:12:35 +00:00
8a50cefc6b
LUCENE-3748: EnglishPossessiveFilter did not work with a proper right quotation mark
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1242740 13f79535-47bb-0310-9956-ffa450edef68
2012-02-10 11:01:11 +00:00
9f783ead67
SOLR-3115: improve japanese stopwords.txt description
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1242557 13f79535-47bb-0310-9956-ffa450edef68
2012-02-09 22:17:44 +00:00
509f4c557d
LUCENE-3751: align default japanese configurations for lucene/solr
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1242543 13f79535-47bb-0310-9956-ffa450edef68
2012-02-09 21:45:41 +00:00
72ae3171be
LUCENE-3765: Trappy behavior with StopFilter/ignoreCase
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1242497 13f79535-47bb-0310-9956-ffa450edef68
2012-02-09 19:59:50 +00:00
c0319d5928
SOLR-3056: document expectations in these files
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1241960 13f79535-47bb-0310-9956-ffa450edef68
2012-02-08 16:27:47 +00:00
dac1b58277
SOLR-3097, SOLR-3105: add fieldtypes for different languages to the example
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1241878 13f79535-47bb-0310-9956-ffa450edef68
2012-02-08 12:07:52 +00:00
bef6e3664d
LUCENE-3726: additional tests
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1240760 13f79535-47bb-0310-9956-ffa450edef68
2012-02-05 16:16:02 +00:00
03497e7595
LUCENE-3745: add proper Japanese stopping
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1240714 13f79535-47bb-0310-9956-ffa450edef68
2012-02-05 13:05:42 +00:00
009608d9f2
LUCENE-3726: default Kuromoji to search mode
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1240710 13f79535-47bb-0310-9956-ffa450edef68
2012-02-05 12:41:13 +00:00
6d3bb736f3
[LUCENE-3744] - applied patch for whiteList usage in TypeTokenFilter
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1240034 13f79535-47bb-0310-9956-ffa450edef68
2012-02-03 09:13:17 +00:00
60c36c24fb
don't let prefix's output bleed into full string's output (potential/latent bug)
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1239658 13f79535-47bb-0310-9956-ffa450edef68
2012-02-02 15:01:13 +00:00
995c5b9ef1
LUCENE-3730: improve Kuromoji search mode heuristics
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1239061 13f79535-47bb-0310-9956-ffa450edef68
2012-02-01 11:03:17 +00:00
8e40ea5bf8
LUCENE-3742: fix token offset for hangs-off-end output in SynonymFilter
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1238851 13f79535-47bb-0310-9956-ffa450edef68
2012-01-31 23:01:55 +00:00
10ba9abeb2
Reverse merged revision(s) from lucene/dev/trunk up to 1237502
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene2858@1237505 13f79535-47bb-0310-9956-ffa450edef68
2012-01-29 23:19:05 +00:00
d1165b1972
LUCENE-3725: add optional packing to FSTs
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1237500 13f79535-47bb-0310-9956-ffa450edef68
2012-01-29 22:48:45 +00:00
d7fe56ddae
LUCENE-2858: fix analyzer
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene2858@1237312 13f79535-47bb-0310-9956-ffa450edef68
2012-01-29 15:16:04 +00:00
97d62cc383
Fix offset array assertion off-by-one
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1236912 13f79535-47bb-0310-9956-ffa450edef68
2012-01-27 22:43:48 +00:00
f640687877
LUCENE-3720: add warning+experimental and disable test
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1236341 13f79535-47bb-0310-9956-ffa450edef68
2012-01-26 18:26:07 +00:00
6edfe4f157
LUCENE-3717: add tests
...
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1235199 13f79535-47bb-0310-9956-ffa450edef68
2012-01-24 10:40:46 +00:00
Robert Muir
Michael McCandless
Jan Høydahl
Steven Rowe
Uwe Schindler
Christian Moen
Dawid Weiss
Tommaso Teofili
Ryan McKinley