Commit Graph

21 Commits

Author SHA1 Message Date
Robert Muir 7dc025bdce LUCENE-3642: fix invalid offsets from CharTokenizer, [Edge]NGramFilters, SmartChinese, add sanity check to BaseTokenStreamTestCase
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1213329 13f79535-47bb-0310-9956-ffa450edef68
2011-12-12 17:28:09 +00:00
Simon Willnauer c0a7abbec0 LUCENE-2564: Cut over WordListLoader to CharArrayMap/Set and use CharSetDecoder to detect encoding problems early
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1200091 13f79535-47bb-0310-9956-ffa450edef68
2011-11-10 01:52:48 +00:00
Simon Willnauer dc6b4b6533 LUCENE-2564: Cut over WordListLoader to CharArrayMap/Set and use CharSetDecoder to detect encoding problems early
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1200080 13f79535-47bb-0310-9956-ffa450edef68
2011-11-10 01:21:25 +00:00
Christopher John Male 67c13bd2fe LUCENE-3455: Renamed Analyzer.reusableTokenStream to Analyzer.tokenStream
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1176728 13f79535-47bb-0310-9956-ffa450edef68
2011-09-28 05:26:54 +00:00
Christopher John Male 4ff0b2f82c LUCENE-3396: Collapsing Analyzer and ReusableAnalyzerBase together, mandating use of TokenStreamComponents
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1175297 13f79535-47bb-0310-9956-ffa450edef68
2011-09-25 05:10:25 +00:00
Christopher John Male 4c5606ee29 LUCENE-3396: Converted most Analyzers over to using ReusableAnalyzerBase
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1169607 13f79535-47bb-0310-9956-ffa450edef68
2011-09-12 05:50:26 +00:00
Robert Muir efbdae6dd2 LUCENE-3378: remove unneeded special test-classpaths in build
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1158821 13f79535-47bb-0310-9956-ffa450edef68
2011-08-17 16:45:37 +00:00
Steven Rowe 23d22e4d47 LUCENE-3337: avoid building jar files unless necessary in build
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1151720 13f79535-47bb-0310-9956-ffa450edef68
2011-07-28 04:02:09 +00:00
Steven Rowe 4505c08643 SOLR-2452: merged with trunk up r1144161; applied the svn movement script and the latest version of the post-svn-movement patch
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/solr2452@1144174 13f79535-47bb-0310-9956-ffa450edef68
2011-07-08 06:41:23 +00:00
Robert Muir 68061ef921 LUCENE-2560: add basic stress tests for analyzers
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1096178 13f79535-47bb-0310-9956-ffa450edef68
2011-04-23 16:55:15 +00:00
Robert Muir ecd795c585 LUCENE-3026: SmartChineseAnalyzer's WordTokenFilter threw NullPointerException on sentences longer than 32,767 characters
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1092328 13f79535-47bb-0310-9956-ffa450edef68
2011-04-14 15:15:31 +00:00
Shai Erera ece1524805 LUCENE-2609: Generate jar containing test classes (trunk)
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1067738 13f79535-47bb-0310-9956-ffa450edef68
2011-02-06 19:48:54 +00:00
Steven Rowe 11146b8c3c changed generate-maven-artifacts target to place all maven artifacts in one place: modules/dist/maven/; added modules/dist/ to list of dirs to remove with the 'clean' target; added modules/dist/ to svn:ignore list on modules/
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1062308 13f79535-47bb-0310-9956-ffa450edef68
2011-01-23 01:42:19 +00:00
Steven Rowe 74360c80f5 LUCENE-2657: Replace Maven POM templates with full POMs
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1061613 13f79535-47bb-0310-9956-ffa450edef68
2011-01-21 03:44:13 +00:00
Robert Muir 8696f549d4 LUCENE-2020: Remove unused imports
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1052926 13f79535-47bb-0310-9956-ffa450edef68
2010-12-26 19:16:42 +00:00
Robert Muir c8b7a21b4b clear up more compiler warnings
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1003906 13f79535-47bb-0310-9956-ffa450edef68
2010-10-02 22:20:26 +00:00
Ryan McKinley 3be9fedd84 getting 'generate-maven-artifacts' to work with analysis module
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@987286 13f79535-47bb-0310-9956-ffa450edef68
2010-08-19 19:58:36 +00:00
Robert Muir ad0e495911 LUCENE-2372: switch over remaining uses of TermAttribute
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@950008 13f79535-47bb-0310-9956-ffa450edef68
2010-06-01 10:35:13 +00:00
Robert Muir 71b59ca566 LUCENE-2413: consolidate remaining concrete core analyzers to modules/analysis
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@948195 13f79535-47bb-0310-9956-ffa450edef68
2010-05-25 20:16:44 +00:00
Robert Muir 5259d7d90b LUCENE-2413: move KeywordMarkerFilter to analyzers module
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@946621 13f79535-47bb-0310-9956-ffa450edef68
2010-05-20 13:23:12 +00:00
Robert Muir f6e9cc9f32 LUCENE-2444: move contrib/analyzers to modules/analysis
git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@941308 13f79535-47bb-0310-9956-ffa450edef68
2010-05-05 14:26:59 +00:00