mirror of https://github.com/apache/lucene.git
268 lines
10 KiB
Plaintext
268 lines
10 KiB
Plaintext
Lucene contrib change Log
|
|
|
|
======================= Trunk (not yet released) =======================
|
|
|
|
Changes in runtime behavior
|
|
|
|
API Changes
|
|
|
|
* LUCENE-1936: Deprecated RussianLowerCaseFilter, because it transforms
|
|
text exactly the same as LowerCaseFilter. Please use LowerCaseFilter
|
|
instead, which has the same functionality. (Robert Muir)
|
|
|
|
Bug fixes
|
|
|
|
* LUCENE-1781: Fixed various issues with the lat/lng bounding box
|
|
distance filter created for radius search in contrib/spatial.
|
|
(Bill Bell via Mike McCandless)
|
|
|
|
New features
|
|
|
|
* LUCENE-1924: Added BalancedSegmentMergePolicy to contrib/misc,
|
|
which is a merge policy that tries to avoid doing very large
|
|
segment merges to give better search performance in a mixed
|
|
indexing/searching environment. (John Wang via Mike McCandless)
|
|
|
|
Optimizations
|
|
|
|
Documentation
|
|
|
|
Build
|
|
|
|
Test Cases
|
|
======================= Release 2.9.0 2009-09-23 =======================
|
|
|
|
Changes in runtime behavior
|
|
|
|
* LUCENE-1505: Local lucene now uses org.apache.lucene.util.NumericUtils for all
|
|
number conversion. You'll need to fully re-index any previously created indexes.
|
|
This isn't a break in back-compatibility because local Lucene has not yet
|
|
been released. (Mike McCandless)
|
|
|
|
* LUCENE-1758: ArabicAnalyzer now uses the light10 algorithm, has a refined
|
|
default stopword list, and lowercases non-Arabic text.
|
|
You'll need to fully re-index any previously created indexes. This isn't a
|
|
break in back-compatibility because ArabicAnalyzer has not yet been
|
|
released. (Robert Muir)
|
|
|
|
|
|
API Changes
|
|
|
|
* LUCENE-1695: Update the Highlighter to use the new TokenStream API. This issue breaks backwards
|
|
compatibility with some public classes. If you have implemented custom Fragmenters or Scorers,
|
|
you will need to adjust them to work with the new TokenStream API. Rather than getting passed a
|
|
Token at a time, you will be given a TokenStream to init your impl with - store the Attributes
|
|
you are interested in locally and access them on each call to the method that used to pass a new
|
|
Token. Look at the included updated impls for examples. (Mark Miller)
|
|
|
|
* LUCENE-1460: Change contrib TokenStreams/Filters to use the new
|
|
TokenStream API. (Robert Muir, Michael Busch)
|
|
|
|
* LUCENE-1775, LUCENE-1903: Change remaining TokenFilters (shingle, prefix-suffix)
|
|
to use the new TokenStream API. ShingleFilter is much more efficient now,
|
|
it clones much less often and computes the tokens mostly on the fly now.
|
|
Also added more tests. (Robert Muir, Michael Busch, Uwe Schindler, Chris Harris)
|
|
|
|
* LUCENE-1685: The position aware SpanScorer has become the default scorer
|
|
for Highlighting. The SpanScorer implementation has replaced QueryScorer
|
|
and the old term highlighting QueryScorer has been renamed to
|
|
QueryTermScorer. Multi-term queries are also now expanded by default. If
|
|
you were previously rewriting the query for multi-term query highlighting,
|
|
you should no longer do that (unless you switch to using QueryTermScorer).
|
|
The SpanScorer API (now QueryScorer) has also been improved to more closely
|
|
match the API of the previous QueryScorer implementation. (Mark Miller)
|
|
|
|
* LUCENE-1793: Deprecate the custom encoding support in the Greek and Russian
|
|
Analyzers. If you need to index text in these encodings, please use Java's
|
|
character set conversion facilities (InputStreamReader, etc) during I/O,
|
|
so that Lucene can analyze this text as Unicode instead. (Robert Muir)
|
|
|
|
Bug fixes
|
|
|
|
* LUCENE-1423: InstantiatedTermEnum#skipTo(Term) throws ArrayIndexOutOfBounds on empty index.
|
|
(Karl Wettin)
|
|
|
|
* LUCENE-1462: InstantiatedIndexWriter did not reset pre analyzed TokenStreams the
|
|
same way IndexWriter does. Parts of InstantiatedIndex was not Serializable.
|
|
(Karl Wettin)
|
|
|
|
* LUCENE-1510: InstantiatedIndexReader#norms methods throws NullPointerException on empty index.
|
|
(Karl Wettin, Robert Newson)
|
|
|
|
* LUCENE-1514: ShingleMatrixFilter#next(Token) easily throws a StackOverflowException
|
|
due to recursive invocation. (Karl Wettin)
|
|
|
|
* LUCENE-1548: Fix distance normalization in LevenshteinDistance to
|
|
not produce negative distances (Thomas Morton via Mike McCandless)
|
|
|
|
* LUCENE-1490: Fix latin1 conversion of HALFWIDTH_AND_FULLWIDTH_FORMS
|
|
characters to only apply to the correct subset (Daniel Cheng via
|
|
Mike McCandless)
|
|
|
|
* LUCENE-1576: Fix BrazilianAnalyzer to downcase tokens after
|
|
StandardTokenizer so that stop words with mixed case are filtered
|
|
out. (Rafael Cunha de Almeida, Douglas Campos via Mike McCandless)
|
|
|
|
* LUCENE-1491: EdgeNGramTokenFilter no longer stops on tokens shorter than minimum n-gram size.
|
|
(Todd Teak via Otis Gospodnetic)
|
|
|
|
* LUCENE-1683: Fixed JavaUtilRegexCapabilities (an impl used by
|
|
RegexQuery) to use Matcher.matches() not Matcher.lookingAt() so
|
|
that the regexp must match the entire string, not just a prefix.
|
|
(Trejkaz via Mike McCandless)
|
|
|
|
* LUCENE-1792: Fix new query parser to set rewrite method for
|
|
multi-term queries. (Luis Alves, Mike McCandless via Michael Busch)
|
|
|
|
* LUCENE-1828: Fix memory index to call TokenStream.reset() and
|
|
TokenStream.end(). (Tim Smith via Michael Busch)
|
|
|
|
* LUCENE-1912: Fix fast-vector-highlighter issue when two or more
|
|
terms are concatenated (Koji Sekiguchi via Mike McCandless)
|
|
|
|
New features
|
|
|
|
* LUCENE-1531: Added support for BoostingTermQuery to XML query parser. (Karl Wettin)
|
|
|
|
* LUCENE-1435: Added contrib/collation, a CollationKeyFilter
|
|
allowing you to convert tokens into CollationKeys encoded using
|
|
IndexableBinaryStringTools. This allows for faster RangeQuery when
|
|
a field needs to use a custom Collator. (Steven Rowe via Mike
|
|
McCandless)
|
|
|
|
* LUCENE-1591: EnWikiDocMaker, LineDocMaker, WriteLineDoc can now
|
|
read/write bz2 using Apache commons compress library. This means
|
|
you can download the .bz2 export from http://wikipedia.org and
|
|
immediately index it. (Shai Erera via Mike McCandless)
|
|
|
|
* LUCENE-1629: Add SmartChineseAnalyzer to contrib/analyzers. It
|
|
improves on CJKAnalyzer and ChineseAnalyzer by handling Chinese
|
|
sentences properly. SmartChineseAnalyzer uses a Hidden Markov
|
|
Model to tokenize Chinese words in a more intelligent way.
|
|
(Xiaoping Gao via Mike McCandless)
|
|
|
|
* LUCENE-1676: Added DelimitedPayloadTokenFilter class for automatically adding payloads "in-stream" (Grant Ingersoll)
|
|
|
|
* LUCENE-1578: Support for loading unoptimized readers to the
|
|
constructor of InstantiatedIndex. (Karl Wettin)
|
|
|
|
* LUCENE-1704: Allow specifying the Tidy configuration file when
|
|
parsing HTML docs with contrib/ant. (Keith Sprochi via Mike
|
|
McCandless)
|
|
|
|
* LUCENE-1522: Added contrib/fast-vector-highlighter, a new alternative
|
|
highlighter. (Koji Sekiguchi via Mike McCandless)
|
|
|
|
* LUCENE-1740: Added "analyzer" command to Lucli, enabling changing
|
|
the analyzer from the default StandardAnalyzer. (Bernd Fondermann
|
|
via Mike McCandless)
|
|
|
|
* LUCENE-1272: Add get/setBoost to MoreLikeThis. (Jonathan
|
|
Leibiusky via Mike McCandless)
|
|
|
|
* LUCENE-1745: Added constructors to JakartaRegexpCapabilities and
|
|
JavaUtilRegexCapabilities as well as static flags to support
|
|
configuring a RegexCapabilities implementation with the
|
|
implementation-specific modifier flags. Allows for callers to
|
|
customize the RegexQuery using the implementation-specific options
|
|
and fine tune how regular expressions are compiled and
|
|
matched. (Marc Zampetti zampettim@aim.com via Mike McCandless)
|
|
|
|
* LUCENE-1567: Added a new QueryParser framework, that allows
|
|
implementing a new query syntax in a flexible and efficient way.
|
|
This new QueryParser will be moved to Lucene's core in release
|
|
3.0 and will then replace the current core QueryParser, which
|
|
has been deprecated with this patch.
|
|
(Luis Alves and Adriano Campos via Michael Busch)
|
|
|
|
* LUCENE-1486: Added ComplexPhraseQueryParser, an extension of QueryParser
|
|
that allows a subset of the Lucene query language to be embedded in
|
|
PhraseQuerys. Wildcard, Range, and Fuzzy queries, as well as limited
|
|
boolean logic, can be used within quote operators with this parser, ie:
|
|
"(jo* -john) smyth~". (Mark Harwood via Mark Miller)
|
|
|
|
* Added web-based demo of functionality in contrib's XML Query Parser
|
|
packaged as War file (Mark Harwood)
|
|
|
|
* LUCENE-1406: Added Arabic analyzer. (Robert Muir via Grant Ingersoll)
|
|
|
|
* LUCENE-1628: Added Persian analyzer. (Robert Muir)
|
|
|
|
* LUCENE-1813: Add option to ReverseStringFilter to mark reversed tokens.
|
|
(Andrzej Bialecki via Robert Muir)
|
|
|
|
Optimizations
|
|
|
|
* LUCENE-1643: Re-use the collation key (RawCollationKey) for
|
|
better performance, in ICUCollationKeyFilter. (Robert Muir via
|
|
Mike McCandless)
|
|
|
|
* LUCENE-1794: Implement TokenStream reuse for contrib Analyzers,
|
|
and implement reset() for TokenStreams to support reuse. (Robert Muir)
|
|
|
|
Documentation
|
|
|
|
* LUCENE-1876: added missing package level documentation for numerous
|
|
contrib packages.
|
|
(Steven Rowe & Robert Muir)
|
|
|
|
Build
|
|
|
|
* LUCENE-1728: Split contrib/analyzers into common and smartcn modules.
|
|
Contrib/analyzers now builds an additional lucene-smartcn Jar file. All
|
|
smartcn classes are not included in the lucene-analyzers JAR file.
|
|
(Robert Muir via Simon Willnauer)
|
|
|
|
* LUCENE-1829: Fix contrib query parser to properly create javacc files.
|
|
(Jan-Pascal and Luis Alves via Michael Busch)
|
|
|
|
Test Cases
|
|
|
|
|
|
======================= Release 2.4.0 2008-10-06 =======================
|
|
|
|
Changes in runtime behavior
|
|
|
|
(None)
|
|
|
|
API Changes
|
|
|
|
1.
|
|
|
|
(None)
|
|
|
|
Bug fixes
|
|
|
|
1. LUCENE-1312: Added full support for InstantiatedIndexReader#getFieldNames()
|
|
and tests that assert that deleted documents behaves as they should (they did).
|
|
(Jason Rutherglen, Karl Wettin)
|
|
|
|
2. LUCENE-1318: InstantiatedIndexReader.norms(String, b[], int) didn't treat
|
|
the array offset right. (Jason Rutherglen via Karl Wettin)
|
|
|
|
New features
|
|
|
|
1. LUCENE-1320: ShingleMatrixFilter, multidimensional shingle token filter. (Karl Wettin)
|
|
|
|
2. LUCENE-1142: Updated Snowball package, org.tartarus distribution revision 500.
|
|
Introducing Hungarian, Turkish and Romanian support, updated older stemmers
|
|
and optimized (reflectionless) SnowballFilter.
|
|
IMPORTANT NOTICE ON BACKWARDS COMPATIBILITY: an index created using the 2.3.2 (or older)
|
|
might not be compatible with these updated classes as some algorithms have changed.
|
|
(Karl Wettin)
|
|
|
|
3. LUCENE-1016: TermVectorAccessor, transparent vector space access via stored vectors
|
|
or by resolving the inverted index. (Karl Wettin)
|
|
|
|
Documentation
|
|
|
|
(None)
|
|
|
|
Build
|
|
|
|
(None)
|
|
|
|
Test Cases
|
|
|
|
(None)
|