Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Let WordDelimiterGraphFilterFactory propagate ignoreKeywords flag (#12525) 

* Let WordDelimiterGraphFilterFactory propagate ignoreKeywords flag

fixes https://github.com/apache/lucene/issues/12522

* Document changes

* Align with default in code
This commit is contained in:
ThomasDC 2023-11-28 18:28:07 +01:00 committed by GitHub
parent 203f506130
commit 502f15a89b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
lucene/CHANGES.txt
View File

@ -404,6 +404,8 @@ New Features
* GITHUB#12479: Add new Maximum Inner Product vector similarity function for non-normalized dot-product * GITHUB#12479: Add new Maximum Inner Product vector similarity function for non-normalized dot-product
vector search. (Jack Mazanec, Ben Trent) vector search. (Jack Mazanec, Ben Trent)
* GITHUB#12525: `WordDelimiterGraphFilterFactory` now supports the `ignoreKeywords` flag (Thomas De Craemer)
* GITHUB#12489: Add support for recursive graph bisection, also called * GITHUB#12489: Add support for recursive graph bisection, also called
bipartite graph partitioning, and often abbreviated BP, an algorithm for bipartite graph partitioning, and often abbreviated BP, an algorithm for
reordering doc IDs that results in more compact postings and faster queries, reordering doc IDs that results in more compact postings and faster queries,

5
lucene/analysis/common/src/java/org/apache/lucene/analysis/miscellaneous/WordDelimiterGraphFilterFactory.java
View File

@ -45,7 +45,7 @@ import org.apache.lucene.util.ResourceLoaderAware;
* preserveOriginal="0" splitOnNumerics="1" splitOnCaseChange="1" * preserveOriginal="0" splitOnNumerics="1" splitOnCaseChange="1"
* catenateWords="0" catenateNumbers="0" catenateAll="0" * catenateWords="0" catenateNumbers="0" catenateAll="0"
* generateWordParts="1" generateNumberParts="1" stemEnglishPossessive="1" * generateWordParts="1" generateNumberParts="1" stemEnglishPossessive="1"
* types="wdfftypes.txt" /> * types="wdfftypes.txt" ignoreKeywords="0" />
* </analyzer> * </analyzer>
* &lt;/fieldType&gt;</pre> * &lt;/fieldType&gt;</pre>
* *
@ -100,6 +100,9 @@ public class WordDelimiterGraphFilterFactory extends TokenFilterFactory
if (getInt(args, "stemEnglishPossessive", 1) != 0) { if (getInt(args, "stemEnglishPossessive", 1) != 0) {
flags |= STEM_ENGLISH_POSSESSIVE; flags |= STEM_ENGLISH_POSSESSIVE;
} }
if (getInt(args, "ignoreKeywords", 0) != 0) {
flags |= IGNORE_KEYWORDS;
}
wordFiles = get(args, PROTECTED_TOKENS); wordFiles = get(args, PROTECTED_TOKENS);
types = get(args, TYPES); types = get(args, TYPES);
this.flags = flags; this.flags = flags;