LUCENE-7873: The SPI lookup of Codecs, PostingsFormats, DocValuesFormats and all analysis factories was changed to only inspect the current classloader that defined the interface class (lucene-core.jar)

lucene/CHANGES.txt

@@ -133,6 +133,12 @@ Other
 * LUCENE-7719: Generalized the UnifiedHighlighter's support for AutomatonQuery
   for character & binary automata. Added AutomatonQuery.isBinary. (David Smiley)
 
+* LUCENE-7873: Due to serious problems with context class loaders in several
+  frameworks (OSGI, Java 9 Jigsaw), the lookup of Codecs, PostingsFormats,
+  DocValuesFormats and all analysis factories was changed to only inspect the
+  current classloader that defined the interface class (lucene-core.jar).
+  See MIGRATE.txt for more information!  (Uwe Schindler, Dawid Weiss)
+
 ======================= Lucene 6.7.0 =======================
 
 New Features

lucene/MIGRATE.txt

@@ -1,5 +1,46 @@
 # Apache Lucene Migration Guide
 
+## Changed SPI lookups for codecs and analysis changed (LUCENE-7873) ##
+
+Due to serious problems with context class loaders in several frameworks
+(OSGI, Java 9 Jigsaw), the lookup of Codecs, PostingsFormats, DocValuesFormats
+and all analysis factories was changed to only inspect the current classloader
+that defined the interface class (`lucene-core.jar`). Normal applications
+should not encounter any issues with that change, because the application
+classloader (unnamed module in Java 9) can load all SPIs from all JARs
+from classpath.
+
+For any code that relies on the old behaviour (e.g., certain web applications
+or components in application servers) one can manually instruct the Lucene
+SPI implementation to also inspect the context classloader. To do this,
+add this code to the early startup phase of your application before any
+Apache Lucene component is used:
+
+    ClassLoader cl = Thread.currentThread().getContextClassLoader();
+    // Codecs:
+    PostingsFormat.reloadPostingsFormats(cl);
+    DocValuesFormat.reloadDocValuesFormats(cl);
+    Codec.reloadCodecs(cl);
+    // Analysis:
+    CharFilterFactory.reloadCharFilters(cl);
+    TokenFilterFactory.reloadTokenFilters(cl);
+    TokenizerFactory.reloadTokenizers(cl);
+
+This code will reload all service providers from the given class loader
+(in our case the context class loader). Of course, instead of specifying
+the context class loader, it is receommended to use the application's main
+class loader or the module class loader.
+
+If you are migrating your project to Java 9 Jigsaw module system, keep in mind
+that Lucene currently does not yet support `module-info.java` declarations of
+service provider impls (`provides` statement). It is therefore recommended
+to keep all of Lucene in one Uber-Module and not try to split Lucene into
+several modules. As soon as Lucene will migrate to Java 9 as minimum requirement,
+we will work on improving that.
+
+For OSGI, the same applies. You have to create a bundle with all of Lucene for
+SPI to work correctly.
+
 ## Query.hashCode and Query.equals are now abstract methods (LUCENE-7277)
 
 Any custom query subclasses should redeclare equivalence relationship according

lucene/analysis/common/src/java/org/apache/lucene/analysis/util/AnalysisSPILoader.java

@@ -48,7 +48,7 @@ public final class AnalysisSPILoader<S extends AbstractAnalysisFactory> {
   }
 
   public AnalysisSPILoader(Class<S> clazz, String[] suffixes) {
-    this(clazz, suffixes, Thread.currentThread().getContextClassLoader());
+    this(clazz, suffixes, null);
   }
   
   public AnalysisSPILoader(Class<S> clazz, String[] suffixes, ClassLoader classloader) {

lucene/core/src/java/org/apache/lucene/util/NamedSPILoader.java

@@ -35,7 +35,7 @@ public final class NamedSPILoader<S extends NamedSPILoader.NamedSPI> implements
   private final Class<S> clazz;
 
   public NamedSPILoader(Class<S> clazz) {
-    this(clazz, Thread.currentThread().getContextClassLoader());
+    this(clazz, null);
   }
   
   public NamedSPILoader(Class<S> clazz, ClassLoader classloader) {

lucene/core/src/java/org/apache/lucene/util/SPIClassIterator.java

@@ -48,13 +48,11 @@ public final class SPIClassIterator<S> implements Iterator<Class<? extends S>> {
   private final Enumeration<URL> profilesEnum;
   private Iterator<String> linesIterator;
   
-  /** Creates a new SPI iterator to lookup services of type {@code clazz} using the context classloader. */
+  /** Creates a new SPI iterator to lookup services of type {@code clazz} using
+   * the same {@link ClassLoader} as the argument. */
   public static <S> SPIClassIterator<S> get(Class<S> clazz) {
-    ClassLoader cl = Thread.currentThread().getContextClassLoader();
-    if (cl == null) {
-      cl = clazz.getClassLoader();
-    }
-    return new SPIClassIterator<>(clazz, cl);
+    return new SPIClassIterator<>(clazz,
+        Objects.requireNonNull(clazz.getClassLoader(), () -> clazz + " has no classloader."));
   }
   
   /** Creates a new SPI iterator to lookup services of type {@code clazz} using the given classloader. */