mirror of https://github.com/apache/lucene.git
SOLR-769: Add clustering contrib capabilities to Solr
git-svn-id: https://svn.apache.org/repos/asf/lucene/solr/trunk@776692 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Grant Ingersoll
parent
ea3399a19a
commit
1fc5ecc6c2
95
LICENSE.txt
95
LICENSE.txt
|
|
@ -736,6 +736,101 @@ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
|||
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
||||
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
==========================================================================
|
||||
contrib/clustering
|
||||
==========================================================================
|
||||
Carrot2 Project
|
||||
|
||||
Copyright (C) 2002-2008, Dawid Weiss, Stanislaw Osinski.
|
||||
Portions (C) Contributors listed in "carrot2.CONTRIBUTORS" file.
|
||||
All rights reserved.
|
||||
|
||||
Redistribution and use in source and binary forms, with or without modification,
|
||||
are permitted provided that the following conditions are met:
|
||||
|
||||
- Redistributions of source code must retain the above copyright notice, this
|
||||
list of conditions and the following disclaimer.
|
||||
|
||||
- Redistributions in binary form must reproduce the above copyright notice, this
|
||||
list of conditions and the following disclaimer in the documentation and/or
|
||||
other materials provided with the distribution.
|
||||
|
||||
- Neither the name of the Poznan University of Technology, Poznan, Poland nor
|
||||
the names of its contributors may be used to endorse or promote products
|
||||
derived from this software without specific prior written permission.
|
||||
|
||||
- We request that you include in the end-user documentation provided with the
|
||||
redistribution and/or in the software itself an acknowledgement equivalent to
|
||||
the following: "This product includes software developed by the Carrot2
|
||||
Project."
|
||||
|
||||
- No algorithms or technical solutions in the project may be patented or claimed
|
||||
proprietary.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
||||
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
||||
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
|
||||
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
||||
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
||||
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
|
||||
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
||||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
==========================================================================
|
||||
EHCache
|
||||
/**
|
||||
* Copyright 2003-2008 Luck Consulting Pty Ltd
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
==========================================================================
|
||||
Google Collections
|
||||
/**
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
==========================================================================
|
||||
Jackson
|
||||
|
||||
/**
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
|
||||
===========================================================================
|
||||
Apache Tika Licenses - contrib/extraction
|
||||
---------------------------------------------------------------------------
|
||||
|
|
|
|||
43
NOTICE.txt
43
NOTICE.txt
|
|
@ -12,6 +12,7 @@ including, but not limited to:
|
|||
- Apache Tomcat (lib/servlet-api-2.4.jar)
|
||||
- Apache Commons
|
||||
- Apache Geronimo (stax API jar)
|
||||
- Apache Log4j (contrib/clustering)
|
||||
|
||||
This product includes tests written with EasyMock Copyright 2001-2007
|
||||
Tammo Freese (http://www.easymock.org/)
|
||||
|
|
@ -141,3 +142,45 @@ Copyright (c) 1995-2005 International Business Machines Corporation and others
|
|||
Copyright (c) 2000-2005 INRIA, France Telecom
|
||||
|
||||
|
||||
=========================================================================
|
||||
== Carrot2 Notice ==
|
||||
=========================================================================
|
||||
Copyright (C) 2002-2008, Dawid Weiss, Stanislaw Osinski.
|
||||
Portions (C) Contributors listed in "carrot2.CONTRIBUTORS" file.
|
||||
All rights reserved.
|
||||
|
||||
This product includes software developed by the Carrot2 Project.
|
||||
|
||||
See http://project.carrot2.org/
|
||||
=======
|
||||
=========================================================================
|
||||
== EHCache Notice ==
|
||||
=========================================================================
|
||||
Copyright 2003-2008 Luck Consulting Pty Ltd
|
||||
|
||||
This product includes software developed by the EHCache Project
|
||||
|
||||
See ????
|
||||
|
||||
=========================================================================
|
||||
== Google Collections Notice ==
|
||||
|
||||
=========================================================================
|
||||
|
||||
Copyright ???? Google, Inc.
|
||||
|
||||
This product includes software developed by the Google Collections project.
|
||||
|
||||
See ????
|
||||
|
||||
=========================================================================
|
||||
== Jackson Notice ==
|
||||
=========================================================================
|
||||
Copyright ????
|
||||
|
||||
This product includes software developed by the Jackson project.
|
||||
|
||||
See ????
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
34
build.xml
34
build.xml
|
|
@ -188,6 +188,15 @@
|
|||
<path id="javadoc.classpath">
|
||||
<path refid="compile.classpath"/>
|
||||
<path refid="compile.classpath.solrj"/>
|
||||
<fileset dir="contrib/clustering/lib">
|
||||
<include name="*.jar"/>
|
||||
</fileset>
|
||||
<fileset dir="contrib/clustering/lib/downloads">
|
||||
<include name="*.jar"/>
|
||||
</fileset>
|
||||
<fileset dir="contrib/extraction/lib">
|
||||
<include name="*.jar"/>
|
||||
</fileset>
|
||||
<pathelement location="${dest}/client/solrj"/>
|
||||
</path>
|
||||
|
||||
|
|
@ -198,11 +207,13 @@
|
|||
<packageset dir="${src}/java" />
|
||||
<packageset dir="${src}/webapp/src" />
|
||||
<packageset dir="contrib/dataimporthandler/src/main/java" />
|
||||
<packageset dir="contrib/clustering/src/main/java" />
|
||||
<packageset dir="contrib/extraction/src/main/java" />
|
||||
<group title="Core" packages="org.apache.*" />
|
||||
<group title="Common" packages="org.apache.solr.common.*" />
|
||||
<group title="SolrJ" packages="org.apache.solr.client.solrj*" />
|
||||
<group title="contrib: DataImportHandler" packages="org.apache.solr.handler.dataimport*" />
|
||||
<group title="contrib: Clustering" packages="org.apache.solr.handler.clustering*" />
|
||||
<group title="contrib: Solr Cell" packages="org.apache.solr.handler.extraction*" />
|
||||
</sources>
|
||||
</invoke-javadoc>
|
||||
|
|
@ -397,6 +408,7 @@
|
|||
<fileset dir="src/java"/>
|
||||
<fileset dir="src/webapp/src"/>
|
||||
<fileset dir="contrib/dataimporthandler/src/main/java" />
|
||||
<fileset dir="contrib/clustering/src/main/java" />
|
||||
<fileset dir="contrib/extraction/src/main/java" />
|
||||
</clover-setup>
|
||||
</target>
|
||||
|
|
@ -493,6 +505,8 @@
|
|||
|
||||
<solr-jar destfile="${dist}/apache-solr-cell-src-${version}.jar"
|
||||
basedir="contrib/extraction/src" />
|
||||
<solr-jar destfile="${dist}/apache-solr-clustering-src-${version}.jar"
|
||||
basedir="contrib/clustering/src" />
|
||||
</target>
|
||||
|
||||
<target name="dist-javadoc" description="Creates the Solr javadoc distribution files"
|
||||
|
|
@ -505,6 +519,8 @@
|
|||
basedir="${build.javadoc}/solrj" />
|
||||
<solr-jar destfile="${dist}/apache-solr-dataimporthandler-docs-${version}.jar"
|
||||
basedir="${build.javadoc}/contrib-solr-dataimporthandler" />
|
||||
<solr-jar destfile="${dist}/apache-solr-clustering-docs-${version}.jar"
|
||||
basedir="${build.javadoc}/contrib-solr-clustering" />
|
||||
<solr-jar destfile="${dist}/apache-solr-cell-docs-${version}.jar"
|
||||
basedir="${build.javadoc}/contrib-solr-cell" />
|
||||
</target>
|
||||
|
|
@ -686,6 +702,8 @@
|
|||
<sign-maven-artifacts artifact.id="solr-core"/>
|
||||
<sign-maven-artifacts artifact.id="solr-dataimporthandler"/>
|
||||
<sign-maven-artifacts artifact.id="solr-dataimporthandler-extras"/>
|
||||
<sign-maven-artifacts artifact.id="solr-clustering"/>
|
||||
|
||||
<sign-maven-artifacts artifact.id="solr-cell"/>
|
||||
<sign-maven-dependency-artifacts artifact.id="solr-lucene-analyzers"/>
|
||||
<sign-maven-dependency-artifacts artifact.id="solr-lucene-core"/>
|
||||
|
|
@ -761,9 +779,9 @@
|
|||
|
||||
<!-- ========== SOLR ARTIFACTS ========== -->
|
||||
|
||||
|
||||
<m2-deploy pom.xml="contrib/dataimporthandler/solr-dataimporthandler-pom.xml.template"
|
||||
jar.file="${dist}/apache-solr-dataimporthandler-${version}.jar">
|
||||
|
||||
<artifact-attachments>
|
||||
<attach file="${dist}/apache-solr-dataimporthandler-src-${version}.jar" classifier="sources"/>
|
||||
<attach file="${dist}/apache-solr-dataimporthandler-docs-${version}.jar" classifier="javadoc"/>
|
||||
|
|
@ -781,13 +799,23 @@
|
|||
|
||||
<m2-deploy pom.xml="contrib/extraction/solr-cell-pom.xml.template"
|
||||
jar.file="${dist}/apache-solr-cell-${version}.jar">
|
||||
|
||||
<artifact-attachments>
|
||||
<attach file="${dist}/apache-solr-cell-src-${version}.jar" classifier="sources"/>
|
||||
<attach file="${dist}/apache-solr-cell-docs-${version}.jar" classifier="javadoc"/>
|
||||
</artifact-attachments>
|
||||
</m2-deploy>
|
||||
|
||||
<m2-deploy pom.xml="contrib/clustering/solr-clustering-pom.xml.template"
|
||||
jar.file="${dist}/apache-solr-clustering-${version}.jar">
|
||||
|
||||
<artifact-attachments>
|
||||
<attach file="${dist}/apache-solr-clustering-src-${version}.jar" classifier="sources"/>
|
||||
<attach file="${dist}/apache-solr-clustering-docs-${version}.jar" classifier="javadoc"/>
|
||||
</artifact-attachments>
|
||||
</m2-deploy>
|
||||
<!-- Clustring specific -->
|
||||
<!-- TODO: MORE NEEDED HERE ONCE WE FINALIZE THE LIBS FOR CARROT -->
|
||||
<!-- end clustering specific -->
|
||||
|
||||
<m2-deploy pom.xml="${src}/maven/solr-core-pom.xml.template"
|
||||
jar.file="${dist}/apache-solr-core-${version}.jar">
|
||||
|
|
@ -834,6 +862,8 @@
|
|||
</fileset>
|
||||
<fileset dir="contrib/dataimporthandler/src/main/java"/>
|
||||
<fileset dir="contrib/dataimporthandler/src/test/java"/>
|
||||
<fileset dir="contrib/clustering/src/main/java"/>
|
||||
<fileset dir="contrib/clustering/src/test/java"/>
|
||||
<fileset dir="contrib/extraction/src/main/java"/>
|
||||
<fileset dir="contrib/extraction/src/test/java"/>
|
||||
</rat:report>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,8 @@
|
|||
The Clustering contrib plugin for Solr provides a generic mechanism for plugging in third party clustering implementations.
|
||||
It currently provides clustering support for search results using the Carrot2 project.
|
||||
|
||||
See http://wiki.apache.org/solr/ClusteringComponent for how to get started.
|
||||
|
||||
Also, note, some of the Carrot2 libraries cannot be distributed in binary form because they are LGPL. Thus, you will have
|
||||
to download those components. See the build.xml file located in this directory for the location of the libraries.
|
||||
The libraries you will need are: nni.jar, Colt and PNJ.
|
||||
|
|
@ -0,0 +1,172 @@
|
|||
<?xml version="1.0"?>
|
||||
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<project name="solr-clustering" default="build">
|
||||
|
||||
<property name="solr-path" value="../.." />
|
||||
|
||||
<import file="../../common-build.xml"/>
|
||||
|
||||
<description>
|
||||
Clustering Integraton
|
||||
</description>
|
||||
|
||||
<path id="common.classpath">
|
||||
<fileset dir="lib"/>
|
||||
<fileset dir="lib/downloads"/>
|
||||
<pathelement location="${solr-path}/build/solr" />
|
||||
<pathelement location="${solr-path}/build/solrj" />
|
||||
<fileset dir="${solr-path}/lib" includes="*.jar"/>
|
||||
</path>
|
||||
|
||||
<path id="test.classpath">
|
||||
<pathelement path="${dest}/classes" />
|
||||
<pathelement path="${dest}/test-classes" />
|
||||
<pathelement path="${java.class.path}"/>
|
||||
<path refid="common.classpath" />
|
||||
</path>
|
||||
|
||||
<target name="clean">
|
||||
<delete failonerror="false" dir="${dest}"/>
|
||||
|
||||
</target>
|
||||
<target name="clean-downloads">
|
||||
<delete dir="lib/downloads"/>
|
||||
</target>
|
||||
|
||||
<target name="init">
|
||||
<mkdir dir="${dest}/classes"/>
|
||||
<mkdir dir="${build.javadoc}" />
|
||||
<ant dir="../../" inheritall="false" target="compile" />
|
||||
<ant dir="../../" inheritall="false" target="make-manifest" />
|
||||
<mkdir dir="lib/downloads" />
|
||||
</target>
|
||||
|
||||
<target name="check-files">
|
||||
<available file="lib/downloads/colt-1.2.0.jar" property="colt.exists"/>
|
||||
<available file="lib/downloads/pcj-1.2.jar" property="pcj.exists"/>
|
||||
<available file="lib/downloads/nni-1.0.0.jar" property="nni.exists"/>
|
||||
</target>
|
||||
<target name="get-colt" depends="check-files" unless="colt.exists">
|
||||
<!-- Get the LGPL deps and put them in a separate dir -->
|
||||
<get src="http://repo1.maven.org/maven2/colt/colt/1.2.0/colt-1.2.0.jar" dest="lib/downloads/colt-1.2.0.jar"/>
|
||||
</target>
|
||||
<target name="get-nni" depends="check-files" unless="nni.exists">
|
||||
<!-- Get the LGPL deps and put them in a separate dir -->
|
||||
<get src="http://download.carrot2.org/maven2/org/carrot2/nni/1.0.0/nni-1.0.0.jar" dest="lib/downloads/nni-1.0.0.jar"/>
|
||||
|
||||
</target>
|
||||
<target name="get-pcj" depends="check-files" unless="pcj.exists">
|
||||
<!-- Get the LGPL deps and put them in a separate dir -->
|
||||
<get src="http://repo1.maven.org/maven2/pcj/pcj/1.2/pcj-1.2.jar" dest="lib/downloads/pcj-1.2.jar"/>
|
||||
</target>
|
||||
<target name="get-libraries" depends="init, get-colt, get-pcj, get-nni"/>
|
||||
|
||||
<target name="compile" depends="init, get-libraries">
|
||||
<solr-javac destdir="${dest}/classes"
|
||||
classpathref="common.classpath">
|
||||
<src path="src/main/java" />
|
||||
</solr-javac>
|
||||
</target>
|
||||
|
||||
<target name="build" depends="compile">
|
||||
<solr-jar destfile="${dest}/${fullnamever}.jar" basedir="${dest}/classes"
|
||||
manifest="${common.dir}/${dest}/META-INF/MANIFEST.MF" />
|
||||
</target>
|
||||
|
||||
<target name="compileTests" depends="compile">
|
||||
<solr-javac destdir="${dest}/test-classes"
|
||||
classpathref="test.classpath">
|
||||
<src path="src/test/java" />
|
||||
</solr-javac>
|
||||
</target>
|
||||
|
||||
<target name="example" depends="build">
|
||||
<!-- Copy the jar into example/solr/lib -->
|
||||
<copy file="${dest}/${fullnamever}.jar" todir="${example}/clustering/solr/lib"/>
|
||||
<copy todir="${example}/clustering/solr/lib">
|
||||
<fileset dir="lib">
|
||||
<include name="*.jar"/>
|
||||
</fileset>
|
||||
<fileset dir="lib/downloads">
|
||||
<include name="*.jar"/>
|
||||
</fileset>
|
||||
</copy>
|
||||
<copy todir="${example}/clustering/solr/lib">
|
||||
<fileset dir="lib/downloads">
|
||||
<include name="*.jar"/>
|
||||
</fileset>
|
||||
</copy>
|
||||
</target>
|
||||
|
||||
|
||||
<target name="test" depends="compileTests">
|
||||
<mkdir dir="${junit.output.dir}"/>
|
||||
|
||||
<junit printsummary="on"
|
||||
haltonfailure="no"
|
||||
errorProperty="tests.failed"
|
||||
failureProperty="tests.failed"
|
||||
dir="src/test/resources/"
|
||||
>
|
||||
<formatter type="brief" usefile="false" if="junit.details"/>
|
||||
<classpath refid="test.classpath"/>
|
||||
<formatter type="xml"/>
|
||||
<batchtest fork="yes" todir="${junit.output.dir}" unless="testcase" >
|
||||
<fileset dir="src/test/java" includes="${junit.includes}">
|
||||
<exclude name="**/AbstractClusteringTest*"/>
|
||||
</fileset>
|
||||
</batchtest>
|
||||
<batchtest fork="yes" todir="${junit.output.dir}" if="testcase">
|
||||
<fileset dir="src/test/java" includes="**/${testcase}.java"/>
|
||||
</batchtest>
|
||||
</junit>
|
||||
|
||||
<fail if="tests.failed">Tests failed!</fail>
|
||||
</target>
|
||||
|
||||
<target name="dist" depends="build">
|
||||
<copy file="${dest}/${fullnamever}.jar" todir="${solr-path}/build/web/WEB-INF/lib"/>
|
||||
<copy todir="${solr-path}/build/web/WEB-INF/lib" flatten="true">
|
||||
<fileset dir="lib">
|
||||
<include name="**/*.jar"/>
|
||||
</fileset>
|
||||
</copy>
|
||||
<copy file="${dest}/${fullnamever}.jar" todir="${solr-path}/dist"/>
|
||||
</target>
|
||||
|
||||
<target name="javadoc">
|
||||
<sequential>
|
||||
<mkdir dir="${build.javadoc}/contrib-${name}"/>
|
||||
|
||||
<path id="javadoc.classpath">
|
||||
<path refid="common.classpath"/>
|
||||
</path>
|
||||
|
||||
<invoke-javadoc
|
||||
destdir="${build.javadoc}/contrib-${name}"
|
||||
title="${Name} ${version} contrib-${fullnamever} API">
|
||||
<sources>
|
||||
<packageset dir="src/main/java"/>
|
||||
</sources>
|
||||
</invoke-javadoc>
|
||||
</sequential>
|
||||
</target>
|
||||
|
||||
</project>
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
AnyObjectId[06a00de18c7ed6f96313327cc435db48975d9e19] was removed in git history.
|
||||
Apache SVN contains full history.
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
AnyObjectId[532939ecab6b77ccb77af3635c55ff9752b70ab7] was removed in git history.
|
||||
Apache SVN contains full history.
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
AnyObjectId[bb1dcd314deb13c412f2eab8abd54784d36ebd5a] was removed in git history.
|
||||
Apache SVN contains full history.
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
AnyObjectId[967c8045c23297a8e7fd4fe565f71734e48fe3bb] was removed in git history.
|
||||
Apache SVN contains full history.
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
AnyObjectId[cce3e8929ae214ea1221890ea4cd8d67d4b54fde] was removed in git history.
|
||||
Apache SVN contains full history.
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
AnyObjectId[625130719013f195869881a36dcb8d2b14d64d1e] was removed in git history.
|
||||
Apache SVN contains full history.
|
||||
|
|
@ -0,0 +1,63 @@
|
|||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
|
||||
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one
|
||||
or more contributor license agreements. See the NOTICE file
|
||||
distributed with this work for additional information
|
||||
regarding copyright ownership. The ASF licenses this file
|
||||
to you under the Apache License, Version 2.0 (the
|
||||
"License"); you may not use this file except in compliance
|
||||
with the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing,
|
||||
software distributed under the License is distributed on an
|
||||
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
||||
KIND, either express or implied. See the License for the
|
||||
specific language governing permissions and limitations
|
||||
under the License.
|
||||
-->
|
||||
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.apache.solr</groupId>
|
||||
<artifactId>solr-parent</artifactId>
|
||||
<version>@maven_version@</version>
|
||||
</parent>
|
||||
|
||||
<groupId>org.apache.solr</groupId>
|
||||
<artifactId>solr-clustering</artifactId>
|
||||
<name>Apache Solr Clustering</name>
|
||||
<version>@maven_version@</version>
|
||||
<description>Apache Solr Clustering</description>
|
||||
<packaging>jar</packaging>
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.apache.solr</groupId>
|
||||
<artifactId>solr-common</artifactId>
|
||||
<version>@maven_version@</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.solr</groupId>
|
||||
<artifactId>solr-core</artifactId>
|
||||
<version>@maven_version@</version>
|
||||
</dependency>
|
||||
<!-- Carrot -->
|
||||
<dependency>
|
||||
<groupId>org.carrot2</groupId>
|
||||
<artifactId>carrot2-mini</artifactId>
|
||||
<version>3.0.1</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
<repositories>
|
||||
<repository>
|
||||
<id>carrot2.org</id>
|
||||
<name>Carrot2 Maven2 repository</name>
|
||||
<url>http://download.carrot2.org/maven2/</url>
|
||||
</repository>
|
||||
</repositories>
|
||||
</project>
|
||||
|
|
@ -0,0 +1,190 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import org.apache.solr.common.params.SolrParams;
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.core.SolrCore;
|
||||
import org.apache.solr.core.SolrResourceLoader;
|
||||
import org.apache.solr.handler.clustering.carrot2.CarrotClusteringEngine;
|
||||
import org.apache.solr.handler.component.ResponseBuilder;
|
||||
import org.apache.solr.handler.component.SearchComponent;
|
||||
import org.apache.solr.search.DocListAndSet;
|
||||
import org.apache.solr.util.plugin.SolrCoreAware;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.Collections;
|
||||
import java.util.HashMap;
|
||||
import java.util.Map;
|
||||
|
||||
|
||||
/**
|
||||
* Provide a plugin for clustering results. Can either be for search results (i.e. via Carrot2) or for
|
||||
* clustering documents (i.e. via Mahout)
|
||||
* <p/>
|
||||
* This engine is experimental. Output from this engine is subject to change in future releases.
|
||||
*
|
||||
*/
|
||||
public class ClusteringComponent extends SearchComponent implements SolrCoreAware {
|
||||
private transient static Logger log = LoggerFactory.getLogger(ClusteringComponent.class);
|
||||
|
||||
private Map<String, SearchClusteringEngine> searchClusteringEngines = new HashMap<String, SearchClusteringEngine>();
|
||||
private Map<String, DocumentClusteringEngine> documentClusteringEngines = new HashMap<String, DocumentClusteringEngine>();
|
||||
/**
|
||||
* Base name for all spell checker query parameters. This name is also used to
|
||||
* register this component with SearchHandler.
|
||||
*/
|
||||
public static final String COMPONENT_NAME = "clustering";
|
||||
private NamedList initParams;
|
||||
|
||||
|
||||
public void prepare(ResponseBuilder rb) throws IOException {
|
||||
SolrParams params = rb.req.getParams();
|
||||
if (!params.getBool(COMPONENT_NAME, false)) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
public void process(ResponseBuilder rb) throws IOException {
|
||||
SolrParams params = rb.req.getParams();
|
||||
if (!params.getBool(COMPONENT_NAME, false)) {
|
||||
return;
|
||||
}
|
||||
String name = params.get(ClusteringParams.ENGINE_NAME, ClusteringEngine.DEFAULT_ENGINE_NAME);
|
||||
boolean useResults = params.getBool(ClusteringParams.USE_SEARCH_RESULTS, false);
|
||||
if (useResults == true) {
|
||||
SearchClusteringEngine engine = searchClusteringEngines.get(name);
|
||||
if (engine != null) {
|
||||
DocListAndSet results = rb.getResults();
|
||||
NamedList nl = engine.cluster(rb.getQuery(), results.docList, params);
|
||||
rb.rsp.add("clusters", nl);
|
||||
} else {
|
||||
log.warn("No engine for: " + name);
|
||||
}
|
||||
}
|
||||
boolean useCollection = params.getBool(ClusteringParams.USE_COLLECTION, false);
|
||||
if (useCollection == true) {
|
||||
DocumentClusteringEngine engine = documentClusteringEngines.get(name);
|
||||
if (engine != null) {
|
||||
boolean useDocSet = params.getBool(ClusteringParams.USE_DOC_SET, false);
|
||||
NamedList nl = null;
|
||||
|
||||
//TODO: This likely needs to be made into a background task that runs in an executor
|
||||
if (useDocSet == true) {
|
||||
nl = engine.cluster(rb.getResults().docSet, params);
|
||||
} else {
|
||||
nl = engine.cluster(params);
|
||||
}
|
||||
rb.rsp.add("clusters", nl);
|
||||
} else {
|
||||
log.warn("No engine for " + name);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
@SuppressWarnings("unchecked")
|
||||
public void init(NamedList args) {
|
||||
super.init(args);
|
||||
this.initParams = args;
|
||||
}
|
||||
|
||||
public void inform(SolrCore core) {
|
||||
if (initParams != null) {
|
||||
log.info("Initializing Clustering Engines");
|
||||
boolean searchHasDefault = false;
|
||||
boolean documentHasDefault = false;
|
||||
for (int i = 0; i < initParams.size(); i++) {
|
||||
if (initParams.getName(i).equals("engine")) {
|
||||
NamedList engineNL = (NamedList) initParams.getVal(i);
|
||||
String className = (String) engineNL.get("classname");
|
||||
if (className == null) {
|
||||
className = CarrotClusteringEngine.class.getName();
|
||||
}
|
||||
SolrResourceLoader loader = core.getResourceLoader();
|
||||
ClusteringEngine clusterer = (ClusteringEngine) loader.newInstance(className);
|
||||
if (clusterer != null) {
|
||||
String name = clusterer.init(engineNL, core);
|
||||
if (name != null) {
|
||||
boolean isDefault = name.equals(ClusteringEngine.DEFAULT_ENGINE_NAME);
|
||||
if (clusterer instanceof SearchClusteringEngine) {
|
||||
if (isDefault == true && searchHasDefault == false) {
|
||||
searchHasDefault = true;
|
||||
} else if (isDefault == true && searchHasDefault == true) {
|
||||
throw new RuntimeException("More than one engine is missing name: " + engineNL);
|
||||
}
|
||||
searchClusteringEngines.put(name, (SearchClusteringEngine) clusterer);
|
||||
} else if (clusterer instanceof DocumentClusteringEngine) {
|
||||
if (isDefault == true && documentHasDefault == false) {
|
||||
searchHasDefault = true;
|
||||
} else if (isDefault == true && documentHasDefault == true) {
|
||||
throw new RuntimeException("More than one engine is missing name: " + engineNL);
|
||||
}
|
||||
documentClusteringEngines.put(name, (DocumentClusteringEngine) clusterer);
|
||||
}
|
||||
} else {
|
||||
if (clusterer instanceof SearchClusteringEngine && searchHasDefault == false) {
|
||||
searchClusteringEngines.put(ClusteringEngine.DEFAULT_ENGINE_NAME, (SearchClusteringEngine) clusterer);
|
||||
searchHasDefault = true;
|
||||
} else if (clusterer instanceof DocumentClusteringEngine && documentHasDefault == false) {
|
||||
documentClusteringEngines.put(ClusteringEngine.DEFAULT_ENGINE_NAME, (DocumentClusteringEngine) clusterer);
|
||||
documentHasDefault = true;
|
||||
} else {
|
||||
throw new RuntimeException("More than one engine is missing name: " + engineNL);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
log.info("Finished Initializing Clustering Engines");
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* @return Unmodifiable Map of the engines, key is the name from the config, value is the engine
|
||||
* */
|
||||
public Map<String, SearchClusteringEngine> getSearchClusteringEngines() {
|
||||
return Collections.unmodifiableMap(searchClusteringEngines);
|
||||
}
|
||||
|
||||
// ///////////////////////////////////////////
|
||||
// / SolrInfoMBean
|
||||
// //////////////////////////////////////////
|
||||
|
||||
@Override
|
||||
public String getDescription() {
|
||||
return "A Clustering component";
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getVersion() {
|
||||
return "$Revision:$";
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getSourceId() {
|
||||
return "$Id:$";
|
||||
}
|
||||
|
||||
@Override
|
||||
public String getSource() {
|
||||
return "$URL:$";
|
||||
}
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.core.SolrCore;
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*
|
||||
**/
|
||||
public class ClusteringEngine {
|
||||
private String name;
|
||||
public static final String ENGINE_NAME = "name";
|
||||
public static final String DEFAULT_ENGINE_NAME = "default";
|
||||
|
||||
public String init(NamedList config, SolrCore core) {
|
||||
name = (String) config.get(ENGINE_NAME);
|
||||
|
||||
return name;
|
||||
}
|
||||
|
||||
public String getName() {
|
||||
return name;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*
|
||||
**/
|
||||
public interface ClusteringParams {
|
||||
|
||||
public static final String CLUSTERING_PREFIX = "clustering.";
|
||||
|
||||
public static final String ENGINE_NAME = CLUSTERING_PREFIX + "engine";
|
||||
|
||||
public static final String USE_SEARCH_RESULTS = CLUSTERING_PREFIX + "results";
|
||||
|
||||
public static final String USE_COLLECTION = CLUSTERING_PREFIX + "collection";
|
||||
/**
|
||||
* When document clustering, cluster on the Doc Set
|
||||
*/
|
||||
public static final String USE_DOC_SET = CLUSTERING_PREFIX + "docs.useDocSet";
|
||||
}
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.common.params.SolrParams;
|
||||
import org.apache.solr.core.SolrCore;
|
||||
import org.apache.solr.search.DocList;
|
||||
import org.apache.solr.search.DocSet;
|
||||
import org.apache.lucene.search.Query;
|
||||
|
||||
|
||||
/**
|
||||
* Experimental. Subject to change before the next release.
|
||||
*
|
||||
**/
|
||||
public abstract class DocumentClusteringEngine extends ClusteringEngine {
|
||||
|
||||
/**
|
||||
* Experimental. Subject to change before the next release
|
||||
*
|
||||
* Cluster all the documents in the index. Clustering is often an expensive task that can take a long time.
|
||||
* @param solrParams The params controlling clustering
|
||||
* @return The clustering results
|
||||
*/
|
||||
public abstract NamedList cluster(SolrParams solrParams);
|
||||
|
||||
/**
|
||||
* Experimental. Subject to change before the next release
|
||||
*
|
||||
*
|
||||
* Cluster the set of docs. Clustering of documents is often an expensive task that can take a long time.
|
||||
* @param docs The docs to cluster. If null, cluster all docs as in {@link #cluster(org.apache.solr.common.params.SolrParams)}
|
||||
* @param solrParams The params controlling the clustering
|
||||
* @return The results.
|
||||
*/
|
||||
public abstract NamedList cluster(DocSet docs, SolrParams solrParams);
|
||||
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.common.params.SolrParams;
|
||||
import org.apache.solr.core.SolrCore;
|
||||
import org.apache.solr.search.DocList;
|
||||
import org.apache.lucene.search.Query;
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*
|
||||
**/
|
||||
public abstract class SearchClusteringEngine extends ClusteringEngine {
|
||||
|
||||
|
||||
public abstract NamedList cluster(Query query, DocList docList, SolrParams solrParams);
|
||||
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,254 @@
|
|||
package org.apache.solr.handler.clustering.carrot2;
|
||||
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.*;
|
||||
|
||||
import org.apache.commons.lang.StringUtils;
|
||||
import org.apache.lucene.document.FieldSelector;
|
||||
import org.apache.lucene.document.SetBasedFieldSelector;
|
||||
import org.apache.lucene.search.Query;
|
||||
import org.apache.solr.common.params.HighlightParams;
|
||||
import org.apache.solr.common.params.SolrParams;
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.common.SolrException;
|
||||
import org.apache.solr.core.SolrCore;
|
||||
import org.apache.solr.handler.clustering.SearchClusteringEngine;
|
||||
import org.apache.solr.highlight.SolrHighlighter;
|
||||
import org.apache.solr.request.LocalSolrQueryRequest;
|
||||
import org.apache.solr.request.SolrQueryRequest;
|
||||
import org.apache.solr.search.*;
|
||||
import org.apache.solr.util.RefCounted;
|
||||
import org.carrot2.core.*;
|
||||
import org.carrot2.core.attribute.AttributeNames;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import com.google.common.collect.Sets;
|
||||
|
||||
/**
|
||||
* Search results clustering engine based on Carrot2 clustering algorithms.
|
||||
*
|
||||
* Output from this class is subject to change.
|
||||
*
|
||||
* @link http://project.carrot2.org
|
||||
*/
|
||||
@SuppressWarnings("unchecked")
|
||||
public class CarrotClusteringEngine extends SearchClusteringEngine {
|
||||
private transient static Logger log = LoggerFactory
|
||||
.getLogger(CarrotClusteringEngine.class);
|
||||
|
||||
/** Carrot2 controller that manages instances of clustering algorithms */
|
||||
private CachingController controller = new CachingController();
|
||||
private Class<? extends IClusteringAlgorithm> clusteringAlgorithmClass;
|
||||
|
||||
private SolrCore core;
|
||||
private String idFieldName;
|
||||
|
||||
public NamedList cluster(Query query, DocList docList, SolrParams solrParams) {
|
||||
try {
|
||||
// Prepare attributes for Carrot2 clustering call
|
||||
Map<String, Object> attributes = new HashMap<String, Object>();
|
||||
List<Document> documents = getDocuments(docList, core, query, solrParams);
|
||||
attributes.put(AttributeNames.DOCUMENTS, documents);
|
||||
attributes.put(AttributeNames.QUERY, query.toString());
|
||||
|
||||
// Pass extra overriding attributes from the request, if any
|
||||
extractCarrotAttributes(solrParams, attributes);
|
||||
|
||||
// Perform clustering and convert to named list
|
||||
return clustersToNamedList(controller.process(attributes,
|
||||
clusteringAlgorithmClass).getClusters(), solrParams);
|
||||
} catch (Exception e) {
|
||||
log.error("Carrot2 clustering failed", e);
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public String init(NamedList config, final SolrCore core) {
|
||||
String result = super.init(config, core);
|
||||
SolrParams initParams = SolrParams.toSolrParams(config);
|
||||
|
||||
// Initialize Carrot2 controller. Pass initialization attributes, if any.
|
||||
HashMap<String, Object> initAttributes = new HashMap<String, Object>();
|
||||
extractCarrotAttributes(initParams, initAttributes);
|
||||
this.controller.init(initAttributes);
|
||||
|
||||
this.core = core;
|
||||
this.idFieldName = core.getSchema().getUniqueKeyField().getName();
|
||||
|
||||
// Make sure the requested Carrot2 clustering algorithm class is available
|
||||
String carrotAlgorithmClassName = initParams.get(CarrotParams.ALGORITHM);
|
||||
try {
|
||||
Class<?> algorithmClass = Thread.currentThread().getContextClassLoader()
|
||||
.loadClass(carrotAlgorithmClassName);
|
||||
if (!IClusteringAlgorithm.class.isAssignableFrom(algorithmClass)) {
|
||||
throw new IllegalArgumentException("Class provided as "
|
||||
+ CarrotParams.ALGORITHM + " must implement "
|
||||
+ IClusteringAlgorithm.class.getName());
|
||||
}
|
||||
this.clusteringAlgorithmClass = (Class<? extends IClusteringAlgorithm>) algorithmClass;
|
||||
} catch (ClassNotFoundException e) {
|
||||
throw new RuntimeException(
|
||||
"Failed to load Carrot clustering algorithm class", e);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Prepares Carrot2 documents for clustering.
|
||||
*/
|
||||
private List<Document> getDocuments(DocList docList, SolrCore core,
|
||||
Query query, SolrParams solrParams) {
|
||||
SolrHighlighter highligher = null;
|
||||
|
||||
// Names of fields to deliver content for clustering
|
||||
String urlField = solrParams.get(CarrotParams.URL_FIELD_NAME, "url");
|
||||
String titleField = solrParams.get(CarrotParams.TITLE_FIELD_NAME, "title");
|
||||
String snippetField = solrParams.get(CarrotParams.SNIPPET_FIELD_NAME,
|
||||
titleField);
|
||||
if (StringUtils.isBlank(snippetField)) {
|
||||
throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, CarrotParams.SNIPPET_FIELD_NAME
|
||||
+ " must not be blank.");
|
||||
}
|
||||
Set<String> fieldsToLoad = Sets.newHashSet(urlField, titleField,
|
||||
snippetField, idFieldName);
|
||||
|
||||
// Get the documents
|
||||
DocIterator docsIter = docList.iterator();
|
||||
boolean produceSummary = solrParams.getBool(CarrotParams.PRODUCE_SUMMARY,
|
||||
false);
|
||||
|
||||
SolrQueryRequest req = null;
|
||||
String[] snippetFieldAry = null;
|
||||
if (produceSummary == true) {
|
||||
highligher = core.getHighlighter();
|
||||
Map args = new HashMap();
|
||||
snippetFieldAry = new String[] { snippetField };
|
||||
args.put(HighlightParams.FIELDS, snippetFieldAry);
|
||||
args.put(HighlightParams.HIGHLIGHT, "true");
|
||||
req = new LocalSolrQueryRequest(core, query.toString(), "", 0, 1, args);
|
||||
}
|
||||
|
||||
RefCounted<SolrIndexSearcher> refCounter = core.getSearcher();
|
||||
SolrIndexSearcher searcher = refCounter.get();
|
||||
List<Document> result = new ArrayList<Document>(docList.size());
|
||||
try {
|
||||
FieldSelector fieldSelector = new SetBasedFieldSelector(fieldsToLoad,
|
||||
Collections.emptySet());
|
||||
float[] scores = { 1.0f };
|
||||
int[] docsHolder = new int[1];
|
||||
Query theQuery = query;
|
||||
|
||||
while (docsIter.hasNext()) {
|
||||
Integer id = docsIter.next();
|
||||
org.apache.lucene.document.Document doc = searcher.doc(id,
|
||||
fieldSelector);
|
||||
String snippet = getValue(doc, snippetField);
|
||||
if (produceSummary == true) {
|
||||
docsHolder[0] = id.intValue();
|
||||
DocList docAsList = new DocSlice(0, 1, docsHolder, scores, 1, 1.0f);
|
||||
highligher.doHighlighting(docAsList, theQuery, req, snippetFieldAry);
|
||||
}
|
||||
Document carrotDocument = new Document(getValue(doc, titleField),
|
||||
snippet, doc.get(urlField));
|
||||
carrotDocument.addField("solrId", doc.get(idFieldName));
|
||||
result.add(carrotDocument);
|
||||
}
|
||||
} catch (IOException e) {
|
||||
log.error("IOException", e);
|
||||
} finally {
|
||||
refCounter.decref();
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
protected String getValue(org.apache.lucene.document.Document doc,
|
||||
String field) {
|
||||
StringBuilder result = new StringBuilder();
|
||||
String[] vals = doc.getValues(field);
|
||||
for (int i = 0; i < vals.length; i++) {
|
||||
// Join multiple values with a period so that Carrot2 does not pick up
|
||||
// phrases that cross field value boundaries (in most cases it would
|
||||
// create useless phrases).
|
||||
result.append(vals[i]).append(" . ");
|
||||
}
|
||||
return result.toString().trim();
|
||||
}
|
||||
|
||||
private NamedList clustersToNamedList(List<Cluster> carrotClusters,
|
||||
SolrParams solrParams) {
|
||||
NamedList result = new NamedList();
|
||||
clustersToNamedList(carrotClusters, result, solrParams.getBool(
|
||||
CarrotParams.OUTPUT_SUB_CLUSTERS, false), solrParams.getInt(
|
||||
CarrotParams.NUM_DESCRIPTIONS, Integer.MAX_VALUE));
|
||||
return result;
|
||||
}
|
||||
|
||||
private void clustersToNamedList(List<Cluster> outputClusters,
|
||||
NamedList parent, boolean outputSubClusters, int maxLabels) {
|
||||
for (Cluster outCluster : outputClusters) {
|
||||
NamedList cluster = new NamedList();
|
||||
parent.add("cluster", cluster);
|
||||
|
||||
List<String> labels = outCluster.getPhrases();
|
||||
NamedList labelsNL = new NamedList();
|
||||
cluster.add("labels", labelsNL);
|
||||
int labelsAdded = 0;
|
||||
for (String label : labels) {
|
||||
if (++labelsAdded > maxLabels) {
|
||||
break;
|
||||
}
|
||||
labelsNL.add("label", label);
|
||||
}
|
||||
|
||||
List<Document> docs = outCluster.getDocuments();
|
||||
NamedList docsNL = new NamedList();
|
||||
cluster.add("docs", docsNL);
|
||||
for (Document doc : docs) {
|
||||
docsNL.add("doc", doc.getField("solrId"));
|
||||
}
|
||||
|
||||
if (outputSubClusters) {
|
||||
NamedList subclustersNL = new NamedList();
|
||||
cluster.add("clusters", subclustersNL);
|
||||
clustersToNamedList(outCluster.getSubclusters(), subclustersNL,
|
||||
outputSubClusters, maxLabels);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Extracts parameters that can possibly match some attributes of Carrot2 algorithms.
|
||||
*/
|
||||
private void extractCarrotAttributes(SolrParams solrParams,
|
||||
Map<String, Object> attributes) {
|
||||
// Extract all non-predefined parameters. This way, we'll be able to set all
|
||||
// parameters of Carrot2 algorithms without defining their names as constants.
|
||||
for (Iterator<String> paramNames = solrParams.getParameterNamesIterator(); paramNames
|
||||
.hasNext();) {
|
||||
String paramName = paramNames.next();
|
||||
if (!CarrotParams.CARROT_PARAM_NAMES.contains(paramName)) {
|
||||
attributes.put(paramName, solrParams.get(paramName));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
package org.apache.solr.handler.clustering.carrot2;
|
||||
|
||||
import java.util.Set;
|
||||
|
||||
import com.google.common.collect.ImmutableSet;
|
||||
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
public interface CarrotParams {
|
||||
|
||||
String CARROT_PREFIX = "carrot.";
|
||||
|
||||
String ALGORITHM = CARROT_PREFIX + "algorithm";
|
||||
String TITLE_FIELD_NAME = CARROT_PREFIX + "title";
|
||||
String URL_FIELD_NAME = CARROT_PREFIX + "url";
|
||||
String SNIPPET_FIELD_NAME = CARROT_PREFIX + "snippet";
|
||||
String PRODUCE_SUMMARY = CARROT_PREFIX + "produceSummary";
|
||||
String NUM_DESCRIPTIONS = CARROT_PREFIX + "numDescriptions";
|
||||
String OUTPUT_SUB_CLUSTERS = CARROT_PREFIX + "outputSubClusters";
|
||||
|
||||
public static final Set<String> CARROT_PARAM_NAMES = ImmutableSet.of(
|
||||
ALGORITHM, TITLE_FIELD_NAME, URL_FIELD_NAME, SNIPPET_FIELD_NAME,
|
||||
PRODUCE_SUMMARY, NUM_DESCRIPTIONS, OUTPUT_SUB_CLUSTERS);
|
||||
}
|
||||
|
|
@ -0,0 +1,198 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import org.apache.solr.util.AbstractSolrTestCase;
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
public class AbstractClusteringTest extends AbstractSolrTestCase {
|
||||
protected int numberOfDocs = 0;
|
||||
|
||||
@Override
|
||||
public void setUp() throws Exception {
|
||||
super.setUp();
|
||||
|
||||
numberOfDocs = 0;
|
||||
for (String[] doc : DOCUMENTS) {
|
||||
assertU("add failed", adoc("id", Integer.toString(numberOfDocs), "url", doc[0], "title", doc[1], "snippet", doc[2]));
|
||||
numberOfDocs++;
|
||||
}
|
||||
assertU("commit", commit());
|
||||
}
|
||||
|
||||
public String getSchemaFile() {
|
||||
return "schema.xml";
|
||||
}
|
||||
|
||||
public String getSolrConfigFile() {
|
||||
return "solrconfig.xml";
|
||||
}
|
||||
|
||||
final String [][] DOCUMENTS = new String[][] {
|
||||
{ "http://en.wikipedia.org/wiki/Data_mining",
|
||||
"Data Mining - Wikipedia",
|
||||
"Article about knowledge-discovery in databases (KDD), the practice of automatically searching large stores of data for patterns." },
|
||||
|
||||
|
||||
{ "http://en.wikipedia.org/wiki/Datamining",
|
||||
"Data mining - Wikipedia, the free encyclopedia",
|
||||
"Data mining is the entire process of applying computer-based methodology, ... Moreover, some data-mining systems such as neural networks are inherently geared ..." },
|
||||
|
||||
|
||||
{ "http://www.statsoft.com/textbook/stdatmin.html",
|
||||
"Electronic Statistics Textbook: Data Mining Techniques",
|
||||
"Outlines the crucial concepts in data mining, defines the data warehousing process, and offers examples of computational and graphical exploratory data analysis techniques." },
|
||||
|
||||
|
||||
{ "http://www.thearling.com/text/dmwhite/dmwhite.htm",
|
||||
"An Introduction to Data Mining",
|
||||
"Data mining, the extraction of hidden predictive information from large ... Data mining tools predict future trends and behaviors, allowing businesses to ..." },
|
||||
|
||||
|
||||
{ "http://www.anderson.ucla.edu/faculty/jason.frand/teacher/technologies/palace/datamining.htm",
|
||||
"Data Mining: What is Data Mining?",
|
||||
"Outlines what knowledge discovery, the process of analyzing data from different perspectives and summarizing it into useful information, can do and how it works." },
|
||||
|
||||
|
||||
{ "http://www.spss.com/datamine",
|
||||
"Data Mining Software, Data Mining Applications and Data Mining Solutions",
|
||||
"The patterns uncovered using data mining help organizations make better and ... data mining customer ... Data mining applications, on the other hand, embed ..." },
|
||||
|
||||
|
||||
{ "http://www.kdnuggets.com/",
|
||||
"KD Nuggets",
|
||||
"Newsletter on the data mining and knowledge industries, offering information on data mining, knowledge discovery, text mining, and web mining software, courses, jobs, publications, and meetings." },
|
||||
|
||||
|
||||
{ "http://www.answers.com/topic/data-mining",
|
||||
"data mining: Definition from Answers.com",
|
||||
"data mining n. The automatic extraction of useful, often previously unknown information from large databases or data ... Data Mining For Investing ..." },
|
||||
|
||||
|
||||
{ "http://www.statsoft.com/products/dataminer.htm",
|
||||
"STATISTICA Data Mining and Predictive Modeling Solutions",
|
||||
"GRC site-wide menuing system research and development. ... Contact a Data Mining Solutions Consultant. News and Success Stories. Events ..." },
|
||||
|
||||
|
||||
{ "http://datamining.typepad.com/",
|
||||
"Data Mining: Text Mining, Visualization and Social Media",
|
||||
"Commentary on text mining, data mining, social media and data visualization. ... While mining Twitter data for business and marketing intelligence (trend/buzz ..." },
|
||||
|
||||
|
||||
{ "http://www.twocrows.com/",
|
||||
"Two Crows Corporation",
|
||||
"Dedicated to the development, marketing, sales and support of tools for knowledge discovery to make data mining accessible and easy to use." },
|
||||
|
||||
|
||||
{ "http://www.thearling.com/",
|
||||
"Thearling.com",
|
||||
"Kurt Thearling's site dedicated to sharing information about data mining, the automated extraction of hidden predictive information from databases, and other analytic technologies." },
|
||||
|
||||
|
||||
{ "http://www.ccsu.edu/datamining/",
|
||||
"CCSU - Data Mining",
|
||||
"Offers degrees and certificates in data mining. Allows students to explore cutting-edge data mining techniques and applications: market basket analysis, decision trees, neural networks, machine learning, web mining, and data modeling." },
|
||||
|
||||
|
||||
{ "http://www.oracle.com/technology/products/bi/odm",
|
||||
"Oracle Data Mining",
|
||||
"Oracle Data Mining Product Center ... New Oracle Data Mining Powers New Social CRM Application (more information ... Mining High-Dimensional Data for ..." },
|
||||
|
||||
|
||||
{ "http://databases.about.com/od/datamining/a/datamining.htm",
|
||||
"Data Mining: An Introduction",
|
||||
"About.com article on how businesses are discovering new trends and patterns of behavior that previously went unnoticed through data mining, automated statistical analysis techniques." },
|
||||
|
||||
|
||||
{ "http://www.dmoz.org/Computers/Software/Databases/Data_Mining/",
|
||||
"Open Directory - Computers: Software: Databases: Data Mining",
|
||||
"Data Mining and Knowledge Discovery - A peer-reviewed journal publishing ... Data mining creates information assets that an organization can leverage to ..." },
|
||||
|
||||
|
||||
{ "http://www.cs.wisc.edu/dmi/",
|
||||
"DMI:Data Mining Institute",
|
||||
"Data Mining Institute at UW-Madison ... The Data Mining Institute (DMI) was started on June 1, 1999 at the Computer ... of the Data Mining Group of Microsoft ..." },
|
||||
|
||||
|
||||
{ "http://www.the-data-mine.com/",
|
||||
"The Data Mine",
|
||||
"Provides information about data mining also known as knowledge discovery in databases (KDD) or simply knowledge discovery. List software, events, organizations, and people working in data mining." },
|
||||
|
||||
|
||||
{ "http://www.statserv.com/datamining.html",
|
||||
"St@tServ - About Data Mining",
|
||||
"St@tServ Data Mining page ... Data mining in molecular biology, by Alvis Brazma. Graham Williams page. Knowledge Discovery and Data Mining Resources, ..." },
|
||||
|
||||
|
||||
{ "http://ocw.mit.edu/OcwWeb/Sloan-School-of-Management/15-062Data-MiningSpring2003/CourseHome/index.htm",
|
||||
"MIT OpenCourseWare | Sloan School of Management | 15.062 Data Mining ...",
|
||||
"Introduces students to a class of methods known as data mining that assists managers in recognizing patterns and making intelligent use of massive amounts of ..." },
|
||||
|
||||
|
||||
{ "http://www.pentaho.com/products/data_mining/",
|
||||
"Pentaho Commercial Open Source Business Intelligence: Data Mining",
|
||||
"For example, data mining can warn you there's a high probability a specific ... Pentaho Data Mining is differentiated by its open, standards-compliant nature, ..." },
|
||||
|
||||
|
||||
{ "http://www.investorhome.com/mining.htm",
|
||||
"Investor Home - Data Mining",
|
||||
"Data Mining or Data Snooping is the practice of searching for relationships and ... Data mining involves searching through databases for correlations and patterns ..." },
|
||||
|
||||
|
||||
{ "http://www.datamining.com/",
|
||||
"Predictive Modeling and Predictive Analytics Solutions | Enterprise ...",
|
||||
"Insightful Enterprise Miner - Enterprise data mining for predictive modeling and predictive analytics." },
|
||||
|
||||
|
||||
{ "http://www.sourcewatch.org/index.php?title=Data_mining",
|
||||
"Data mining - SourceWatch",
|
||||
"These agencies reported 199 data mining projects, of which 68 ... Office, \"DATA MINING. ... powerful technology known as data mining -- and how, in the ..." },
|
||||
|
||||
|
||||
{ "http://www.autonlab.org/tutorials/",
|
||||
"Statistical Data Mining Tutorials",
|
||||
"Includes a set of tutorials on many aspects of statistical data mining, including the foundations of probability, the foundations of statistical data analysis, and most of the classic machine learning and data mining algorithms." },
|
||||
|
||||
|
||||
{ "http://www.microstrategy.com/data-mining/index.asp",
|
||||
"Data Mining",
|
||||
"With MicroStrategy, data mining scoring is fully integrated into mainstream ... The integration of data mining models from other applications is accomplished by ..." },
|
||||
|
||||
|
||||
{ "http://www.datamininglab.com/",
|
||||
"Elder Research",
|
||||
"Provides consulting and short courses in data mining and pattern discovery patterns in data." },
|
||||
|
||||
|
||||
{ "http://www.sqlserverdatamining.com/",
|
||||
"SQL Server Data Mining > Home",
|
||||
"SQL Server Data Mining Portal ... Data Mining as an Application Platform (Whitepaper) Creating a Web Cross-sell Application with SQL Server 2005 Data Mining (Article) ..." },
|
||||
|
||||
|
||||
{ "http://databases.about.com/cs/datamining/g/dmining.htm",
|
||||
"Data Mining",
|
||||
"What is data mining? Find out here! ... Book Review: Data Mining and Statistical Analysis Using SQL. What is Data Mining, and What Does it Have to Do with ..." },
|
||||
|
||||
|
||||
{ "http://www.sas.com/technologies/analytics/datamining/index.html",
|
||||
"Data Mining Software and Text Mining | SAS",
|
||||
"... raw data to smarter ... Data Mining is an iterative process of creating ... The knowledge gleaned from data and text mining can be used to fuel ..." }
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import org.apache.solr.util.AbstractSolrTestCase;
|
||||
import org.apache.solr.core.SolrCore;
|
||||
import org.apache.solr.handler.component.SearchComponent;
|
||||
import org.apache.solr.handler.component.SpellCheckComponent;
|
||||
import org.apache.solr.handler.component.QueryComponent;
|
||||
import org.apache.solr.common.params.ModifiableSolrParams;
|
||||
import org.apache.solr.common.params.CommonParams;
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.common.util.SimpleOrderedMap;
|
||||
import org.apache.solr.request.SolrRequestHandler;
|
||||
import org.apache.solr.request.SolrQueryResponse;
|
||||
import org.apache.solr.request.LocalSolrQueryRequest;
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*
|
||||
**/
|
||||
public class ClusteringComponentTest extends AbstractClusteringTest {
|
||||
|
||||
public void testComponent() throws Exception {
|
||||
SolrCore core = h.getCore();
|
||||
|
||||
SearchComponent sc = core.getSearchComponent("clustering");
|
||||
assertTrue("sc is null and it shouldn't be", sc != null);
|
||||
ModifiableSolrParams params = new ModifiableSolrParams();
|
||||
|
||||
params.add(ClusteringComponent.COMPONENT_NAME, "true");
|
||||
params.add(CommonParams.Q, "*:*");
|
||||
|
||||
params.add(ClusteringParams.USE_SEARCH_RESULTS, "true");
|
||||
|
||||
|
||||
SolrRequestHandler handler = core.getRequestHandler("standard");
|
||||
SolrQueryResponse rsp;
|
||||
rsp = new SolrQueryResponse();
|
||||
rsp.add("responseHeader", new SimpleOrderedMap());
|
||||
handler.handleRequest(new LocalSolrQueryRequest(core, params), rsp);
|
||||
NamedList values = rsp.getValues();
|
||||
NamedList clusters = (NamedList) values.get("clusters");
|
||||
//System.out.println("Clusters: " + clusters);
|
||||
assertTrue("clusters is null and it shouldn't be", clusters != null);
|
||||
|
||||
|
||||
params = new ModifiableSolrParams();
|
||||
params.add(ClusteringComponent.COMPONENT_NAME, "true");
|
||||
params.add(ClusteringParams.ENGINE_NAME, "mock");
|
||||
params.add(ClusteringParams.USE_COLLECTION, "true");
|
||||
params.add(QueryComponent.COMPONENT_NAME, "false");
|
||||
|
||||
handler = core.getRequestHandler("docClustering");
|
||||
|
||||
rsp = new SolrQueryResponse();
|
||||
rsp.add("responseHeader", new SimpleOrderedMap());
|
||||
handler.handleRequest(new LocalSolrQueryRequest(core, params), rsp);
|
||||
values = rsp.getValues();
|
||||
clusters = (NamedList) values.get("clusters");
|
||||
//System.out.println("Clusters: " + clusters);
|
||||
assertTrue("clusters is null and it shouldn't be", clusters != null);
|
||||
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
package org.apache.solr.handler.clustering;
|
||||
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.common.params.SolrParams;
|
||||
import org.apache.solr.search.DocSet;
|
||||
|
||||
|
||||
/**
|
||||
*
|
||||
*
|
||||
**/
|
||||
public class MockDocumentClusteringEngine extends DocumentClusteringEngine{
|
||||
public NamedList cluster(DocSet docs, SolrParams solrParams) {
|
||||
NamedList result = new NamedList();
|
||||
return result;
|
||||
}
|
||||
|
||||
public NamedList cluster(SolrParams solrParams) {
|
||||
NamedList result = new NamedList();
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,168 @@
|
|||
package org.apache.solr.handler.clustering.carrot2;
|
||||
|
||||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
* contributor license agreements. See the NOTICE file distributed with
|
||||
* this work for additional information regarding copyright ownership.
|
||||
* The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
* (the "License"); you may not use this file except in compliance with
|
||||
* the License. You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
import java.io.IOException;
|
||||
|
||||
import org.apache.lucene.search.*;
|
||||
import org.apache.solr.common.params.ModifiableSolrParams;
|
||||
import org.apache.solr.common.params.SolrParams;
|
||||
import org.apache.solr.common.util.NamedList;
|
||||
import org.apache.solr.handler.clustering.AbstractClusteringTest;
|
||||
import org.apache.solr.handler.clustering.ClusteringComponent;
|
||||
import org.apache.solr.search.DocList;
|
||||
import org.apache.solr.search.SolrIndexSearcher;
|
||||
import org.apache.solr.util.RefCounted;
|
||||
import org.carrot2.util.attribute.AttributeUtils;
|
||||
|
||||
/**
|
||||
*
|
||||
*/
|
||||
@SuppressWarnings("unchecked")
|
||||
public class CarrotClusteringEngineTest extends AbstractClusteringTest {
|
||||
public void testCarrotLingo() throws Exception {
|
||||
checkEngine(getClusteringEngine("default"), 9);
|
||||
}
|
||||
|
||||
public void testCarrotStc() throws Exception {
|
||||
checkEngine(getClusteringEngine("stc"), 2);
|
||||
}
|
||||
|
||||
public void testWithoutSubclusters() throws Exception {
|
||||
checkClusters(checkEngine(getClusteringEngine("mock"), this.numberOfDocs),
|
||||
1, 1, 0);
|
||||
}
|
||||
|
||||
public void testWithSubclusters() throws Exception {
|
||||
ModifiableSolrParams params = new ModifiableSolrParams();
|
||||
params.set(CarrotParams.OUTPUT_SUB_CLUSTERS, true);
|
||||
checkClusters(checkEngine(getClusteringEngine("mock"), this.numberOfDocs,
|
||||
params), 1, 1, 2);
|
||||
}
|
||||
|
||||
public void testNumDescriptions() throws Exception {
|
||||
ModifiableSolrParams params = new ModifiableSolrParams();
|
||||
params.set(AttributeUtils.getKey(MockClusteringAlgorithm.class, "labels"), 5);
|
||||
params.set(CarrotParams.NUM_DESCRIPTIONS, 3);
|
||||
checkClusters(checkEngine(getClusteringEngine("mock"), this.numberOfDocs,
|
||||
params), 1, 3, 0);
|
||||
}
|
||||
|
||||
public void testCarrotAttributePassing() throws Exception {
|
||||
ModifiableSolrParams params = new ModifiableSolrParams();
|
||||
params.set(AttributeUtils.getKey(MockClusteringAlgorithm.class, "depth"), 1);
|
||||
params.set(AttributeUtils.getKey(MockClusteringAlgorithm.class, "labels"), 3);
|
||||
checkClusters(checkEngine(getClusteringEngine("mock"), this.numberOfDocs,
|
||||
params), 1, 3, 0);
|
||||
}
|
||||
|
||||
private CarrotClusteringEngine getClusteringEngine(String engineName) {
|
||||
ClusteringComponent comp = (ClusteringComponent) h.getCore()
|
||||
.getSearchComponent("clustering");
|
||||
assertNotNull("clustering component should not be null", comp);
|
||||
CarrotClusteringEngine engine = (CarrotClusteringEngine) comp
|
||||
.getSearchClusteringEngines().get(engineName);
|
||||
assertNotNull("clustering engine for name: " + engineName
|
||||
+ " should not be null", engine);
|
||||
return engine;
|
||||
}
|
||||
|
||||
private NamedList checkEngine(CarrotClusteringEngine engine,
|
||||
int expectedNumClusters) throws IOException {
|
||||
return checkEngine(engine, expectedNumClusters, new ModifiableSolrParams());
|
||||
}
|
||||
|
||||
private NamedList checkEngine(CarrotClusteringEngine engine,
|
||||
int expectedNumClusters, SolrParams clusteringParams) throws IOException {
|
||||
// Get all documents to cluster
|
||||
RefCounted<SolrIndexSearcher> ref = h.getCore().getSearcher();
|
||||
MatchAllDocsQuery query = new MatchAllDocsQuery();
|
||||
DocList docList;
|
||||
try {
|
||||
SolrIndexSearcher searcher = ref.get();
|
||||
docList = searcher.getDocList(query, (Query) null, new Sort(), 0,
|
||||
numberOfDocs);
|
||||
assertEquals("docList size", this.numberOfDocs, docList.matches());
|
||||
} finally {
|
||||
ref.decref();
|
||||
}
|
||||
|
||||
ModifiableSolrParams solrParams = new ModifiableSolrParams();
|
||||
solrParams.add(CarrotParams.PRODUCE_SUMMARY, "true");
|
||||
solrParams.add(clusteringParams);
|
||||
|
||||
// Perform clustering
|
||||
NamedList results = engine.cluster(query, docList, solrParams);
|
||||
assertEquals("number of clusters", expectedNumClusters, results.size());
|
||||
checkClusters(results, false);
|
||||
return results;
|
||||
}
|
||||
|
||||
private void checkClusters(NamedList results, int expectedDocCount,
|
||||
int expectedLabelCount, int expectedSubclusterCount) {
|
||||
for (int i = 0; i < results.size(); i++) {
|
||||
if (results.getName(i).equals("cluster")) {
|
||||
NamedList cluster = (NamedList) results.getVal(i);
|
||||
checkCluster(cluster, expectedDocCount, expectedLabelCount,
|
||||
expectedSubclusterCount);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void checkClusters(NamedList results, boolean hasSubclusters) {
|
||||
for (int i = 0; i < results.size(); i++) {
|
||||
if (results.getName(i).equals("cluster")) {
|
||||
NamedList cluster = (NamedList) results.getVal(i);
|
||||
checkCluster(cluster, hasSubclusters);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void checkCluster(NamedList cluster, boolean hasSubclusters) {
|
||||
NamedList docs = (NamedList) cluster.get("docs");
|
||||
assertNotNull("docs is null and it shouldn't be", docs);
|
||||
for (int j = 0; j < docs.size(); j++) {
|
||||
String id = (String) docs.getVal(j);
|
||||
assertNotNull("id is null and it shouldn't be", id);
|
||||
}
|
||||
|
||||
NamedList labels = (NamedList) cluster.get("labels");
|
||||
assertNotNull("labels is null but it shouldn't be", labels);
|
||||
|
||||
if (hasSubclusters) {
|
||||
NamedList subclusters = (NamedList) cluster.get("clusters");
|
||||
assertNotNull("subclusters is null but it shouldn't be", subclusters);
|
||||
}
|
||||
}
|
||||
|
||||
private void checkCluster(NamedList cluster, int expectedDocCount,
|
||||
int expectedLabelCount, int expectedSubclusterCount) {
|
||||
checkCluster(cluster, expectedSubclusterCount > 0);
|
||||
assertEquals("number of docs in cluster", expectedDocCount,
|
||||
((NamedList) cluster.get("docs")).size());
|
||||
assertEquals("number of labels in cluster", expectedLabelCount,
|
||||
((NamedList) cluster.get("labels")).size());
|
||||
|
||||
if (expectedSubclusterCount > 0) {
|
||||
NamedList subclusters = (NamedList) cluster.get("clusters");
|
||||
assertEquals("numClusters", expectedSubclusterCount, subclusters.size());
|
||||
assertEquals("number of subclusters in cluster",
|
||||
expectedSubclusterCount, subclusters.size());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
package org.apache.solr.handler.clustering.carrot2;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import org.carrot2.core.*;
|
||||
import org.carrot2.core.attribute.AttributeNames;
|
||||
import org.carrot2.core.attribute.Processing;
|
||||
import org.carrot2.util.attribute.*;
|
||||
import org.carrot2.util.attribute.constraint.IntRange;
|
||||
|
||||
import com.google.common.collect.Lists;
|
||||
|
||||
@Bindable(prefix = "MockClusteringAlgorithm")
|
||||
public class MockClusteringAlgorithm extends ProcessingComponentBase implements
|
||||
IClusteringAlgorithm {
|
||||
@Input
|
||||
@Processing
|
||||
@Attribute(key = AttributeNames.DOCUMENTS)
|
||||
private List<Document> documents;
|
||||
|
||||
@Output
|
||||
@Processing
|
||||
@Attribute(key = AttributeNames.CLUSTERS)
|
||||
private List<Cluster> clusters;
|
||||
|
||||
@Input
|
||||
@Processing
|
||||
@Attribute
|
||||
@IntRange(min = 1, max = 5)
|
||||
private int depth = 2;
|
||||
|
||||
@Input
|
||||
@Processing
|
||||
@Attribute
|
||||
@IntRange(min = 1, max = 5)
|
||||
private int labels = 1;
|
||||
|
||||
@Override
|
||||
public void process() throws ProcessingException {
|
||||
clusters = Lists.newArrayList();
|
||||
if (documents == null) {
|
||||
return;
|
||||
}
|
||||
|
||||
int documentIndex = 1;
|
||||
for (Document document : documents) {
|
||||
StringBuilder label = new StringBuilder("Cluster " + documentIndex);
|
||||
Cluster cluster = createCluster(label.toString(), document);
|
||||
clusters.add(cluster);
|
||||
for (int i = 1; i <= depth; i++) {
|
||||
label.append(".");
|
||||
label.append(i);
|
||||
Cluster newCluster = createCluster(label.toString(), document);
|
||||
cluster.addSubclusters(createCluster(label.toString(), document), newCluster);
|
||||
cluster = newCluster;
|
||||
}
|
||||
documentIndex++;
|
||||
}
|
||||
}
|
||||
|
||||
private Cluster createCluster(String labelBase, Document... documents) {
|
||||
Cluster cluster = new Cluster();
|
||||
for (int i = 0; i < labels; i++) {
|
||||
cluster.addPhrases(labelBase + "#" + (i + 1));
|
||||
}
|
||||
cluster.addDocuments(documents);
|
||||
return cluster;
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,246 @@
|
|||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# Syntax:
|
||||
# "source" => "target"
|
||||
# "source".length() > 0 (source cannot be empty.)
|
||||
# "target".length() >= 0 (target can be empty.)
|
||||
|
||||
# example:
|
||||
# "À" => "A"
|
||||
# "\u00C0" => "A"
|
||||
# "\u00C0" => "\u0041"
|
||||
# "ß" => "ss"
|
||||
# "\t" => " "
|
||||
# "\n" => ""
|
||||
|
||||
# À => A
|
||||
"\u00C0" => "A"
|
||||
|
||||
# Á => A
|
||||
"\u00C1" => "A"
|
||||
|
||||
# Â => A
|
||||
"\u00C2" => "A"
|
||||
|
||||
# Ã => A
|
||||
"\u00C3" => "A"
|
||||
|
||||
# Ä => A
|
||||
"\u00C4" => "A"
|
||||
|
||||
# Å => A
|
||||
"\u00C5" => "A"
|
||||
|
||||
# Æ => AE
|
||||
"\u00C6" => "AE"
|
||||
|
||||
# Ç => C
|
||||
"\u00C7" => "C"
|
||||
|
||||
# È => E
|
||||
"\u00C8" => "E"
|
||||
|
||||
# É => E
|
||||
"\u00C9" => "E"
|
||||
|
||||
# Ê => E
|
||||
"\u00CA" => "E"
|
||||
|
||||
# Ë => E
|
||||
"\u00CB" => "E"
|
||||
|
||||
# Ì => I
|
||||
"\u00CC" => "I"
|
||||
|
||||
# Í => I
|
||||
"\u00CD" => "I"
|
||||
|
||||
# Î => I
|
||||
"\u00CE" => "I"
|
||||
|
||||
# Ï => I
|
||||
"\u00CF" => "I"
|
||||
|
||||
# IJ => IJ
|
||||
"\u0132" => "IJ"
|
||||
|
||||
# Ð => D
|
||||
"\u00D0" => "D"
|
||||
|
||||
# Ñ => N
|
||||
"\u00D1" => "N"
|
||||
|
||||
# Ò => O
|
||||
"\u00D2" => "O"
|
||||
|
||||
# Ó => O
|
||||
"\u00D3" => "O"
|
||||
|
||||
# Ô => O
|
||||
"\u00D4" => "O"
|
||||
|
||||
# Õ => O
|
||||
"\u00D5" => "O"
|
||||
|
||||
# Ö => O
|
||||
"\u00D6" => "O"
|
||||
|
||||
# Ø => O
|
||||
"\u00D8" => "O"
|
||||
|
||||
# Π=> OE
|
||||
"\u0152" => "OE"
|
||||
|
||||
# Þ
|
||||
"\u00DE" => "TH"
|
||||
|
||||
# Ù => U
|
||||
"\u00D9" => "U"
|
||||
|
||||
# Ú => U
|
||||
"\u00DA" => "U"
|
||||
|
||||
# Û => U
|
||||
"\u00DB" => "U"
|
||||
|
||||
# Ü => U
|
||||
"\u00DC" => "U"
|
||||
|
||||
# Ý => Y
|
||||
"\u00DD" => "Y"
|
||||
|
||||
# Ÿ => Y
|
||||
"\u0178" => "Y"
|
||||
|
||||
# à => a
|
||||
"\u00E0" => "a"
|
||||
|
||||
# á => a
|
||||
"\u00E1" => "a"
|
||||
|
||||
# â => a
|
||||
"\u00E2" => "a"
|
||||
|
||||
# ã => a
|
||||
"\u00E3" => "a"
|
||||
|
||||
# ä => a
|
||||
"\u00E4" => "a"
|
||||
|
||||
# å => a
|
||||
"\u00E5" => "a"
|
||||
|
||||
# æ => ae
|
||||
"\u00E6" => "ae"
|
||||
|
||||
# ç => c
|
||||
"\u00E7" => "c"
|
||||
|
||||
# è => e
|
||||
"\u00E8" => "e"
|
||||
|
||||
# é => e
|
||||
"\u00E9" => "e"
|
||||
|
||||
# ê => e
|
||||
"\u00EA" => "e"
|
||||
|
||||
# ë => e
|
||||
"\u00EB" => "e"
|
||||
|
||||
# ì => i
|
||||
"\u00EC" => "i"
|
||||
|
||||
# í => i
|
||||
"\u00ED" => "i"
|
||||
|
||||
# î => i
|
||||
"\u00EE" => "i"
|
||||
|
||||
# ï => i
|
||||
"\u00EF" => "i"
|
||||
|
||||
# ij => ij
|
||||
"\u0133" => "ij"
|
||||
|
||||
# ð => d
|
||||
"\u00F0" => "d"
|
||||
|
||||
# ñ => n
|
||||
"\u00F1" => "n"
|
||||
|
||||
# ò => o
|
||||
"\u00F2" => "o"
|
||||
|
||||
# ó => o
|
||||
"\u00F3" => "o"
|
||||
|
||||
# ô => o
|
||||
"\u00F4" => "o"
|
||||
|
||||
# õ => o
|
||||
"\u00F5" => "o"
|
||||
|
||||
# ö => o
|
||||
"\u00F6" => "o"
|
||||
|
||||
# ø => o
|
||||
"\u00F8" => "o"
|
||||
|
||||
# œ => oe
|
||||
"\u0153" => "oe"
|
||||
|
||||
# ß => ss
|
||||
"\u00DF" => "ss"
|
||||
|
||||
# þ => th
|
||||
"\u00FE" => "th"
|
||||
|
||||
# ù => u
|
||||
"\u00F9" => "u"
|
||||
|
||||
# ú => u
|
||||
"\u00FA" => "u"
|
||||
|
||||
# û => u
|
||||
"\u00FB" => "u"
|
||||
|
||||
# ü => u
|
||||
"\u00FC" => "u"
|
||||
|
||||
# ý => y
|
||||
"\u00FD" => "y"
|
||||
|
||||
# ÿ => y
|
||||
"\u00FF" => "y"
|
||||
|
||||
# ff => ff
|
||||
"\uFB00" => "ff"
|
||||
|
||||
# fi => fi
|
||||
"\uFB01" => "fi"
|
||||
|
||||
# fl => fl
|
||||
"\uFB02" => "fl"
|
||||
|
||||
# ffi => ffi
|
||||
"\uFB03" => "ffi"
|
||||
|
||||
# ffl => ffl
|
||||
"\uFB04" => "ffl"
|
||||
|
||||
# ſt => ft
|
||||
"\uFB05" => "ft"
|
||||
|
||||
# st => st
|
||||
"\uFB06" => "st"
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
# Use a protected word file to protect against the stemmer reducing two
|
||||
# unrelated words to the same base word.
|
||||
|
||||
# Some non-words that normally won't be encountered,
|
||||
# just to test that they won't be stemmed.
|
||||
dontstems
|
||||
zwhacky
|
||||
|
||||
|
|
@ -0,0 +1,347 @@
|
|||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<!--
|
||||
This is the Solr schema file. This file should be named "schema.xml" and
|
||||
should be in the conf directory under the solr home
|
||||
(i.e. ./solr/conf/schema.xml by default)
|
||||
or located where the classloader for the Solr webapp can find it.
|
||||
|
||||
This example schema is the recommended starting point for users.
|
||||
It should be kept correct and concise, usable out-of-the-box.
|
||||
|
||||
For more information, on how to customize this file, please see
|
||||
http://wiki.apache.org/solr/SchemaXml
|
||||
-->
|
||||
|
||||
<schema name="example" version="1.1">
|
||||
<!-- attribute "name" is the name of this schema and is only used for display purposes.
|
||||
Applications should change this to reflect the nature of the search collection.
|
||||
version="1.1" is Solr's version number for the schema syntax and semantics. It should
|
||||
not normally be changed by applications.
|
||||
1.0: multiValued attribute did not exist, all fields are multiValued by nature
|
||||
1.1: multiValued attribute introduced, false by default -->
|
||||
|
||||
<types>
|
||||
<!-- field type definitions. The "name" attribute is
|
||||
just a label to be used by field definitions. The "class"
|
||||
attribute and any other attributes determine the real
|
||||
behavior of the fieldType.
|
||||
Class names starting with "solr" refer to java classes in the
|
||||
org.apache.solr.analysis package.
|
||||
-->
|
||||
|
||||
<!-- The StrField type is not analyzed, but indexed/stored verbatim.
|
||||
- StrField and TextField support an optional compressThreshold which
|
||||
limits compression (if enabled in the derived fields) to values which
|
||||
exceed a certain size (in characters).
|
||||
-->
|
||||
<fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
<!-- boolean type: "true" or "false" -->
|
||||
<fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
<!-- The optional sortMissingLast and sortMissingFirst attributes are
|
||||
currently supported on types that are sorted internally as strings.
|
||||
- If sortMissingLast="true", then a sort on this field will cause documents
|
||||
without the field to come after documents with the field,
|
||||
regardless of the requested sort order (asc or desc).
|
||||
- If sortMissingFirst="true", then a sort on this field will cause documents
|
||||
without the field to come before documents with the field,
|
||||
regardless of the requested sort order.
|
||||
- If sortMissingLast="false" and sortMissingFirst="false" (the default),
|
||||
then default lucene sorting will be used which places docs without the
|
||||
field first in an ascending sort and last in a descending sort.
|
||||
-->
|
||||
|
||||
|
||||
<!-- numeric field types that store and index the text
|
||||
value verbatim (and hence don't support range queries, since the
|
||||
lexicographic ordering isn't equal to the numeric ordering) -->
|
||||
<fieldType name="integer" class="solr.IntField" omitNorms="true"/>
|
||||
<fieldType name="long" class="solr.LongField" omitNorms="true"/>
|
||||
<fieldType name="float" class="solr.FloatField" omitNorms="true"/>
|
||||
<fieldType name="double" class="solr.DoubleField" omitNorms="true"/>
|
||||
|
||||
|
||||
<!-- Numeric field types that manipulate the value into
|
||||
a string value that isn't human-readable in its internal form,
|
||||
but with a lexicographic ordering the same as the numeric ordering,
|
||||
so that range queries work correctly. -->
|
||||
<fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
|
||||
<fieldType name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/>
|
||||
<fieldType name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/>
|
||||
<fieldType name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
|
||||
<!-- The format for this date field is of the form 1995-12-31T23:59:59Z, and
|
||||
is a more restricted form of the canonical representation of dateTime
|
||||
http://www.w3.org/TR/xmlschema-2/#dateTime
|
||||
The trailing "Z" designates UTC time and is mandatory.
|
||||
Optional fractional seconds are allowed: 1995-12-31T23:59:59.999Z
|
||||
All other components are mandatory.
|
||||
|
||||
Expressions can also be used to denote calculations that should be
|
||||
performed relative to "NOW" to determine the value, ie...
|
||||
|
||||
NOW/HOUR
|
||||
... Round to the start of the current hour
|
||||
NOW-1DAY
|
||||
... Exactly 1 day prior to now
|
||||
NOW/DAY+6MONTHS+3DAYS
|
||||
... 6 months and 3 days in the future from the start of
|
||||
the current day
|
||||
|
||||
Consult the DateField javadocs for more information.
|
||||
-->
|
||||
<fieldType name="date" class="solr.DateField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
|
||||
<!-- The "RandomSortField" is not used to store or search any
|
||||
data. You can declare fields of this type it in your schema
|
||||
to generate psuedo-random orderings of your docs for sorting
|
||||
purposes. The ordering is generated based on the field name
|
||||
and the version of the index, As long as the index version
|
||||
remains unchanged, and the same field name is reused,
|
||||
the ordering of the docs will be consistent.
|
||||
If you want differend psuedo-random orderings of documents,
|
||||
for the same version of the index, use a dynamicField and
|
||||
change the name
|
||||
-->
|
||||
<fieldType name="random" class="solr.RandomSortField" indexed="true" />
|
||||
|
||||
<!-- solr.TextField allows the specification of custom text analyzers
|
||||
specified as a tokenizer and a list of token filters. Different
|
||||
analyzers may be specified for indexing and querying.
|
||||
|
||||
The optional positionIncrementGap puts space between multiple fields of
|
||||
this type on the same document, with the purpose of preventing false phrase
|
||||
matching across fields.
|
||||
|
||||
For more info on customizing your analyzer chain, please see
|
||||
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
|
||||
-->
|
||||
|
||||
<!-- One can also specify an existing Analyzer class that has a
|
||||
default constructor via the class attribute on the analyzer element
|
||||
<fieldType name="text_greek" class="solr.TextField">
|
||||
<analyzer class="org.apache.lucene.analysis.el.GreekAnalyzer"/>
|
||||
</fieldType>
|
||||
-->
|
||||
|
||||
<!-- A text field that only splits on whitespace for exact matching of words -->
|
||||
<fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
|
||||
<analyzer>
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!-- A text field that uses WordDelimiterFilter to enable splitting and matching of
|
||||
words on case-change, alpha numeric boundaries, and non-alphanumeric chars,
|
||||
so that a query of "wifi" or "wi fi" could match a document containing "Wi-Fi".
|
||||
Synonyms and stopwords are customized by external files, and stemming is enabled.
|
||||
Duplicate tokens at the same position (which may result from Stemmed Synonyms or
|
||||
WordDelim parts) are removed.
|
||||
-->
|
||||
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
|
||||
<analyzer type="index">
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
<!-- in this example, we will only use synonyms at query time
|
||||
<filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
|
||||
-->
|
||||
<!-- Case insensitive stop word removal.
|
||||
enablePositionIncrements=true ensures that a 'gap' is left to
|
||||
allow for accurate phrase queries.
|
||||
-->
|
||||
<filter class="solr.StopFilterFactory"
|
||||
ignoreCase="true"
|
||||
words="stopwords.txt"
|
||||
enablePositionIncrements="true"
|
||||
/>
|
||||
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
<analyzer type="query">
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
<!--<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>-->
|
||||
<!--<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>-->
|
||||
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
|
||||
<!-- Less flexible matching, but less false matches. Probably not ideal for product names,
|
||||
but may be good for SKUs. Can insert dashes in the wrong place and still match. -->
|
||||
<fieldType name="textTight" class="solr.TextField" positionIncrementGap="100" >
|
||||
<analyzer>
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
<!--<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="false"/>
|
||||
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>-->
|
||||
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="0" generateNumberParts="0" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!--
|
||||
Setup simple analysis for spell checking
|
||||
-->
|
||||
<fieldType name="textSpell" class="solr.TextField" positionIncrementGap="100" >
|
||||
<analyzer>
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!-- This is an example of using the KeywordTokenizer along
|
||||
With various TokenFilterFactories to produce a sortable field
|
||||
that does not include some properties of the source text
|
||||
-->
|
||||
<fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">
|
||||
<analyzer>
|
||||
<!-- KeywordTokenizer does no actual tokenizing, so the entire
|
||||
input string is preserved as a single token
|
||||
-->
|
||||
<tokenizer class="solr.KeywordTokenizerFactory"/>
|
||||
<!-- The LowerCase TokenFilter does what you expect, which can be
|
||||
when you want your sorting to be case insensitive
|
||||
-->
|
||||
<filter class="solr.LowerCaseFilterFactory" />
|
||||
<!-- The TrimFilter removes any leading or trailing whitespace -->
|
||||
<filter class="solr.TrimFilterFactory" />
|
||||
<!-- The PatternReplaceFilter gives you the flexibility to use
|
||||
Java Regular expression to replace any sequence of characters
|
||||
matching a pattern with an arbitrary replacement string,
|
||||
which may include back refrences to portions of the orriginal
|
||||
string matched by the pattern.
|
||||
|
||||
See the Java Regular Expression documentation for more
|
||||
infomation on pattern and replacement string syntax.
|
||||
|
||||
http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.html
|
||||
-->
|
||||
<filter class="solr.PatternReplaceFilterFactory"
|
||||
pattern="([^a-z])" replacement="" replace="all"
|
||||
/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!-- since fields of this type are by default not stored or indexed, any data added to
|
||||
them will be ignored outright
|
||||
-->
|
||||
<fieldtype name="ignored" stored="false" indexed="false" class="solr.StrField" />
|
||||
|
||||
</types>
|
||||
|
||||
|
||||
<fields>
|
||||
<!-- Valid attributes for fields:
|
||||
name: mandatory - the name for the field
|
||||
type: mandatory - the name of a previously defined type from the <types> section
|
||||
indexed: true if this field should be indexed (searchable or sortable)
|
||||
stored: true if this field should be retrievable
|
||||
compressed: [false] if this field should be stored using gzip compression
|
||||
(this will only apply if the field type is compressable; among
|
||||
the standard field types, only TextField and StrField are)
|
||||
multiValued: true if this field may contain multiple values per document
|
||||
omitNorms: (expert) set to true to omit the norms associated with
|
||||
this field (this disables length normalization and index-time
|
||||
boosting for the field, and saves some memory). Only full-text
|
||||
fields or fields that need an index-time boost need norms.
|
||||
termVectors: [false] set to true to store the term vector for a given field.
|
||||
When using MoreLikeThis, fields used for similarity should be stored for
|
||||
best performance.
|
||||
-->
|
||||
|
||||
<field name="id" type="string" indexed="true" stored="true" required="true" />
|
||||
<field name="url" type="string" indexed="true" stored="true" required="true" />
|
||||
|
||||
<field name="title" type="text" indexed="true" stored="true" multiValued="true"/>
|
||||
<field name="snippet" type="text" indexed="true" stored="true" multiValued="true"/>
|
||||
<field name="body" type="text" indexed="true" stored="true" multiValued="true"/>
|
||||
<!-- catchall field, containing all other searchable text fields (implemented
|
||||
via copyField further on in this schema -->
|
||||
<field name="text" type="text" indexed="true" stored="false" multiValued="true"/>
|
||||
<!-- Dynamic field definitions. If a field name is not found, dynamicFields
|
||||
will be used if the name matches any of the patterns.
|
||||
RESTRICTION: the glob-like pattern in the name attribute must have
|
||||
a "*" only at the start or the end.
|
||||
EXAMPLE: name="*_i" will match any field ending in _i (like myid_i, z_i)
|
||||
Longer patterns will be matched first. if equal size patterns
|
||||
both match, the first appearing in the schema will be used. -->
|
||||
<dynamicField name="*_i" type="sint" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_s" type="string" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_l" type="slong" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_t" type="text" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_b" type="boolean" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_f" type="sfloat" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_d" type="sdouble" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_dt" type="date" indexed="true" stored="true"/>
|
||||
|
||||
<dynamicField name="random*" type="random" />
|
||||
|
||||
<!-- uncomment the following to ignore any fields that don't already match an existing
|
||||
field name or dynamic field, rather than reporting them as an error.
|
||||
alternately, change the type="ignored" to some other type e.g. "text" if you want
|
||||
unknown fields indexed and/or stored by default -->
|
||||
<!--dynamicField name="*" type="ignored" /-->
|
||||
|
||||
</fields>
|
||||
|
||||
<!-- Field to use to determine and enforce document uniqueness.
|
||||
Unless this field is marked with required="false", it will be a required field
|
||||
-->
|
||||
<uniqueKey>id</uniqueKey>
|
||||
|
||||
<!-- field for the QueryParser to use when an explicit fieldname is absent -->
|
||||
<defaultSearchField>text</defaultSearchField>
|
||||
|
||||
<!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
|
||||
<solrQueryParser defaultOperator="OR"/>
|
||||
|
||||
<!-- copyField commands copy one field to another at the time a document
|
||||
is added to the index. It's used either to index the same field differently,
|
||||
or to add multiple fields to the same field for easier/faster searching. -->
|
||||
<copyField source="url" dest="text"/>
|
||||
<copyField source="title" dest="text"/>
|
||||
<copyField source="body" dest="text"/>
|
||||
<copyField source="snippet" dest="text"/>
|
||||
|
||||
<!-- Similarity is the scoring routine for each document vs. a query.
|
||||
A custom similarity may be specified here, but the default is fine
|
||||
for most applications. -->
|
||||
<!-- <similarity class="org.apache.lucene.search.DefaultSimilarity"/> -->
|
||||
<!-- ... OR ...
|
||||
Specify a SimilarityFactory class name implementation
|
||||
allowing parameters to be used.
|
||||
-->
|
||||
<!--
|
||||
<similarity class="com.example.solr.CustomSimilarityFactory">
|
||||
<str name="paramkey">param value</str>
|
||||
</similarity>
|
||||
-->
|
||||
|
||||
|
||||
</schema>
|
||||
|
|
@ -0,0 +1,561 @@
|
|||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<config>
|
||||
<!-- Set this to 'false' if you want solr to continue working after it has
|
||||
encountered an severe configuration error. In a production environment,
|
||||
you may want solr to keep working even if one handler is mis-configured.
|
||||
|
||||
You may also set this to false using by setting the system property:
|
||||
-Dsolr.abortOnConfigurationError=false
|
||||
-->
|
||||
<abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
|
||||
|
||||
<!-- Used to specify an alternate directory to hold all index data
|
||||
other than the default ./data under the Solr home.
|
||||
If replication is in use, this should match the replication configuration. -->
|
||||
<dataDir>${solr.data.dir:./solr/data}</dataDir>
|
||||
|
||||
|
||||
<indexDefaults>
|
||||
<!-- Values here affect all index writers and act as a default unless overridden. -->
|
||||
<useCompoundFile>false</useCompoundFile>
|
||||
|
||||
<mergeFactor>10</mergeFactor>
|
||||
<!--
|
||||
If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first.
|
||||
|
||||
-->
|
||||
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
|
||||
<!-- Tell Lucene when to flush documents to disk.
|
||||
Giving Lucene more memory for indexing means faster indexing at the cost of more RAM
|
||||
|
||||
If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first.
|
||||
|
||||
-->
|
||||
<ramBufferSizeMB>32</ramBufferSizeMB>
|
||||
<maxMergeDocs>2147483647</maxMergeDocs>
|
||||
<maxFieldLength>10000</maxFieldLength>
|
||||
<writeLockTimeout>1000</writeLockTimeout>
|
||||
<commitLockTimeout>10000</commitLockTimeout>
|
||||
|
||||
<!--
|
||||
Expert: Turn on Lucene's auto commit capability.
|
||||
This causes intermediate segment flushes to write a new lucene
|
||||
index descriptor, enabling it to be opened by an external
|
||||
IndexReader.
|
||||
NOTE: Despite the name, this value does not have any relation to Solr's autoCommit functionality
|
||||
-->
|
||||
<!--<luceneAutoCommit>false</luceneAutoCommit>-->
|
||||
<!--
|
||||
Expert:
|
||||
The Merge Policy in Lucene controls how merging is handled by Lucene. The default in 2.3 is the LogByteSizeMergePolicy, previous
|
||||
versions used LogDocMergePolicy.
|
||||
|
||||
LogByteSizeMergePolicy chooses segments to merge based on their size. The Lucene 2.2 default, LogDocMergePolicy chose when
|
||||
to merge based on number of documents
|
||||
|
||||
Other implementations of MergePolicy must have a no-argument constructor
|
||||
-->
|
||||
<!--<mergePolicy>org.apache.lucene.index.LogByteSizeMergePolicy</mergePolicy>-->
|
||||
|
||||
<!--
|
||||
Expert:
|
||||
The Merge Scheduler in Lucene controls how merges are performed. The ConcurrentMergeScheduler (Lucene 2.3 default)
|
||||
can perform merges in the background using separate threads. The SerialMergeScheduler (Lucene 2.2 default) does not.
|
||||
-->
|
||||
<!--<mergeScheduler>org.apache.lucene.index.ConcurrentMergeScheduler</mergeScheduler>-->
|
||||
|
||||
<!--
|
||||
This option specifies which Lucene LockFactory implementation to use.
|
||||
|
||||
single = SingleInstanceLockFactory - suggested for a read-only index
|
||||
or when there is no possibility of another process trying
|
||||
to modify the index.
|
||||
native = NativeFSLockFactory
|
||||
simple = SimpleFSLockFactory
|
||||
|
||||
(For backwards compatibility with Solr 1.2, 'simple' is the default
|
||||
if not specified.)
|
||||
-->
|
||||
<lockType>single</lockType>
|
||||
</indexDefaults>
|
||||
|
||||
<mainIndex>
|
||||
<!-- options specific to the main on-disk lucene index -->
|
||||
<useCompoundFile>false</useCompoundFile>
|
||||
<ramBufferSizeMB>32</ramBufferSizeMB>
|
||||
<mergeFactor>10</mergeFactor>
|
||||
<!-- Deprecated -->
|
||||
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
|
||||
<maxMergeDocs>2147483647</maxMergeDocs>
|
||||
<maxFieldLength>10000</maxFieldLength>
|
||||
|
||||
<!-- If true, unlock any held write or commit locks on startup.
|
||||
This defeats the locking mechanism that allows multiple
|
||||
processes to safely access a lucene index, and should be
|
||||
used with care.
|
||||
This is not needed if lock type is 'none' or 'single'
|
||||
-->
|
||||
<unlockOnStartup>false</unlockOnStartup>
|
||||
</mainIndex>
|
||||
|
||||
<!-- Enables JMX if and only if an existing MBeanServer is found, use
|
||||
this if you want to configure JMX through JVM parameters. Remove
|
||||
this to disable exposing Solr configuration and statistics to JMX.
|
||||
|
||||
If you want to connect to a particular server, specify the agentId
|
||||
e.g. <jmx agentId="myAgent" />
|
||||
|
||||
If you want to start a new MBeanServer, specify the serviceUrl
|
||||
e.g <jmx serviceurl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr" />
|
||||
|
||||
For more details see http://wiki.apache.org/solr/SolrJmx
|
||||
-->
|
||||
<jmx />
|
||||
|
||||
<!-- the default high-performance update handler -->
|
||||
<updateHandler class="solr.DirectUpdateHandler2">
|
||||
|
||||
<!-- A prefix of "solr." for class names is an alias that
|
||||
causes solr to search appropriate packages, including
|
||||
org.apache.solr.(search|update|request|core|analysis)
|
||||
-->
|
||||
|
||||
<!-- Perform a <commit/> automatically under certain conditions:
|
||||
maxDocs - number of updates since last commit is greater than this
|
||||
maxTime - oldest uncommited update (in ms) is this long ago
|
||||
<autoCommit>
|
||||
<maxDocs>10000</maxDocs>
|
||||
<maxTime>1000</maxTime>
|
||||
</autoCommit>
|
||||
-->
|
||||
|
||||
<!-- The RunExecutableListener executes an external command.
|
||||
exe - the name of the executable to run
|
||||
dir - dir to use as the current working directory. default="."
|
||||
wait - the calling thread waits until the executable returns. default="true"
|
||||
args - the arguments to pass to the program. default=nothing
|
||||
env - environment variables to set. default=nothing
|
||||
-->
|
||||
<!-- A postCommit event is fired after every commit or optimize command
|
||||
<listener event="postCommit" class="solr.RunExecutableListener">
|
||||
<str name="exe">solr/bin/snapshooter</str>
|
||||
<str name="dir">.</str>
|
||||
<bool name="wait">true</bool>
|
||||
<arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
|
||||
<arr name="env"> <str>MYVAR=val1</str> </arr>
|
||||
</listener>
|
||||
-->
|
||||
<!-- A postOptimize event is fired only after every optimize command, useful
|
||||
in conjunction with index distribution to only distribute optimized indicies
|
||||
<listener event="postOptimize" class="solr.RunExecutableListener">
|
||||
<str name="exe">snapshooter</str>
|
||||
<str name="dir">solr/bin</str>
|
||||
<bool name="wait">true</bool>
|
||||
</listener>
|
||||
-->
|
||||
|
||||
</updateHandler>
|
||||
|
||||
|
||||
<query>
|
||||
<!-- Maximum number of clauses in a boolean query... can affect
|
||||
range or prefix queries that expand to big boolean
|
||||
queries. An exception is thrown if exceeded. -->
|
||||
<maxBooleanClauses>1024</maxBooleanClauses>
|
||||
|
||||
|
||||
<!-- Cache used by SolrIndexSearcher for filters (DocSets),
|
||||
unordered sets of *all* documents that match a query.
|
||||
When a new searcher is opened, its caches may be prepopulated
|
||||
or "autowarmed" using data from caches in the old searcher.
|
||||
autowarmCount is the number of items to prepopulate. For LRUCache,
|
||||
the autowarmed items will be the most recently accessed items.
|
||||
Parameters:
|
||||
class - the SolrCache implementation (currently only LRUCache)
|
||||
size - the maximum number of entries in the cache
|
||||
initialSize - the initial capacity (number of entries) of
|
||||
the cache. (seel java.util.HashMap)
|
||||
autowarmCount - the number of entries to prepopulate from
|
||||
and old cache.
|
||||
-->
|
||||
<filterCache
|
||||
class="solr.LRUCache"
|
||||
size="512"
|
||||
initialSize="512"
|
||||
autowarmCount="128"/>
|
||||
|
||||
<!-- queryResultCache caches results of searches - ordered lists of
|
||||
document ids (DocList) based on a query, a sort, and the range
|
||||
of documents requested. -->
|
||||
<queryResultCache
|
||||
class="solr.LRUCache"
|
||||
size="512"
|
||||
initialSize="512"
|
||||
autowarmCount="32"/>
|
||||
|
||||
<!-- documentCache caches Lucene Document objects (the stored fields for each document).
|
||||
Since Lucene internal document ids are transient, this cache will not be autowarmed. -->
|
||||
<documentCache
|
||||
class="solr.LRUCache"
|
||||
size="512"
|
||||
initialSize="512"
|
||||
autowarmCount="0"/>
|
||||
|
||||
<!-- If true, stored fields that are not requested will be loaded lazily.
|
||||
|
||||
This can result in a significant speed improvement if the usual case is to
|
||||
not load all stored fields, especially if the skipped fields are large compressed
|
||||
text fields.
|
||||
-->
|
||||
<enableLazyFieldLoading>true</enableLazyFieldLoading>
|
||||
|
||||
<!-- Example of a generic cache. These caches may be accessed by name
|
||||
through SolrIndexSearcher.getCache(),cacheLookup(), and cacheInsert().
|
||||
The purpose is to enable easy caching of user/application level data.
|
||||
The regenerator argument should be specified as an implementation
|
||||
of solr.search.CacheRegenerator if autowarming is desired. -->
|
||||
<!--
|
||||
<cache name="myUserCache"
|
||||
class="solr.LRUCache"
|
||||
size="4096"
|
||||
initialSize="1024"
|
||||
autowarmCount="1024"
|
||||
regenerator="org.mycompany.mypackage.MyRegenerator"
|
||||
/>
|
||||
-->
|
||||
|
||||
<!-- An optimization that attempts to use a filter to satisfy a search.
|
||||
If the requested sort does not include score, then the filterCache
|
||||
will be checked for a filter matching the query. If found, the filter
|
||||
will be used as the source of document ids, and then the sort will be
|
||||
applied to that.
|
||||
<useFilterForSortedQuery>true</useFilterForSortedQuery>
|
||||
-->
|
||||
|
||||
<!-- An optimization for use with the queryResultCache. When a search
|
||||
is requested, a superset of the requested number of document ids
|
||||
are collected. For example, if a search for a particular query
|
||||
requests matching documents 10 through 19, and queryWindowSize is 50,
|
||||
then documents 0 through 49 will be collected and cached. Any further
|
||||
requests in that range can be satisfied via the cache. -->
|
||||
<queryResultWindowSize>50</queryResultWindowSize>
|
||||
|
||||
<!-- Maximum number of documents to cache for any entry in the
|
||||
queryResultCache. -->
|
||||
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>
|
||||
|
||||
<!-- This entry enables an int hash representation for filters (DocSets)
|
||||
when the number of items in the set is less than maxSize. For smaller
|
||||
sets, this representation is more memory efficient, more efficient to
|
||||
iterate over, and faster to take intersections. -->
|
||||
<HashDocSet maxSize="3000" loadFactor="0.75"/>
|
||||
|
||||
<!-- a newSearcher event is fired whenever a new searcher is being prepared
|
||||
and there is a current searcher handling requests (aka registered). -->
|
||||
<!-- QuerySenderListener takes an array of NamedList and executes a
|
||||
local query request for each NamedList in sequence. -->
|
||||
<listener event="newSearcher" class="solr.QuerySenderListener">
|
||||
<arr name="queries">
|
||||
<lst> <str name="q">solr</str> <str name="start">0</str> <str name="rows">10</str> </lst>
|
||||
<lst> <str name="q">rocks</str> <str name="start">0</str> <str name="rows">10</str> </lst>
|
||||
<lst><str name="q">static newSearcher warming query from solrconfig.xml</str></lst>
|
||||
</arr>
|
||||
</listener>
|
||||
|
||||
<!-- a firstSearcher event is fired whenever a new searcher is being
|
||||
prepared but there is no current registered searcher to handle
|
||||
requests or to gain autowarming data from. -->
|
||||
<listener event="firstSearcher" class="solr.QuerySenderListener">
|
||||
<arr name="queries">
|
||||
<lst> <str name="q">fast_warm</str> <str name="start">0</str> <str name="rows">10</str> </lst>
|
||||
<lst><str name="q">static firstSearcher warming query from solrconfig.xml</str></lst>
|
||||
</arr>
|
||||
</listener>
|
||||
|
||||
<!-- If a search request comes in and there is no current registered searcher,
|
||||
then immediately register the still warming searcher and use it. If
|
||||
"false" then all requests will block until the first searcher is done
|
||||
warming. -->
|
||||
<useColdSearcher>false</useColdSearcher>
|
||||
|
||||
<!-- Maximum number of searchers that may be warming in the background
|
||||
concurrently. An error is returned if this limit is exceeded. Recommend
|
||||
1-2 for read-only slaves, higher for masters w/o cache warming. -->
|
||||
<maxWarmingSearchers>2</maxWarmingSearchers>
|
||||
|
||||
</query>
|
||||
|
||||
<!--
|
||||
Let the dispatch filter handler /select?qt=XXX
|
||||
handleSelect=true will use consistent error handling for /select and /update
|
||||
handleSelect=false will use solr1.1 style error formatting
|
||||
-->
|
||||
<requestDispatcher handleSelect="true" >
|
||||
<!--Make sure your system has some authentication before enabling remote streaming! -->
|
||||
<requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
|
||||
|
||||
<!-- Set HTTP caching related parameters (for proxy caches and clients).
|
||||
|
||||
To get the behaviour of Solr 1.2 (ie: no caching related headers)
|
||||
use the never304="true" option and do not specify a value for
|
||||
<cacheControl>
|
||||
-->
|
||||
<!-- <httpCaching never304="true"> -->
|
||||
<httpCaching lastModifiedFrom="openTime"
|
||||
etagSeed="Solr">
|
||||
<!-- lastModFrom="openTime" is the default, the Last-Modified value
|
||||
(and validation against If-Modified-Since requests) will all be
|
||||
relative to when the current Searcher was opened.
|
||||
You can change it to lastModFrom="dirLastMod" if you want the
|
||||
value to exactly corrispond to when the physical index was last
|
||||
modified.
|
||||
|
||||
etagSeed="..." is an option you can change to force the ETag
|
||||
header (and validation against If-None-Match requests) to be
|
||||
differnet even if the index has not changed (ie: when making
|
||||
significant changes to your config file)
|
||||
|
||||
lastModifiedFrom and etagSeed are both ignored if you use the
|
||||
never304="true" option.
|
||||
-->
|
||||
<!-- If you include a <cacheControl> directive, it will be used to
|
||||
generate a Cache-Control header, as well as an Expires header
|
||||
if the value contains "max-age="
|
||||
|
||||
By default, no Cache-Control header is generated.
|
||||
|
||||
You can use the <cacheControl> option even if you have set
|
||||
never304="true"
|
||||
-->
|
||||
<!-- <cacheControl>max-age=30, public</cacheControl> -->
|
||||
</httpCaching>
|
||||
</requestDispatcher>
|
||||
|
||||
|
||||
<!-- requestHandler plugins... incoming queries will be dispatched to the
|
||||
correct handler based on the path or the qt (query type) param.
|
||||
Names starting with a '/' are accessed with the a path equal to the
|
||||
registered name. Names without a leading '/' are accessed with:
|
||||
http://host/app/select?qt=name
|
||||
If no qt is defined, the requestHandler that declares default="true"
|
||||
will be used.
|
||||
-->
|
||||
<requestHandler name="standard" class="solr.SearchHandler" default="true">
|
||||
<!-- default values for query parameters -->
|
||||
<lst name="defaults">
|
||||
<str name="echoParams">explicit</str>
|
||||
<!--
|
||||
<int name="rows">10</int>
|
||||
<str name="fl">*</str>
|
||||
<str name="version">2.1</str>
|
||||
-->
|
||||
</lst>
|
||||
<arr name="last-components">
|
||||
<str>clustering</str>
|
||||
</arr>
|
||||
</requestHandler>
|
||||
|
||||
|
||||
<requestHandler name="docClustering" class="solr.SearchHandler">
|
||||
<!-- default values for query parameters -->
|
||||
<lst name="defaults">
|
||||
<str name="echoParams">explicit</str>
|
||||
<!--
|
||||
<int name="rows">10</int>
|
||||
<str name="fl">*</str>
|
||||
<str name="version">2.1</str>
|
||||
-->
|
||||
</lst>
|
||||
<arr name="last-components">
|
||||
<str>doc-clustering</str>
|
||||
</arr>
|
||||
</requestHandler>
|
||||
|
||||
<!-- DisMaxRequestHandler allows easy searching across multiple fields
|
||||
for simple user-entered phrases. It's implementation is now
|
||||
just the standard SearchHandler with a default query type
|
||||
of "dismax".
|
||||
see http://wiki.apache.org/solr/DisMaxRequestHandler
|
||||
-->
|
||||
|
||||
|
||||
<searchComponent class="org.apache.solr.handler.clustering.ClusteringComponent" name="clustering">
|
||||
<!-- Declare an engine -->
|
||||
<lst name="engine">
|
||||
<!-- The name, only one can be named "default" -->
|
||||
<str name="name">default</str>
|
||||
<str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
|
||||
</lst>
|
||||
<lst name="engine">
|
||||
<str name="name">stc</str>
|
||||
<str name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
|
||||
</lst>
|
||||
<lst name="engine">
|
||||
<str name="name">mock</str>
|
||||
<str name="carrot.algorithm">org.apache.solr.handler.clustering.carrot2.MockClusteringAlgorithm</str>
|
||||
</lst>
|
||||
</searchComponent>
|
||||
|
||||
<searchComponent class="org.apache.solr.handler.clustering.ClusteringComponent" name="doc-clustering">
|
||||
<!-- Declare an engine -->
|
||||
<lst name="engine">
|
||||
<!-- The name, only one can be named "default" -->
|
||||
<str name="name">mock</str>
|
||||
<str name="classname">org.apache.solr.handler.clustering.MockDocumentClusteringEngine</str>
|
||||
</lst>
|
||||
</searchComponent>
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- Update request handler.
|
||||
|
||||
Note: Since solr1.1 requestHandlers requires a valid content type header if posted in
|
||||
the body. For example, curl now requires: -H 'Content-type:text/xml; charset=utf-8'
|
||||
The response format differs from solr1.1 formatting and returns a standard error code.
|
||||
|
||||
To enable solr1.1 behavior, remove the /update handler or change its path
|
||||
-->
|
||||
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
|
||||
|
||||
<!--
|
||||
Analysis request handler. Since Solr 1.3. Use to returnhow a document is analyzed. Useful
|
||||
for debugging and as a token server for other types of applications
|
||||
-->
|
||||
<requestHandler name="/analysis" class="solr.AnalysisRequestHandler" />
|
||||
|
||||
|
||||
<!-- CSV update handler, loaded on demand -->
|
||||
<requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" />
|
||||
|
||||
|
||||
<!--
|
||||
Admin Handlers - This will register all the standard admin RequestHandlers. Adding
|
||||
this single handler is equivolent to registering:
|
||||
|
||||
<requestHandler name="/admin/luke" class="org.apache.solr.handler.admin.LukeRequestHandler" />
|
||||
<requestHandler name="/admin/system" class="org.apache.solr.handler.admin.SystemInfoHandler" />
|
||||
<requestHandler name="/admin/plugins" class="org.apache.solr.handler.admin.PluginInfoHandler" />
|
||||
<requestHandler name="/admin/threads" class="org.apache.solr.handler.admin.ThreadDumpHandler" />
|
||||
<requestHandler name="/admin/properties" class="org.apache.solr.handler.admin.PropertiesRequestHandler" />
|
||||
<requestHandler name="/admin/file" class="org.apache.solr.handler.admin.ShowFileRequestHandler" >
|
||||
|
||||
If you wish to hide files under ${solr.home}/conf, explicitly register the ShowFileRequestHandler using:
|
||||
<requestHandler name="/admin/file" class="org.apache.solr.handler.admin.ShowFileRequestHandler" >
|
||||
<lst name="invariants">
|
||||
<str name="hidden">synonyms.txt</str>
|
||||
<str name="hidden">anotherfile.txt</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
-->
|
||||
<requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
|
||||
|
||||
<!-- ping/healthcheck -->
|
||||
<requestHandler name="/admin/ping" class="PingRequestHandler">
|
||||
<lst name="defaults">
|
||||
<str name="qt">standard</str>
|
||||
<str name="q">solrpingquery</str>
|
||||
<str name="echoParams">all</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<!-- Echo the request contents back to the client -->
|
||||
<requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
|
||||
<lst name="defaults">
|
||||
<str name="echoParams">explicit</str> <!-- for all params (including the default etc) use: 'all' -->
|
||||
<str name="echoHandler">true</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<highlighting>
|
||||
<!-- Configure the standard fragmenter -->
|
||||
<!-- This could most likely be commented out in the "default" case -->
|
||||
<fragmenter name="gap" class="org.apache.solr.highlight.GapFragmenter" default="true">
|
||||
<lst name="defaults">
|
||||
<int name="hl.fragsize">100</int>
|
||||
</lst>
|
||||
</fragmenter>
|
||||
|
||||
<!-- A regular-expression-based fragmenter (f.i., for sentence extraction) -->
|
||||
<fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter">
|
||||
<lst name="defaults">
|
||||
<!-- slightly smaller fragsizes work better because of slop -->
|
||||
<int name="hl.fragsize">70</int>
|
||||
<!-- allow 50% slop on fragment sizes -->
|
||||
<float name="hl.regex.slop">0.5</float>
|
||||
<!-- a basic sentence pattern -->
|
||||
<str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
|
||||
</lst>
|
||||
</fragmenter>
|
||||
|
||||
<!-- Configure the standard formatter -->
|
||||
<formatter name="html" class="org.apache.solr.highlight.HtmlFormatter" default="true">
|
||||
<lst name="defaults">
|
||||
<str name="hl.simple.pre"><![CDATA[<em>]]></str>
|
||||
<str name="hl.simple.post"><![CDATA[</em>]]></str>
|
||||
</lst>
|
||||
</formatter>
|
||||
</highlighting>
|
||||
|
||||
|
||||
<!-- queryResponseWriter plugins... query responses will be written using the
|
||||
writer specified by the 'wt' request parameter matching the name of a registered
|
||||
writer.
|
||||
The "default" writer is the default and will be used if 'wt' is not specified
|
||||
in the request. XMLResponseWriter will be used if nothing is specified here.
|
||||
The json, python, and ruby writers are also available by default.
|
||||
|
||||
<queryResponseWriter name="xml" class="org.apache.solr.request.XMLResponseWriter" default="true"/>
|
||||
<queryResponseWriter name="json" class="org.apache.solr.request.JSONResponseWriter"/>
|
||||
<queryResponseWriter name="python" class="org.apache.solr.request.PythonResponseWriter"/>
|
||||
<queryResponseWriter name="ruby" class="org.apache.solr.request.RubyResponseWriter"/>
|
||||
<queryResponseWriter name="php" class="org.apache.solr.request.PHPResponseWriter"/>
|
||||
<queryResponseWriter name="phps" class="org.apache.solr.request.PHPSerializedResponseWriter"/>
|
||||
|
||||
<queryResponseWriter name="custom" class="com.example.MyResponseWriter"/>
|
||||
-->
|
||||
|
||||
<!-- XSLT response writer transforms the XML output by any xslt file found
|
||||
in Solr's conf/xslt directory. Changes to xslt files are checked for
|
||||
every xsltCacheLifetimeSeconds.
|
||||
-->
|
||||
<queryResponseWriter name="xslt" class="org.apache.solr.request.XSLTResponseWriter">
|
||||
<int name="xsltCacheLifetimeSeconds">5</int>
|
||||
</queryResponseWriter>
|
||||
|
||||
|
||||
<!-- example of registering a query parser
|
||||
<queryParser name="lucene" class="org.apache.solr.search.LuceneQParserPlugin"/>
|
||||
-->
|
||||
|
||||
<!-- example of registering a custom function parser
|
||||
<valueSourceParser name="myfunc" class="com.mycompany.MyValueSourceParser" />
|
||||
-->
|
||||
|
||||
<!-- config for the admin interface -->
|
||||
<admin>
|
||||
<defaultQuery>solr</defaultQuery>
|
||||
|
||||
<!-- configure a healthcheck file for servers behind a loadbalancer
|
||||
<healthcheck type="file">server-enabled</healthcheck>
|
||||
-->
|
||||
</admin>
|
||||
|
||||
</config>
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
pizza
|
||||
history
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
# a couple of test stopwords to test that the words are really being
|
||||
# configured from this file:
|
||||
stopworda
|
||||
stopwordb
|
||||
|
||||
#Standard english stop words taken from Lucene's StopAnalyzer
|
||||
a
|
||||
an
|
||||
and
|
||||
are
|
||||
as
|
||||
at
|
||||
be
|
||||
but
|
||||
by
|
||||
for
|
||||
if
|
||||
in
|
||||
into
|
||||
is
|
||||
it
|
||||
no
|
||||
not
|
||||
of
|
||||
on
|
||||
or
|
||||
s
|
||||
such
|
||||
t
|
||||
that
|
||||
the
|
||||
their
|
||||
then
|
||||
there
|
||||
these
|
||||
they
|
||||
this
|
||||
to
|
||||
was
|
||||
will
|
||||
with
|
||||
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
#some test synonym mappings unlikely to appear in real input text
|
||||
aaa => aaaa
|
||||
bbb => bbbb1 bbbb2
|
||||
ccc => cccc1,cccc2
|
||||
a\=>a => b\=>b
|
||||
a\,a => b\,b
|
||||
fooaaa,baraaa,bazaaa
|
||||
|
||||
# Some synonym groups specific to this example
|
||||
GB,gib,gigabyte,gigabytes
|
||||
MB,mib,megabyte,megabytes
|
||||
Television, Televisions, TV, TVs
|
||||
#notice we use "gib" instead of "GiB" so any WordDelimiterFilter coming
|
||||
#after us won't split it into two words.
|
||||
|
||||
# Synonym mappings can be used for spelling correction too
|
||||
pixima => pixma
|
||||
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
This Solr Home provides configuration for using the clustering contrib.
|
||||
|
||||
See http://wiki.apache.org/solr/ClusteringComponent for more info.
|
||||
|
||||
To run:
|
||||
|
||||
From the top level example dir
|
||||
java -Dsolr.solr.home=./clustering/solr -jar start.jar
|
||||
|
||||
|
||||
Add the docs just like you do for the main example.
|
||||
This Solr Home provides configuration for using the clustering contrib.
|
||||
|
||||
See http://wiki.apache.org/solr/ClusteringComponent for more info.
|
||||
|
||||
To run:
|
||||
|
||||
From the top level example dir
|
||||
java -Dsolr.solr.home=./clustering/solr -jar start.jar
|
||||
|
||||
|
||||
Add the docs just like you do for the main example.
|
||||
This Solr Home provides configuration for using the clustering contrib.
|
||||
|
||||
See http://wiki.apache.org/solr/ClusteringComponent for more info.
|
||||
|
||||
To run:
|
||||
|
||||
From the top level example dir
|
||||
java -Dsolr.home.home=./clustering/solr -jar start.jar
|
||||
|
||||
|
||||
Add the docs just like you do for the main example.
|
||||
|
|
@ -0,0 +1,246 @@
|
|||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# Syntax:
|
||||
# "source" => "target"
|
||||
# "source".length() > 0 (source cannot be empty.)
|
||||
# "target".length() >= 0 (target can be empty.)
|
||||
|
||||
# example:
|
||||
# "À" => "A"
|
||||
# "\u00C0" => "A"
|
||||
# "\u00C0" => "\u0041"
|
||||
# "ß" => "ss"
|
||||
# "\t" => " "
|
||||
# "\n" => ""
|
||||
|
||||
# À => A
|
||||
"\u00C0" => "A"
|
||||
|
||||
# Á => A
|
||||
"\u00C1" => "A"
|
||||
|
||||
# Â => A
|
||||
"\u00C2" => "A"
|
||||
|
||||
# Ã => A
|
||||
"\u00C3" => "A"
|
||||
|
||||
# Ä => A
|
||||
"\u00C4" => "A"
|
||||
|
||||
# Å => A
|
||||
"\u00C5" => "A"
|
||||
|
||||
# Æ => AE
|
||||
"\u00C6" => "AE"
|
||||
|
||||
# Ç => C
|
||||
"\u00C7" => "C"
|
||||
|
||||
# È => E
|
||||
"\u00C8" => "E"
|
||||
|
||||
# É => E
|
||||
"\u00C9" => "E"
|
||||
|
||||
# Ê => E
|
||||
"\u00CA" => "E"
|
||||
|
||||
# Ë => E
|
||||
"\u00CB" => "E"
|
||||
|
||||
# Ì => I
|
||||
"\u00CC" => "I"
|
||||
|
||||
# Í => I
|
||||
"\u00CD" => "I"
|
||||
|
||||
# Î => I
|
||||
"\u00CE" => "I"
|
||||
|
||||
# Ï => I
|
||||
"\u00CF" => "I"
|
||||
|
||||
# IJ => IJ
|
||||
"\u0132" => "IJ"
|
||||
|
||||
# Ð => D
|
||||
"\u00D0" => "D"
|
||||
|
||||
# Ñ => N
|
||||
"\u00D1" => "N"
|
||||
|
||||
# Ò => O
|
||||
"\u00D2" => "O"
|
||||
|
||||
# Ó => O
|
||||
"\u00D3" => "O"
|
||||
|
||||
# Ô => O
|
||||
"\u00D4" => "O"
|
||||
|
||||
# Õ => O
|
||||
"\u00D5" => "O"
|
||||
|
||||
# Ö => O
|
||||
"\u00D6" => "O"
|
||||
|
||||
# Ø => O
|
||||
"\u00D8" => "O"
|
||||
|
||||
# Π=> OE
|
||||
"\u0152" => "OE"
|
||||
|
||||
# Þ
|
||||
"\u00DE" => "TH"
|
||||
|
||||
# Ù => U
|
||||
"\u00D9" => "U"
|
||||
|
||||
# Ú => U
|
||||
"\u00DA" => "U"
|
||||
|
||||
# Û => U
|
||||
"\u00DB" => "U"
|
||||
|
||||
# Ü => U
|
||||
"\u00DC" => "U"
|
||||
|
||||
# Ý => Y
|
||||
"\u00DD" => "Y"
|
||||
|
||||
# Ÿ => Y
|
||||
"\u0178" => "Y"
|
||||
|
||||
# à => a
|
||||
"\u00E0" => "a"
|
||||
|
||||
# á => a
|
||||
"\u00E1" => "a"
|
||||
|
||||
# â => a
|
||||
"\u00E2" => "a"
|
||||
|
||||
# ã => a
|
||||
"\u00E3" => "a"
|
||||
|
||||
# ä => a
|
||||
"\u00E4" => "a"
|
||||
|
||||
# å => a
|
||||
"\u00E5" => "a"
|
||||
|
||||
# æ => ae
|
||||
"\u00E6" => "ae"
|
||||
|
||||
# ç => c
|
||||
"\u00E7" => "c"
|
||||
|
||||
# è => e
|
||||
"\u00E8" => "e"
|
||||
|
||||
# é => e
|
||||
"\u00E9" => "e"
|
||||
|
||||
# ê => e
|
||||
"\u00EA" => "e"
|
||||
|
||||
# ë => e
|
||||
"\u00EB" => "e"
|
||||
|
||||
# ì => i
|
||||
"\u00EC" => "i"
|
||||
|
||||
# í => i
|
||||
"\u00ED" => "i"
|
||||
|
||||
# î => i
|
||||
"\u00EE" => "i"
|
||||
|
||||
# ï => i
|
||||
"\u00EF" => "i"
|
||||
|
||||
# ij => ij
|
||||
"\u0133" => "ij"
|
||||
|
||||
# ð => d
|
||||
"\u00F0" => "d"
|
||||
|
||||
# ñ => n
|
||||
"\u00F1" => "n"
|
||||
|
||||
# ò => o
|
||||
"\u00F2" => "o"
|
||||
|
||||
# ó => o
|
||||
"\u00F3" => "o"
|
||||
|
||||
# ô => o
|
||||
"\u00F4" => "o"
|
||||
|
||||
# õ => o
|
||||
"\u00F5" => "o"
|
||||
|
||||
# ö => o
|
||||
"\u00F6" => "o"
|
||||
|
||||
# ø => o
|
||||
"\u00F8" => "o"
|
||||
|
||||
# œ => oe
|
||||
"\u0153" => "oe"
|
||||
|
||||
# ß => ss
|
||||
"\u00DF" => "ss"
|
||||
|
||||
# þ => th
|
||||
"\u00FE" => "th"
|
||||
|
||||
# ù => u
|
||||
"\u00F9" => "u"
|
||||
|
||||
# ú => u
|
||||
"\u00FA" => "u"
|
||||
|
||||
# û => u
|
||||
"\u00FB" => "u"
|
||||
|
||||
# ü => u
|
||||
"\u00FC" => "u"
|
||||
|
||||
# ý => y
|
||||
"\u00FD" => "y"
|
||||
|
||||
# ÿ => y
|
||||
"\u00FF" => "y"
|
||||
|
||||
# ff => ff
|
||||
"\uFB00" => "ff"
|
||||
|
||||
# fi => fi
|
||||
"\uFB01" => "fi"
|
||||
|
||||
# fl => fl
|
||||
"\uFB02" => "fl"
|
||||
|
||||
# ffi => ffi
|
||||
"\uFB03" => "ffi"
|
||||
|
||||
# ffl => ffl
|
||||
"\uFB04" => "ffl"
|
||||
|
||||
# ſt => ft
|
||||
"\uFB05" => "ft"
|
||||
|
||||
# st => st
|
||||
"\uFB06" => "st"
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
# Use a protected word file to protect against the stemmer reducing two
|
||||
# unrelated words to the same base word.
|
||||
|
||||
# Some non-words that normally won't be encountered,
|
||||
# just to test that they won't be stemmed.
|
||||
dontstems
|
||||
zwhacky
|
||||
|
|
@ -0,0 +1,388 @@
|
|||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<!--
|
||||
This is the Solr schema file. This file should be named "schema.xml" and
|
||||
should be in the conf directory under the solr home
|
||||
(i.e. ./solr/conf/schema.xml by default)
|
||||
or located where the classloader for the Solr webapp can find it.
|
||||
|
||||
This example schema is the recommended starting point for users.
|
||||
It should be kept correct and concise, usable out-of-the-box.
|
||||
|
||||
For more information, on how to customize this file, please see
|
||||
http://wiki.apache.org/solr/SchemaXml
|
||||
-->
|
||||
|
||||
<schema name="example" version="1.1">
|
||||
<!-- attribute "name" is the name of this schema and is only used for display purposes.
|
||||
Applications should change this to reflect the nature of the search collection.
|
||||
version="1.1" is Solr's version number for the schema syntax and semantics. It should
|
||||
not normally be changed by applications.
|
||||
1.0: multiValued attribute did not exist, all fields are multiValued by nature
|
||||
1.1: multiValued attribute introduced, false by default -->
|
||||
|
||||
<types>
|
||||
<!-- field type definitions. The "name" attribute is
|
||||
just a label to be used by field definitions. The "class"
|
||||
attribute and any other attributes determine the real
|
||||
behavior of the fieldType.
|
||||
Class names starting with "solr" refer to java classes in the
|
||||
org.apache.solr.analysis package.
|
||||
-->
|
||||
|
||||
<!-- The StrField type is not analyzed, but indexed/stored verbatim.
|
||||
- StrField and TextField support an optional compressThreshold which
|
||||
limits compression (if enabled in the derived fields) to values which
|
||||
exceed a certain size (in characters).
|
||||
-->
|
||||
<fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
<!-- boolean type: "true" or "false" -->
|
||||
<fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
<!-- The optional sortMissingLast and sortMissingFirst attributes are
|
||||
currently supported on types that are sorted internally as strings.
|
||||
- If sortMissingLast="true", then a sort on this field will cause documents
|
||||
without the field to come after documents with the field,
|
||||
regardless of the requested sort order (asc or desc).
|
||||
- If sortMissingFirst="true", then a sort on this field will cause documents
|
||||
without the field to come before documents with the field,
|
||||
regardless of the requested sort order.
|
||||
- If sortMissingLast="false" and sortMissingFirst="false" (the default),
|
||||
then default lucene sorting will be used which places docs without the
|
||||
field first in an ascending sort and last in a descending sort.
|
||||
-->
|
||||
|
||||
|
||||
<!-- numeric field types that store and index the text
|
||||
value verbatim (and hence don't support range queries, since the
|
||||
lexicographic ordering isn't equal to the numeric ordering) -->
|
||||
<fieldType name="integer" class="solr.IntField" omitNorms="true"/>
|
||||
<fieldType name="long" class="solr.LongField" omitNorms="true"/>
|
||||
<fieldType name="float" class="solr.FloatField" omitNorms="true"/>
|
||||
<fieldType name="double" class="solr.DoubleField" omitNorms="true"/>
|
||||
|
||||
|
||||
<!-- Numeric field types that manipulate the value into
|
||||
a string value that isn't human-readable in its internal form,
|
||||
but with a lexicographic ordering the same as the numeric ordering,
|
||||
so that range queries work correctly. -->
|
||||
<fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/>
|
||||
<fieldType name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/>
|
||||
<fieldType name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/>
|
||||
<fieldType name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
|
||||
<!-- The format for this date field is of the form 1995-12-31T23:59:59Z, and
|
||||
is a more restricted form of the canonical representation of dateTime
|
||||
http://www.w3.org/TR/xmlschema-2/#dateTime
|
||||
The trailing "Z" designates UTC time and is mandatory.
|
||||
Optional fractional seconds are allowed: 1995-12-31T23:59:59.999Z
|
||||
All other components are mandatory.
|
||||
|
||||
Expressions can also be used to denote calculations that should be
|
||||
performed relative to "NOW" to determine the value, ie...
|
||||
|
||||
NOW/HOUR
|
||||
... Round to the start of the current hour
|
||||
NOW-1DAY
|
||||
... Exactly 1 day prior to now
|
||||
NOW/DAY+6MONTHS+3DAYS
|
||||
... 6 months and 3 days in the future from the start of
|
||||
the current day
|
||||
|
||||
Consult the DateField javadocs for more information.
|
||||
-->
|
||||
<fieldType name="date" class="solr.DateField" sortMissingLast="true" omitNorms="true"/>
|
||||
|
||||
|
||||
<!-- The "RandomSortField" is not used to store or search any
|
||||
data. You can declare fields of this type it in your schema
|
||||
to generate psuedo-random orderings of your docs for sorting
|
||||
purposes. The ordering is generated based on the field name
|
||||
and the version of the index, As long as the index version
|
||||
remains unchanged, and the same field name is reused,
|
||||
the ordering of the docs will be consistent.
|
||||
If you want differend psuedo-random orderings of documents,
|
||||
for the same version of the index, use a dynamicField and
|
||||
change the name
|
||||
-->
|
||||
<fieldType name="random" class="solr.RandomSortField" indexed="true" />
|
||||
|
||||
<!-- solr.TextField allows the specification of custom text analyzers
|
||||
specified as a tokenizer and a list of token filters. Different
|
||||
analyzers may be specified for indexing and querying.
|
||||
|
||||
The optional positionIncrementGap puts space between multiple fields of
|
||||
this type on the same document, with the purpose of preventing false phrase
|
||||
matching across fields.
|
||||
|
||||
For more info on customizing your analyzer chain, please see
|
||||
http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters
|
||||
-->
|
||||
|
||||
<!-- One can also specify an existing Analyzer class that has a
|
||||
default constructor via the class attribute on the analyzer element
|
||||
<fieldType name="text_greek" class="solr.TextField">
|
||||
<analyzer class="org.apache.lucene.analysis.el.GreekAnalyzer"/>
|
||||
</fieldType>
|
||||
-->
|
||||
|
||||
<!-- A text field that only splits on whitespace for exact matching of words -->
|
||||
<fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
|
||||
<analyzer>
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!-- A text field that uses WordDelimiterFilter to enable splitting and matching of
|
||||
words on case-change, alpha numeric boundaries, and non-alphanumeric chars,
|
||||
so that a query of "wifi" or "wi fi" could match a document containing "Wi-Fi".
|
||||
Synonyms and stopwords are customized by external files, and stemming is enabled.
|
||||
Duplicate tokens at the same position (which may result from Stemmed Synonyms or
|
||||
WordDelim parts) are removed.
|
||||
-->
|
||||
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
|
||||
<analyzer type="index">
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
<!-- in this example, we will only use synonyms at query time
|
||||
<filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
|
||||
-->
|
||||
<!-- Case insensitive stop word removal.
|
||||
enablePositionIncrements=true ensures that a 'gap' is left to
|
||||
allow for accurate phrase queries.
|
||||
-->
|
||||
<filter class="solr.StopFilterFactory"
|
||||
ignoreCase="true"
|
||||
words="stopwords.txt"
|
||||
enablePositionIncrements="true"
|
||||
/>
|
||||
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
<analyzer type="query">
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
|
||||
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
|
||||
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
|
||||
<!-- Less flexible matching, but less false matches. Probably not ideal for product names,
|
||||
but may be good for SKUs. Can insert dashes in the wrong place and still match. -->
|
||||
<fieldType name="textTight" class="solr.TextField" positionIncrementGap="100" >
|
||||
<analyzer>
|
||||
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
|
||||
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="false"/>
|
||||
<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
|
||||
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="0" generateNumberParts="0" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!--
|
||||
Setup simple analysis for spell checking
|
||||
-->
|
||||
<fieldType name="textSpell" class="solr.TextField" positionIncrementGap="100" >
|
||||
<analyzer>
|
||||
<tokenizer class="solr.StandardTokenizerFactory"/>
|
||||
<filter class="solr.LowerCaseFilterFactory"/>
|
||||
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!-- This is an example of using the KeywordTokenizer along
|
||||
With various TokenFilterFactories to produce a sortable field
|
||||
that does not include some properties of the source text
|
||||
-->
|
||||
<fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true">
|
||||
<analyzer>
|
||||
<!-- KeywordTokenizer does no actual tokenizing, so the entire
|
||||
input string is preserved as a single token
|
||||
-->
|
||||
<tokenizer class="solr.KeywordTokenizerFactory"/>
|
||||
<!-- The LowerCase TokenFilter does what you expect, which can be
|
||||
when you want your sorting to be case insensitive
|
||||
-->
|
||||
<filter class="solr.LowerCaseFilterFactory" />
|
||||
<!-- The TrimFilter removes any leading or trailing whitespace -->
|
||||
<filter class="solr.TrimFilterFactory" />
|
||||
<!-- The PatternReplaceFilter gives you the flexibility to use
|
||||
Java Regular expression to replace any sequence of characters
|
||||
matching a pattern with an arbitrary replacement string,
|
||||
which may include back refrences to portions of the orriginal
|
||||
string matched by the pattern.
|
||||
|
||||
See the Java Regular Expression documentation for more
|
||||
infomation on pattern and replacement string syntax.
|
||||
|
||||
http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/package-summary.html
|
||||
-->
|
||||
<filter class="solr.PatternReplaceFilterFactory"
|
||||
pattern="([^a-z])" replacement="" replace="all"
|
||||
/>
|
||||
</analyzer>
|
||||
</fieldType>
|
||||
|
||||
<!-- since fields of this type are by default not stored or indexed, any data added to
|
||||
them will be ignored outright
|
||||
-->
|
||||
<fieldtype name="ignored" stored="false" indexed="false" class="solr.StrField" />
|
||||
|
||||
</types>
|
||||
|
||||
|
||||
<fields>
|
||||
<!-- Valid attributes for fields:
|
||||
name: mandatory - the name for the field
|
||||
type: mandatory - the name of a previously defined type from the <types> section
|
||||
indexed: true if this field should be indexed (searchable or sortable)
|
||||
stored: true if this field should be retrievable
|
||||
compressed: [false] if this field should be stored using gzip compression
|
||||
(this will only apply if the field type is compressable; among
|
||||
the standard field types, only TextField and StrField are)
|
||||
multiValued: true if this field may contain multiple values per document
|
||||
omitNorms: (expert) set to true to omit the norms associated with
|
||||
this field (this disables length normalization and index-time
|
||||
boosting for the field, and saves some memory). Only full-text
|
||||
fields or fields that need an index-time boost need norms.
|
||||
termVectors: [false] set to true to store the term vector for a given field.
|
||||
When using MoreLikeThis, fields used for similarity should be stored for
|
||||
best performance.
|
||||
-->
|
||||
|
||||
<field name="id" type="string" indexed="true" stored="true" required="true" />
|
||||
<field name="sku" type="textTight" indexed="true" stored="true" omitNorms="true"/>
|
||||
<field name="name" type="text" indexed="true" stored="true"/>
|
||||
<field name="nameSort" type="string" indexed="true" stored="false"/>
|
||||
<field name="alphaNameSort" type="alphaOnlySort" indexed="true" stored="false"/>
|
||||
<field name="manu" type="text" indexed="true" stored="true" omitNorms="true"/>
|
||||
<field name="cat" type="text_ws" indexed="true" stored="true" multiValued="true" omitNorms="true" termVectors="true" />
|
||||
<field name="features" type="text" indexed="true" stored="true" multiValued="true"/>
|
||||
<field name="includes" type="text" indexed="true" stored="true"/>
|
||||
|
||||
<field name="weight" type="sfloat" indexed="true" stored="true"/>
|
||||
<field name="price" type="sfloat" indexed="true" stored="true"/>
|
||||
<!-- "default" values can be specified for fields, indicating which
|
||||
value should be used if no value is specified when adding a document.
|
||||
-->
|
||||
<field name="popularity" type="sint" indexed="true" stored="true" default="0"/>
|
||||
<field name="inStock" type="boolean" indexed="true" stored="true"/>
|
||||
|
||||
<!-- Some sample docs exists solely to demonstrate the spellchecker
|
||||
functionality, this is the only field they container.
|
||||
Typically you might build the spellchecker of "catchall" type field
|
||||
containing all of the text in each document
|
||||
-->
|
||||
<field name="word" type="string" indexed="true" stored="true"/>
|
||||
|
||||
|
||||
<!-- catchall field, containing all other searchable text fields (implemented
|
||||
via copyField further on in this schema -->
|
||||
<field name="text" type="text" indexed="true" stored="false" multiValued="true"/>
|
||||
|
||||
<!-- non-tokenized version of manufacturer to make it easier to sort or group
|
||||
results by manufacturer. copied from "manu" via copyField -->
|
||||
<field name="manu_exact" type="string" indexed="true" stored="false"/>
|
||||
|
||||
<!-- Here, default is used to create a "timestamp" field indicating
|
||||
When each document was indexed.
|
||||
-->
|
||||
<field name="timestamp" type="date" indexed="true" stored="true" default="NOW" multiValued="false"/>
|
||||
|
||||
<field name="spell" type="textSpell" indexed="true" stored="true" multiValued="true"/>
|
||||
<!-- Dynamic field definitions. If a field name is not found, dynamicFields
|
||||
will be used if the name matches any of the patterns.
|
||||
RESTRICTION: the glob-like pattern in the name attribute must have
|
||||
a "*" only at the start or the end.
|
||||
EXAMPLE: name="*_i" will match any field ending in _i (like myid_i, z_i)
|
||||
Longer patterns will be matched first. if equal size patterns
|
||||
both match, the first appearing in the schema will be used. -->
|
||||
<dynamicField name="*_i" type="sint" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_s" type="string" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_l" type="slong" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_t" type="text" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_b" type="boolean" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_f" type="sfloat" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_d" type="sdouble" indexed="true" stored="true"/>
|
||||
<dynamicField name="*_dt" type="date" indexed="true" stored="true"/>
|
||||
|
||||
<dynamicField name="random*" type="random" />
|
||||
|
||||
<!-- uncomment the following to ignore any fields that don't already match an existing
|
||||
field name or dynamic field, rather than reporting them as an error.
|
||||
alternately, change the type="ignored" to some other type e.g. "text" if you want
|
||||
unknown fields indexed and/or stored by default -->
|
||||
<!--dynamicField name="*" type="ignored" /-->
|
||||
|
||||
</fields>
|
||||
|
||||
<!-- Field to use to determine and enforce document uniqueness.
|
||||
Unless this field is marked with required="false", it will be a required field
|
||||
-->
|
||||
<uniqueKey>id</uniqueKey>
|
||||
|
||||
<!-- field for the QueryParser to use when an explicit fieldname is absent -->
|
||||
<defaultSearchField>text</defaultSearchField>
|
||||
|
||||
<!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
|
||||
<solrQueryParser defaultOperator="OR"/>
|
||||
|
||||
<!-- copyField commands copy one field to another at the time a document
|
||||
is added to the index. It's used either to index the same field differently,
|
||||
or to add multiple fields to the same field for easier/faster searching. -->
|
||||
<copyField source="id" dest="sku"/>
|
||||
|
||||
<copyField source="incubationdate_dt" dest="incubationdate_s"/>
|
||||
<copyField source="cat" dest="text"/>
|
||||
<copyField source="name" dest="text"/>
|
||||
<copyField source="name" dest="nameSort"/>
|
||||
<copyField source="name" dest="alphaNameSort"/>
|
||||
<copyField source="manu" dest="text"/>
|
||||
<copyField source="features" dest="text"/>
|
||||
<copyField source="includes" dest="text"/>
|
||||
|
||||
<copyField source="manu" dest="manu_exact"/>
|
||||
|
||||
<copyField source="name" dest="spell"/>
|
||||
|
||||
<!-- Similarity is the scoring routine for each document vs. a query.
|
||||
A custom similarity may be specified here, but the default is fine
|
||||
for most applications. -->
|
||||
<!-- <similarity class="org.apache.lucene.search.DefaultSimilarity"/> -->
|
||||
<!-- ... OR ...
|
||||
Specify a SimilarityFactory class name implementation
|
||||
allowing parameters to be used.
|
||||
-->
|
||||
<!--
|
||||
<similarity class="com.example.solr.CustomSimilarityFactory">
|
||||
<str name="paramkey">param value</str>
|
||||
</similarity>
|
||||
-->
|
||||
|
||||
|
||||
</schema>
|
||||
|
|
@ -0,0 +1,558 @@
|
|||
<?xml version="1.0" encoding="UTF-8" ?>
|
||||
<!--
|
||||
Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
contributor license agreements. See the NOTICE file distributed with
|
||||
this work for additional information regarding copyright ownership.
|
||||
The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
(the "License"); you may not use this file except in compliance with
|
||||
the License. You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<config>
|
||||
<!-- Set this to 'false' if you want solr to continue working after it has
|
||||
encountered an severe configuration error. In a production environment,
|
||||
you may want solr to keep working even if one handler is mis-configured.
|
||||
|
||||
You may also set this to false using by setting the system property:
|
||||
-Dsolr.abortOnConfigurationError=false
|
||||
-->
|
||||
<abortOnConfigurationError>${solr.abortOnConfigurationError:true}</abortOnConfigurationError>
|
||||
|
||||
<!-- Used to specify an alternate directory to hold all index data
|
||||
other than the default ./data under the Solr home.
|
||||
If replication is in use, this should match the replication configuration. -->
|
||||
<dataDir>${solr.data.dir:./solr/data}</dataDir>
|
||||
|
||||
|
||||
<indexDefaults>
|
||||
<!-- Values here affect all index writers and act as a default unless overridden. -->
|
||||
<useCompoundFile>false</useCompoundFile>
|
||||
|
||||
<mergeFactor>10</mergeFactor>
|
||||
<!--
|
||||
If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first.
|
||||
|
||||
-->
|
||||
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
|
||||
<!-- Tell Lucene when to flush documents to disk.
|
||||
Giving Lucene more memory for indexing means faster indexing at the cost of more RAM
|
||||
|
||||
If both ramBufferSizeMB and maxBufferedDocs is set, then Lucene will flush based on whichever limit is hit first.
|
||||
|
||||
-->
|
||||
<ramBufferSizeMB>32</ramBufferSizeMB>
|
||||
<maxMergeDocs>2147483647</maxMergeDocs>
|
||||
<maxFieldLength>10000</maxFieldLength>
|
||||
<writeLockTimeout>1000</writeLockTimeout>
|
||||
<commitLockTimeout>10000</commitLockTimeout>
|
||||
|
||||
<!--
|
||||
Expert: Turn on Lucene's auto commit capability.
|
||||
This causes intermediate segment flushes to write a new lucene
|
||||
index descriptor, enabling it to be opened by an external
|
||||
IndexReader.
|
||||
NOTE: Despite the name, this value does not have any relation to Solr's autoCommit functionality
|
||||
-->
|
||||
<!--<luceneAutoCommit>false</luceneAutoCommit>-->
|
||||
<!--
|
||||
Expert:
|
||||
The Merge Policy in Lucene controls how merging is handled by Lucene. The default in 2.3 is the LogByteSizeMergePolicy, previous
|
||||
versions used LogDocMergePolicy.
|
||||
|
||||
LogByteSizeMergePolicy chooses segments to merge based on their size. The Lucene 2.2 default, LogDocMergePolicy chose when
|
||||
to merge based on number of documents
|
||||
|
||||
Other implementations of MergePolicy must have a no-argument constructor
|
||||
-->
|
||||
<!--<mergePolicy>org.apache.lucene.index.LogByteSizeMergePolicy</mergePolicy>-->
|
||||
|
||||
<!--
|
||||
Expert:
|
||||
The Merge Scheduler in Lucene controls how merges are performed. The ConcurrentMergeScheduler (Lucene 2.3 default)
|
||||
can perform merges in the background using separate threads. The SerialMergeScheduler (Lucene 2.2 default) does not.
|
||||
-->
|
||||
<!--<mergeScheduler>org.apache.lucene.index.ConcurrentMergeScheduler</mergeScheduler>-->
|
||||
|
||||
<!--
|
||||
This option specifies which Lucene LockFactory implementation to use.
|
||||
|
||||
single = SingleInstanceLockFactory - suggested for a read-only index
|
||||
or when there is no possibility of another process trying
|
||||
to modify the index.
|
||||
native = NativeFSLockFactory
|
||||
simple = SimpleFSLockFactory
|
||||
|
||||
(For backwards compatibility with Solr 1.2, 'simple' is the default
|
||||
if not specified.)
|
||||
-->
|
||||
<lockType>single</lockType>
|
||||
</indexDefaults>
|
||||
|
||||
<mainIndex>
|
||||
<!-- options specific to the main on-disk lucene index -->
|
||||
<useCompoundFile>false</useCompoundFile>
|
||||
<ramBufferSizeMB>32</ramBufferSizeMB>
|
||||
<mergeFactor>10</mergeFactor>
|
||||
<!-- Deprecated -->
|
||||
<!--<maxBufferedDocs>1000</maxBufferedDocs>-->
|
||||
<maxMergeDocs>2147483647</maxMergeDocs>
|
||||
<maxFieldLength>10000</maxFieldLength>
|
||||
|
||||
<!-- If true, unlock any held write or commit locks on startup.
|
||||
This defeats the locking mechanism that allows multiple
|
||||
processes to safely access a lucene index, and should be
|
||||
used with care.
|
||||
This is not needed if lock type is 'none' or 'single'
|
||||
-->
|
||||
<unlockOnStartup>false</unlockOnStartup>
|
||||
</mainIndex>
|
||||
|
||||
<!-- Enables JMX if and only if an existing MBeanServer is found, use
|
||||
this if you want to configure JMX through JVM parameters. Remove
|
||||
this to disable exposing Solr configuration and statistics to JMX.
|
||||
|
||||
If you want to connect to a particular server, specify the agentId
|
||||
e.g. <jmx agentId="myAgent" />
|
||||
|
||||
If you want to start a new MBeanServer, specify the serviceUrl
|
||||
e.g <jmx serviceurl="service:jmx:rmi:///jndi/rmi://localhost:9999/solr" />
|
||||
|
||||
For more details see http://wiki.apache.org/solr/SolrJmx
|
||||
-->
|
||||
<jmx />
|
||||
|
||||
<!-- the default high-performance update handler -->
|
||||
<updateHandler class="solr.DirectUpdateHandler2">
|
||||
|
||||
<!-- A prefix of "solr." for class names is an alias that
|
||||
causes solr to search appropriate packages, including
|
||||
org.apache.solr.(search|update|request|core|analysis)
|
||||
-->
|
||||
|
||||
<!-- Perform a <commit/> automatically under certain conditions:
|
||||
maxDocs - number of updates since last commit is greater than this
|
||||
maxTime - oldest uncommited update (in ms) is this long ago
|
||||
<autoCommit>
|
||||
<maxDocs>10000</maxDocs>
|
||||
<maxTime>1000</maxTime>
|
||||
</autoCommit>
|
||||
-->
|
||||
|
||||
<!-- The RunExecutableListener executes an external command.
|
||||
exe - the name of the executable to run
|
||||
dir - dir to use as the current working directory. default="."
|
||||
wait - the calling thread waits until the executable returns. default="true"
|
||||
args - the arguments to pass to the program. default=nothing
|
||||
env - environment variables to set. default=nothing
|
||||
-->
|
||||
<!-- A postCommit event is fired after every commit or optimize command
|
||||
<listener event="postCommit" class="solr.RunExecutableListener">
|
||||
<str name="exe">solr/bin/snapshooter</str>
|
||||
<str name="dir">.</str>
|
||||
<bool name="wait">true</bool>
|
||||
<arr name="args"> <str>arg1</str> <str>arg2</str> </arr>
|
||||
<arr name="env"> <str>MYVAR=val1</str> </arr>
|
||||
</listener>
|
||||
-->
|
||||
<!-- A postOptimize event is fired only after every optimize command, useful
|
||||
in conjunction with index distribution to only distribute optimized indicies
|
||||
<listener event="postOptimize" class="solr.RunExecutableListener">
|
||||
<str name="exe">snapshooter</str>
|
||||
<str name="dir">solr/bin</str>
|
||||
<bool name="wait">true</bool>
|
||||
</listener>
|
||||
-->
|
||||
|
||||
</updateHandler>
|
||||
|
||||
|
||||
<query>
|
||||
<!-- Maximum number of clauses in a boolean query... can affect
|
||||
range or prefix queries that expand to big boolean
|
||||
queries. An exception is thrown if exceeded. -->
|
||||
<maxBooleanClauses>1024</maxBooleanClauses>
|
||||
|
||||
|
||||
<!-- Cache used by SolrIndexSearcher for filters (DocSets),
|
||||
unordered sets of *all* documents that match a query.
|
||||
When a new searcher is opened, its caches may be prepopulated
|
||||
or "autowarmed" using data from caches in the old searcher.
|
||||
autowarmCount is the number of items to prepopulate. For LRUCache,
|
||||
the autowarmed items will be the most recently accessed items.
|
||||
Parameters:
|
||||
class - the SolrCache implementation (currently only LRUCache)
|
||||
size - the maximum number of entries in the cache
|
||||
initialSize - the initial capacity (number of entries) of
|
||||
the cache. (seel java.util.HashMap)
|
||||
autowarmCount - the number of entries to prepopulate from
|
||||
and old cache.
|
||||
-->
|
||||
<filterCache
|
||||
class="solr.LRUCache"
|
||||
size="512"
|
||||
initialSize="512"
|
||||
autowarmCount="128"/>
|
||||
|
||||
<!-- queryResultCache caches results of searches - ordered lists of
|
||||
document ids (DocList) based on a query, a sort, and the range
|
||||
of documents requested. -->
|
||||
<queryResultCache
|
||||
class="solr.LRUCache"
|
||||
size="512"
|
||||
initialSize="512"
|
||||
autowarmCount="32"/>
|
||||
|
||||
<!-- documentCache caches Lucene Document objects (the stored fields for each document).
|
||||
Since Lucene internal document ids are transient, this cache will not be autowarmed. -->
|
||||
<documentCache
|
||||
class="solr.LRUCache"
|
||||
size="512"
|
||||
initialSize="512"
|
||||
autowarmCount="0"/>
|
||||
|
||||
<!-- If true, stored fields that are not requested will be loaded lazily.
|
||||
|
||||
This can result in a significant speed improvement if the usual case is to
|
||||
not load all stored fields, especially if the skipped fields are large compressed
|
||||
text fields.
|
||||
-->
|
||||
<enableLazyFieldLoading>true</enableLazyFieldLoading>
|
||||
|
||||
<!-- Example of a generic cache. These caches may be accessed by name
|
||||
through SolrIndexSearcher.getCache(),cacheLookup(), and cacheInsert().
|
||||
The purpose is to enable easy caching of user/application level data.
|
||||
The regenerator argument should be specified as an implementation
|
||||
of solr.search.CacheRegenerator if autowarming is desired. -->
|
||||
<!--
|
||||
<cache name="myUserCache"
|
||||
class="solr.LRUCache"
|
||||
size="4096"
|
||||
initialSize="1024"
|
||||
autowarmCount="1024"
|
||||
regenerator="org.mycompany.mypackage.MyRegenerator"
|
||||
/>
|
||||
-->
|
||||
|
||||
<!-- An optimization that attempts to use a filter to satisfy a search.
|
||||
If the requested sort does not include score, then the filterCache
|
||||
will be checked for a filter matching the query. If found, the filter
|
||||
will be used as the source of document ids, and then the sort will be
|
||||
applied to that.
|
||||
<useFilterForSortedQuery>true</useFilterForSortedQuery>
|
||||
-->
|
||||
|
||||
<!-- An optimization for use with the queryResultCache. When a search
|
||||
is requested, a superset of the requested number of document ids
|
||||
are collected. For example, if a search for a particular query
|
||||
requests matching documents 10 through 19, and queryWindowSize is 50,
|
||||
then documents 0 through 49 will be collected and cached. Any further
|
||||
requests in that range can be satisfied via the cache. -->
|
||||
<queryResultWindowSize>50</queryResultWindowSize>
|
||||
|
||||
<!-- Maximum number of documents to cache for any entry in the
|
||||
queryResultCache. -->
|
||||
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>
|
||||
|
||||
<!-- This entry enables an int hash representation for filters (DocSets)
|
||||
when the number of items in the set is less than maxSize. For smaller
|
||||
sets, this representation is more memory efficient, more efficient to
|
||||
iterate over, and faster to take intersections. -->
|
||||
<HashDocSet maxSize="3000" loadFactor="0.75"/>
|
||||
|
||||
<!-- a newSearcher event is fired whenever a new searcher is being prepared
|
||||
and there is a current searcher handling requests (aka registered). -->
|
||||
<!-- QuerySenderListener takes an array of NamedList and executes a
|
||||
local query request for each NamedList in sequence. -->
|
||||
<listener event="newSearcher" class="solr.QuerySenderListener">
|
||||
<arr name="queries">
|
||||
<lst> <str name="q">solr</str> <str name="start">0</str> <str name="rows">10</str> </lst>
|
||||
<lst> <str name="q">rocks</str> <str name="start">0</str> <str name="rows">10</str> </lst>
|
||||
<lst><str name="q">static newSearcher warming query from solrconfig.xml</str></lst>
|
||||
</arr>
|
||||
</listener>
|
||||
|
||||
<!-- a firstSearcher event is fired whenever a new searcher is being
|
||||
prepared but there is no current registered searcher to handle
|
||||
requests or to gain autowarming data from. -->
|
||||
<listener event="firstSearcher" class="solr.QuerySenderListener">
|
||||
<arr name="queries">
|
||||
<lst> <str name="q">fast_warm</str> <str name="start">0</str> <str name="rows">10</str> </lst>
|
||||
<lst><str name="q">static firstSearcher warming query from solrconfig.xml</str></lst>
|
||||
</arr>
|
||||
</listener>
|
||||
|
||||
<!-- If a search request comes in and there is no current registered searcher,
|
||||
then immediately register the still warming searcher and use it. If
|
||||
"false" then all requests will block until the first searcher is done
|
||||
warming. -->
|
||||
<useColdSearcher>false</useColdSearcher>
|
||||
|
||||
<!-- Maximum number of searchers that may be warming in the background
|
||||
concurrently. An error is returned if this limit is exceeded. Recommend
|
||||
1-2 for read-only slaves, higher for masters w/o cache warming. -->
|
||||
<maxWarmingSearchers>2</maxWarmingSearchers>
|
||||
|
||||
</query>
|
||||
|
||||
<!--
|
||||
Let the dispatch filter handler /select?qt=XXX
|
||||
handleSelect=true will use consistent error handling for /select and /update
|
||||
handleSelect=false will use solr1.1 style error formatting
|
||||
-->
|
||||
<requestDispatcher handleSelect="true" >
|
||||
<!--Make sure your system has some authentication before enabling remote streaming! -->
|
||||
<requestParsers enableRemoteStreaming="false" multipartUploadLimitInKB="2048" />
|
||||
|
||||
<!-- Set HTTP caching related parameters (for proxy caches and clients).
|
||||
|
||||
To get the behaviour of Solr 1.2 (ie: no caching related headers)
|
||||
use the never304="true" option and do not specify a value for
|
||||
<cacheControl>
|
||||
-->
|
||||
<!-- <httpCaching never304="true"> -->
|
||||
<httpCaching lastModifiedFrom="openTime"
|
||||
etagSeed="Solr">
|
||||
<!-- lastModFrom="openTime" is the default, the Last-Modified value
|
||||
(and validation against If-Modified-Since requests) will all be
|
||||
relative to when the current Searcher was opened.
|
||||
You can change it to lastModFrom="dirLastMod" if you want the
|
||||
value to exactly corrispond to when the physical index was last
|
||||
modified.
|
||||
|
||||
etagSeed="..." is an option you can change to force the ETag
|
||||
header (and validation against If-None-Match requests) to be
|
||||
differnet even if the index has not changed (ie: when making
|
||||
significant changes to your config file)
|
||||
|
||||
lastModifiedFrom and etagSeed are both ignored if you use the
|
||||
never304="true" option.
|
||||
-->
|
||||
<!-- If you include a <cacheControl> directive, it will be used to
|
||||
generate a Cache-Control header, as well as an Expires header
|
||||
if the value contains "max-age="
|
||||
|
||||
By default, no Cache-Control header is generated.
|
||||
|
||||
You can use the <cacheControl> option even if you have set
|
||||
never304="true"
|
||||
-->
|
||||
<!-- <cacheControl>max-age=30, public</cacheControl> -->
|
||||
</httpCaching>
|
||||
</requestDispatcher>
|
||||
|
||||
|
||||
<!-- requestHandler plugins... incoming queries will be dispatched to the
|
||||
correct handler based on the path or the qt (query type) param.
|
||||
Names starting with a '/' are accessed with the a path equal to the
|
||||
registered name. Names without a leading '/' are accessed with:
|
||||
http://host/app/select?qt=name
|
||||
If no qt is defined, the requestHandler that declares default="true"
|
||||
will be used.
|
||||
-->
|
||||
<requestHandler name="standard" class="solr.SearchHandler" default="true">
|
||||
<!-- default values for query parameters -->
|
||||
<lst name="defaults">
|
||||
<str name="echoParams">explicit</str>
|
||||
<!--
|
||||
<int name="rows">10</int>
|
||||
<str name="fl">*</str>
|
||||
<str name="version">2.1</str>
|
||||
-->
|
||||
<!--<bool name="clustering">true</bool>-->
|
||||
<str name="clustering.engine">default</str>
|
||||
<bool name="clustering.results">true</bool>
|
||||
<!-- The title field -->
|
||||
<str name="carrot.title">name</str>
|
||||
<str name="carrot.url">id</str>
|
||||
<!-- The field to cluster on -->
|
||||
<str name="carrot.snippet">features</str>
|
||||
<!-- produce summaries -->
|
||||
<bool name="carrot.produceSummary">true</bool>
|
||||
<!-- the maximum number of labels per cluster -->
|
||||
<!--<int name="carrot.numDescriptions">5</int>-->
|
||||
<!-- produce sub clusters -->
|
||||
<bool name="carrot.outputSubClusters">false</bool>
|
||||
|
||||
</lst>
|
||||
<arr name="last-components">
|
||||
<str>clustering</str>
|
||||
</arr>
|
||||
</requestHandler>
|
||||
|
||||
|
||||
|
||||
<searchComponent class="org.apache.solr.handler.clustering.ClusteringComponent" name="clustering">
|
||||
<!-- Declare an engine -->
|
||||
<lst name="engine">
|
||||
<!-- The name, only one can be named "default" -->
|
||||
<str name="name">default</str>
|
||||
<!--
|
||||
Class name of Carrot2 clustering algorithm. Currently available algorithms are:
|
||||
|
||||
* org.carrot2.clustering.lingo.LingoClusteringAlgorithm
|
||||
* org.carrot2.clustering.stc.STCClusteringAlgorithm
|
||||
|
||||
See http://project.carrot2.org/algorithms.html for the algorithm's characteristics.
|
||||
-->
|
||||
<str name="carrot.algorithm">org.carrot2.clustering.lingo.LingoClusteringAlgorithm</str>
|
||||
<!--
|
||||
Overriding values for Carrot2 default algorithm attributes. For a description
|
||||
of all available attributes, see: http://download.carrot2.org/stable/manual/#chapter.components.
|
||||
Use attribute key as name attribute of str elements below. These can be further
|
||||
overridden for individual requests by specifying attribute key as request
|
||||
parameter name and attribute value as parameter value.
|
||||
-->
|
||||
<str name="LingoClusteringAlgorithm.desiredClusterCountBase">20</str>
|
||||
</lst>
|
||||
<lst name="engine">
|
||||
<str name="name">stc</str>
|
||||
<str name="carrot.algorithm">org.carrot2.clustering.stc.STCClusteringAlgorithm</str>
|
||||
</lst>
|
||||
</searchComponent>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- Update request handler.
|
||||
|
||||
Note: Since solr1.1 requestHandlers requires a valid content type header if posted in
|
||||
the body. For example, curl now requires: -H 'Content-type:text/xml; charset=utf-8'
|
||||
The response format differs from solr1.1 formatting and returns a standard error code.
|
||||
|
||||
To enable solr1.1 behavior, remove the /update handler or change its path
|
||||
-->
|
||||
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
|
||||
|
||||
<!--
|
||||
Analysis request handler. Since Solr 1.3. Use to returnhow a document is analyzed. Useful
|
||||
for debugging and as a token server for other types of applications
|
||||
-->
|
||||
<requestHandler name="/analysis" class="solr.AnalysisRequestHandler" />
|
||||
|
||||
|
||||
<!-- CSV update handler, loaded on demand -->
|
||||
<requestHandler name="/update/csv" class="solr.CSVRequestHandler" startup="lazy" />
|
||||
|
||||
|
||||
<!--
|
||||
Admin Handlers - This will register all the standard admin RequestHandlers. Adding
|
||||
this single handler is equivolent to registering:
|
||||
|
||||
<requestHandler name="/admin/luke" class="org.apache.solr.handler.admin.LukeRequestHandler" />
|
||||
<requestHandler name="/admin/system" class="org.apache.solr.handler.admin.SystemInfoHandler" />
|
||||
<requestHandler name="/admin/plugins" class="org.apache.solr.handler.admin.PluginInfoHandler" />
|
||||
<requestHandler name="/admin/threads" class="org.apache.solr.handler.admin.ThreadDumpHandler" />
|
||||
<requestHandler name="/admin/properties" class="org.apache.solr.handler.admin.PropertiesRequestHandler" />
|
||||
<requestHandler name="/admin/file" class="org.apache.solr.handler.admin.ShowFileRequestHandler" >
|
||||
|
||||
If you wish to hide files under ${solr.home}/conf, explicitly register the ShowFileRequestHandler using:
|
||||
<requestHandler name="/admin/file" class="org.apache.solr.handler.admin.ShowFileRequestHandler" >
|
||||
<lst name="invariants">
|
||||
<str name="hidden">synonyms.txt</str>
|
||||
<str name="hidden">anotherfile.txt</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
-->
|
||||
<requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" />
|
||||
|
||||
<!-- ping/healthcheck -->
|
||||
<requestHandler name="/admin/ping" class="PingRequestHandler">
|
||||
<lst name="defaults">
|
||||
<str name="qt">standard</str>
|
||||
<str name="q">solrpingquery</str>
|
||||
<str name="echoParams">all</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<!-- Echo the request contents back to the client -->
|
||||
<requestHandler name="/debug/dump" class="solr.DumpRequestHandler" >
|
||||
<lst name="defaults">
|
||||
<str name="echoParams">explicit</str> <!-- for all params (including the default etc) use: 'all' -->
|
||||
<str name="echoHandler">true</str>
|
||||
</lst>
|
||||
</requestHandler>
|
||||
|
||||
<highlighting>
|
||||
<!-- Configure the standard fragmenter -->
|
||||
<!-- This could most likely be commented out in the "default" case -->
|
||||
<fragmenter name="gap" class="org.apache.solr.highlight.GapFragmenter" default="true">
|
||||
<lst name="defaults">
|
||||
<int name="hl.fragsize">100</int>
|
||||
</lst>
|
||||
</fragmenter>
|
||||
|
||||
<!-- A regular-expression-based fragmenter (f.i., for sentence extraction) -->
|
||||
<fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter">
|
||||
<lst name="defaults">
|
||||
<!-- slightly smaller fragsizes work better because of slop -->
|
||||
<int name="hl.fragsize">70</int>
|
||||
<!-- allow 50% slop on fragment sizes -->
|
||||
<float name="hl.regex.slop">0.5</float>
|
||||
<!-- a basic sentence pattern -->
|
||||
<str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
|
||||
</lst>
|
||||
</fragmenter>
|
||||
|
||||
<!-- Configure the standard formatter -->
|
||||
<formatter name="html" class="org.apache.solr.highlight.HtmlFormatter" default="true">
|
||||
<lst name="defaults">
|
||||
<str name="hl.simple.pre"><![CDATA[<em>]]></str>
|
||||
<str name="hl.simple.post"><![CDATA[</em>]]></str>
|
||||
</lst>
|
||||
</formatter>
|
||||
</highlighting>
|
||||
|
||||
|
||||
<!-- queryResponseWriter plugins... query responses will be written using the
|
||||
writer specified by the 'wt' request parameter matching the name of a registered
|
||||
writer.
|
||||
The "default" writer is the default and will be used if 'wt' is not specified
|
||||
in the request. XMLResponseWriter will be used if nothing is specified here.
|
||||
The json, python, and ruby writers are also available by default.
|
||||
|
||||
<queryResponseWriter name="xml" class="org.apache.solr.request.XMLResponseWriter" default="true"/>
|
||||
<queryResponseWriter name="json" class="org.apache.solr.request.JSONResponseWriter"/>
|
||||
<queryResponseWriter name="python" class="org.apache.solr.request.PythonResponseWriter"/>
|
||||
<queryResponseWriter name="ruby" class="org.apache.solr.request.RubyResponseWriter"/>
|
||||
<queryResponseWriter name="php" class="org.apache.solr.request.PHPResponseWriter"/>
|
||||
<queryResponseWriter name="phps" class="org.apache.solr.request.PHPSerializedResponseWriter"/>
|
||||
|
||||
<queryResponseWriter name="custom" class="com.example.MyResponseWriter"/>
|
||||
-->
|
||||
|
||||
<!-- XSLT response writer transforms the XML output by any xslt file found
|
||||
in Solr's conf/xslt directory. Changes to xslt files are checked for
|
||||
every xsltCacheLifetimeSeconds.
|
||||
-->
|
||||
<queryResponseWriter name="xslt" class="org.apache.solr.request.XSLTResponseWriter">
|
||||
<int name="xsltCacheLifetimeSeconds">5</int>
|
||||
</queryResponseWriter>
|
||||
|
||||
|
||||
<!-- example of registering a query parser
|
||||
<queryParser name="lucene" class="org.apache.solr.search.LuceneQParserPlugin"/>
|
||||
-->
|
||||
|
||||
<!-- example of registering a custom function parser
|
||||
<valueSourceParser name="myfunc" class="com.mycompany.MyValueSourceParser" />
|
||||
-->
|
||||
|
||||
<!-- config for the admin interface -->
|
||||
<admin>
|
||||
<defaultQuery>solr</defaultQuery>
|
||||
|
||||
<!-- configure a healthcheck file for servers behind a loadbalancer
|
||||
<healthcheck type="file">server-enabled</healthcheck>
|
||||
-->
|
||||
</admin>
|
||||
|
||||
</config>
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
pizza
|
||||
history
|
||||
|
|
@ -0,0 +1,116 @@
|
|||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
# a couple of test stopwords to test that the words are really being
|
||||
# configured from this file:
|
||||
stopworda
|
||||
stopwordb
|
||||
|
||||
#Standard english stop words taken from Lucene's StopAnalyzer
|
||||
a
|
||||
an
|
||||
and
|
||||
are
|
||||
as
|
||||
at
|
||||
be
|
||||
but
|
||||
by
|
||||
for
|
||||
if
|
||||
in
|
||||
into
|
||||
is
|
||||
it
|
||||
no
|
||||
not
|
||||
of
|
||||
on
|
||||
or
|
||||
s
|
||||
such
|
||||
t
|
||||
that
|
||||
the
|
||||
their
|
||||
then
|
||||
there
|
||||
these
|
||||
they
|
||||
this
|
||||
to
|
||||
was
|
||||
will
|
||||
with
|
||||
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
# a couple of test stopwords to test that the words are really being
|
||||
# configured from this file:
|
||||
stopworda
|
||||
stopwordb
|
||||
|
||||
#Standard english stop words taken from Lucene's StopAnalyzer
|
||||
a
|
||||
an
|
||||
and
|
||||
are
|
||||
as
|
||||
at
|
||||
be
|
||||
but
|
||||
by
|
||||
for
|
||||
if
|
||||
in
|
||||
into
|
||||
is
|
||||
it
|
||||
no
|
||||
not
|
||||
of
|
||||
on
|
||||
or
|
||||
s
|
||||
such
|
||||
t
|
||||
that
|
||||
the
|
||||
their
|
||||
then
|
||||
there
|
||||
these
|
||||
they
|
||||
this
|
||||
to
|
||||
was
|
||||
will
|
||||
with
|
||||
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#-----------------------------------------------------------------------
|
||||
#some test synonym mappings unlikely to appear in real input text
|
||||
aaa => aaaa
|
||||
bbb => bbbb1 bbbb2
|
||||
ccc => cccc1,cccc2
|
||||
a\=>a => b\=>b
|
||||
a\,a => b\,b
|
||||
fooaaa,baraaa,bazaaa
|
||||
|
||||
# Some synonym groups specific to this example
|
||||
GB,gib,gigabyte,gigabytes
|
||||
MB,mib,megabyte,megabytes
|
||||
Television, Televisions, TV, TVs
|
||||
#notice we use "gib" instead of "GiB" so any WordDelimiterFilter coming
|
||||
#after us won't split it into two words.
|
||||
|
||||
# Synonym mappings can be used for spelling correction too
|
||||
pixima => pixma
|
||||
Loading…
Reference in New Issue