lucene

History

Dawid Weiss 087f1e3126 LUCENE-3774: Optimized and streamlined license and notice file validation by refactoring the build task into an ANT task and modifying build scripts to perform top-level checks. (Dawid Weiss, Steve Rowe, Robert Muir) git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/trunk@1243527 13f79535-47bb-0310-9956-ffa450edef68		2012-02-13 14:12:59 +00:00
..
lib	LUCENE-3774: Optimized and streamlined license and notice file validation	2012-02-13 14:12:59 +00:00
src	SOLR-3107: hardwire seed in langdetect langid impl	2012-02-07 18:58:43 +00:00
CHANGES.txt	SOLR-3107: hardwire seed in langdetect langid impl	2012-02-07 18:58:43 +00:00
README.txt	SOLR-1979: Create LanguageIdentifierUpdateProcessor	2011-10-05 20:21:59 +00:00
build.xml	LUCENE-3753: Restructure the Lucene build system	2012-02-07 19:59:05 +00:00

README.txt

Apache Solr Language Identifier


Introduction
------------
This module is intended to be used while indexing documents.
It is implemented as an UpdateProcessor to be placed in an UpdateChain.
Its purpose is to identify language from documents and tag the document with language code.
The module can optionally map field names to their language specific counterpart,
e.g. if the input is "title" and language is detected as "en", map to "title_en".
Language may be detected globally for the document, and/or individually per field.

The module currently relies on Tika's language identification capabilities.

Getting Started
---------------
Please refer to the module documentation at http://wiki.apache.org/solr/LanguageDetection

Dependencies
------------
This contrib depends on Tika Core, which is part of extraction contrib.
The easiest is thus to first install extraction contrib and then langid.
Alternatively you can include tika-core manually on your classpath.