Apache
/
lucene
mirror of https://github.com/apache/lucene.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

LUCENE-1135. Added Javadoc marking the WikipediaTokenizer as experimental 

git-svn-id: https://svn.apache.org/repos/asf/lucene/java/trunk@612244 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Grant Ingersoll 2008-01-15 22:01:21 +00:00
parent f7740afe84
commit 0a54ca1920
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
contrib/wikipedia/src/java/org/apache/lucene/wikipedia/analysis/WikipediaTokenizer.java
View File

@ -20,14 +20,17 @@ package org.apache.lucene.wikipedia.analysis;
import org.apache.lucene.analysis.Token;
import org.apache.lucene.analysis.Tokenizer;
import java.io.Reader;
import java.io.IOException;
import java.io.Reader;
/**
* Extension of StandardTokenizer that is aware of Wikipedia syntax. It is based off of the
* Wikipedia tutorial available at http://en.wikipedia.org/wiki/Wikipedia:Tutorial, but it may not be complete.
*
* <p/>
* EXPERIMENTAL !!!!!!!!!
* NOTE: This Tokenizer is considered experimental and the grammar is subject to change in the trunk and in follow up releases.
*
**/
public class WikipediaTokenizer extends Tokenizer {