cleaned up the javadocs a bit to make more readable

git-svn-id: https://svn.apache.org/repos/asf/lucene/solr/trunk@633211 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Grant Ingersoll 2008-03-03 18:03:56 +00:00
parent 7f9cddcd01
commit 56f85a877d
1 changed files with 15 additions and 12 deletions

View File

@ -35,26 +35,29 @@ import java.util.regex.Pattern;
/**
* This tokenizer uses regex pattern matching to construct distinct tokens
* for the input stream. It takes two arguments: "pattern" and "group"
*
* "pattern" is the regular expression.
* "group" says which group to extract into tokens.
*
* for the input stream. It takes two arguments: "pattern" and "group".
* <p/>
* <ul>
* <li>"pattern" is the regular expression.</li>
* <li>"group" says which group to extract into tokens.</li>
* </ul>
* <p>
* group=-1 (the default) is equivalent to "split". In this case, the tokens will
* be equivalent to the output from:
*
*
* http://java.sun.com/j2se/1.4.2/docs/api/java/lang/String.html#split(java.lang.String)
*
* Using group >= 0 selects the matching group as the token. For example, if you have:
*
* </p>
* <p>
* Using group >= 0 selects the matching group as the token. For example, if you have:<br/>
* <pre>
* pattern = \'([^\']+)\'
* group = 0
* input = aaa 'bbb' 'ccc'
*
*</pre>
* the output will be two tokens: 'bbb' and 'ccc' (including the ' marks). With the same input
* but using group=1, the output would be: bbb and ccc (no ' marks)
*
*
* </p>
*
* @since solr1.2
* @version $Id:$
*/