org.dspace.search
Class DSTokenizer

java.lang.Object
  extended by org.apache.lucene.analysis.TokenStream
      extended by org.apache.lucene.analysis.Tokenizer
          extended by org.apache.lucene.analysis.CharTokenizer
              extended by org.dspace.search.DSTokenizer

public final class DSTokenizer
extends org.apache.lucene.analysis.CharTokenizer

Customized Lucene Tokenizer, since the standard one rejects numbers from indexing/querying.


Field Summary
 
Fields inherited from class org.apache.lucene.analysis.Tokenizer
input
 
Constructor Summary
DSTokenizer(Reader in)
          Construct a new LowerCaseTokenizer.
 
Method Summary
protected  boolean isTokenChar(char c)
          Collects only characters which do not satisfy Character.isWhitespace(char).
protected  char normalize(char c)
          Collects only characters which satisfy Character.isLetter(char).
 
Methods inherited from class org.apache.lucene.analysis.CharTokenizer
next, reset
 
Methods inherited from class org.apache.lucene.analysis.Tokenizer
close
 
Methods inherited from class org.apache.lucene.analysis.TokenStream
next, reset
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

DSTokenizer

public DSTokenizer(Reader in)
Construct a new LowerCaseTokenizer.

Method Detail

normalize

protected char normalize(char c)
Collects only characters which satisfy Character.isLetter(char).

Overrides:
normalize in class org.apache.lucene.analysis.CharTokenizer

isTokenChar

protected boolean isTokenChar(char c)
Collects only characters which do not satisfy Character.isWhitespace(char).

Specified by:
isTokenChar in class org.apache.lucene.analysis.CharTokenizer


Copyright © 2010 DuraSpace. All Rights Reserved.