|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
Packages that use TokenStream | |
---|---|
org.apache.lucene.analysis | API and code to convert text into indexable/searchable tokens. |
org.apache.lucene.analysis.standard | A fast grammar-based tokenizer constructed with JFlex. |
org.apache.lucene.document | The logical representation of a Document for indexing and searching. |
Uses of TokenStream in org.apache.lucene.analysis |
---|
Subclasses of TokenStream in org.apache.lucene.analysis | |
---|---|
class |
CachingTokenFilter
This class can be used if the Tokens of a TokenStream are intended to be consumed more than once. |
class |
CharTokenizer
An abstract base class for simple, character-oriented tokenizers. |
class |
ISOLatin1AccentFilter
A filter that replaces accented characters in the ISO Latin 1 character set (ISO-8859-1) by their unaccented equivalent. |
class |
KeywordTokenizer
Emits the entire input as a single token. |
class |
LengthFilter
Removes words that are too long and too short from the stream. |
class |
LetterTokenizer
A LetterTokenizer is a tokenizer that divides text at non-letters. |
class |
LowerCaseFilter
Normalizes token text to lower case. |
class |
LowerCaseTokenizer
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. |
class |
PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm. |
class |
SinkTokenizer
A SinkTokenizer can be used to cache Tokens for use in an Analyzer |
class |
StopFilter
Removes stop words from a token stream. |
class |
TeeTokenFilter
Works in conjunction with the SinkTokenizer to provide the ability to set aside tokens that have already been analyzed. |
class |
TokenFilter
A TokenFilter is a TokenStream whose input is another token stream. |
class |
Tokenizer
A Tokenizer is a TokenStream whose input is a Reader. |
class |
WhitespaceTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. |
Fields in org.apache.lucene.analysis declared as TokenStream | |
---|---|
protected TokenStream |
TokenFilter.input
The source of tokens for this filter. |
Methods in org.apache.lucene.analysis that return TokenStream | |
---|---|
TokenStream |
WhitespaceAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
StopAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
KeywordAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
SimpleAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
PerFieldAnalyzerWrapper.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
Analyzer.reusableTokenStream(String fieldName,
Reader reader)
Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. |
TokenStream |
WhitespaceAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
StopAnalyzer.tokenStream(String fieldName,
Reader reader)
Filters LowerCaseTokenizer with StopFilter. |
TokenStream |
KeywordAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
SimpleAnalyzer.tokenStream(String fieldName,
Reader reader)
|
TokenStream |
PerFieldAnalyzerWrapper.tokenStream(String fieldName,
Reader reader)
|
abstract TokenStream |
Analyzer.tokenStream(String fieldName,
Reader reader)
Creates a TokenStream which tokenizes all the text in the provided Reader. |
Constructors in org.apache.lucene.analysis with parameters of type TokenStream | |
---|---|
CachingTokenFilter(TokenStream input)
|
|
ISOLatin1AccentFilter(TokenStream input)
|
|
LengthFilter(TokenStream in,
int min,
int max)
Build a filter that removes words that are too long or too short from the text. |
|
LowerCaseFilter(TokenStream in)
|
|
PorterStemFilter(TokenStream in)
|
|
StopFilter(TokenStream in,
Set stopWords)
Constructs a filter which removes words from the input TokenStream that are named in the Set. |
|
StopFilter(TokenStream input,
Set stopWords,
boolean ignoreCase)
Construct a token stream filtering the given input. |
|
StopFilter(TokenStream input,
String[] stopWords)
Construct a token stream filtering the given input. |
|
StopFilter(TokenStream in,
String[] stopWords,
boolean ignoreCase)
Constructs a filter which removes words from the input TokenStream that are named in the array of words. |
|
TeeTokenFilter(TokenStream input,
SinkTokenizer sink)
|
|
TokenFilter(TokenStream input)
Construct a token stream filtering the given input. |
Uses of TokenStream in org.apache.lucene.analysis.standard |
---|
Subclasses of TokenStream in org.apache.lucene.analysis.standard | |
---|---|
class |
StandardFilter
Normalizes tokens extracted with StandardTokenizer . |
class |
StandardTokenizer
A grammar-based tokenizer constructed with JFlex |
Methods in org.apache.lucene.analysis.standard that return TokenStream | |
---|---|
TokenStream |
StandardAnalyzer.reusableTokenStream(String fieldName,
Reader reader)
|
TokenStream |
StandardAnalyzer.tokenStream(String fieldName,
Reader reader)
Constructs a StandardTokenizer filtered by a StandardFilter , a LowerCaseFilter and a StopFilter . |
Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream | |
---|---|
StandardFilter(TokenStream in)
Construct filtering in. |
Uses of TokenStream in org.apache.lucene.document |
---|
Methods in org.apache.lucene.document that return TokenStream | |
---|---|
TokenStream |
Fieldable.tokenStreamValue()
The value of the field as a TokenStream, or null. |
TokenStream |
Field.tokenStreamValue()
The value of the field as a TokesStream, or null. |
Methods in org.apache.lucene.document with parameters of type TokenStream | |
---|---|
void |
Field.setValue(TokenStream value)
Expert: change the value of this field. |
Constructors in org.apache.lucene.document with parameters of type TokenStream | |
---|---|
Field(String name,
TokenStream tokenStream)
Create a tokenized and indexed field that is not stored. |
|
Field(String name,
TokenStream tokenStream,
Field.TermVector termVector)
Create a tokenized and indexed field that is not stored, optionally with storing term vectors. |
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |