Package org.apache.lucene.demo.knn
Class KnnVectorDictFilter
java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.demo.knn.KnnVectorDictFilter
- All Implemented Interfaces:
Closeable,AutoCloseable,Unwrappable<TokenStream>
Looks up each tokens in a dictionary, and sums the token vectors. Unrecognized tokens are
ignored. The resulting vector is normalized to unit length.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.State -
Field Summary
Fields inherited from class org.apache.lucene.analysis.TokenFilter
inputFields inherited from class org.apache.lucene.analysis.TokenStream
DEFAULT_TOKEN_ATTRIBUTE_FACTORY -
Constructor Summary
Constructors -
Method Summary
Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, unwrapMethods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
-
Constructor Details
-
KnnVectorDictFilter
sole constructor- Parameters:
input- the input token stream to filter.dict- a token to vector dictionary, used to look up the token vectors.
-
-
Method Details
-
incrementToken
- Specified by:
incrementTokenin classTokenStream- Throws:
IOException
-
reset
- Overrides:
resetin classTokenFilter- Throws:
IOException
-
end
- Overrides:
endin classTokenFilter- Throws:
IOException
-
getResult
Get the vector computed from the input- Returns:
- the resultant sum of the vectors of each term.
-