Class ParallelLeafReader
- All Implemented Interfaces:
Closeable,AutoCloseable
LeafReader which reads multiple, parallel indexes. Each index added must have the same
number of documents, but typically each contains different fields. Deletions are taken from the
first reader. Each document contains the union of the fields of all documents with the same
document number. When searching, matches for a query term are from the first index added that has
the field.
This is useful, e.g., with collections that have large fields which change rarely and small fields that change more frequently. The smaller fields may be re-indexed in a new index and both indexes may be searched together.
Warning: It is up to you to make sure all indexes are created and modified the same way. For example, if you add documents to one index, you need to add the same documents in the same order to the other indexes. Failure to do so will result in undefined behavior.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.index.IndexReader
IndexReader.CacheHelper, IndexReader.CacheKey, IndexReader.ClosedListener -
Constructor Summary
ConstructorsConstructorDescriptionParallelLeafReader(boolean closeSubReaders, LeafReader... readers) Create a ParallelLeafReader based on the provided readers.ParallelLeafReader(boolean closeSubReaders, LeafReader[] readers, LeafReader[] storedFieldsReaders) Expert: create a ParallelLeafReader based on the provided readers and storedFieldReaders; when a document is loaded, only storedFieldsReaders will be used.ParallelLeafReader(LeafReader... readers) Create a ParallelLeafReader based on the provided readers; auto-closes the given readers onIndexReader.close(). -
Method Summary
Modifier and TypeMethodDescriptionvoidChecks consistency of this reader.protected voiddoClose()Implements close.getBinaryDocValues(String field) ReturnsBinaryDocValuesfor this field, or null if no binary doc values were indexed for this field.getByteVectorValues(String fieldName) ReturnsByteVectorValuesfor this field, or null if noByteVectorValueswere indexed.Optional method: Return aIndexReader.CacheHelperthat can be used to cache based on the content of this leaf regardless of deletions.getDocValuesSkipper(String field) Returns aDocValuesSkipperallowing skipping ranges of doc IDs that are not of interest, ornullif a skip index was not indexed.Get theFieldInfosdescribing all fields in this reader.getFloatVectorValues(String fieldName) ReturnsFloatVectorValuesfor this field, or null if noFloatVectorValueswere indexed.Returns theBitsrepresenting live (not deleted) docs.Return metadata about this leaf.getNormValues(String field) ReturnsNumericDocValuesrepresenting norms for this field, or null if noNumericDocValueswere indexed.getNumericDocValues(String field) ReturnsNumericDocValuesfor this field, or null if no numeric doc values were indexed for this field.Returns theLeafReaders that were passed on init.getPointValues(String fieldName) Returns thePointValuesused for numeric or spatial searches for the given field, or null if there are no point fields.Optional method: Return aIndexReader.CacheHelperthat can be used to cache based on the content of this reader.getSortedDocValues(String field) ReturnsSortedDocValuesfor this field, or null if noSortedDocValueswere indexed for this field.getSortedNumericDocValues(String field) ReturnsSortedNumericDocValuesfor this field, or null if noSortedNumericDocValueswere indexed for this field.getSortedSetDocValues(String field) ReturnsSortedSetDocValuesfor this field, or null if noSortedSetDocValueswere indexed for this field.intmaxDoc()Returns one greater than the largest possible document number.intnumDocs()Returns the number of documents in this index.voidsearchNearestVectors(String fieldName, byte[] target, KnnCollector knnCollector, Bits acceptDocs) Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.voidsearchNearestVectors(String fieldName, float[] target, KnnCollector knnCollector, Bits acceptDocs) Return the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function.Returns aStoredFieldsreader for the stored fields of this index.Returns theTermsindex for this field, or null if it has none.Returns aTermVectorsreader for the term vectors of this index.toString()Methods inherited from class org.apache.lucene.index.LeafReader
docFreq, getContext, getDocCount, getSumDocFreq, getSumTotalTermFreq, postings, postings, searchNearestVectors, searchNearestVectors, totalTermFreqMethods inherited from class org.apache.lucene.index.IndexReader
close, decRef, ensureOpen, equals, getRefCount, hasDeletions, hashCode, incRef, leaves, notifyReaderClosedListeners, numDeletedDocs, registerParentReader, tryIncRef
-
Constructor Details
-
ParallelLeafReader
Create a ParallelLeafReader based on the provided readers; auto-closes the given readers onIndexReader.close().- Throws:
IOException
-
ParallelLeafReader
Create a ParallelLeafReader based on the provided readers.- Throws:
IOException
-
ParallelLeafReader
public ParallelLeafReader(boolean closeSubReaders, LeafReader[] readers, LeafReader[] storedFieldsReaders) throws IOException Expert: create a ParallelLeafReader based on the provided readers and storedFieldReaders; when a document is loaded, only storedFieldsReaders will be used.- Throws:
IOException
-
-
Method Details
-
toString
-
getFieldInfos
Get theFieldInfosdescribing all fields in this reader.Note: Implementations should cache the FieldInfos instance returned by this method such that subsequent calls to this method return the same instance.
NOTE: the returned field numbers will likely not correspond to the actual field numbers in the underlying readers, and codec metadata (
FieldInfo.getAttribute(String)will be unavailable.- Specified by:
getFieldInfosin classLeafReader
-
getLiveDocs
Description copied from class:LeafReaderReturns theBitsrepresenting live (not deleted) docs. A set bit indicates the doc ID has not been deleted. If this method returns null it means there are no deleted documents (all documents are live).The returned instance has been safely published for use by multiple threads without additional synchronization.
- Specified by:
getLiveDocsin classLeafReader
-
terms
Description copied from class:LeafReaderReturns theTermsindex for this field, or null if it has none.- Specified by:
termsin classLeafReader- Throws:
IOException
-
numDocs
public int numDocs()Description copied from class:IndexReaderReturns the number of documents in this index.NOTE: This operation may run in O(maxDoc). Implementations that can't return this number in constant-time should cache it.
- Specified by:
numDocsin classIndexReader
-
maxDoc
public int maxDoc()Description copied from class:IndexReaderReturns one greater than the largest possible document number. This may be used to, e.g., determine how big to allocate an array which will have an element for every document number in an index.- Specified by:
maxDocin classIndexReader
-
storedFields
Description copied from class:IndexReaderReturns aStoredFieldsreader for the stored fields of this index.This call never returns
null, even if no stored fields were indexed. The returned instance should only be used by a single thread.Example:
TopDocs hits = searcher.search(query, 10); StoredFields storedFields = reader.storedFields(); for (ScoreDoc hit : hits.scoreDocs) { Document doc = storedFields.document(hit.doc); }- Specified by:
storedFieldsin classIndexReader- Throws:
IOException- If there is a low-level IO error
-
getCoreCacheHelper
Description copied from class:LeafReaderOptional method: Return aIndexReader.CacheHelperthat can be used to cache based on the content of this leaf regardless of deletions. Two readers that have the same data but different sets of deleted documents or doc values updates may be considered equal. Consider usingIndexReader.getReaderCacheHelper()if you need deletions or dv updates to be taken into account.A return value of
nullindicates that this reader is not suited for caching, which is typically the case for short-lived wrappers that alter the content of the wrapped leaf reader.- Specified by:
getCoreCacheHelperin classLeafReader
-
getReaderCacheHelper
Description copied from class:IndexReaderOptional method: Return aIndexReader.CacheHelperthat can be used to cache based on the content of this reader. Two readers that have different data or different sets of deleted documents will be considered different.A return value of
nullindicates that this reader is not suited for caching, which is typically the case for short-lived wrappers that alter the content of the wrapped reader.- Specified by:
getReaderCacheHelperin classIndexReader
-
termVectors
Description copied from class:IndexReaderReturns aTermVectorsreader for the term vectors of this index.This call never returns
null, even if no term vectors were indexed. The returned instance should only be used by a single thread.Example:
TopDocs hits = searcher.search(query, 10); TermVectors termVectors = reader.termVectors(); for (ScoreDoc hit : hits.scoreDocs) { Fields vector = termVectors.get(hit.doc); }- Specified by:
termVectorsin classIndexReader- Throws:
IOException- If there is a low-level IO error
-
doClose
Description copied from class:IndexReaderImplements close.- Specified by:
doClosein classIndexReader- Throws:
IOException
-
getNumericDocValues
Description copied from class:LeafReaderReturnsNumericDocValuesfor this field, or null if no numeric doc values were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getNumericDocValuesin classLeafReader- Throws:
IOException
-
getBinaryDocValues
Description copied from class:LeafReaderReturnsBinaryDocValuesfor this field, or null if no binary doc values were indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getBinaryDocValuesin classLeafReader- Throws:
IOException
-
getSortedDocValues
Description copied from class:LeafReaderReturnsSortedDocValuesfor this field, or null if noSortedDocValueswere indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getSortedDocValuesin classLeafReader- Throws:
IOException
-
getSortedNumericDocValues
Description copied from class:LeafReaderReturnsSortedNumericDocValuesfor this field, or null if noSortedNumericDocValueswere indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getSortedNumericDocValuesin classLeafReader- Throws:
IOException
-
getSortedSetDocValues
Description copied from class:LeafReaderReturnsSortedSetDocValuesfor this field, or null if noSortedSetDocValueswere indexed for this field. The returned instance should only be used by a single thread.- Specified by:
getSortedSetDocValuesin classLeafReader- Throws:
IOException
-
getDocValuesSkipper
Description copied from class:LeafReaderReturns aDocValuesSkipperallowing skipping ranges of doc IDs that are not of interest, ornullif a skip index was not indexed. The returned instance should be confined to the thread that created it.- Specified by:
getDocValuesSkipperin classLeafReader- Throws:
IOException
-
getNormValues
Description copied from class:LeafReaderReturnsNumericDocValuesrepresenting norms for this field, or null if noNumericDocValueswere indexed. The returned instance should only be used by a single thread.- Specified by:
getNormValuesin classLeafReader- Throws:
IOException
-
getPointValues
Description copied from class:LeafReaderReturns thePointValuesused for numeric or spatial searches for the given field, or null if there are no point fields.- Specified by:
getPointValuesin classLeafReader- Throws:
IOException
-
getFloatVectorValues
Description copied from class:LeafReaderReturnsFloatVectorValuesfor this field, or null if noFloatVectorValueswere indexed. The returned instance should only be used by a single thread.- Specified by:
getFloatVectorValuesin classLeafReader- Throws:
IOException
-
getByteVectorValues
Description copied from class:LeafReaderReturnsByteVectorValuesfor this field, or null if noByteVectorValueswere indexed. The returned instance should only be used by a single thread.- Specified by:
getByteVectorValuesin classLeafReader- Throws:
IOException
-
searchNearestVectors
public void searchNearestVectors(String fieldName, float[] target, KnnCollector knnCollector, Bits acceptDocs) throws IOException Description copied from class:LeafReaderReturn the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.
The returned
TopDocswill contain aScoreDocfor each nearest neighbor, in order of their similarity to the query vector (decreasing scores). TheTotalHitscontains the number of documents visited during the search. If the search stopped early because it hitvisitedLimit, it is indicated through the relationTotalHits.Relation.GREATER_THAN_OR_EQUAL_TO.The behavior is undefined if the given field doesn't have KNN vectors enabled on its
FieldInfo. The return value is nevernull.- Specified by:
searchNearestVectorsin classLeafReader- Parameters:
fieldName- the vector field to searchtarget- the vector-valued queryknnCollector- collector with settings for gathering the vector results.acceptDocs-Bitsthat represents the allowed documents to match, ornullif they are all allowed to match.- Throws:
IOException
-
searchNearestVectors
public void searchNearestVectors(String fieldName, byte[] target, KnnCollector knnCollector, Bits acceptDocs) throws IOException Description copied from class:LeafReaderReturn the k nearest neighbor documents as determined by comparison of their vector values for this field, to the given vector, by the field's similarity function. The score of each document is derived from the vector similarity in a way that ensures scores are positive and that a larger score corresponds to a higher ranking.The search is allowed to be approximate, meaning the results are not guaranteed to be the true k closest neighbors. For large values of k (for example when k is close to the total number of documents), the search may also retrieve fewer than k documents.
The returned
TopDocswill contain aScoreDocfor each nearest neighbor, in order of their similarity to the query vector (decreasing scores). TheTotalHitscontains the number of documents visited during the search. If the search stopped early because it hitvisitedLimit, it is indicated through the relationTotalHits.Relation.GREATER_THAN_OR_EQUAL_TO.The behavior is undefined if the given field doesn't have KNN vectors enabled on its
FieldInfo. The return value is nevernull.- Specified by:
searchNearestVectorsin classLeafReader- Parameters:
fieldName- the vector field to searchtarget- the vector-valued queryknnCollector- collector with settings for gathering the vector results.acceptDocs-Bitsthat represents the allowed documents to match, ornullif they are all allowed to match.- Throws:
IOException
-
checkIntegrity
Description copied from class:LeafReaderChecks consistency of this reader.Note that this may be costly in terms of I/O, e.g. may involve computing a checksum value against large data files.
- Specified by:
checkIntegrityin classLeafReader- Throws:
IOException
-
getParallelReaders
Returns theLeafReaders that were passed on init. -
getMetaData
Description copied from class:LeafReaderReturn metadata about this leaf.- Specified by:
getMetaDatain classLeafReader
-