Class BlockWriter
java.lang.Object
org.apache.lucene.codecs.uniformsplit.BlockWriter
- Direct Known Subclasses:
STBlockWriter
Writes blocks in the block file.
According the Uniform Split technique, the writing combines three steps per block, and it is repeated for all the field blocks:
- Select the term with the shortest
minimal distinguishing prefix(MDP) in the neighborhood of thetarget block size(+-delta size) - The selected term becomes the first term of the next block, and its MDP is the next block key.
- The current block is written to the
block file. And its block key isaddedto theindex dictionary.
This stateful BlockWriter is called repeatedly to add all the BlockLine terms of a field. Then
finishLastBlock(org.apache.lucene.codecs.uniformsplit.IndexDictionary.Builder) is called. And then this BlockWriter can be reused to add the
terms of another field.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final BlockEncoderprotected final BlockHeader.Serializerprotected final ByteBuffersDataOutputprotected final BlockLine.Serializerprotected final IndexOutputprotected final ByteBuffersDataOutputprotected final intprotected FieldMetadataprotected BytesRefprotected final BlockHeaderprotected BytesRefprotected final intprotected final DeltaBaseTermStateSerializerprotected final ByteBuffersDataOutput -
Constructor Summary
ConstructorsModifierConstructorDescriptionprotectedBlockWriter(IndexOutput blockOutput, int targetNumBlockLines, int deltaNumLines, BlockEncoder blockEncoder) -
Method Summary
Modifier and TypeMethodDescriptionprotected voidaddBlockKey(List<BlockLine> blockLines, IndexDictionary.Builder dictionaryBuilder) Adds a new block key with its corresponding block file pointer to theIndexDictionary.Builder.protected voidaddLine(BytesRef term, BlockTermState blockTermState, IndexDictionary.Builder dictionaryBuilder) Adds a newBlockLineterm for the current field.protected BlockHeader.Serializerprotected BlockLine.Serializerprotected DeltaBaseTermStateSerializerprotected voidfinishLastBlock(IndexDictionary.Builder dictionaryBuilder) This method is called when there is no more term for the field.protected voidsplitAndWriteBlock(IndexDictionary.Builder dictionaryBuilder) Defines the new block start according totargetNumBlockLinesanddeltaNumLines.protected voidupdateFieldMetadata(long blockStartFP) updates the field metadata after all lines were written for the block.protected voidwriteBlock(List<BlockLine> blockLines, IndexDictionary.Builder dictionaryBuilder) Writes a block and adds its block key to the dictionary builder.protected voidwriteBlockLine(boolean isIncrementalEncodingSeed, BlockLine line, BlockLine previousLine)
-
Field Details
-
targetNumBlockLines
protected final int targetNumBlockLines -
deltaNumLines
protected final int deltaNumLines -
blockLines
-
blockOutput
-
blockLinesWriteBuffer
-
termStatesWriteBuffer
-
blockHeaderWriter
-
blockLineWriter
-
termStateSerializer
-
blockEncoder
-
blockWriteBuffer
-
fieldMetadata
-
lastTerm
-
reusableBlockHeader
-
scratchBytesRef
-
-
Constructor Details
-
BlockWriter
protected BlockWriter(IndexOutput blockOutput, int targetNumBlockLines, int deltaNumLines, BlockEncoder blockEncoder)
-
-
Method Details
-
createBlockHeaderSerializer
-
createBlockLineSerializer
-
createDeltaBaseTermStateSerializer
-
addLine
protected void addLine(BytesRef term, BlockTermState blockTermState, IndexDictionary.Builder dictionaryBuilder) throws IOException Adds a newBlockLineterm for the current field.This method determines whether the new term is part of the current block, or if it is part of the next block. In the latter case, a new block is started (including one or more of the lastly added lines), the current block is written to the block file, and the current block key is added to the
IndexDictionary.Builder.- Parameters:
term- The block line term. TheBytesRefinstance is used directly, the caller is responsible to make a deep copy if needed. This is required because we keep a list of block lines until we decide to write the current block, and each line must have a different term instance.blockTermState- Block line details.dictionaryBuilder- to which the block keys are added.- Throws:
IOException
-
finishLastBlock
This method is called when there is no more term for the field. It writes the remaining lines added withaddLine(org.apache.lucene.util.BytesRef, org.apache.lucene.codecs.BlockTermState, org.apache.lucene.codecs.uniformsplit.IndexDictionary.Builder)as the last block of the field and resets thisBlockWriterstate. Then thisBlockWritercan be used for another field.- Throws:
IOException
-
splitAndWriteBlock
Defines the new block start according totargetNumBlockLinesanddeltaNumLines. The new block is started (including one or more of the lastly added lines), the current block is written to the block file, and the current block key is added to theIndexDictionary.Builder.- Throws:
IOException
-
writeBlock
protected void writeBlock(List<BlockLine> blockLines, IndexDictionary.Builder dictionaryBuilder) throws IOException Writes a block and adds its block key to the dictionary builder.- Throws:
IOException
-
updateFieldMetadata
protected void updateFieldMetadata(long blockStartFP) updates the field metadata after all lines were written for the block. -
writeBlockLine
protected void writeBlockLine(boolean isIncrementalEncodingSeed, BlockLine line, BlockLine previousLine) throws IOException - Throws:
IOException
-
addBlockKey
protected void addBlockKey(List<BlockLine> blockLines, IndexDictionary.Builder dictionaryBuilder) throws IOException Adds a new block key with its corresponding block file pointer to theIndexDictionary.Builder. The block key is the MDP (seeTermBytes) of the block first term.- Throws:
IOException
-