Package org.apache.arrow.memory
Class DatabricksArrowBuf
- java.lang.Object
-
- org.apache.arrow.memory.ArrowBuf
-
- org.apache.arrow.memory.DatabricksArrowBuf
-
- All Implemented Interfaces:
AutoCloseable
public class DatabricksArrowBuf extends ArrowBuf
A ByteBuffer-backed implementation of ArrowBuf that does not use unsafe memory operations. This implementation uses standard java.nio.ByteBuffer for all memory operations instead of MemoryUtil/Unsafe-based direct memory access.
-
-
Field Summary
-
Fields inherited from class org.apache.arrow.memory.ArrowBuf
DEBUG, DEBUG_ALLOCATOR, DEBUG_LOG_LENGTH
-
-
Constructor Summary
Constructors Constructor Description DatabricksArrowBuf(org.apache.arrow.memory.ReferenceManager referenceManager, org.apache.arrow.memory.BufferManager bufferManager, long capacity)Constructs a new DatabricksArrowBuf backed by a heap ByteBuffer.
-
Method Summary
All Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description longcapacity()ArrowBufcapacity(long newCapacity)Adjusts the capacity of this buffer.voidcheckBytes(long start, long end)Allows a function to determine whether not reading a particular string of bytes is valid.ArrowBufclear()voidclose()booleanequals(Object obj)longgetActualMemoryConsumed()Return that is Accounted for by this buffer (and its potentially shared siblings within the context of the associated allocator).bytegetByte(long index)Get byte value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.voidgetBytes(long index, byte[] dst)Copy data from this ArrowBuf at a given index in into destination byte array.voidgetBytes(long index, byte[] dst, int dstIndex, int length)Copy data from this ArrowBuf at a given index into destination byte array.voidgetBytes(long index, OutputStream out, int length)Copy a certain length of bytes from this ArrowBuf at a given index into the given OutputStream.voidgetBytes(long index, ByteBuffer dst)Copy data from this ArrowBuf at a given index into the destination ByteBuffer.voidgetBytes(long index, ArrowBuf dst, long dstIndex, int length)Copy a given length of data from this ArrowBuf starting at a given index into a dst ArrowBuf at dstIndex.chargetChar(long index)Get char value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.doublegetDouble(long index)Get double value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.floatgetFloat(long index)Get float value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.intgetInt(long index)Get int value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.longgetLong(long index)Get long value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.intgetOffset()Returns the offset within the underlying ByteBuffer where this buffer's data starts.longgetPossibleMemoryConsumed()Returns the possible memory consumed by this ArrowBuf in the worse case scenario.org.apache.arrow.memory.ReferenceManagergetReferenceManager()Get reference manager for this ArrowBuf.shortgetShort(long index)Get short value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.inthashCode()longmemoryAddress()ByteBuffernioBuffer()Make a nio byte buffer from this arrowbuf.ByteBuffernioBuffer(long index, int length)Make a nio byte buffer from this ArrowBuf.ByteOrderorder()Returns the byte order of elements in this buffer.voidprint(StringBuilder sb, int indent)Print detailed information of this buffer intosb.longreadableBytes()Returns the number of bytes still available to read in this buffer.bytereadByte()Read the byte at readerIndex.voidreadBytes(byte[] dst)Read dst.length bytes at readerIndex into dst byte array.longreaderIndex()Get the index at which the next byte will be read from.ArrowBufreaderIndex(long readerIndex)Set the reader index for this ArrowBuf.ArrowBufreallocIfNeeded(long size)Returnsthisif size is less thanArrowBuf.capacity(), otherwise delegates toBufferManager.replace(ArrowBuf, long)to get a new buffer.intrefCnt()voidsetByte(long index, byte value)Set byte value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetByte(long index, int value)Set byte value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetBytes(long index, byte[] src)Copy data from a given byte array into this ArrowBuf starting at a given index.voidsetBytes(long index, byte[] src, int srcIndex, long length)Copy data from a given byte array starting at the given source index into this ArrowBuf at a given index.intsetBytes(long index, InputStream in, int length)Copy a certain length of bytes from given InputStream into this ArrowBuf at the provided index.voidsetBytes(long index, ByteBuffer src)Copy data into this ArrowBuf at a given index onwards from a source ByteBuffer.voidsetBytes(long index, ByteBuffer src, int srcIndex, int length)Copy data into this ArrowBuf at a given index onwards from a source ByteBuffer starting at a given srcIndex for a certain length.voidsetBytes(long index, ArrowBuf src)Copy readableBytes() number of bytes from src ArrowBuf starting from its readerIndex into this ArrowBuf starting at the given index.voidsetBytes(long index, ArrowBuf src, long srcIndex, long length)Copy data from src ArrowBuf starting at index srcIndex into this ArrowBuf at given index.voidsetChar(long index, int value)Set char value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetDouble(long index, double value)Set double value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetFloat(long index, float value)Set float value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetInt(long index, int value)Set int value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetLong(long index, long value)Set long value at a particular index in the underlying memory chunk this ArrowBuf has access to.ArrowBufsetOne(int index, int length)Deprecated.ArrowBufsetOne(long index, long length)Sets all bits to one in the specified range.voidsetShort(long index, int value)Set short value at a particular index in the underlying memory chunk this ArrowBuf has access to.voidsetShort(long index, short value)Set short value at a particular index in the underlying memory chunk this ArrowBuf has access to.ArrowBufsetZero(long index, long length)Zero-out the bytes in this ArrowBuf starting at the given index for the given length.ArrowBufslice()Returns a slice of only the readable bytes in the buffer.ArrowBufslice(long index, long length)Returns a slice (view) starting atindexwith the givenlength.StringtoHexString(long start, int length)Return the buffer's byte contents in the form of a hex dump.StringtoString()longwritableBytes()Returns the number of bytes still available to write into this buffer before capacity is reached.voidwriteByte(byte value)Set the provided byte value at the writerIndex.voidwriteByte(int value)Set the lower order byte for the provided value at the writerIndex.voidwriteBytes(byte[] src)Write the bytes from given byte array into this ArrowBuf starting at writerIndex.voidwriteBytes(byte[] src, int srcIndex, int length)Write the bytes from given byte array starting at srcIndex into this ArrowBuf starting at writerIndex.voidwriteDouble(double value)Set the provided double value at the writerIndex.voidwriteFloat(float value)Set the provided float value at the writerIndex.voidwriteInt(int value)Set the provided int value at the writerIndex.voidwriteLong(long value)Set the provided long value at the writerIndex.longwriterIndex()Get the index at which next byte will be written to.ArrowBufwriterIndex(long writerIndex)Set the writer index for this ArrowBuf.voidwriteShort(int value)Set the provided int value as short at the writerIndex.
-
-
-
Constructor Detail
-
DatabricksArrowBuf
public DatabricksArrowBuf(org.apache.arrow.memory.ReferenceManager referenceManager, org.apache.arrow.memory.BufferManager bufferManager, long capacity)Constructs a new DatabricksArrowBuf backed by a heap ByteBuffer.- Parameters:
referenceManager- The memory manager to track memory usage and reference countbufferManager- The buffer manager for reallocation supportcapacity- The capacity in bytes of this buffer
-
-
Method Detail
-
checkBytes
public void checkBytes(long start, long end)Description copied from class:ArrowBufAllows a function to determine whether not reading a particular string of bytes is valid.Will throw an exception if the memory is not readable for some reason. Only doesn't something in the case that AssertionUtil.BOUNDS_CHECKING_ENABLED is true.
- Overrides:
checkBytesin classArrowBuf- Parameters:
start- The starting position of the bytes to be read.end- The exclusive endpoint of the bytes to be read.
-
getReferenceManager
public org.apache.arrow.memory.ReferenceManager getReferenceManager()
Description copied from class:ArrowBufGet reference manager for this ArrowBuf.- Overrides:
getReferenceManagerin classArrowBuf- Returns:
- user provided implementation of
ReferenceManager
-
capacity
public ArrowBuf capacity(long newCapacity)
Description copied from class:ArrowBufAdjusts the capacity of this buffer. Size increases are NOT supported.
-
order
public ByteOrder order()
Description copied from class:ArrowBufReturns the byte order of elements in this buffer.
-
readableBytes
public long readableBytes()
Description copied from class:ArrowBufReturns the number of bytes still available to read in this buffer.- Overrides:
readableBytesin classArrowBuf
-
writableBytes
public long writableBytes()
Description copied from class:ArrowBufReturns the number of bytes still available to write into this buffer before capacity is reached.- Overrides:
writableBytesin classArrowBuf
-
slice
public ArrowBuf slice()
Description copied from class:ArrowBufReturns a slice of only the readable bytes in the buffer.
-
slice
public ArrowBuf slice(long index, long length)
Description copied from class:ArrowBufReturns a slice (view) starting atindexwith the givenlength.
-
nioBuffer
public ByteBuffer nioBuffer()
Description copied from class:ArrowBufMake a nio byte buffer from this arrowbuf.
-
nioBuffer
public ByteBuffer nioBuffer(long index, int length)
Description copied from class:ArrowBufMake a nio byte buffer from this ArrowBuf.
-
memoryAddress
public long memoryAddress()
- Overrides:
memoryAddressin classArrowBuf
-
getLong
public long getLong(long index)
Description copied from class:ArrowBufGet long value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setLong
public void setLong(long index, long value)Description copied from class:ArrowBufSet long value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
getFloat
public float getFloat(long index)
Description copied from class:ArrowBufGet float value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setFloat
public void setFloat(long index, float value)Description copied from class:ArrowBufSet float value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
getDouble
public double getDouble(long index)
Description copied from class:ArrowBufGet double value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setDouble
public void setDouble(long index, double value)Description copied from class:ArrowBufSet double value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
getChar
public char getChar(long index)
Description copied from class:ArrowBufGet char value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setChar
public void setChar(long index, int value)Description copied from class:ArrowBufSet char value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
getInt
public int getInt(long index)
Description copied from class:ArrowBufGet int value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setInt
public void setInt(long index, int value)Description copied from class:ArrowBufSet int value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
getShort
public short getShort(long index)
Description copied from class:ArrowBufGet short value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setShort
public void setShort(long index, int value)Description copied from class:ArrowBufSet short value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setShort
public void setShort(long index, short value)Description copied from class:ArrowBufSet short value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setByte
public void setByte(long index, int value)Description copied from class:ArrowBufSet byte value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
setByte
public void setByte(long index, byte value)Description copied from class:ArrowBufSet byte value at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
getByte
public byte getByte(long index)
Description copied from class:ArrowBufGet byte value stored at a particular index in the underlying memory chunk this ArrowBuf has access to.
-
readByte
public byte readByte()
Description copied from class:ArrowBufRead the byte at readerIndex.
-
readBytes
public void readBytes(byte[] dst)
Description copied from class:ArrowBufRead dst.length bytes at readerIndex into dst byte array.
-
writeByte
public void writeByte(byte value)
Description copied from class:ArrowBufSet the provided byte value at the writerIndex.
-
writeByte
public void writeByte(int value)
Description copied from class:ArrowBufSet the lower order byte for the provided value at the writerIndex.
-
writeBytes
public void writeBytes(byte[] src)
Description copied from class:ArrowBufWrite the bytes from given byte array into this ArrowBuf starting at writerIndex.- Overrides:
writeBytesin classArrowBuf- Parameters:
src- src byte array
-
writeBytes
public void writeBytes(byte[] src, int srcIndex, int length)Description copied from class:ArrowBufWrite the bytes from given byte array starting at srcIndex into this ArrowBuf starting at writerIndex.- Overrides:
writeBytesin classArrowBuf- Parameters:
src- src byte arraysrcIndex- index in the byte array where the copy will being fromlength- length of data to copy
-
writeShort
public void writeShort(int value)
Description copied from class:ArrowBufSet the provided int value as short at the writerIndex.- Overrides:
writeShortin classArrowBuf- Parameters:
value- value to set
-
writeInt
public void writeInt(int value)
Description copied from class:ArrowBufSet the provided int value at the writerIndex.
-
writeLong
public void writeLong(long value)
Description copied from class:ArrowBufSet the provided long value at the writerIndex.
-
writeFloat
public void writeFloat(float value)
Description copied from class:ArrowBufSet the provided float value at the writerIndex.- Overrides:
writeFloatin classArrowBuf- Parameters:
value- value to set
-
writeDouble
public void writeDouble(double value)
Description copied from class:ArrowBufSet the provided double value at the writerIndex.- Overrides:
writeDoublein classArrowBuf- Parameters:
value- value to set
-
getBytes
public void getBytes(long index, byte[] dst)Description copied from class:ArrowBufCopy data from this ArrowBuf at a given index in into destination byte array.
-
getBytes
public void getBytes(long index, byte[] dst, int dstIndex, int length)Description copied from class:ArrowBufCopy data from this ArrowBuf at a given index into destination byte array.
-
setBytes
public void setBytes(long index, byte[] src)Description copied from class:ArrowBufCopy data from a given byte array into this ArrowBuf starting at a given index.
-
setBytes
public void setBytes(long index, byte[] src, int srcIndex, long length)Description copied from class:ArrowBufCopy data from a given byte array starting at the given source index into this ArrowBuf at a given index.
-
getBytes
public void getBytes(long index, ByteBuffer dst)Description copied from class:ArrowBufCopy data from this ArrowBuf at a given index into the destination ByteBuffer.
-
setBytes
public void setBytes(long index, ByteBuffer src)Description copied from class:ArrowBufCopy data into this ArrowBuf at a given index onwards from a source ByteBuffer.
-
setBytes
public void setBytes(long index, ByteBuffer src, int srcIndex, int length)Description copied from class:ArrowBufCopy data into this ArrowBuf at a given index onwards from a source ByteBuffer starting at a given srcIndex for a certain length.- Overrides:
setBytesin classArrowBuf- Parameters:
index- index (0 based relative to the portion of memory this ArrowBuf has access to)src- src ByteBuffer where the data will be copied fromsrcIndex- starting index in the src ByteBuffer where the data copy will start fromlength- length of data to copy from src ByteBuffer
-
getBytes
public void getBytes(long index, ArrowBuf dst, long dstIndex, int length)Description copied from class:ArrowBufCopy a given length of data from this ArrowBuf starting at a given index into a dst ArrowBuf at dstIndex.- Overrides:
getBytesin classArrowBuf- Parameters:
index- index (0 based relative to the portion of memory this ArrowBuf has access to)dst- dst ArrowBuf where the data will be copied intodstIndex- index (0 based relative to the portion of memory dst ArrowBuf has access to)length- length of data to copy
-
setBytes
public void setBytes(long index, ArrowBuf src, long srcIndex, long length)Description copied from class:ArrowBufCopy data from src ArrowBuf starting at index srcIndex into this ArrowBuf at given index.- Overrides:
setBytesin classArrowBuf- Parameters:
index- index index (0 based relative to the portion of memory this ArrowBuf has access to)src- src ArrowBuf where the data will be copied fromsrcIndex- starting index in the src ArrowBuf where the copy will begin fromlength- length of data to copy from src ArrowBuf
-
setBytes
public void setBytes(long index, ArrowBuf src)Description copied from class:ArrowBufCopy readableBytes() number of bytes from src ArrowBuf starting from its readerIndex into this ArrowBuf starting at the given index.
-
setBytes
public int setBytes(long index, InputStream in, int length) throws IOExceptionDescription copied from class:ArrowBufCopy a certain length of bytes from given InputStream into this ArrowBuf at the provided index.- Overrides:
setBytesin classArrowBuf- Parameters:
index- index index (0 based relative to the portion of memory this ArrowBuf has access to)in- src stream to copy fromlength- length of data to copy- Returns:
- number of bytes copied from stream into ArrowBuf
- Throws:
IOException- on failing to read from stream
-
getBytes
public void getBytes(long index, OutputStream out, int length) throws IOExceptionDescription copied from class:ArrowBufCopy a certain length of bytes from this ArrowBuf at a given index into the given OutputStream.- Overrides:
getBytesin classArrowBuf- Parameters:
index- index index (0 based relative to the portion of memory this ArrowBuf has access to)out- dst stream to copy data intolength- length of data to copy- Throws:
IOException- on failing to write to stream
-
close
public void close()
- Specified by:
closein interfaceAutoCloseable- Overrides:
closein classArrowBuf
-
getPossibleMemoryConsumed
public long getPossibleMemoryConsumed()
Description copied from class:ArrowBufReturns the possible memory consumed by this ArrowBuf in the worse case scenario. (not shared, connected to larger underlying buffer of allocated memory)- Overrides:
getPossibleMemoryConsumedin classArrowBuf- Returns:
- Size in bytes.
-
getActualMemoryConsumed
public long getActualMemoryConsumed()
Description copied from class:ArrowBufReturn that is Accounted for by this buffer (and its potentially shared siblings within the context of the associated allocator).- Overrides:
getActualMemoryConsumedin classArrowBuf- Returns:
- Size in bytes.
-
toHexString
public String toHexString(long start, int length)
Description copied from class:ArrowBufReturn the buffer's byte contents in the form of a hex dump.- Overrides:
toHexStringin classArrowBuf- Parameters:
start- the starting byte indexlength- how many bytes to log- Returns:
- A hex dump in a String.
-
print
public void print(StringBuilder sb, int indent)
Description copied from class:ArrowBufPrint detailed information of this buffer intosb.Most information will only be present if BaseAllocator.DEBUG is true.
-
readerIndex
public long readerIndex()
Description copied from class:ArrowBufGet the index at which the next byte will be read from.- Overrides:
readerIndexin classArrowBuf- Returns:
- reader index
-
writerIndex
public long writerIndex()
Description copied from class:ArrowBufGet the index at which next byte will be written to.- Overrides:
writerIndexin classArrowBuf- Returns:
- writer index
-
readerIndex
public ArrowBuf readerIndex(long readerIndex)
Description copied from class:ArrowBufSet the reader index for this ArrowBuf.- Overrides:
readerIndexin classArrowBuf- Parameters:
readerIndex- new reader index- Returns:
- this ArrowBuf
-
writerIndex
public ArrowBuf writerIndex(long writerIndex)
Description copied from class:ArrowBufSet the writer index for this ArrowBuf.- Overrides:
writerIndexin classArrowBuf- Parameters:
writerIndex- new writer index- Returns:
- this ArrowBuf
-
setZero
public ArrowBuf setZero(long index, long length)
Description copied from class:ArrowBufZero-out the bytes in this ArrowBuf starting at the given index for the given length.
-
setOne
@Deprecated public ArrowBuf setOne(int index, int length)
Deprecated.Description copied from class:ArrowBufSets all bits to one in the specified range.
-
setOne
public ArrowBuf setOne(long index, long length)
Description copied from class:ArrowBufSets all bits to one in the specified range.
-
reallocIfNeeded
public ArrowBuf reallocIfNeeded(long size)
Description copied from class:ArrowBufReturnsthisif size is less thanArrowBuf.capacity(), otherwise delegates toBufferManager.replace(ArrowBuf, long)to get a new buffer.- Overrides:
reallocIfNeededin classArrowBuf
-
getOffset
public int getOffset()
Returns the offset within the underlying ByteBuffer where this buffer's data starts. This is used for sliced buffers that share the same underlying ByteBuffer.- Returns:
- the offset in bytes
-
-