net.openhft.chronicle.bytes

Class MappedBytes

java.lang.Object

net.openhft.chronicle.core.io.AbstractReferenceCounted

net.openhft.chronicle.bytes.AbstractBytes<Void>

net.openhft.chronicle.bytes.MappedBytes

All Implemented Interfaces:

Closeable, Appendable, AutoCloseable, CharSequence, Bytes<Void>, BytesIn<Void>, BytesOut<Void>, BytesPrepender<Bytes<Void>>, BytesStore<Bytes<Void>,Void>, ByteStringAppender<Bytes<Void>>, ByteStringParser<Bytes<Void>>, HexDumpBytesDescription<BytesOut<Void>>, net.openhft.chronicle.bytes.internal.HasUncheckedRandomDataInput, RandomCommon, RandomDataInput, RandomDataOutput<Bytes<Void>>, DecimalAppender, StreamingCommon<Bytes<Void>>, StreamingDataInput<Bytes<Void>>, StreamingDataOutput<Bytes<Void>>, net.openhft.chronicle.core.io.ManagedCloseable, net.openhft.chronicle.core.io.Monitorable, net.openhft.chronicle.core.io.QueryCloseable, net.openhft.chronicle.core.io.ReferenceCounted, net.openhft.chronicle.core.io.ReferenceCountedTracer, net.openhft.chronicle.core.io.ReferenceOwner, net.openhft.chronicle.core.io.SingleThreadedChecked, net.openhft.chronicle.core.io.Syncable

public abstract class MappedBytes
extends AbstractBytes<Void>
implements net.openhft.chronicle.core.io.Closeable, net.openhft.chronicle.core.io.ManagedCloseable, net.openhft.chronicle.core.io.Syncable

A specialised Bytes implementation backed by a memory-mapped file.

The underlying file is accessed in chunks (64 MiB by default) so very large files can be treated as if they were in memory. Only the most recently accessed chunk is retained; previous chunks are released. Call ReferenceCounted.releaseLast() when finished with a MappedBytes instance to free system resources.

Instances are single threaded and should be reserved before use.

Field Summary

Fields

Modifier and Type	Field and Description
protected static boolean	TRACE System property flag trace.mapped.bytes for enabling trace logging of MappedBytes operations.

Fields inherited from class net.openhft.chronicle.bytes.AbstractBytes

bytesStore, isPresent, readPosition, writeLimit

Fields inherited from class net.openhft.chronicle.core.io.AbstractReferenceCounted

referenceCounted, WARN_COUNT, WARN_NS

Fields inherited from interface net.openhft.chronicle.bytes.Bytes

DEFAULT_BYTE_BUFFER_CAPACITY, MAX_CAPACITY, MAX_HEAP_CAPACITY

Fields inherited from interface net.openhft.chronicle.bytes.StreamingDataOutput

JAVA9_STRING_CODER_LATIN, JAVA9_STRING_CODER_UTF16

Fields inherited from interface net.openhft.chronicle.core.io.SingleThreadedChecked

DISABLE_SINGLE_THREADED_CHECK

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MappedBytes() Constructs an instance for use by subclasses.
protected	MappedBytes(String name) Constructs an instance for use by subclasses with a descriptive name.

Method Summary

Modifier and Type	Method and Description
@NotNull Bytes<Void>	bytesForRead() Returns a view for reading from the current mapping.
@NotNull Bytes<Void>	bytesForWrite() Returns a view for writing to the current mapping.
abstract void	chunkCount(long[] chunkCount) Populates the supplied array with the number of chunks held by the underlying MappedFile.
abstract boolean	isBackingFileReadOnly()
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize) Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize, long overlapSize) Creates a chunked mapping with a specified overlap between chunks.
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize, long overlapSize, boolean readOnly) Convenience overload using the default page size.
static @NotNull MappedBytes	mappedBytes(@NotNull File file, long chunkSize, long overlapSize, int pageSize, boolean readOnly) Creates a chunked mapping with explicit overlap, page size and read-only option.
static @NotNull MappedBytes	mappedBytes(@NotNull MappedFile rw) Creates a MappedBytes view for a pre-existing MappedFile.
static @NotNull MappedBytes	mappedBytes(@NotNull String filename, long chunkSize) Creates a chunked mapping for the given file.
abstract MappedFile	mappedFile()
static @NotNull MappedBytes	readOnly(@NotNull File file) Maps the given file in read-only mode.
boolean	sharedMemory() Memory-mapped files are a form of shared memory, hence this returns true.
static @NotNull MappedBytes	singleMappedBytes(@NotNull File file, long capacity) As singleMappedBytes(String, long) but accepting a File instance.
static @NotNull MappedBytes	singleMappedBytes(@NotNull File file, long capacity, boolean readOnly) Creates a single mapping for the whole file with explicit read-only option.
static @NotNull MappedBytes	singleMappedBytes(@NotNull String filename, long capacity) Creates a MappedBytes covering the entire file as a single mapping.
void	sync() Forces any changes made to this mapping to be written to the storage device.

Methods inherited from class net.openhft.chronicle.bytes.AbstractBytes

acquireUncheckedInput, addressForRead, addressForWrite, addressForWritePosition, append, append, append, append, append, append, appendAndReturnLength, byteCheckSum, byteCheckSum, byteCheckSum, bytesStore, bytesStore, canReadDirect, canWriteDirect, capacity, clear, clearAndPad, compact, compareAndSwapInt, compareAndSwapLong, copyTo, decimaliser, decimaliser, equals, fpAppend0, fpAppend0, hashCode, isDirectMemory, isElastic, isImmutableEmptyByteStore, lastDecimalPlaces, lastDecimalPlaces, lastNumberHadDigits, lastNumberHadDigits, length, lenient, lenient, move, nativeRead, nativeWrite, peekUnsignedByte, peekUnsignedByte, performRelease, prewrite, prewrite, prewriteByte, prewriteInt, prewriteLong, prewriteOffsetPositionMoved, prewriteShort, read, read, readByte, readByte, readCheckOffset, readDouble, readDouble, readFloat, readFloat, readInt, readInt, readLimit, readLimit, readLong, readLong, readOffsetPositionMoved, readPosition, readPosition, readPositionForHeader, readShort, readShort, readSkip, readUnsignedByte, readUnsignedByte, readVolatileByte, readVolatileInt, readVolatileInt, readVolatileLong, readVolatileLong, readVolatileShort, realCapacity, safeCopySize, start, startsWith, testAndSetInt, toString, uncheckedReadSkipBackOne, uncheckedReadSkipOne, uncheckedReadUnsignedByte, uncheckedWritePosition, underlyingObject, unmonitor, write, write, write, write, write, write8bit, write8bit, write8bit, write8bit, writeBoolean, writeByte, writeByte, writeCheckOffset, writeDouble, writeDouble, writeDoubleAndInt, writeFloat, writeFloat, writeInt, writeInt, writeIntAdv, writeLimit, writeLimit, writeLong, writeLong, writeLongAdv, writeOffsetPositionMoved, writeOffsetPositionMoved, writeOrderedInt, writeOrderedInt, writeOrderedLong, writeOrderedLong, writePosition, writePosition, writeShort, writeShort, writeSkip, writeSome, writeVolatileByte, writeVolatileInt, writeVolatileLong, writeVolatileShort

Methods inherited from class net.openhft.chronicle.core.io.AbstractReferenceCounted

addReferenceChangeListener, assertReferencesReleased, backgroundPerformRelease, canReleaseInBackground, createdHere, disableReferenceTracing, enableReferenceTracing, refCount, referenceCountedUnmonitored, referenceId, release, releaseLast, removeReferenceChangeListener, reserve, reserveTransfer, singleThreadedCheckDisabled, singleThreadedCheckReset, threadSafetyCheck, throwExceptionIfNotReleased, throwExceptionIfReleased, tryReserve, unmonitor, warnAndReleaseIfNotReleased

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface net.openhft.chronicle.core.io.ManagedCloseable

createdHere, throwExceptionIfClosed, warnAndCloseIfNotClosed

Methods inherited from interface net.openhft.chronicle.core.io.Closeable

close, closeQuietly, closeQuietly

Methods inherited from interface net.openhft.chronicle.core.io.QueryCloseable

isClosed, isClosing

Methods inherited from interface net.openhft.chronicle.core.io.Syncable

syncIfAvailable

Methods inherited from interface net.openhft.chronicle.bytes.Bytes

allocateDirect, allocateDirect, allocateElastic, allocateElastic, allocateElasticDirect, allocateElasticDirect, allocateElasticOnHeap, allocateElasticOnHeap, copy, copyTo, copyTo, directFrom, elasticByteBuffer, elasticByteBuffer, elasticByteBuffer, elasticHeapByteBuffer, elasticHeapByteBuffer, empty, ensureCapacity, forFieldGroup, from, from, fromDirect, fromHexString, indexOf, indexOf, isClear, isEqual, readMarshallableLength16, readWithLength, readWrite, safeLimit, toHexString, toHexString, toHexString, toString, toString, toString, unchecked, unchecked, unwrite, valueOf, wrapForRead, wrapForRead, wrapForWrite, wrapForWrite, write, writeMarshallableLength16

Methods inherited from interface net.openhft.chronicle.bytes.BytesStore

addAndGetDouble, addAndGetDoubleNotAtomic, addAndGetFloat, addAndGetFloatNotAtomic, addAndGetInt, addAndGetIntNotAtomic, addAndGetLong, addAndGetShortNotAtomic, addAndGetUnsignedByteNotAtomic, charAt, cipher, cipher, compareAndSwapDouble, compareAndSwapFloat, contentEquals, elasticByteBuffer, endsWith, equalBytes, follow, forFields, from, hash, inside, inside, isEmpty, isEqual, lazyNativeBytesStoreWithFixedCapacity, nativePointer, nativeStore, nativeStoreFrom, nativeStoreWithFixedCapacity, startsWith, subSequence, to8bitString, toDebugString, toDebugString, toUtf8String, wrap, wrap, wrap, writeMaxInt, writeMaxLong, zeroOut

Methods inherited from interface net.openhft.chronicle.bytes.RandomDataOutput

append, append, write, write, writeBoolean, writeByte, writeInt24, writeOrderedDouble, writeOrderedFloat, writeUnsignedByte, writeUnsignedInt, writeUnsignedShort, writeUtf8, writeUtf8Limited, writeVolatileDouble, writeVolatileFloat

Methods inherited from interface java.lang.CharSequence

chars, codePoints

Methods inherited from interface net.openhft.chronicle.bytes.BytesIn

bytesMethodReader, bytesMethodReaderBuilder, readObject

Methods inherited from interface net.openhft.chronicle.bytes.RandomDataInput

canReadDirect, compareUtf8, copyTo, fastHash, findByte, parseLong, peekVolatileInt, printable, readBoolean, readIncompleteLong, readUnsignedInt, readUnsignedInt24, readUnsignedShort, readUtf8, readUtf8Limited, readUtf8Limited, readVolatileDouble, readVolatileFloat, subBytes, toByteArray, toTemporaryDirectByteBuffer

Methods inherited from interface net.openhft.chronicle.bytes.ByteStringParser

parse8bit, parse8bit, parse8bit, parse8bit, parse8bit, parseBigDecimal, parseBoolean, parseBoolean, parseDouble, parseFlexibleLong, parseFloat, parseInt, parseLong, parseLongDecimal, parseUtf8, parseUtf8, parseUtf8, reader, skipTo

Methods inherited from interface net.openhft.chronicle.bytes.StreamingDataInput

inputStream, parseHexLong, parseUtf8, parseUtf8, rawReadByte, rawReadInt, rawReadLong, read, read, read, read, read, read8bit, read8bit, read8bit, readBigDecimal, readBigInteger, readBoolean, readChar, readEnum, readHistogram, readIncompleteLong, readInt24, readLimitToCapacity, readPositionRemaining, readPositionUnlimited, readStopBit, readStopBitChar, readStopBitDecimal, readStopBitDouble, readUnsignedInt, readUnsignedInt24, readUnsignedShort, readUtf8, readUtf8, readUtf8, readUtf8, readWithLength, readWithLength, readWithLength0, unsafeRead, unsafeReadObject

Methods inherited from interface net.openhft.chronicle.bytes.BytesOut

bytesMethodWriter, writeObject

Methods inherited from interface net.openhft.chronicle.bytes.ByteStringAppender

append, append, append, append, append, append8bit, append8bit, append8bit, append8bit, append8bit, appendBase, appendBase16, appendBase16, appendDateMillis, appendDecimal, appendTimeMillis, writer

Methods inherited from interface net.openhft.chronicle.bytes.StreamingDataOutput

appendUtf8, appendUtf8, appendUtf8, appendUtf8, appendUtf8, appendUtf8, copyFrom, outputStream, rawWriteByte, rawWriteInt, rawWriteLong, unsafeWrite, unsafeWriteObject, write, write, write, write, write8bit, write8bit, write8bit, writeBigDecimal, writeBigInteger, writeChar, writeEnum, writeHistogram, writeInt24, writePositionForHeader, writePositionRemaining, writeStopBit, writeStopBit, writeStopBit, writeStopBitDecimal, writeUnsignedByte, writeUnsignedInt, writeUnsignedInt24, writeUnsignedShort, writeUtf8, writeUtf8, writeWithLength, writeWithLength

Methods inherited from interface net.openhft.chronicle.bytes.BytesPrepender

prepend

Methods inherited from interface net.openhft.chronicle.bytes.HexDumpBytesDescription

adjustHexDumpIndentation, retainedHexDumpDescription, writeHexDumpDescription

Methods inherited from interface net.openhft.chronicle.core.io.SingleThreadedChecked

singleThreadedCheckDisabled, singleThreadedCheckReset

Methods inherited from interface net.openhft.chronicle.core.io.ReferenceCountedTracer

onReleased

Methods inherited from interface net.openhft.chronicle.core.io.Monitorable

unmonitor

Field Detail

TRACE

protected static final boolean TRACE

System property flag trace.mapped.bytes for enabling trace logging of MappedBytes operations.

Constructor Detail

MappedBytes

protected MappedBytes()
               throws net.openhft.chronicle.core.io.ClosedIllegalStateException,
                      net.openhft.chronicle.core.io.ThreadingIllegalStateException

Constructs an instance for use by subclasses. The resulting object is backed by an empty BytesStore until a real mapping is assigned.

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException

net.openhft.chronicle.core.io.ThreadingIllegalStateException

MappedBytes

protected MappedBytes(String name)
               throws net.openhft.chronicle.core.io.ClosedIllegalStateException,
                      net.openhft.chronicle.core.io.ThreadingIllegalStateException

Constructs an instance for use by subclasses with a descriptive name. The instance initially references an empty store.

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException

net.openhft.chronicle.core.io.ThreadingIllegalStateException

Method Detail

singleMappedBytes

@NotNull
public static @NotNull MappedBytes singleMappedBytes(@NotNull
                                                              @NotNull String filename,
                                                              long capacity)
                                                       throws FileNotFoundException,
                                                              IllegalStateException

Creates a MappedBytes covering the entire file as a single mapping.

Parameters:

filename - path of the file to map

capacity - total capacity to map

Returns:

a new MappedBytes

Throws:

FileNotFoundException - if the file cannot be found

net.openhft.chronicle.core.io.ClosedIllegalStateException - if the mapped file is already closed

net.openhft.chronicle.core.io.ThreadingIllegalStateException - if accessed from multiple threads

IllegalStateException

singleMappedBytes

@NotNull
public static @NotNull MappedBytes singleMappedBytes(@NotNull
                                                              @NotNull File file,
                                                              long capacity)
                                                       throws FileNotFoundException

As singleMappedBytes(String, long) but accepting a File instance.

Throws:

FileNotFoundException

singleMappedBytes

@NotNull
public static @NotNull MappedBytes singleMappedBytes(@NotNull
                                                              @NotNull File file,
                                                              long capacity,
                                                              boolean readOnly)
                                                       throws FileNotFoundException

Creates a single mapping for the whole file with explicit read-only option.

Parameters:

file - file to map

capacity - total capacity

readOnly - true for read only

Returns:

a new MappedBytes

Throws:

FileNotFoundException - if the file does not exist

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull String filename,
                                                        long chunkSize)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a chunked mapping for the given file.

Parameters:

filename - path of the file to map

chunkSize - size of each chunk in bytes

Returns:

a new MappedBytes

Throws:

FileNotFoundException - if the file does not exist

net.openhft.chronicle.core.io.ClosedIllegalStateException - if the mapped file is closed

net.openhft.chronicle.core.io.ThreadingIllegalStateException - if accessed from multiple threads

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a MappedBytes instance that wraps a memory-mapped file divided into chunks of a specified size.

Parameters:

file - The name of the file to be memory-mapped.

chunkSize - The size of each chunk in bytes.

Returns:

A new MappedBytes instance.

Throws:

FileNotFoundException - If the file does not exist.

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

net.openhft.chronicle.core.io.ThreadingIllegalStateException - If this resource was accessed by multiple threads in an unsafe way

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize,
                                                        long overlapSize)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a chunked mapping with a specified overlap between chunks.

Parameters:

file - file to map

chunkSize - size of each chunk

overlapSize - overlap between adjacent chunks

Returns:

a new MappedBytes

Throws:

FileNotFoundException - if the file does not exist

net.openhft.chronicle.core.io.ClosedIllegalStateException - if the mapped file is closed

net.openhft.chronicle.core.io.ThreadingIllegalStateException - if accessed from multiple threads

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize,
                                                        long overlapSize,
                                                        int pageSize,
                                                        boolean readOnly)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a chunked mapping with explicit overlap, page size and read-only option.

Parameters:

file - file to map

chunkSize - size of each chunk

overlapSize - overlap between adjacent chunks

pageSize - page size to align mappings

readOnly - true for read only

Returns:

a new MappedBytes

Throws:

FileNotFoundException - if the file does not exist

net.openhft.chronicle.core.io.ClosedIllegalStateException - if the mapped file is closed

net.openhft.chronicle.core.io.ThreadingIllegalStateException - if accessed from multiple threads

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull File file,
                                                        long chunkSize,
                                                        long overlapSize,
                                                        boolean readOnly)
                                                 throws FileNotFoundException,
                                                        net.openhft.chronicle.core.io.ClosedIllegalStateException

Convenience overload using the default page size.

Throws:

FileNotFoundException

net.openhft.chronicle.core.io.ClosedIllegalStateException

See Also:

mappedBytes(File, long, long, int, boolean)

mappedBytes

@NotNull
public static @NotNull MappedBytes mappedBytes(@NotNull
                                                        @NotNull MappedFile rw)
                                                 throws net.openhft.chronicle.core.io.ClosedIllegalStateException

Creates a MappedBytes view for a pre-existing MappedFile.

Parameters:

rw - the mapped file to use

Returns:

the created MappedBytes

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException - if the mapped file has been closed

net.openhft.chronicle.core.io.ThreadingIllegalStateException - if accessed from multiple threads

readOnly

@NotNull
public static @NotNull MappedBytes readOnly(@NotNull
                                                     @NotNull File file)
                                              throws FileNotFoundException,
                                                     net.openhft.chronicle.core.io.ClosedIllegalStateException,
                                                     net.openhft.chronicle.core.io.ThreadingIllegalStateException

Maps the given file in read-only mode.

Parameters:

file - file to map

Returns:

a new read only MappedBytes

Throws:

FileNotFoundException - if the file does not exist

net.openhft.chronicle.core.io.ClosedIllegalStateException - if the mapped file is closed

net.openhft.chronicle.core.io.ThreadingIllegalStateException - if accessed from multiple threads

isBackingFileReadOnly

public abstract boolean isBackingFileReadOnly()

Returns:

true if the underlying file was mapped read only

sharedMemory

public boolean sharedMemory()

Memory-mapped files are a form of shared memory, hence this returns true.

Specified by:

sharedMemory in interface Bytes<Void>

Specified by:

sharedMemory in interface RandomCommon

Returns:

true if this Bytes object uses memory that can be shared across processes, false otherwise.

chunkCount

public abstract void chunkCount(long[] chunkCount)

Populates the supplied array with the number of chunks held by the underlying MappedFile.

mappedFile

public abstract MappedFile mappedFile()

Returns:

the underlying MappedFile

sync

public void sync()

Forces any changes made to this mapping to be written to the storage device.

Specified by:

sync in interface net.openhft.chronicle.core.io.Syncable

bytesForRead

@NotNull
public @NotNull Bytes<Void> bytesForRead()
                                           throws net.openhft.chronicle.core.io.ClosedIllegalStateException

Returns a view for reading from the current mapping.

Specified by:

bytesForRead in interface Bytes<Void>

Specified by:

bytesForRead in interface BytesStore<Bytes<Void>,Void>

Specified by:

bytesForRead in interface RandomCommon

Returns:

a Bytes that wraps this ByteStore

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.

bytesForWrite

@NotNull
public @NotNull Bytes<Void> bytesForWrite()
                                            throws net.openhft.chronicle.core.io.ClosedIllegalStateException

Returns a view for writing to the current mapping.

Specified by:

bytesForWrite in interface Bytes<Void>

Specified by:

bytesForWrite in interface BytesStore<Bytes<Void>,Void>

Specified by:

bytesForWrite in interface RandomCommon

Returns:

a Bytes that wraps this BytesStore

Throws:

net.openhft.chronicle.core.io.ClosedIllegalStateException - If the resource has been released or closed.