com.aoindustries.util.persistent

Class TwoCopyBarrierBuffer

java.lang.Object

com.aoindustries.util.persistent.AbstractPersistentBuffer

com.aoindustries.util.persistent.TwoCopyBarrierBuffer

All Implemented Interfaces:

PersistentBuffer

public class TwoCopyBarrierBuffer
extends AbstractPersistentBuffer

Java does not support write barriers without a complete force call, this class works-around this issue by maintaining two copies of the file and updating the older copy to be the newer copy occasionally on barrier(false) and immediately on barrier(true) (if protectionLevel is high enough).

All instances also share a Timer to perform automatic background flushing of caches. Automatic flushing is single-threaded to favor low load averages over timely flushes.

This class also acts as a write cache that may batch or delay writes for potentially long periods of time. This is useful for media such as flash memory where write throughput is quite limited and the number of writes on the media is also limited. This also turns many random writes into fewer, sequential writes. This may help spinning platter-based media.

Since this class is intended for flash-based media, it will not rewrite the same data to a section of a file. Even if this means that it has to read the section of the file, compare the values, and then write only when changed. This is an attempt to help the wear leveling technology built into the media by dispatching a minimum number of writes.

An additional benefit may be that reads from cached data are performed directly from the write cache, although this is not the purpose of this buffer. Writes that would not change anything, however, are not cached and would not help as a read cache.

Two copies of the file are maintained. The most recent version of the file will normally be named with the regular name, but other versions will exist with .old or .new appended to the filename. The exact order of update is:

1. Rename filename.old to filename.new
2. Write new version of all data to filename.new
3. Rename filename to filename.old
4. Rename filename.new to filename

The filename states, in order:

          Complete      Complete Old  Partial
 Normal:  filename      filename.old
          filename                    filename.new
          filename.new  filename.old
 Normal:  filename      filename.old

 

This implementation assumes an atomic file rename for correct recovery.

To reduce the chance of data loss, this registers a JVM shutdown hook to flush all caches on JVM shutdown. If it takes a long time to flush the data this can cause significant delays when stopping a JVM.

TODO: Should we run on top of RandomAccessBuffer/LargeMappedPersistentBuffer/MappedPersistentBuffer for memory-mapped read performance?

Author:

AO Industries, Inc.

Field Summary

Fields inherited from class com.aoindustries.util.persistent.AbstractPersistentBuffer

protectionLevel

Constructor Summary

Constructors

Constructor and Description
TwoCopyBarrierBuffer() Creates a read-write buffer backed by temporary files.
TwoCopyBarrierBuffer(File file) Creates a read-write buffer with BARRIER protection level.
TwoCopyBarrierBuffer(File file, ProtectionLevel protectionLevel) Creates a buffer.
TwoCopyBarrierBuffer(File file, ProtectionLevel protectionLevel, int sectorSize, long asynchronousCommitDelay, long synchronousCommitDelay) Creates a buffer.
TwoCopyBarrierBuffer(String name) Creates a read-write buffer with BARRIER protection level.
TwoCopyBarrierBuffer(String name, ProtectionLevel protectionLevel) Creates a buffer.

Method Summary

Modifier and Type	Method and Description
void	barrier(boolean force) Ensures that all writes before this barrier occur before all writes after this barrier.
long	capacity() Gets the capacity of this buffer.
void	close() Closes this buffer.
void	ensureZeros(long position, long len) Ensures that all values from the position for the provided length are zeros.
protected void	finalize()
byte	get(long position) Implemented as call to get(long,byte[],int,int).
int	getSome(long position, byte[] buff, int off, int len) Reads to the provided byte[], may read fewer than len bytes, but will always read at least one byte.
boolean	isClosed() Checks if this buffer is closed.
void	put(long position, byte value) Implemented as call to put(long,byte[],int,int).
void	put(long position, byte[] buff, int off, int len) Writes the bytes to the provided position.
void	setCapacity(long newCapacity) Sets the capacity of this buffer.

Methods inherited from class com.aoindustries.util.persistent.AbstractPersistentBuffer

get, getBoolean, getInputStream, getInt, getLong, getOutputStream, getProtectionLevel, putInt, putLong

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TwoCopyBarrierBuffer

public TwoCopyBarrierBuffer()
                     throws IOException

Creates a read-write buffer backed by temporary files. The protection level is set to NONE. The temporary file will be deleted when this buffer is closed or on JVM shutdown. Uses default sectorSize of 4096, asynchronous commit delay of 5 seconds, and synchronous commit delay of 60 seconds. A shutdown hook is not registered.

Throws:

IOException

TwoCopyBarrierBuffer

public TwoCopyBarrierBuffer(String name)
                     throws IOException

Creates a read-write buffer with BARRIER protection level. Uses default sectorSize of 4096, asynchronous commit delay of 5 seconds, and synchronous commit delay of 60 seconds.

Throws:

IOException

TwoCopyBarrierBuffer

public TwoCopyBarrierBuffer(String name,
                            ProtectionLevel protectionLevel)
                     throws IOException

Creates a buffer. Uses default sectorSize of 4096, asynchronous commit delay of 5 seconds, and synchronous commit delay of 60 seconds.

Throws:

IOException

TwoCopyBarrierBuffer

public TwoCopyBarrierBuffer(File file)
                     throws IOException

Creates a read-write buffer with BARRIER protection level. Uses default sectorSize of 4096, asynchronous commit delay of 5 seconds, and synchronous commit delay of 60 seconds.

Throws:

IOException

TwoCopyBarrierBuffer

public TwoCopyBarrierBuffer(File file,
                            ProtectionLevel protectionLevel)
                     throws IOException

Creates a buffer. Uses default sectorSize of 4096, asynchronous commit delay of 5 seconds, and synchronous commit delay of 60 seconds.

Throws:

IOException

TwoCopyBarrierBuffer

public TwoCopyBarrierBuffer(File file,
                            ProtectionLevel protectionLevel,
                            int sectorSize,
                            long asynchronousCommitDelay,
                            long synchronousCommitDelay)
                     throws IOException

Creates a buffer. Synchronizes the two copies of the file. Populates the oldWriteCache with any data the doesn't match the newer version of the file. This means that both files are read completely at start-up in order to provide the most efficient synchronization later.

Parameters:

file - The base filename, will be appended with ".old" and ".new" while committing changes.

protectionLevel - The protection level for this buffer.

sectorSize - The size of the sectors cached and written. For best results this should match the underlying filesystem block size. Must be a power of two >= 1.

asynchronousCommitDelay - The number of milliseconds before a background thread syncs uncommitted data to the underlying storage. A value of Long.MAX_VALUE will avoid any overhead of background thread management.

synchronousCommitDelay - The number of milliseconds before a the calling thread syncs uncommitted data to the underlying storage.

Throws:

IOException

Method Detail

finalize

protected void finalize()
                 throws Throwable

Overrides:

finalize in class Object

Throws:

Throwable

isClosed

public boolean isClosed()

Description copied from interface: PersistentBuffer

Checks if this buffer is closed.

close

public void close()
           throws IOException

Description copied from interface: PersistentBuffer

Closes this buffer. It is OK to close an already closed buffer.

Throws:

IOException

capacity

public long capacity()
              throws IOException

Description copied from interface: PersistentBuffer

Gets the capacity of this buffer.

Throws:

IOException

setCapacity

public void setCapacity(long newCapacity)
                 throws IOException

Description copied from interface: PersistentBuffer

Sets the capacity of this buffer. If the buffer is increased in size, the new space will be zero-filled. Setting the capacity may impose an automatic barrier(true), depending on implementation. This should be considered an expensive operation.

Throws:

IOException

get

public byte get(long position)
         throws IOException

Description copied from class: AbstractPersistentBuffer

Implemented as call to get(long,byte[],int,int). For performance reasons, it is strongly recommended to provide a more efficient implementation of this method.

Specified by:

get in interface PersistentBuffer

Overrides:

get in class AbstractPersistentBuffer

Throws:

IOException

See Also:

AbstractPersistentBuffer.get(long, byte[], int, int)

getSome

public int getSome(long position,
                   byte[] buff,
                   int off,
                   int len)
            throws IOException

Description copied from interface: PersistentBuffer

Reads to the provided byte[], may read fewer than len bytes, but will always read at least one byte. Blocks if no data is available.

Throws:

IOException

put

public void put(long position,
                byte value)
         throws IOException

Description copied from class: AbstractPersistentBuffer

Implemented as call to put(long,byte[],int,int). For performance reasons, it is strongly recommended to provide a more efficient implementation of this method.

Specified by:

put in interface PersistentBuffer

Overrides:

put in class AbstractPersistentBuffer

Throws:

IOException

See Also:

PersistentBuffer.put(long, byte[], int, int)

ensureZeros

public void ensureZeros(long position,
                        long len)
                 throws IOException

Description copied from interface: PersistentBuffer

Ensures that all values from the position for the provided length are zeros. This may or may not modify the buffer in the process. The values will all be zero upon return. Some implementations may choose to overwrite zeros and return modified, others may choose to detect zeros and avoid modifications. Thus it is possible for existing zeros to still result in a modification.

Throws:

IOException

put

public void put(long position,
                byte[] buff,
                int off,
                int len)
         throws IOException

Description copied from interface: PersistentBuffer

Writes the bytes to the provided position. The buffer will not be expanded automatically.

Throws:

IOException

barrier

public void barrier(boolean force)
             throws IOException

Description copied from interface: PersistentBuffer

Ensures that all writes before this barrier occur before all writes after this barrier. If force is true, will also commit to physical media synchronously before returning. This request may be ignored or force downgraded to barrier-only depending on the current protection level.

Throws:

IOException

See Also:

PersistentBuffer.getProtectionLevel()