journal.io.api
Class Journal

java.lang.Object
  journal.io.api.Journal

public class Journal
extends Object

Journal implementation based on append-only rotating logs and checksummed records, with concurrent writes and reads, dynamic batching and logs compaction.

Journal records can be written, read and deleted by providing a Location object.

The whole Journal can be replayed forward or backward by simply obtaining a redo or undo iterable and going through it in a for-each block.

Author:

Hiram Chirino, Sergio Bossa

Nested Class Summary

static class	Journal.ReadType
static class	Journal.WriteType

Constructor Summary

Journal()

Method Summary

void	close() Close the journal.
void	compact() Compact the journal, reducing size of logs containing deleted entries and completely removing empty (with only deleted entries) logs.
void	delete(Location location) Delete the record at the given Location. Deletes cause first a batch sync and always are logical: records will be actually deleted at log cleanup time.
File	getDirectory() Get the journal directory containing log files.
File	getDirectoryArchive() Get the optional archive directory used to archive cleaned up log files.
long	getDisposeInterval() Get the milliseconds interval for resources disposal.
String	getFilePrefix() Get the prefix for log files.
Set<File>	getFiles() Get the files part of this journal.
String	getFileSuffix() Get the suffix for log files.
int	getMaxFileLength() Get the max length of each log file.
int	getMaxWriteBatchSize() Get the max size in bytes of the write batch: must always be equal or less than the max file length.
ReplicationTarget	getReplicationTarget() Get the ReplicationTarget to replicate batch writes to.
boolean	isArchiveFiles() Return true if cleaned up log files should be archived, false otherwise.
boolean	isChecksum() Return true if records checksum is enabled, false otherwise.
boolean	isPhysicalSync() Return true if every disk write is followed by a physical disk sync, synchronizing file descriptor properties and flushing hardware buffers, false otherwise.
void	open() Open the journal, eventually recovering it if already existent.
byte[]	read(Location location, Journal.ReadType read) Read the record stored at the given Location, either by syncing with the disk state (if ReadType.SYNC) or by taking advantage of speculative disk reads (if ReadType.ASYNC); the latter is faster, while the former is slower but will suddenly detect deleted records.
Iterable<Location>	redo() Return an iterable to replay the journal by going through all records locations.
Iterable<Location>	redo(Location start) Return an iterable to replay the journal by going through all records locations starting from the given one.
void	setArchiveFiles(boolean archiveFiles) Set true if cleaned up log files should be archived, false otherwise.
void	setChecksum(boolean checksumWrites) Set true if records checksum is enabled, false otherwise.
void	setDirectory(File directory) Set the journal directory containing log files.
void	setDirectoryArchive(File directoryArchive) Set the optional archive directory used to archive cleaned up log files.
void	setDisposeInterval(long disposeInterval) Set the milliseconds interval for resources disposal: i.e., un-accessed files will be closed.
void	setDisposer(ScheduledExecutorService disposer) Set the ScheduledExecutorService to use for internal resources disposing.
void	setFilePrefix(String filePrefix) Set the prefix for log files.
void	setFileSuffix(String fileSuffix) Set the suffix for log files.
void	setMaxFileLength(int maxFileLength) Set the max length of each log file.
void	setMaxWriteBatchSize(int maxWriteBatchSize) Set the max size in bytes of the write batch: must always be equal or less than the max file length.
void	setPhysicalSync(boolean physicalSync) Set true if every disk write must be followed by a physical disk sync, synchronizing file descriptor properties and flushing hardware buffers, false otherwise.
void	setRecoveryErrorHandler(RecoveryErrorHandler recoveryErrorHandler) Set the RecoveryErrorHandler to invoke in case of checksum errors.
void	setReplicationTarget(ReplicationTarget replicationTarget) Set the ReplicationTarget to replicate batch writes to.
void	setWriter(Executor writer) Set the Executor to use for writing new record entries.
void	sync() Sync asynchronously written records on disk.
String	toString()
void	truncate() Truncate the journal, removing all log files.
Iterable<Location>	undo() Return an iterable to replay the journal in reverse, starting with the newest location and ending with the first.
Iterable<Location>	undo(Location end) Return an iterable to replay the journal in reverse, starting with the newest location and ending with the specified end location.
Location	write(byte[] data, Journal.WriteType write) Write the given byte buffer record, either sync (if WriteType.SYNC) or async (if WriteType.ASYNC), and returns the stored Location. A sync write causes all previously batched async writes to be synced too.
Location	write(byte[] data, Journal.WriteType write, WriteCallback callback) Write the given byte buffer record, either sync (if WriteType.SYNC) or async (if WriteType.ASYNC), and returns the stored Location. A sync write causes all previously batched async writes to be synced too. The provided callback will be invoked if sync is completed or if some error occurs during syncing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Journal

public Journal()

Method Detail

open

public void open()
          throws IOException

Open the journal, eventually recovering it if already existent.

Throws:

IOException

close

public void close()
           throws IOException

Close the journal.

Throws:

IOException

compact

public void compact()
             throws ClosedJournalException,
                    IOException

Compact the journal, reducing size of logs containing deleted entries and completely removing empty (with only deleted entries) logs.

Throws:

IOException

ClosedJournalException

sync

public void sync()
          throws ClosedJournalException,
                 IOException

Sync asynchronously written records on disk.

Throws:

IOException

ClosedJournalException

truncate

public void truncate()
              throws OpenJournalException,
                     IOException

Truncate the journal, removing all log files. Please note truncate requires the journal to be closed.

Throws:

IOException

OpenJournalException

read

public byte[] read(Location location,
                   Journal.ReadType read)
            throws ClosedJournalException,
                   CompactedDataFileException,
                   IOException

Read the record stored at the given Location, either by syncing with the disk state (if ReadType.SYNC) or by taking advantage of speculative disk reads (if ReadType.ASYNC); the latter is faster, while the former is slower but will suddenly detect deleted records.

Parameters:

location -

read -

Returns:

Throws:

IOException

ClosedJournalException

CompactedDataFileException

write

public Location write(byte[] data,
                      Journal.WriteType write)
               throws ClosedJournalException,
                      IOException

Write the given byte buffer record, either sync (if WriteType.SYNC) or async (if WriteType.ASYNC), and returns the stored Location.
A sync write causes all previously batched async writes to be synced too.

Parameters:

data -

write -

Returns:

Throws:

IOException

ClosedJournalException

write

public Location write(byte[] data,
                      Journal.WriteType write,
                      WriteCallback callback)
               throws ClosedJournalException,
                      IOException

Write the given byte buffer record, either sync (if WriteType.SYNC) or async (if WriteType.ASYNC), and returns the stored Location.
A sync write causes all previously batched async writes to be synced too.
The provided callback will be invoked if sync is completed or if some error occurs during syncing.

Parameters:

data -

write -

callback -

Returns:

Throws:

IOException

ClosedJournalException

delete

public void delete(Location location)
            throws ClosedJournalException,
                   CompactedDataFileException,
                   IOException

Delete the record at the given Location.
Deletes cause first a batch sync and always are logical: records will be actually deleted at log cleanup time.

Parameters:

location -

Throws:

IOException

ClosedJournalException

CompactedDataFileException

redo

public Iterable<Location> redo()
                        throws ClosedJournalException,
                               CompactedDataFileException,
                               IOException

Return an iterable to replay the journal by going through all records locations.

Returns:

Throws:

IOException

ClosedJournalException

CompactedDataFileException

redo

public Iterable<Location> redo(Location start)
                        throws ClosedJournalException,
                               CompactedDataFileException,
                               IOException

Return an iterable to replay the journal by going through all records locations starting from the given one.

Parameters:

start -

Returns:

Throws:

IOException

CompactedDataFileException

ClosedJournalException

undo

public Iterable<Location> undo()
                        throws ClosedJournalException,
                               CompactedDataFileException,
                               IOException

Return an iterable to replay the journal in reverse, starting with the newest location and ending with the first. The iterable does not include future writes - writes that happen after its creation.

Returns:

Throws:

IOException

CompactedDataFileException

ClosedJournalException

undo

public Iterable<Location> undo(Location end)
                        throws ClosedJournalException,
                               CompactedDataFileException,
                               IOException

Return an iterable to replay the journal in reverse, starting with the newest location and ending with the specified end location. The iterable does not include future writes - writes that happen after its creation.

Parameters:

end -

Returns:

Throws:

IOException

CompactedDataFileException

ClosedJournalException

getFiles

public Set<File> getFiles()

Get the files part of this journal.

Returns:

getMaxFileLength

public int getMaxFileLength()

Get the max length of each log file.

Returns:

setMaxFileLength

public void setMaxFileLength(int maxFileLength)

Set the max length of each log file.

getDirectory

public File getDirectory()

Get the journal directory containing log files.

Returns:

setDirectory

public void setDirectory(File directory)

Set the journal directory containing log files.

getFilePrefix

public String getFilePrefix()

Get the prefix for log files.

Returns:

setFilePrefix

public void setFilePrefix(String filePrefix)

Set the prefix for log files.

Parameters:

filePrefix -

getDirectoryArchive

public File getDirectoryArchive()

Get the optional archive directory used to archive cleaned up log files.

Returns:

setDirectoryArchive

public void setDirectoryArchive(File directoryArchive)

Set the optional archive directory used to archive cleaned up log files.

Parameters:

directoryArchive -

isArchiveFiles

public boolean isArchiveFiles()

Return true if cleaned up log files should be archived, false otherwise.

Returns:

setArchiveFiles

public void setArchiveFiles(boolean archiveFiles)

Set true if cleaned up log files should be archived, false otherwise.

Parameters:

archiveFiles -

setReplicationTarget

public void setReplicationTarget(ReplicationTarget replicationTarget)

Set the ReplicationTarget to replicate batch writes to.

Parameters:

replicationTarget -

getReplicationTarget

public ReplicationTarget getReplicationTarget()

Get the ReplicationTarget to replicate batch writes to.

Returns:

getFileSuffix

public String getFileSuffix()

Get the suffix for log files.

Returns:

setFileSuffix

public void setFileSuffix(String fileSuffix)

Set the suffix for log files.

Parameters:

fileSuffix -

isChecksum

public boolean isChecksum()

Return true if records checksum is enabled, false otherwise.

Returns:

setChecksum

public void setChecksum(boolean checksumWrites)

Set true if records checksum is enabled, false otherwise.

Parameters:

checksumWrites -

isPhysicalSync

public boolean isPhysicalSync()

Return true if every disk write is followed by a physical disk sync, synchronizing file descriptor properties and flushing hardware buffers, false otherwise.

Returns:

setPhysicalSync

public void setPhysicalSync(boolean physicalSync)

Set true if every disk write must be followed by a physical disk sync, synchronizing file descriptor properties and flushing hardware buffers, false otherwise.

getMaxWriteBatchSize

public int getMaxWriteBatchSize()

Get the max size in bytes of the write batch: must always be equal or less than the max file length.

Returns:

setMaxWriteBatchSize

public void setMaxWriteBatchSize(int maxWriteBatchSize)

Set the max size in bytes of the write batch: must always be equal or less than the max file length.

Parameters:

maxWriteBatchSize -

setDisposeInterval

public void setDisposeInterval(long disposeInterval)

Set the milliseconds interval for resources disposal: i.e., un-accessed files will be closed.

Parameters:

disposeInterval -

getDisposeInterval

public long getDisposeInterval()

Get the milliseconds interval for resources disposal.

Returns:

setWriter

public void setWriter(Executor writer)

Set the Executor to use for writing new record entries. Important note: the provided Executor must be manually closed.

Parameters:

writer -

setDisposer

public void setDisposer(ScheduledExecutorService disposer)

Set the ScheduledExecutorService to use for internal resources disposing. Important note: the provided ScheduledExecutorService must be manually closed.

Parameters:

writer -

setRecoveryErrorHandler

public void setRecoveryErrorHandler(RecoveryErrorHandler recoveryErrorHandler)

Set the RecoveryErrorHandler to invoke in case of checksum errors.

Parameters:

recoveryErrorHandler -

toString

public String toString()

Overrides:

toString in class Object