org.apache.flink.runtime.state

Class DuplicatingCheckpointOutputStream

java.lang.Object

java.io.OutputStream

org.apache.flink.core.fs.FSDataOutputStream

org.apache.flink.runtime.state.CheckpointStreamFactory.CheckpointStateOutputStream

org.apache.flink.runtime.state.DuplicatingCheckpointOutputStream

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

public class DuplicatingCheckpointOutputStream
extends CheckpointStreamFactory.CheckpointStateOutputStream

A CheckpointStateOutputStream that wraps a primary and a secondary CheckpointStateOutputStream and duplicates all writes into both streams. This stream applies buffering to reduce the amount of dual-method calling. Furthermore, exceptions that happen in interactions with the secondary stream are not exposed, until the user calls closeAndGetSecondaryHandle(). In contrast to that, exceptions from interactions with the primary stream are immediately returned to the user. This class is used to write state for local recovery as a local (secondary) copy of the (primary) state snapshot that is written to a (slower but highly-available) remote filesystem.

Constructor Summary

Constructors

Constructor and Description
DuplicatingCheckpointOutputStream(CheckpointStreamFactory.CheckpointStateOutputStream primaryOutputStream, CheckpointStreamFactory.CheckpointStateOutputStream secondaryOutputStream)
DuplicatingCheckpointOutputStream(CheckpointStreamFactory.CheckpointStateOutputStream primaryOutputStream, CheckpointStreamFactory.CheckpointStateOutputStream secondaryOutputStream, int bufferSize)

Method Summary

Modifier and Type	Method and Description
void	close() This method should close the stream, if has not been closed before.
StreamStateHandle	closeAndGetHandle() Closes the stream and gets a state handle that can create an input stream producing the data written to this stream.
StreamStateHandle	closeAndGetPrimaryHandle() Returns the state handle from the primaryOutputStream.
StreamStateHandle	closeAndGetSecondaryHandle() Returns the state handle from the secondaryOutputStream.
void	flush()
long	getPos()
Exception	getSecondaryStreamException()
void	sync()
void	write(byte[] b)
void	write(byte[] b, int off, int len)
void	write(int b)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DuplicatingCheckpointOutputStream

public DuplicatingCheckpointOutputStream(CheckpointStreamFactory.CheckpointStateOutputStream primaryOutputStream,
                                         CheckpointStreamFactory.CheckpointStateOutputStream secondaryOutputStream)
                                  throws IOException

Throws:

IOException

DuplicatingCheckpointOutputStream

public DuplicatingCheckpointOutputStream(CheckpointStreamFactory.CheckpointStateOutputStream primaryOutputStream,
                                         CheckpointStreamFactory.CheckpointStateOutputStream secondaryOutputStream,
                                         int bufferSize)
                                  throws IOException

Throws:

IOException

Method Detail

write

public void write(int b)
           throws IOException

Specified by:

write in class OutputStream

Throws:

IOException

write

public void write(byte[] b)
           throws IOException

Overrides:

write in class OutputStream

Throws:

IOException

write

public void write(byte[] b,
                  int off,
                  int len)
           throws IOException

Overrides:

write in class OutputStream

Throws:

IOException

getPos

public long getPos()
            throws IOException

Specified by:

getPos in class org.apache.flink.core.fs.FSDataOutputStream

Throws:

IOException

flush

public void flush()
           throws IOException

Specified by:

flush in interface Flushable

Specified by:

flush in class org.apache.flink.core.fs.FSDataOutputStream

Throws:

IOException

sync

public void sync()
          throws IOException

Specified by:

sync in class org.apache.flink.core.fs.FSDataOutputStream

Throws:

IOException

close

public void close()
           throws IOException

Description copied from class: CheckpointStreamFactory.CheckpointStateOutputStream

This method should close the stream, if has not been closed before. If this method actually closes the stream, it should delete/release the resource behind the stream, such as the file that the stream writes to.

The above implies that this method is intended to be the "unsuccessful close", such as when cancelling the stream writing, or when an exception occurs. Closing the stream for the successful case must go through CheckpointStreamFactory.CheckpointStateOutputStream.closeAndGetHandle().

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class CheckpointStreamFactory.CheckpointStateOutputStream

Throws:

IOException - Thrown, if the stream cannot be closed.

closeAndGetHandle

@Nullable
public StreamStateHandle closeAndGetHandle()
                                              throws IOException

Description copied from class: CheckpointStreamFactory.CheckpointStateOutputStream

Closes the stream and gets a state handle that can create an input stream producing the data written to this stream.

This closing must be called (also when the caller is not interested in the handle) to successfully close the stream and retain the produced resource. In contrast, the CheckpointStreamFactory.CheckpointStateOutputStream.close() method removes the target resource when called.

Specified by:

closeAndGetHandle in class CheckpointStreamFactory.CheckpointStateOutputStream

Returns:

A state handle that can create an input stream producing the data written to this stream.

Throws:

IOException - Thrown, if the stream cannot be closed.

closeAndGetPrimaryHandle

public StreamStateHandle closeAndGetPrimaryHandle()
                                           throws IOException

Returns the state handle from the primaryOutputStream.

Throws:

IOException

closeAndGetSecondaryHandle

public StreamStateHandle closeAndGetSecondaryHandle()
                                             throws IOException

Returns the state handle from the secondaryOutputStream. Also reports suppressed exceptions from earlier interactions with that stream.

Throws:

IOException

getSecondaryStreamException

public Exception getSecondaryStreamException()