org.apache.hadoop.hbase.regionserver.wal

Class WALEdit

java.lang.Object

org.apache.hadoop.hbase.regionserver.wal.WALEdit

All Implemented Interfaces:

HeapSize, org.apache.hadoop.io.Writable

@InterfaceAudience.LimitedPrivate(value={"Replication","Coprocesssor"})
public class WALEdit
extends Object
implements org.apache.hadoop.io.Writable, HeapSize

WALEdit: Used in HBase's transaction log (WAL) to represent the collection of edits (KeyValue objects) corresponding to a single transaction. The class implements "Writable" interface for serializing/deserializing a set of KeyValue items. Previously, if a transaction contains 3 edits to c1, c2, c3 for a row R, the WAL would have three log entries as follows: : : : This presents problems because row level atomicity of transactions was not guaranteed. If we crash after few of the above appends make it, then recovery will restore a partial transaction. In the new world, all the edits for a given transaction are written out as a single record, for example: : where, the WALEdit is serialized as: <-1, # of edits, , , ... > For example: <-1, 3, , , > The -1 marker is just a special way of being backward compatible with an old WAL which would have contained a single . The deserializer for WALEdit backward compatibly detects if the record is an old style KeyValue or the new style WALEdit.

Field Summary

Fields

Modifier and Type	Field and Description
static byte[]	BULK_LOAD
static WALEdit	EMPTY_WALEDIT
static org.apache.commons.logging.Log	LOG
static byte[]	METAFAMILY

Constructor Summary

Constructors

Constructor and Description
WALEdit()
WALEdit(boolean isReplay)

Method Summary

Methods

Modifier and Type	Method and Description
WALEdit	add(Cell cell)
static WALEdit	createBulkLoadEvent(HRegionInfo hri, org.apache.hadoop.hbase.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor) Create a bulk loader WALEdit
static WALEdit	createCompaction(HRegionInfo hri, org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor c) Create a compaction WALEdit
static WALEdit	createFlushWALEdit(HRegionInfo hri, org.apache.hadoop.hbase.protobuf.generated.WALProtos.FlushDescriptor f)
static WALEdit	createRegionEventWALEdit(HRegionInfo hri, org.apache.hadoop.hbase.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)
NavigableMap<byte[],Integer>	getAndRemoveScopes()
static org.apache.hadoop.hbase.protobuf.generated.WALProtos.BulkLoadDescriptor	getBulkLoadDescriptor(Cell cell) Deserialized and returns a BulkLoadDescriptor from the passed in Cell
ArrayList<Cell>	getCells()
static org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor	getCompaction(Cell kv) Deserialized and returns a CompactionDescriptor is the KeyValue contains one.
static org.apache.hadoop.hbase.protobuf.generated.WALProtos.FlushDescriptor	getFlushDescriptor(Cell cell)
static org.apache.hadoop.hbase.protobuf.generated.WALProtos.RegionEventDescriptor	getRegionEventDescriptor(Cell cell)
long	heapSize()
boolean	isEmpty()
boolean	isMetaEdit()
static boolean	isMetaEditFamily(byte[] f)
static boolean	isMetaEditFamily(Cell cell)
boolean	isReplay()
void	readFields(DataInput in)
int	readFromCells(Codec.Decoder cellDecoder, int expectedCount) Reads WALEdit from cells.
void	setCompressionContext(CompressionContext compressionContext)
int	size()
String	toString()
void	write(DataOutput out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

METAFAMILY

public static final byte[] METAFAMILY

BULK_LOAD

public static final byte[] BULK_LOAD

EMPTY_WALEDIT

public static final WALEdit EMPTY_WALEDIT

Constructor Detail

WALEdit

public WALEdit()

WALEdit

public WALEdit(boolean isReplay)

Method Detail

isMetaEditFamily

public static boolean isMetaEditFamily(byte[] f)

Parameters:

f -

Returns:

True is f is METAFAMILY

isMetaEditFamily

public static boolean isMetaEditFamily(Cell cell)

isMetaEdit

public boolean isMetaEdit()

isReplay

public boolean isReplay()

Returns:

True when current WALEdit is created by log replay. Replication skips WALEdits from replay.

setCompressionContext

public void setCompressionContext(CompressionContext compressionContext)

add

public WALEdit add(Cell cell)

isEmpty

public boolean isEmpty()

size

public int size()

getCells

public ArrayList<Cell> getCells()

getAndRemoveScopes

public NavigableMap<byte[],Integer> getAndRemoveScopes()

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Throws:

IOException

readFromCells

public int readFromCells(Codec.Decoder cellDecoder,
                int expectedCount)
                  throws IOException

Reads WALEdit from cells.

Parameters:

cellDecoder - Cell decoder.

expectedCount - Expected cell count.

Returns:

Number of KVs read.

Throws:

IOException

heapSize

public long heapSize()

Specified by:

heapSize in interface HeapSize

toString

public String toString()

Overrides:

toString in class Object

createFlushWALEdit

public static WALEdit createFlushWALEdit(HRegionInfo hri,
                         org.apache.hadoop.hbase.protobuf.generated.WALProtos.FlushDescriptor f)

getFlushDescriptor

public static org.apache.hadoop.hbase.protobuf.generated.WALProtos.FlushDescriptor getFlushDescriptor(Cell cell)
                                                                                               throws IOException

Throws:

IOException

createRegionEventWALEdit

public static WALEdit createRegionEventWALEdit(HRegionInfo hri,
                               org.apache.hadoop.hbase.protobuf.generated.WALProtos.RegionEventDescriptor regionEventDesc)

getRegionEventDescriptor

public static org.apache.hadoop.hbase.protobuf.generated.WALProtos.RegionEventDescriptor getRegionEventDescriptor(Cell cell)
                                                                                                           throws IOException

Throws:

IOException

createCompaction

public static WALEdit createCompaction(HRegionInfo hri,
                       org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor c)

Create a compaction WALEdit

Parameters:

c -

Returns:

A WALEdit that has c serialized as its value

getCompaction

public static org.apache.hadoop.hbase.protobuf.generated.WALProtos.CompactionDescriptor getCompaction(Cell kv)
                                                                                               throws IOException

Deserialized and returns a CompactionDescriptor is the KeyValue contains one.

Parameters:

kv - the key value

Returns:

deserialized CompactionDescriptor or null.

Throws:

IOException

createBulkLoadEvent

public static WALEdit createBulkLoadEvent(HRegionInfo hri,
                          org.apache.hadoop.hbase.protobuf.generated.WALProtos.BulkLoadDescriptor bulkLoadDescriptor)

Create a bulk loader WALEdit

Parameters:

hri - The HRegionInfo for the region in which we are bulk loading

bulkLoadDescriptor - The descriptor for the Bulk Loader

Returns:

The WALEdit for the BulkLoad

getBulkLoadDescriptor

public static org.apache.hadoop.hbase.protobuf.generated.WALProtos.BulkLoadDescriptor getBulkLoadDescriptor(Cell cell)
                                                                                                     throws IOException

Deserialized and returns a BulkLoadDescriptor from the passed in Cell

Parameters:

cell - the key value

Returns:

deserialized BulkLoadDescriptor or null.

Throws:

IOException