T - the type of elements handled by this coderpublic class AvroCoder<T>
extends org.apache.beam.sdk.coders.CustomCoder<T>
Coder using Avro binary format.
Each instance of AvroCoder<T> encapsulates an Avro schema for objects of type T.
The Avro schema may be provided explicitly via of(Class, Schema) or omitted
via of(Class), in which case it will be inferred using Avro's ReflectData.
For complete details about schema generation and how it can be controlled please see the
org.apache.avro.reflect package. Only concrete classes with a no-argument constructor can
be mapped to Avro records. All inherited fields that are not static or transient are included.
Fields are not permitted to be null unless annotated by Nullable or a Union
schema containing "null".
To use, specify the Coder type on a PCollection:
PCollection<MyCustomElement> records =
input.apply(...)
.setCoder(AvroCoder.of(MyCustomElement.class));
or annotate the element class using @DefaultCoder.
@DefaultCoder(AvroCoder.class)
public class MyCustomElement {
...
}
The implementation attempts to determine if the Avro encoding of the given type will satisfy
the criteria of Coder.verifyDeterministic() by inspecting both the type and the Schema
provided or generated by Avro. Only coders that are deterministic can be used in GroupByKey operations.
| Modifier and Type | Class and Description |
|---|---|
static class |
AvroCoder.JodaTimestampConversion
Conversion for DateTime.
|
| Modifier | Constructor and Description |
|---|---|
protected |
AvroCoder(java.lang.Class<T> type,
org.apache.avro.Schema schema) |
protected |
AvroCoder(java.lang.Class<T> type,
org.apache.avro.Schema schema,
boolean useReflectApi) |
| Modifier and Type | Method and Description |
|---|---|
T |
decode(java.io.InputStream inStream) |
void |
encode(T value,
java.io.OutputStream outStream) |
boolean |
equals(@Nullable java.lang.Object other) |
static org.apache.beam.sdk.coders.CoderProvider |
getCoderProvider()
Returns a
CoderProvider which uses the AvroCoder if possible for all types. |
org.apache.beam.sdk.values.TypeDescriptor<T> |
getEncodedTypeDescriptor() |
org.apache.avro.Schema |
getSchema()
Returns the schema used by this coder.
|
java.lang.Class<T> |
getType()
Returns the type this coder encodes/decodes.
|
int |
hashCode() |
static <T> AvroCoder<T> |
of(java.lang.Class<T> clazz)
Returns an
AvroCoder instance for the provided element class. |
static <T> AvroCoder<T> |
of(java.lang.Class<T> type,
boolean useReflectApi)
Returns an
AvroCoder instance for the given class, respecting whether to use Avro's
Reflect* or Specific* suite for encoding and decoding. |
static <T> AvroCoder<T> |
of(java.lang.Class<T> type,
org.apache.avro.Schema schema)
Returns an
AvroCoder instance for the provided element type using the provided Avro
schema. |
static <T> AvroCoder<T> |
of(java.lang.Class<T> type,
org.apache.avro.Schema schema,
boolean useReflectApi)
Returns an
AvroCoder instance for the given class and schema, respecting whether to use
Avro's Reflect* or Specific* suite for encoding and decoding. |
static AvroGenericCoder |
of(org.apache.avro.Schema schema)
Returns an
AvroGenericCoder instance for the Avro schema. |
static <T> AvroCoder<T> |
of(org.apache.beam.sdk.values.TypeDescriptor<T> type)
Returns an
AvroCoder instance for the provided element type. |
static <T> AvroCoder<T> |
of(org.apache.beam.sdk.values.TypeDescriptor<T> type,
boolean useReflectApi)
Returns an
AvroCoder instance for the provided element type, respecting whether to use
Avro's Reflect* or Specific* suite for encoding and decoding. |
boolean |
useReflectApi() |
void |
verifyDeterministic() |
protected AvroCoder(java.lang.Class<T> type, org.apache.avro.Schema schema)
protected AvroCoder(java.lang.Class<T> type, org.apache.avro.Schema schema, boolean useReflectApi)
public static <T> AvroCoder<T> of(org.apache.beam.sdk.values.TypeDescriptor<T> type)
AvroCoder instance for the provided element type.T - the element typepublic static <T> AvroCoder<T> of(org.apache.beam.sdk.values.TypeDescriptor<T> type, boolean useReflectApi)
AvroCoder instance for the provided element type, respecting whether to use
Avro's Reflect* or Specific* suite for encoding and decoding.T - the element typepublic static <T> AvroCoder<T> of(java.lang.Class<T> clazz)
AvroCoder instance for the provided element class.T - the element typepublic static AvroGenericCoder of(org.apache.avro.Schema schema)
AvroGenericCoder instance for the Avro schema. The implicit type is
GenericRecord.public static <T> AvroCoder<T> of(java.lang.Class<T> type, boolean useReflectApi)
AvroCoder instance for the given class, respecting whether to use Avro's
Reflect* or Specific* suite for encoding and decoding.T - the element typepublic static <T> AvroCoder<T> of(java.lang.Class<T> type, org.apache.avro.Schema schema)
AvroCoder instance for the provided element type using the provided Avro
schema.
The schema must correspond to the type provided.
T - the element typepublic static <T> AvroCoder<T> of(java.lang.Class<T> type, org.apache.avro.Schema schema, boolean useReflectApi)
AvroCoder instance for the given class and schema, respecting whether to use
Avro's Reflect* or Specific* suite for encoding and decoding.T - the element typepublic static org.apache.beam.sdk.coders.CoderProvider getCoderProvider()
CoderProvider which uses the AvroCoder if possible for all types.
It is unsafe to register this as a CoderProvider because Avro will reflectively
accept dangerous types such as Object.
This method is invoked reflectively from DefaultCoder.
public java.lang.Class<T> getType()
public boolean useReflectApi()
public void encode(T value, java.io.OutputStream outStream) throws java.io.IOException
encode in class org.apache.beam.sdk.coders.Coder<T>java.io.IOExceptionpublic T decode(java.io.InputStream inStream) throws java.io.IOException
decode in class org.apache.beam.sdk.coders.Coder<T>java.io.IOExceptionpublic void verifyDeterministic()
throws org.apache.beam.sdk.coders.Coder.NonDeterministicException
verifyDeterministic in class org.apache.beam.sdk.coders.CustomCoder<T>NonDeterministicException - when the type may not be deterministically encoded using the
given Schema, the directBinaryEncoder, and the ReflectDatumWriter
or GenericDatumWriter.org.apache.beam.sdk.coders.Coder.NonDeterministicExceptionpublic org.apache.avro.Schema getSchema()
public org.apache.beam.sdk.values.TypeDescriptor<T> getEncodedTypeDescriptor()
getEncodedTypeDescriptor in class org.apache.beam.sdk.coders.Coder<T>public boolean equals(@Nullable java.lang.Object other)
equals in class java.lang.Objectpublic int hashCode()
hashCode in class java.lang.Object