A C D E F G H I J N O P R S T U V W

A

apply(Schema) - Method in interface org.apache.beam.sdk.extensions.avro.io.AvroSink.DatumWriterFactory

apply(Schema, Schema) - Method in interface org.apache.beam.sdk.extensions.avro.io.AvroSource.DatumReaderFactory

AvroCoder<T> - Class in org.apache.beam.sdk.extensions.avro.coders

A Coder using Avro binary format.

AvroCoder(Class<T>, Schema) - Constructor for class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

AvroCoder(Class<T>, Schema, boolean) - Constructor for class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

AvroCoder.JodaTimestampConversion - Class in org.apache.beam.sdk.extensions.avro.coders

Conversion for DateTime.

AvroConvertType(boolean) - Constructor for class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.AvroConvertType

AvroGenericCoder - Class in org.apache.beam.sdk.extensions.avro.coders

AvroCoder specialisation for GenericRecord.

AvroIO - Class in org.apache.beam.sdk.extensions.avro.io

PTransforms for reading and writing Avro files.

AvroIO.Parse<T> - Class in org.apache.beam.sdk.extensions.avro.io

Implementation of AvroIO.parseGenericRecords(org.apache.beam.sdk.transforms.SerializableFunction<org.apache.avro.generic.GenericRecord, T>).

AvroIO.ParseAll<T> - Class in org.apache.beam.sdk.extensions.avro.io

Deprecated.

See AvroIO.parseAllGenericRecords(SerializableFunction) for details.

AvroIO.ParseFiles<T> - Class in org.apache.beam.sdk.extensions.avro.io

Implementation of AvroIO.parseFilesGenericRecords(org.apache.beam.sdk.transforms.SerializableFunction<org.apache.avro.generic.GenericRecord, T>).

AvroIO.Read<T> - Class in org.apache.beam.sdk.extensions.avro.io

Implementation of AvroIO.read(java.lang.Class<T>) and AvroIO.readGenericRecords(org.apache.avro.Schema).

AvroIO.ReadAll<T> - Class in org.apache.beam.sdk.extensions.avro.io

Deprecated.

See AvroIO.readAll(Class) for details.

AvroIO.ReadFiles<T> - Class in org.apache.beam.sdk.extensions.avro.io

Implementation of AvroIO.readFiles(java.lang.Class<T>).

AvroIO.RecordFormatter<ElementT> - Interface in org.apache.beam.sdk.extensions.avro.io

Deprecated.

Users can achieve the same by providing this transform in a ParDo before using write in AvroIO AvroIO.write(Class).

AvroIO.Sink<ElementT> - Class in org.apache.beam.sdk.extensions.avro.io

Implementation of AvroIO.sink(java.lang.Class<ElementT>) and AvroIO.sinkViaGenericRecords(org.apache.avro.Schema, org.apache.beam.sdk.extensions.avro.io.AvroIO.RecordFormatter<ElementT>).

AvroIO.TypedWrite<UserT,DestinationT,OutputT> - Class in org.apache.beam.sdk.extensions.avro.io

Implementation of AvroIO.write(java.lang.Class<T>).

AvroIO.Write<T> - Class in org.apache.beam.sdk.extensions.avro.io

This class is used as the default return value of AvroIO.write(java.lang.Class<T>)

AvroPayloadSerializerProvider - Class in org.apache.beam.sdk.extensions.avro.schemas.io.payloads

AvroPayloadSerializerProvider() - Constructor for class org.apache.beam.sdk.extensions.avro.schemas.io.payloads.AvroPayloadSerializerProvider

AvroReader(AvroSource<T>) - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

Reads Avro records of type T from the specified source.

AvroRecordSchema - Class in org.apache.beam.sdk.extensions.avro.schemas

A SchemaProvider for AVRO generated SpecificRecords and POJOs.

AvroRecordSchema() - Constructor for class org.apache.beam.sdk.extensions.avro.schemas.AvroRecordSchema

AvroSchemaIOProvider - Class in org.apache.beam.sdk.extensions.avro.io

An implementation of SchemaIOProvider for reading and writing Avro files with AvroIO.

AvroSchemaIOProvider() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroSchemaIOProvider

AvroSink<UserT,DestinationT,OutputT> - Class in org.apache.beam.sdk.extensions.avro.io

A FileBasedSink for Avro files.

AvroSink.DatumWriterFactory<T> - Interface in org.apache.beam.sdk.extensions.avro.io

AvroSource<T> - Class in org.apache.beam.sdk.extensions.avro.io

Do not use in pipelines directly: most users should use AvroIO.Read.

AvroSource.AvroReader<T> - Class in org.apache.beam.sdk.extensions.avro.io

A BlockBasedReader for reading blocks from Avro files.

AvroSource.DatumReaderFactory<T> - Interface in org.apache.beam.sdk.extensions.avro.io

AvroUtils - Class in org.apache.beam.sdk.extensions.avro.schemas.utils

Utils to convert AVRO records to Beam rows.

AvroUtils.AvroConvertType - Class in org.apache.beam.sdk.extensions.avro.schemas.utils

AvroUtils.AvroConvertValueForGetter - Class in org.apache.beam.sdk.extensions.avro.schemas.utils

AvroUtils.AvroConvertValueForSetter - Class in org.apache.beam.sdk.extensions.avro.schemas.utils

AvroUtils.FixedBytesField - Class in org.apache.beam.sdk.extensions.avro.schemas.utils

Wrapper for fixed byte fields.

AvroUtils.TypeWithNullability - Class in org.apache.beam.sdk.extensions.avro.schemas.utils

C

configurationSchema() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSchemaIOProvider

Returns the expected schema of the configuration object.

constantDestinations(FileBasedSink.FilenamePolicy, Schema, Map<String, Object>, CodecFactory, SerializableFunction<UserT, OutputT>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Returns a DynamicAvroDestinations that always returns the same FileBasedSink.FilenamePolicy, schema, metadata, and codec.

constantDestinations(FileBasedSink.FilenamePolicy, Schema, Map<String, Object>, CodecFactory, SerializableFunction<UserT, OutputT>, AvroSink.DatumWriterFactory<OutputT>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Returns a DynamicAvroDestinations that always returns the same FileBasedSink.FilenamePolicy, schema, metadata, and codec.

convertAvroFieldStrict(Object, Schema, Schema.FieldType) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Strict conversion from AVRO to Beam, strict because it doesn't do widening or narrowing during conversion.

convertDefault(TypeDescriptor<?>) - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.AvroConvertType

convertDefault(TypeDescriptor<?>) - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.AvroConvertValueForGetter

convertDefault(TypeDescriptor<?>) - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.AvroConvertValueForSetter

create(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.TypeWithNullability

createForSubrangeOfFile(String, long, long) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Deprecated.

Used by Dataflow worker

createForSubrangeOfFile(MatchResult.Metadata, long, long) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

createSingleFileReader(PipelineOptions) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

createWriteOperation() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSink

D

decode(InputStream) - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

DynamicAvroDestinations<UserT,DestinationT,OutputT> - Class in org.apache.beam.sdk.extensions.avro.io

A specialization of FileBasedSink.DynamicDestinations for AvroIO.

DynamicAvroDestinations() - Constructor for class org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations

E

encode(T, OutputStream) - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

equals(Object) - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

expand(PBegin) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

expand(PCollection<String>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

expand(PCollection<FileIO.ReadableFile>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseFiles

expand(PBegin) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

expand(PCollection<String>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

expand(PCollection<FileIO.ReadableFile>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

expand(PCollection<UserT>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

expand(PCollection<T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

F

fieldValueGetters(Class<?>, Schema) - Method in class org.apache.beam.sdk.extensions.avro.schemas.AvroRecordSchema

fieldValueTypeInformations(Class<?>, Schema) - Method in class org.apache.beam.sdk.extensions.avro.schemas.AvroRecordSchema

flush() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Sink

formatRecord(ElementT, Schema) - Method in interface org.apache.beam.sdk.extensions.avro.io.AvroIO.RecordFormatter

Deprecated.

from(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Reads from the given filename or filepattern.

from(ValueProvider<String>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Like AvroIO.Parse.from(String).

from(ValueProvider<String>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Reads from the given filename or filepattern.

from(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Like AvroIO.Read.from(ValueProvider).

from(String, Row, Schema) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSchemaIOProvider

Produce a SchemaIO given a String representing the data's location, the schema of the data that resides there, and some IO-specific configuration object.

from(ValueProvider<String>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Reads from the given file name or pattern ("glob").

from(MatchResult.Metadata) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

from(String) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Like AvroSource.from(ValueProvider).

fromAvroType(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.FixedBytesField

Create a AvroUtils.FixedBytesField from an AVRO type.

fromBeamFieldType(Schema.FieldType) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.FixedBytesField

Create a AvroUtils.FixedBytesField from a Beam Schema.FieldType.

fromLong(Long, Schema, LogicalType) - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder.JodaTimestampConversion

G

getAvroBytesToRowFunction(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns a function mapping encoded AVRO GenericRecords to Beam Rows.

getCodec(DestinationT) - Method in class org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations

Return an AVRO codec for a given destination.

getCoderProvider() - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns a CoderProvider which uses the AvroCoder if possible for all types.

getConvertedType() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder.JodaTimestampConversion

getCreator(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Get an object creator for an AVRO-generated SpecificRecord.

getCurrentBlock() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

getCurrentBlockOffset() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

getCurrentBlockSize() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

getCurrentSource() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

getDatumWriterFactory(DestinationT) - Method in class org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations

Return a AvroSink.DatumWriterFactory for a given destination.

getDynamicDestinations() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSink

getEncodedTypeDescriptor() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

getFactory() - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.AvroConvertValueForGetter

getFactory() - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.AvroConvertValueForSetter

getFieldTypes(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Get field types for an AVRO-generated SpecificRecord or a POJO.

getFromRowFunction(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

getGenericRecordToRowFunction(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns a function mapping AVRO GenericRecords to Beam Rows for use in PCollection.setSchema(org.apache.beam.sdk.schemas.Schema, org.apache.beam.sdk.values.TypeDescriptor<T>, org.apache.beam.sdk.transforms.SerializableFunction<T, org.apache.beam.sdk.values.Row>, org.apache.beam.sdk.transforms.SerializableFunction<org.apache.beam.sdk.values.Row, T>).

getGetters(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Get generated getters for an AVRO-generated SpecificRecord or a POJO.

getLogicalTypeName() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder.JodaTimestampConversion

getMetadata(DestinationT) - Method in class org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations

Return AVRO file metadata for a given destination.

getOutputCoder() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

getRowToAvroBytesFunction(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns a function mapping Beam Rows to encoded AVRO GenericRecords.

getRowToGenericRecordFunction(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns a function mapping Beam Rows to AVRO GenericRecords for use in PCollection.setSchema(org.apache.beam.sdk.schemas.Schema, org.apache.beam.sdk.values.TypeDescriptor<T>, org.apache.beam.sdk.transforms.SerializableFunction<T, org.apache.beam.sdk.values.Row>, org.apache.beam.sdk.transforms.SerializableFunction<org.apache.beam.sdk.values.Row, T>).

getSchema() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns the schema used by this coder.

getSchema(DestinationT) - Method in class org.apache.beam.sdk.extensions.avro.io.DynamicAvroDestinations

Return an AVRO schema for a given destination.

getSchema(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

getSerializer(Schema, Map<String, Object>) - Method in class org.apache.beam.sdk.extensions.avro.schemas.io.payloads.AvroPayloadSerializerProvider

getSize() - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.FixedBytesField

Get the size.

getSplitPointsRemaining() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

getToRowFunction(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

getType() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns the type this coder encodes/decodes.

getType() - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.TypeWithNullability

H

hashCode() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

I

identifier() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSchemaIOProvider

Returns an id that uniquely represents this IO.

identifier() - Method in class org.apache.beam.sdk.extensions.avro.schemas.io.payloads.AvroPayloadSerializerProvider

isBounded() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSchemaIOProvider

isNullable() - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.TypeWithNullability

J

JodaTimestampConversion() - Constructor for class org.apache.beam.sdk.extensions.avro.coders.AvroCoder.JodaTimestampConversion

N

nullable - Variable in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.TypeWithNullability

O

of(TypeDescriptor<T>) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroCoder instance for the provided element type.

of(TypeDescriptor<T>, boolean) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroCoder instance for the provided element type, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.

of(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroCoder instance for the provided element class.

of(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroGenericCoder instance for the Avro schema.

of(Class<T>, boolean) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroCoder instance for the given class, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.

of(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroCoder instance for the provided element type using the provided Avro schema.

of(Class<T>, Schema, boolean) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

Returns an AvroCoder instance for the given class and schema, respecting whether to use Avro's Reflect* or Specific* suite for encoding and decoding.

of(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.coders.AvroGenericCoder

open(WritableByteChannel) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Sink

org.apache.beam.sdk.extensions.avro.coders - package org.apache.beam.sdk.extensions.avro.coders

Defines Coders to specify how data is encoded to and decoded from byte strings using Apache Avro.

org.apache.beam.sdk.extensions.avro.io - package org.apache.beam.sdk.extensions.avro.io

Defines transforms for reading and writing Avro storage format.

org.apache.beam.sdk.extensions.avro.schemas - package org.apache.beam.sdk.extensions.avro.schemas

Defines Schema and other classes for representing schema'd data in a Pipeline using Apache Avro.

org.apache.beam.sdk.extensions.avro.schemas.io.payloads - package org.apache.beam.sdk.extensions.avro.schemas.io.payloads

Provides abstractions for schema-aware AvroIO.

org.apache.beam.sdk.extensions.avro.schemas.utils - package org.apache.beam.sdk.extensions.avro.schemas.utils

Defines utilities for deailing with schemas using Apache Avro.

P

Parse() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

ParseAll() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

parseAllGenericRecords(SerializableFunction<GenericRecord, T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Deprecated.

You can achieve The functionality of AvroIO.parseAllGenericRecords(SerializableFunction) using FileIO matching plus AvroIO.parseFilesGenericRecords(SerializableFunction) ()}. This is the preferred method to make composition explicit. AvroIO.ParseAll will not receive upgrades and will be removed in a future version of Beam.

ParseFiles() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseFiles

parseFilesGenericRecords(SerializableFunction<GenericRecord, T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Like AvroIO.parseGenericRecords(SerializableFunction), but reads each FileIO.ReadableFile in the input PCollection.

parseGenericRecords(SerializableFunction<GenericRecord, T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Reads Avro file(s) containing records of an unspecified schema and converting each record to a custom type.

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseFiles

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

populateDisplayData(DisplayData.Builder) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

R

read(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Reads records of the given type from an Avro file (or multiple Avro files matching a pattern).

Read() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

readAll(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Deprecated.

You can achieve The functionality of AvroIO.readAll(java.lang.Class<T>) using FileIO matching plus AvroIO.readFiles(Class). This is the preferred method to make composition explicit. AvroIO.ReadAll will not receive upgrades and will be removed in a future version of Beam.

ReadAll() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

readAllGenericRecords(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Deprecated.

You can achieve The functionality of AvroIO.readAllGenericRecords(Schema) using FileIO matching plus AvroIO.readFilesGenericRecords(Schema). This is the preferred method to make composition explicit. AvroIO.ReadAll will not receive upgrades and will be removed in a future version of Beam.

readAllGenericRecords(String) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Deprecated.

You can achieve The functionality of AvroIO.readAllGenericRecords(String) using FileIO matching plus AvroIO.readFilesGenericRecords(String). This is the preferred method to make composition explicit. AvroIO.ReadAll will not receive upgrades and will be removed in a future version of Beam.

readFiles(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Like AvroIO.read(java.lang.Class<T>), but reads each file in a PCollection of FileIO.ReadableFile, returned by FileIO.readMatches().

ReadFiles() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

readFilesGenericRecords(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Like AvroIO.readGenericRecords(Schema), but for a PCollection of FileIO.ReadableFile, for example, returned by FileIO.readMatches().

readFilesGenericRecords(String) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Like AvroIO.readGenericRecords(String), but for FileIO.ReadableFile collections.

readGenericRecords(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Reads Avro file(s) containing records of the specified schema.

readGenericRecords(String) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Reads Avro file(s) containing records of the specified schema.

readNextBlock() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

requiresDataSchema() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSchemaIOProvider

S

schemaCoder(TypeDescriptor<T>) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns an SchemaCoder instance for the provided element type.

schemaCoder(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns an SchemaCoder instance for the provided element class.

schemaCoder(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns an SchemaCoder instance for the Avro schema.

schemaCoder(Class<T>, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns an SchemaCoder instance for the provided element type using the provided Avro schema.

schemaCoder(AvroCoder<T>) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Returns an SchemaCoder instance based on the provided AvroCoder for the element type.

schemaFor(TypeDescriptor<T>) - Method in class org.apache.beam.sdk.extensions.avro.schemas.AvroRecordSchema

schemaTypeCreator(Class<?>, Schema) - Method in class org.apache.beam.sdk.extensions.avro.schemas.AvroRecordSchema

sink(Class<ElementT>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

A AvroIO.Sink for use with FileIO.write() and FileIO.writeDynamic(), writing elements of the given generated class, like AvroIO.write(Class).

sink(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

A AvroIO.Sink for use with FileIO.write() and FileIO.writeDynamic(), writing elements with a given (common) schema, like AvroIO.writeGenericRecords(Schema).

sink(String) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

A AvroIO.Sink for use with FileIO.write() and FileIO.writeDynamic(), writing elements with a given (common) schema, like AvroIO.writeGenericRecords(String).

Sink() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.Sink

sinkViaGenericRecords(Schema, AvroIO.RecordFormatter<ElementT>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Deprecated.

RecordFormatter will be removed in future versions.

startReading(ReadableByteChannel) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource.AvroReader

T

to(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Writes to file(s) with the given output prefix.

to(ResourceId) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Writes to file(s) with the given output prefix.

to(ValueProvider<String>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Like AvroIO.TypedWrite.to(String).

to(FileBasedSink.FilenamePolicy) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Writes to files named according to the given FileBasedSink.FilenamePolicy.

to(DynamicAvroDestinations<UserT, NewDestinationT, OutputT>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Deprecated.

Use FileIO.write() or FileIO.writeDynamic() instead.

to(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.to(String).

to(ResourceId) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.to(ResourceId) .

to(ValueProvider<String>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.to(ValueProvider).

to(FileBasedSink.FilenamePolicy) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See TypedWrite#to(FilenamePolicy).

to(DynamicAvroDestinations<T, ?, T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

Deprecated.

Use FileIO.write() or FileIO.writeDynamic() instead.

toAvroField(Schema.Field, String) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Get Avro Field from Beam Field.

toAvroSchema(Schema, String, String) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Converts a Beam Schema into an AVRO schema.

toAvroSchema(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

toAvroType(String, String) - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.FixedBytesField

Convert to an AVRO type.

toBeamField(Schema.Field) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Get Beam Field from avro Field.

toBeamRowStrict(GenericRecord, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Strict conversion from AVRO to Beam, strict because it doesn't do widening or narrowing during conversion.

toBeamSchema(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Converts AVRO schema to Beam row schema.

toBeamType() - Method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.FixedBytesField

Convert to a Beam type.

toGenericRecord(Row) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Convert from a Beam Row to an AVRO GenericRecord.

toGenericRecord(Row, Schema) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils

Convert from a Beam Row to an AVRO GenericRecord.

toLong(DateTime, Schema, LogicalType) - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder.JodaTimestampConversion

toResource(ValueProvider<ResourceId>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Like AvroIO.TypedWrite.to(ResourceId).

toResource(ValueProvider<ResourceId>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.to(ResourceId).

type - Variable in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.TypeWithNullability

TypedWrite() - Constructor for class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

U

useReflectApi() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

V

validate() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

verifyDeterministic() - Method in class org.apache.beam.sdk.extensions.avro.coders.AvroCoder

W

watchForNewFiles(Duration, Watch.Growth.TerminationCondition<String, ?>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Like AvroIO.Read.watchForNewFiles(org.joda.time.Duration, org.apache.beam.sdk.transforms.Watch.Growth.TerminationCondition<java.lang.String, ?>, boolean).

watchForNewFiles(Duration, Watch.Growth.TerminationCondition<String, ?>, boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

Like Read#watchForNewFiles(Duration, TerminationCondition, boolean).

watchForNewFiles(Duration, Watch.Growth.TerminationCondition<String, ?>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

Like Read#watchForNewFiles(Duration, TerminationCondition).

watchForNewFiles(Duration, Watch.Growth.TerminationCondition<String, ?>, boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Continuously watches for new files matching the filepattern, polling it at the given interval, until the given termination condition is reached.

watchForNewFiles(Duration, Watch.Growth.TerminationCondition<String, ?>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Same as Read#watchForNewFiles(Duration, TerminationCondition, boolean) with matchUpdatedFiles=false.

watchForNewFiles(Duration, Watch.Growth.TerminationCondition<String, ?>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

Like AvroIO.Read.watchForNewFiles(org.joda.time.Duration, org.apache.beam.sdk.transforms.Watch.Growth.TerminationCondition<java.lang.String, ?>, boolean).

withBeamSchemas(boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

If set to true, a Beam schema will be inferred from the AVRO schema.

withBeamSchemas(boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

If set to true, a Beam schema will be inferred from the AVRO schema.

withBeamSchemas(boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

If set to true, a Beam schema will be inferred from the AVRO schema.

withCodec(CodecFactory) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Sink

Specifies to use the given CodecFactory for each generated file.

withCodec(CodecFactory) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Writes to Avro file(s) compressed using specified codec.

withCodec(CodecFactory) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withCodec(org.apache.avro.file.CodecFactory).

withCoder(Coder<T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Sets a coder for the result of the parse function.

withCoder(Coder<T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

Specifies the coder for the result of the parseFn.

withCoder(Coder<T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseFiles

Specifies the coder for the result of the parseFn.

withDatumReaderFactory(AvroSource.DatumReaderFactory<T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

withDatumReaderFactory(AvroSource.DatumReaderFactory<?>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

withDatumWriterFactory(AvroSink.DatumWriterFactory<OutputT>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Specifies a AvroSink.DatumWriterFactory to use for creating DatumWriter instances.

withDatumWriterFactory(AvroSink.DatumWriterFactory<T>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withDatumWriterFactory(org.apache.beam.sdk.extensions.avro.io.AvroSink.DatumWriterFactory<OutputT>).

withEmptyMatchTreatment(EmptyMatchTreatment) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Like AvroIO.Read.withEmptyMatchTreatment(org.apache.beam.sdk.io.fs.EmptyMatchTreatment).

withEmptyMatchTreatment(EmptyMatchTreatment) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

Like AvroIO.Read.withEmptyMatchTreatment(org.apache.beam.sdk.io.fs.EmptyMatchTreatment).

withEmptyMatchTreatment(EmptyMatchTreatment) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Configures whether or not a filepattern matching no files is allowed.

withEmptyMatchTreatment(EmptyMatchTreatment) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

Like AvroIO.Read.withEmptyMatchTreatment(org.apache.beam.sdk.io.fs.EmptyMatchTreatment).

withEmptyMatchTreatment(EmptyMatchTreatment) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

withFileExceptionHandler(ReadAllViaFileBasedSource.ReadFileRangesFnExceptionHandler) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseFiles

Specifies if exceptions should be logged only for streaming pipelines.

withFileExceptionHandler(ReadAllViaFileBasedSource.ReadFileRangesFnExceptionHandler) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

Specifies if exceptions should be logged only for streaming pipelines.

withFormatFunction(SerializableFunction<UserT, OutputT>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Specifies a format function to convert UserT to the output type.

withHintMatchesManyFiles() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Like AvroIO.Read.withHintMatchesManyFiles().

withHintMatchesManyFiles() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Hints that the filepattern specified in AvroIO.Read.from(String) matches a very large number of files.

withMatchConfiguration(FileIO.MatchConfiguration) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Parse

Sets the FileIO.MatchConfiguration.

withMatchConfiguration(FileIO.MatchConfiguration) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseAll

Deprecated.

Sets the FileIO.MatchConfiguration.

withMatchConfiguration(FileIO.MatchConfiguration) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Read

Sets the FileIO.MatchConfiguration.

withMatchConfiguration(FileIO.MatchConfiguration) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadAll

Deprecated.

Sets the FileIO.MatchConfiguration.

withMetadata(Map<String, Object>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Sink

Specifies to put the given metadata into each generated file.

withMetadata(Map<String, Object>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Writes to Avro file(s) with the specified metadata.

withMetadata(Map<String, Object>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withMetadata(java.util.Map<java.lang.String, java.lang.Object>) .

withMinBundleSize(long) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Sets the minimum bundle size.

withNoSpilling() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

See WriteFiles.withNoSpilling().

withNumShards(int) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Configures the number of output shards produced overall (when using unwindowed writes) or per-window (when using windowed writes).

withNumShards(int) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withNumShards(int).

withOutputFilenames() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

Specify that output filenames are wanted.

withoutSharding() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Forces a single file as output and empty shard name template.

withoutSharding() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withoutSharding().

withParseFn(SerializableFunction<GenericRecord, X>, Coder<X>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Reads GenericRecord of unspecified schema and maps them to instances of a custom type using the given parseFn and encoded using the given coder.

withSchema(Schema) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Sets the output schema.

withSchema(Schema) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withSchema(org.apache.avro.Schema).

withSchema(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Reads files containing records that conform to the given schema.

withSchema(Schema) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Like AvroSource.withSchema(String).

withSchema(Class<X>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroSource

Reads files containing records of the given class.

withShardNameTemplate(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Uses the given ShardNameTemplate for naming output files.

withShardNameTemplate(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withShardNameTemplate(java.lang.String).

withSize(int) - Static method in class org.apache.beam.sdk.extensions.avro.schemas.utils.AvroUtils.FixedBytesField

Create a AvroUtils.FixedBytesField with the specified size.

withSuffix(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Configures the filename suffix for written files.

withSuffix(String) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withSuffix(java.lang.String).

withSyncInterval(int) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Sets the approximate number of uncompressed bytes to write in each block for the AVRO container format.

withSyncInterval(int) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withSyncInterval(int).

withTempDirectory(ValueProvider<ResourceId>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Set the base directory used to generate temporary files.

withTempDirectory(ResourceId) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Set the base directory used to generate temporary files.

withTempDirectory(ValueProvider<ResourceId>) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withTempDirectory(ValueProvider).

withTempDirectory(ResourceId) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withTempDirectory(ResourceId).

withUsesReshuffle(boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ParseFiles

Specifies if a Reshuffle should run before file reads occur.

withUsesReshuffle(boolean) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.ReadFiles

Specifies if a Reshuffle should run before file reads occur.

withWindowedWrites() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.TypedWrite

Preserves windowing of input elements and writes them to files based on the element's window.

withWindowedWrites() - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Write

See AvroIO.TypedWrite.withWindowedWrites().

write(ElementT) - Method in class org.apache.beam.sdk.extensions.avro.io.AvroIO.Sink

write(Class<T>) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Writes a PCollection to an Avro file (or multiple Avro files matching a sharding pattern).

writeCustomType() - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

A PTransform that writes a PCollection to an avro file (or multiple avro files matching a sharding pattern), with each element of the input collection encoded into its own record of type OutputT.

writeCustomTypeToGenericRecords() - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Similar to AvroIO.writeCustomType(), but specialized for the case where the output type is GenericRecord.

writeGenericRecords(Schema) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Writes Avro records of the specified schema.

writeGenericRecords(String) - Static method in class org.apache.beam.sdk.extensions.avro.io.AvroIO

Writes Avro records of the specified schema.

A C D E F G H I J N O P R S T U V W