org.apache.beam.sdk.extensions.avro.io

Class AvroIO.Write<T>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PDone>

org.apache.beam.sdk.extensions.avro.io.AvroIO.Write<T>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

AvroIO

public static class AvroIO.Write<T>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PDone>

This class is used as the default return value of AvroIO.write(java.lang.Class<T>)

All methods in this class delegate to the appropriate method of AvroIO.TypedWrite. This class exists for backwards compatibility, and will be removed in Beam 3.0.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PDone	expand(org.apache.beam.sdk.values.PCollection<T> input)
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
AvroIO.Write<T>	to(DynamicAvroDestinations<T,?,T> dynamicDestinations) Deprecated. Use FileIO.write() or FileIO.writeDynamic() instead.
AvroIO.Write<T>	to(org.apache.beam.sdk.io.FileBasedSink.FilenamePolicy filenamePolicy) See TypedWrite#to(FilenamePolicy).
AvroIO.Write<T>	to(org.apache.beam.sdk.io.fs.ResourceId outputPrefix) See AvroIO.TypedWrite.to(ResourceId) .
AvroIO.Write<T>	to(java.lang.String outputPrefix) See AvroIO.TypedWrite.to(String).
AvroIO.Write<T>	to(org.apache.beam.sdk.options.ValueProvider<java.lang.String> outputPrefix) See AvroIO.TypedWrite.to(ValueProvider).
AvroIO.Write<T>	toResource(org.apache.beam.sdk.options.ValueProvider<org.apache.beam.sdk.io.fs.ResourceId> outputPrefix) See AvroIO.TypedWrite.to(ResourceId).
AvroIO.Write<T>	withCodec(org.apache.avro.file.CodecFactory codec) See AvroIO.TypedWrite.withCodec(org.apache.avro.file.CodecFactory).
AvroIO.Write<T>	withDatumWriterFactory(AvroSink.DatumWriterFactory<T> datumWriterFactory) See AvroIO.TypedWrite.withDatumWriterFactory(org.apache.beam.sdk.extensions.avro.io.AvroSink.DatumWriterFactory<OutputT>).
AvroIO.Write<T>	withMetadata(java.util.Map<java.lang.String,java.lang.Object> metadata) See AvroIO.TypedWrite.withMetadata(java.util.Map<java.lang.String, java.lang.Object>) .
AvroIO.Write<T>	withNumShards(int numShards) See AvroIO.TypedWrite.withNumShards(int).
<DestinationT> AvroIO.TypedWrite<T,DestinationT,T>	withOutputFilenames() Specify that output filenames are wanted.
AvroIO.Write<T>	withoutSharding() See AvroIO.TypedWrite.withoutSharding().
AvroIO.Write<T>	withSchema(org.apache.avro.Schema schema) See AvroIO.TypedWrite.withSchema(org.apache.avro.Schema).
AvroIO.Write<T>	withShardNameTemplate(java.lang.String shardTemplate) See AvroIO.TypedWrite.withShardNameTemplate(java.lang.String).
AvroIO.Write<T>	withSuffix(java.lang.String filenameSuffix) See AvroIO.TypedWrite.withSuffix(java.lang.String).
AvroIO.Write<T>	withSyncInterval(int syncInterval) See AvroIO.TypedWrite.withSyncInterval(int).
AvroIO.Write<T>	withTempDirectory(org.apache.beam.sdk.io.fs.ResourceId tempDirectory) See AvroIO.TypedWrite.withTempDirectory(ResourceId).
AvroIO.Write<T>	withTempDirectory(org.apache.beam.sdk.options.ValueProvider<org.apache.beam.sdk.io.fs.ResourceId> tempDirectory) See AvroIO.TypedWrite.withTempDirectory(ValueProvider).
AvroIO.Write<T>	withWindowedWrites() See AvroIO.TypedWrite.withWindowedWrites().

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

to

public AvroIO.Write<T> to(java.lang.String outputPrefix)

See AvroIO.TypedWrite.to(String).

to

@Experimental(value=FILESYSTEM)
public AvroIO.Write<T> to(org.apache.beam.sdk.io.fs.ResourceId outputPrefix)

See AvroIO.TypedWrite.to(ResourceId) .

to

public AvroIO.Write<T> to(org.apache.beam.sdk.options.ValueProvider<java.lang.String> outputPrefix)

See AvroIO.TypedWrite.to(ValueProvider).

toResource

@Experimental(value=FILESYSTEM)
public AvroIO.Write<T> toResource(org.apache.beam.sdk.options.ValueProvider<org.apache.beam.sdk.io.fs.ResourceId> outputPrefix)

See AvroIO.TypedWrite.to(ResourceId).

to

public AvroIO.Write<T> to(org.apache.beam.sdk.io.FileBasedSink.FilenamePolicy filenamePolicy)

See TypedWrite#to(FilenamePolicy).

to

@Deprecated
public AvroIO.Write<T> to(DynamicAvroDestinations<T,?,T> dynamicDestinations)

Deprecated. Use FileIO.write() or FileIO.writeDynamic() instead.

See AvroIO.TypedWrite.to(DynamicAvroDestinations).

withSyncInterval

public AvroIO.Write<T> withSyncInterval(int syncInterval)

See AvroIO.TypedWrite.withSyncInterval(int).

withSchema

public AvroIO.Write<T> withSchema(org.apache.avro.Schema schema)

See AvroIO.TypedWrite.withSchema(org.apache.avro.Schema).

withTempDirectory

@Experimental(value=FILESYSTEM)
public AvroIO.Write<T> withTempDirectory(org.apache.beam.sdk.options.ValueProvider<org.apache.beam.sdk.io.fs.ResourceId> tempDirectory)

See AvroIO.TypedWrite.withTempDirectory(ValueProvider).

withTempDirectory

public AvroIO.Write<T> withTempDirectory(org.apache.beam.sdk.io.fs.ResourceId tempDirectory)

See AvroIO.TypedWrite.withTempDirectory(ResourceId).

withShardNameTemplate

public AvroIO.Write<T> withShardNameTemplate(java.lang.String shardTemplate)

See AvroIO.TypedWrite.withShardNameTemplate(java.lang.String).

withSuffix

public AvroIO.Write<T> withSuffix(java.lang.String filenameSuffix)

See AvroIO.TypedWrite.withSuffix(java.lang.String).

withNumShards

public AvroIO.Write<T> withNumShards(int numShards)

See AvroIO.TypedWrite.withNumShards(int).

withoutSharding

public AvroIO.Write<T> withoutSharding()

See AvroIO.TypedWrite.withoutSharding().

withWindowedWrites

public AvroIO.Write<T> withWindowedWrites()

See AvroIO.TypedWrite.withWindowedWrites().

withCodec

public AvroIO.Write<T> withCodec(org.apache.avro.file.CodecFactory codec)

See AvroIO.TypedWrite.withCodec(org.apache.avro.file.CodecFactory).

withDatumWriterFactory

public AvroIO.Write<T> withDatumWriterFactory(AvroSink.DatumWriterFactory<T> datumWriterFactory)

See AvroIO.TypedWrite.withDatumWriterFactory(org.apache.beam.sdk.extensions.avro.io.AvroSink.DatumWriterFactory<OutputT>).

withOutputFilenames

public <DestinationT> AvroIO.TypedWrite<T,DestinationT,T> withOutputFilenames()

Specify that output filenames are wanted.

The nested AvroIO.TypedWritetransform always has access to output filenames, however due to backwards-compatibility concerns, AvroIO.Write cannot return them. This method simply returns the inner AvroIO.TypedWrite transform which has WriteFilesResult as its output type, allowing access to output files.

The supplied DestinationT type must be: the same as that supplied in to(DynamicAvroDestinations) if that method was used, or Void otherwise.

withMetadata

public AvroIO.Write<T> withMetadata(java.util.Map<java.lang.String,java.lang.Object> metadata)

See AvroIO.TypedWrite.withMetadata(java.util.Map<java.lang.String, java.lang.Object>) .

expand

public org.apache.beam.sdk.values.PDone expand(org.apache.beam.sdk.values.PCollection<T> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PDone>

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PDone>