Package org.apache.beam.runners.dataflow.util

Class GcsStager

  • All Implemented Interfaces:
    Stager
    public class GcsStager
extends java.lang.Object
implements Stager
    Utility class for staging files to GCS.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static GcsStager fromOptions​(org.apache.beam.sdk.options.PipelineOptions options)  
      java.util.List<com.google.api.services.dataflow.model.DataflowPackage> stageFiles​(java.util.List<PackageUtil.StagedFile> filesToStage)
      Stages files to DataflowPipelineOptions.getStagingLocation(), suffixed with their md5 hash to avoid collisions.
      com.google.api.services.dataflow.model.DataflowPackage stageToFile​(byte[] bytes, java.lang.String baseName)
      Stage bytes to a target file name wherever this stager stages things.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • fromOptions

        public static GcsStager fromOptions​(org.apache.beam.sdk.options.PipelineOptions options)

      • stageToFile

        public com.google.api.services.dataflow.model.DataflowPackage stageToFile​(byte[] bytes,
                                                                          java.lang.String baseName)
        Description copied from interface: Stager
        Stage bytes to a target file name wherever this stager stages things.
        Specified by:
        stageToFile in interface Stager