Package org.apache.beam.runners.dataflow.util

Interface Stager

  • All Known Implementing Classes:
    GcsStager
    public interface Stager
    Interface for staging files needed for running a Dataflow pipeline.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      java.util.List<com.google.api.services.dataflow.model.DataflowPackage> stageFiles​(java.util.List<PackageUtil.StagedFile> filesToStage)
      Stage files and return a list of packages DataflowPackage objects describing th actual location at which each file was staged.
      com.google.api.services.dataflow.model.DataflowPackage stageToFile​(byte[] bytes, java.lang.String baseName)
      Stage bytes to a target file name wherever this stager stages things.

    • Method Detail

      • stageFiles

        java.util.List<com.google.api.services.dataflow.model.DataflowPackage> stageFiles​(java.util.List<PackageUtil.StagedFile> filesToStage)
        Stage files and return a list of packages DataflowPackage objects describing th actual location at which each file was staged.

        The mechanism for staging is owned by the implementation. The only requirement is that the location specified in the returned DataflowPackage should, in fact, contain the contents of the staged file.

      • stageToFile

        com.google.api.services.dataflow.model.DataflowPackage stageToFile​(byte[] bytes,
                                                                   java.lang.String baseName)
        Stage bytes to a target file name wherever this stager stages things.