Package org.apache.beam.fn.harness

Class ExternalWorkerService

  • All Implemented Interfaces:
    java.lang.AutoCloseable, org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.AsyncService, org.apache.beam.sdk.fn.server.FnService, org.apache.beam.vendor.grpc.v1p69p0.io.grpc.BindableService
    public class ExternalWorkerService
extends org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.BeamFnExternalWorkerPoolImplBase
implements org.apache.beam.sdk.fn.server.FnService
    Implements the BeamFnExternalWorkerPool service by starting a fresh SDK harness for each request.

    • Constructor Summary

      Constructors 
      Constructor Description
      ExternalWorkerService​(org.apache.beam.sdk.options.PipelineOptions options)  

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      static void main​(java.lang.String[] args)
      Worker pool entry point.
      org.apache.beam.sdk.fn.server.GrpcFnServer<ExternalWorkerService> start()  
      void startWorker​(org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerRequest request, org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerResponse> responseObserver)  
      void stopWorker​(org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerRequest request, org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerResponse> responseObserver)  

      • Methods inherited from class org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.BeamFnExternalWorkerPoolImplBase

        bindService

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.apache.beam.vendor.grpc.v1p69p0.io.grpc.BindableService

        bindService

    • Constructor Detail

      • ExternalWorkerService

        public ExternalWorkerService​(org.apache.beam.sdk.options.PipelineOptions options)

    • Method Detail

      • startWorker

        public void startWorker​(org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerRequest request,
                        org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StartWorkerResponse> responseObserver)
        Specified by:
        startWorker in interface org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.AsyncService

      • stopWorker

        public void stopWorker​(org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerRequest request,
                       org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.StopWorkerResponse> responseObserver)
        Specified by:
        stopWorker in interface org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc.AsyncService

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface org.apache.beam.sdk.fn.server.FnService

      • start

        public org.apache.beam.sdk.fn.server.GrpcFnServer<ExternalWorkerService> start()
                                                                        throws java.lang.Exception
        Throws:
        java.lang.Exception

      • main

        public static void main​(java.lang.String[] args)
        Worker pool entry point.

        The worker pool exposes an RPC service that is used with EXTERNAL environment to start and stop the SDK workers.

        The worker pool uses threads for parallelism;

        This entry point is used by the Java SDK container in worker pool mode and expects the following environment variables:

        • PIPELINE_OPTIONS: A serialized form of PipelineOptions. It needs to be known up-front and matches the running job. See PipelineOptions for further details.