Package org.apache.beam.runners.fnexecution.data

Class GrpcDataService

  • All Implemented Interfaces:
    java.lang.AutoCloseable, org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.AsyncService, FnDataService, org.apache.beam.sdk.fn.server.FnService, org.apache.beam.vendor.grpc.v1p69p0.io.grpc.BindableService
    public class GrpcDataService
extends org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.BeamFnDataImplBase
implements org.apache.beam.sdk.fn.server.FnService, FnDataService
    A FnDataService implemented via gRPC.

    This service allows for multiple clients to transmit BeamFnApi.Elements messages.

    This service transmits all outgoing BeamFnApi.Elements messages to the first client that connects.

    • Constructor Summary

      Constructors 
      Constructor Description
      GrpcDataService()
      Deprecated.
      This constructor is for migrating Dataflow purpose only.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      static GrpcDataService create​(org.apache.beam.sdk.options.PipelineOptions options, java.util.concurrent.ExecutorService executor, org.apache.beam.sdk.fn.stream.OutboundObserverFactory outboundObserverFactory)  
      org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator​(java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier, boolean collectElementsIfNoFlushes)
      Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane.
      org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> data​(org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundElementObserver)  
      void registerReceiver​(java.lang.String instructionId, org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)
      Registers a receiver for the provided instruction id.
      void unregisterReceiver​(java.lang.String instructionId)
      Receivers are only expected to be unregistered when bundle processing has completed successfully.

      • Methods inherited from class org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.BeamFnDataImplBase

        bindService

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.apache.beam.vendor.grpc.v1p69p0.io.grpc.BindableService

        bindService

    • Constructor Detail

      • GrpcDataService

        @Deprecated
public GrpcDataService()
        Deprecated.
        This constructor is for migrating Dataflow purpose only.

    • Method Detail

      • create

        public static GrpcDataService create​(org.apache.beam.sdk.options.PipelineOptions options,
                                     java.util.concurrent.ExecutorService executor,
                                     org.apache.beam.sdk.fn.stream.OutboundObserverFactory outboundObserverFactory)

      • data

        public org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> data​(org.apache.beam.vendor.grpc.v1p69p0.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundElementObserver)
        Specified by:
        data in interface org.apache.beam.model.fnexecution.v1.BeamFnDataGrpc.AsyncService

      • close

        public void close()
           throws java.lang.Exception
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface org.apache.beam.sdk.fn.server.FnService
        Throws:
        java.lang.Exception

      • registerReceiver

        public void registerReceiver​(java.lang.String instructionId,
                             org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)
        Description copied from interface: FnDataService
        Registers a receiver for the provided instruction id.

        The receiver is not required to be thread safe.

        Receivers for successfully processed bundles must be unregistered. See FnDataService.unregisterReceiver(java.lang.String) for details.

        Any failure during FnDataReceiver.accept(T) will mark the provided instructionId as invalid and will ignore any future data. It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

        Specified by:
        registerReceiver in interface FnDataService

      • unregisterReceiver

        public void unregisterReceiver​(java.lang.String instructionId)
        Description copied from interface: FnDataService
        Receivers are only expected to be unregistered when bundle processing has completed successfully.

        It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

        Specified by:
        unregisterReceiver in interface FnDataService

      • createOutboundAggregator

        public org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator​(java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier,
                                                                                         boolean collectElementsIfNoFlushes)
        Description copied from interface: FnDataService
        Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane. It is important that BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() is called on the returned BeamFnDataOutboundAggregator at the end of each bundle. If collectElementsIfNoFlushes is set to true, BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() returns the buffered elements instead of sending it through the outbound StreamObserver if there's no previous flush.

        Closing the returned aggregator signals the end of the streams.

        The returned aggregator is not thread safe.

        Specified by:
        createOutboundAggregator in interface FnDataService