Package org.apache.beam.runners.fnexecution.data

Interface FnDataService

  • All Known Implementing Classes:
    GrpcDataService
    public interface FnDataService
    The FnDataService is able to forward inbound elements to a consumer and is also a consumer of outbound elements. Callers can register themselves as consumers for inbound elements or can get a handle for a consumer for outbound elements.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator​(java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier, boolean collectElementsIfNoFlushes)
      Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane.
      void registerReceiver​(java.lang.String instructionId, org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)
      Registers a receiver for the provided instruction id.
      void unregisterReceiver​(java.lang.String instructionId)
      Receivers are only expected to be unregistered when bundle processing has completed successfully.

    • Method Detail

      • registerReceiver

        void registerReceiver​(java.lang.String instructionId,
                      org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> observer)
        Registers a receiver for the provided instruction id.

        The receiver is not required to be thread safe.

        Receivers for successfully processed bundles must be unregistered. See unregisterReceiver(java.lang.String) for details.

        Any failure during FnDataReceiver.accept(T) will mark the provided instructionId as invalid and will ignore any future data. It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

      • unregisterReceiver

        void unregisterReceiver​(java.lang.String instructionId)
        Receivers are only expected to be unregistered when bundle processing has completed successfully.

        It is expected that if a bundle fails during processing then the failure will become visible to the FnDataService during a future FnDataReceiver.accept(T) invocation.

      • createOutboundAggregator

        org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator​(java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier,
                                                                                  boolean collectElementsIfNoFlushes)
        Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane. It is important that BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() is called on the returned BeamFnDataOutboundAggregator at the end of each bundle. If collectElementsIfNoFlushes is set to true, BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() returns the buffered elements instead of sending it through the outbound StreamObserver if there's no previous flush.

        Closing the returned aggregator signals the end of the streams.

        The returned aggregator is not thread safe.