Package org.apache.beam.fn.harness.data

Interface BeamFnDataClient

  • All Known Implementing Classes:
    BeamFnDataGrpcClient
    public interface BeamFnDataClient
    The BeamFnDataClient is able to forward inbound elements to a FnDataReceiver and provide a receiver of outbound elements. Callers can register themselves as receivers for inbound elements or can get a handle for a receiver of outbound elements.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator​(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier, boolean collectElementsIfNoFlushes)
      Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane.
      void poisonInstructionId​(java.lang.String instructionId)
      Poisons the instruction id, indicating that future data arriving for it should be discarded.
      void registerReceiver​(java.lang.String instructionId, java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors, org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver)
      Registers a receiver for the provided instruction id.
      void unregisterReceiver​(java.lang.String instructionId, java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors)
      Receivers are only expected to be unregistered when bundle processing has completed successfully.

    • Method Detail

      • registerReceiver

        void registerReceiver​(java.lang.String instructionId,
                      java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors,
                      org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver)
        Registers a receiver for the provided instruction id.

        The receiver is not required to be thread safe.

        Receivers for successfully processed bundles must be unregistered. See unregisterReceiver(java.lang.String, java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor>) for details.

        Any failure during FnDataReceiver.accept(T) will mark the provided instructionId as invalid and will ignore any future data. It is expected that if a bundle fails during processing then the failure will become visible to the BeamFnDataClient during a future FnDataReceiver.accept(T) invocation.

      • unregisterReceiver

        void unregisterReceiver​(java.lang.String instructionId,
                        java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors)
        Receivers are only expected to be unregistered when bundle processing has completed successfully.

        It is expected that if a bundle fails during processing then the failure will become visible to the BeamFnDataClient during a future FnDataReceiver.accept(T) invocation or via a call to poisonInstructionId(java.lang.String).

      • poisonInstructionId

        void poisonInstructionId​(java.lang.String instructionId)
        Poisons the instruction id, indicating that future data arriving for it should be discarded. Unregisters the receiver if was registered.
        Parameters:
        instructionId -

      • createOutboundAggregator

        org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator​(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                                                                  java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier,
                                                                                  boolean collectElementsIfNoFlushes)
        Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane. It is important that BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() is called on the returned BeamFnDataOutboundAggregator at the end of each bundle. If collectElementsIfNoFlushes is set to true, BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() returns the buffered elements instead of sending it through the outbound StreamObserver if there's no previous flush.

        Closing the returned aggregator signals the end of the streams.

        The returned aggregator is not thread safe.