Package org.apache.beam.fn.harness.debug

Class OutputSampler<T>

  • Type Parameters:
    T - the element type of the PCollection.
    public class OutputSampler<T>
extends java.lang.Object
    This class holds samples for a single PCollection until queried by the parent DataSampler. This class is meant to hold only a limited number of elements in memory. So old values are constantly being overridden in a circular buffer.

    • Constructor Summary

      Constructors 
      Constructor Description
      OutputSampler​(org.apache.beam.sdk.coders.Coder<?> coder, int maxElements, int sampleEveryN, boolean onlySampleExceptions)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void exception​(ElementSample<T> elementSample, java.lang.Exception e, java.lang.String ptransformId, java.lang.String processBundleId)
      Samples an exceptional element to be later queried.
      ElementSample<T> sample​(org.apache.beam.sdk.values.WindowedValue<T> element)
      Samples every sampleEveryNth element or if it is part of the first 10 in the (local) PCollection.
      java.util.List<org.apache.beam.model.fnexecution.v1.BeamFnApi.SampledElement> samples()
      Clears samples at end of call.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • OutputSampler

        public OutputSampler​(org.apache.beam.sdk.coders.Coder<?> coder,
                     int maxElements,
                     int sampleEveryN,
                     boolean onlySampleExceptions)

    • Method Detail

      • sample

        public ElementSample<T> sample​(org.apache.beam.sdk.values.WindowedValue<T> element)
        Samples every sampleEveryNth element or if it is part of the first 10 in the (local) PCollection.

        This method is invoked in parallel by multiple bundle processing threads and in parallel to any samples() being returned to a thread handling a sample request.

        Parameters:
        element - the element to sample.

      • exception

        public void exception​(ElementSample<T> elementSample,
                      java.lang.Exception e,
                      java.lang.String ptransformId,
                      java.lang.String processBundleId)
        Samples an exceptional element to be later queried. The enforces that only one exception occurs per bundle.
        Parameters:
        elementSample - the sampled element to add an exception to.
        e - the exception.
        ptransformId - the source of the exception.
        processBundleId - the failing bundle.

      • samples

        public java.util.List<org.apache.beam.model.fnexecution.v1.BeamFnApi.SampledElement> samples()
                                                                                      throws java.io.IOException
        Clears samples at end of call. This is to help mitigate memory use.

        This method is invoked by a thread handling a data sampling request in parallel to any calls to sample(org.apache.beam.sdk.values.WindowedValue<T>).

        Returns:
        samples taken since last call.
        Throws:
        java.io.IOException