Package org.apache.beam.sdk.extensions.jackson

Class ParseJsons<OutputT>

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    public class ParseJsons<OutputT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<java.lang.String>,​org.apache.beam.sdk.values.PCollection<OutputT>>
    PTransform for parsing JSON Strings. Parse PCollection of Strings in JSON format into a PCollection of objects represented by those JSON Strings using Jackson.
    See Also:
    Serialized Form

    • Field Summary

      • Fields inherited from class org.apache.beam.sdk.transforms.PTransform

        annotations, displayData, name, resourceHints

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      <NewFailureT>
      ParseJsons.ParseJsonsWithFailures<NewFailureT>      		 exceptionsInto​(org.apache.beam.sdk.values.TypeDescriptor<NewFailureT> failureTypeDescriptor)
      Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, with the given type descriptor used for the failure collection but the exception handler yet to be specified using ParseJsons.ParseJsonsWithFailures.exceptionsVia(ProcessFunction).
      ParseJsons.ParseJsonsWithFailures<org.apache.beam.sdk.values.KV<java.lang.String,​java.util.Map<java.lang.String,​java.lang.String>>> exceptionsVia()
      Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, passing the raised exception instance and the input element being processed through the default exception handler ParseJsons.DefaultExceptionAsMapHandler and emitting the result to a failure collection.
      <FailureT> ParseJsons.ParseJsonsWithFailures<FailureT> exceptionsVia​(org.apache.beam.sdk.transforms.InferableFunction<org.apache.beam.sdk.transforms.WithFailures.ExceptionElement<java.lang.String>,​FailureT> exceptionHandler)
      Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, passing the raised exception instance and the input element being processed through the given exceptionHandler and emitting the result to a failure collection.
      org.apache.beam.sdk.values.PCollection<OutputT> expand​(org.apache.beam.sdk.values.PCollection<java.lang.String> input)  
      static <OutputT> ParseJsons<OutputT> of​(java.lang.Class<? extends OutputT> outputClass)
      Creates a ParseJsons PTransform that will parse JSON Strings into a PCollection<OutputT> using a Jackson ObjectMapper.
      ParseJsons<OutputT> withMapper​(com.fasterxml.jackson.databind.ObjectMapper mapper)
      Use custom Jackson ObjectMapper instead of the default one.

      • Methods inherited from class org.apache.beam.sdk.transforms.PTransform

        addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Method Detail

      • of

        public static <OutputT> ParseJsons<OutputT> of​(java.lang.Class<? extends OutputT> outputClass)
        Creates a ParseJsons PTransform that will parse JSON Strings into a PCollection<OutputT> using a Jackson ObjectMapper.

      • withMapper

        public ParseJsons<OutputT> withMapper​(com.fasterxml.jackson.databind.ObjectMapper mapper)
        Use custom Jackson ObjectMapper instead of the default one.

      • exceptionsVia

        public <FailureT> ParseJsons.ParseJsonsWithFailures<FailureT> exceptionsVia​(org.apache.beam.sdk.transforms.InferableFunction<org.apache.beam.sdk.transforms.WithFailures.ExceptionElement<java.lang.String>,​FailureT> exceptionHandler)
        Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, passing the raised exception instance and the input element being processed through the given exceptionHandler and emitting the result to a failure collection.

        See WithFailures documentation for usage patterns of the returned WithFailures.Result.

        Example usage: 

        
 WithFailures.Result<PCollection<MyPojo>, KV<String, Map<String, String>>> result =
     json.apply(
         ParseJsons.of(MyPojo.class)
             .exceptionsVia(new WithFailures.ExceptionAsMapHandler<String>() {}));

 PCollection<MyPojo> output = result.output(); // valid POJOs
 PCollection<KV<String, Map<String, String>>> failures = result.failures();

      • exceptionsVia

        public ParseJsons.ParseJsonsWithFailures<org.apache.beam.sdk.values.KV<java.lang.String,​java.util.Map<java.lang.String,​java.lang.String>>> exceptionsVia()
        Returns a new ParseJsons.ParseJsonsWithFailures transform that catches exceptions raised while parsing elements, passing the raised exception instance and the input element being processed through the default exception handler ParseJsons.DefaultExceptionAsMapHandler and emitting the result to a failure collection.

        See ParseJsons.DefaultExceptionAsMapHandler for more details about default handler behavior.

        See WithFailures documentation for usage patterns of the returned WithFailures.Result.

        Example usage: 

        
 WithFailures.Result<PCollection<MyPojo>, KV<String, Map<String, String>>> result =
     json.apply(
         ParseJsons.of(MyPojo.class)
             .exceptionsVia());

 PCollection<MyPojo> output = result.output(); // valid POJOs
 PCollection<KV<String, Map<String, String>>> failures = result.failures();

      • expand

        public org.apache.beam.sdk.values.PCollection<OutputT> expand​(org.apache.beam.sdk.values.PCollection<java.lang.String> input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<java.lang.String>,​org.apache.beam.sdk.values.PCollection<OutputT>>