software.amazon.awscdk.services.kinesisfirehose

Interface CfnDeliveryStream.OrcSerDeProperty

All Superinterfaces:

software.amazon.jsii.JsiiSerializable

All Known Implementing Classes:

CfnDeliveryStream.OrcSerDeProperty.Jsii$Proxy

Enclosing class:

CfnDeliveryStream

@Stability(value=Stable)
public static interface CfnDeliveryStream.OrcSerDeProperty
extends software.amazon.jsii.JsiiSerializable

A serializer to use for converting data to the ORC format before storing it in Amazon S3.

For more information, see Apache ORC .

Example:

 // The code below shows an example of how to instantiate this type.
 // The values are placeholders you should change.
 import software.amazon.awscdk.services.kinesisfirehose.*;
 OrcSerDeProperty orcSerDeProperty = OrcSerDeProperty.builder()
         .blockSizeBytes(123)
         .bloomFilterColumns(List.of("bloomFilterColumns"))
         .bloomFilterFalsePositiveProbability(123)
         .compression("compression")
         .dictionaryKeyThreshold(123)
         .enablePadding(false)
         .formatVersion("formatVersion")
         .paddingTolerance(123)
         .rowIndexStride(123)
         .stripeSizeBytes(123)
         .build();
 

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	CfnDeliveryStream.OrcSerDeProperty.Builder A builder for CfnDeliveryStream.OrcSerDeProperty
static class	CfnDeliveryStream.OrcSerDeProperty.Jsii$Proxy An implementation for CfnDeliveryStream.OrcSerDeProperty

Method Summary

Modifier and Type	Method and Description
static CfnDeliveryStream.OrcSerDeProperty.Builder	builder()
default Number	getBlockSizeBytes() The Hadoop Distributed File System (HDFS) block size.
default List<String>	getBloomFilterColumns() The column names for which you want Kinesis Data Firehose to create bloom filters.
default Number	getBloomFilterFalsePositiveProbability() The Bloom filter false positive probability (FPP).
default String	getCompression() The compression code to use over data blocks.
default Number	getDictionaryKeyThreshold() Represents the fraction of the total number of non-null rows.
default Object	getEnablePadding() Set this to `true` to indicate that you want stripes to be padded to the HDFS block boundaries.
default String	getFormatVersion() The version of the file to write.
default Number	getPaddingTolerance() A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size.
default Number	getRowIndexStride() The number of rows between index entries.
default Number	getStripeSizeBytes() The number of bytes in each stripe.

Methods inherited from interface software.amazon.jsii.JsiiSerializable

$jsii$toJson

Method Detail

getBlockSizeBytes

@Stability(value=Stable)
 @Nullable
default Number getBlockSizeBytes()

The Hadoop Distributed File System (HDFS) block size.

This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

getBloomFilterColumns

@Stability(value=Stable)
 @Nullable
default List<String> getBloomFilterColumns()

The column names for which you want Kinesis Data Firehose to create bloom filters.

The default is null .

getBloomFilterFalsePositiveProbability

@Stability(value=Stable)
 @Nullable
default Number getBloomFilterFalsePositiveProbability()

The Bloom filter false positive probability (FPP).

The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

getCompression

@Stability(value=Stable)
 @Nullable
default String getCompression()

The compression code to use over data blocks.

The default is SNAPPY .

getDictionaryKeyThreshold

@Stability(value=Stable)
 @Nullable
default Number getDictionaryKeyThreshold()

Represents the fraction of the total number of non-null rows.

To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

getEnablePadding

@Stability(value=Stable)
 @Nullable
default Object getEnablePadding()

Set this to `true` to indicate that you want stripes to be padded to the HDFS block boundaries.

This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false .

getFormatVersion

@Stability(value=Stable)
 @Nullable
default String getFormatVersion()

The version of the file to write.

The possible values are V0_11 and V0_12 . The default is V0_12 .

getPaddingTolerance

@Stability(value=Stable)
 @Nullable
default Number getPaddingTolerance()

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size.

The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when EnablePadding is false .

getRowIndexStride

@Stability(value=Stable)
 @Nullable
default Number getRowIndexStride()

The number of rows between index entries.

The default is 10,000 and the minimum is 1,000.

getStripeSizeBytes

@Stability(value=Stable)
 @Nullable
default Number getStripeSizeBytes()

The number of bytes in each stripe.

The default is 64 MiB and the minimum is 8 MiB.

builder

@Stability(value=Stable)
static CfnDeliveryStream.OrcSerDeProperty.Builder builder()

Returns:

a CfnDeliveryStream.OrcSerDeProperty.Builder of CfnDeliveryStream.OrcSerDeProperty