Package org.apache.parquet.column.page

Interface PageWriter

All Superinterfaces:

AutoCloseable

public interface PageWriter
extends AutoCloseable

a writer for all the pages of a given column chunk

Method Summary

Modifier and Type	Method	Description
long	allocatedSize()
default void	close()
long	getMemSize()
String	memUsageString(String prefix)
void	writeDictionaryPage(DictionaryPage dictionaryPage)	writes a dictionary page
void	writePage(BytesInput bytesInput, int valueCount, int rowCount, Statistics<?> statistics, Encoding rlEncoding, Encoding dlEncoding, Encoding valuesEncoding)	writes a single page
default void	writePage(BytesInput bytesInput, int valueCount, int rowCount, Statistics<?> statistics, SizeStatistics sizeStatistics, Encoding rlEncoding, Encoding dlEncoding, Encoding valuesEncoding)	Deprecated.
default void	writePage(BytesInput bytesInput, int valueCount, int rowCount, Statistics<?> statistics, SizeStatistics sizeStatistics, GeospatialStatistics geospatialStatistics, Encoding rlEncoding, Encoding dlEncoding, Encoding valuesEncoding)	writes a single page
void	writePage(BytesInput bytesInput, int valueCount, Statistics<?> statistics, Encoding rlEncoding, Encoding dlEncoding, Encoding valuesEncoding)	Deprecated. will be removed in 2.0.0.
void	writePageV2(int rowCount, int nullCount, int valueCount, BytesInput repetitionLevels, BytesInput definitionLevels, Encoding dataEncoding, BytesInput data, Statistics<?> statistics)	writes a single page in the new format
default void	writePageV2(int rowCount, int nullCount, int valueCount, BytesInput repetitionLevels, BytesInput definitionLevels, Encoding dataEncoding, BytesInput data, Statistics<?> statistics, SizeStatistics sizeStatistics)	Deprecated.
default void	writePageV2(int rowCount, int nullCount, int valueCount, BytesInput repetitionLevels, BytesInput definitionLevels, Encoding dataEncoding, BytesInput data, Statistics<?> statistics, SizeStatistics sizeStatistics, GeospatialStatistics geospatialStatistics)	writes a single page in the new format

Method Detail

writePage

@Deprecated
void writePage(BytesInput bytesInput,
               int valueCount,
               Statistics<?> statistics,
               Encoding rlEncoding,
               Encoding dlEncoding,
               Encoding valuesEncoding)
        throws IOException

Deprecated.

will be removed in 2.0.0. This method does not support writing column indexes; Use writePage(BytesInput, int, int, Statistics, Encoding, Encoding, Encoding) instead

writes a single page

Parameters:

bytesInput - the bytes for the page

valueCount - the number of values in that page

statistics - the statistics for that page

rlEncoding - repetition level encoding

dlEncoding - definition level encoding

valuesEncoding - values encoding

Throws:

IOException - if there is an exception while writing page data

writePage

void writePage(BytesInput bytesInput,
               int valueCount,
               int rowCount,
               Statistics<?> statistics,
               Encoding rlEncoding,
               Encoding dlEncoding,
               Encoding valuesEncoding)
        throws IOException

writes a single page

Parameters:

bytesInput - the bytes for the page

valueCount - the number of values in that page

rowCount - the number of rows in that page

statistics - the statistics for that page

rlEncoding - repetition level encoding

dlEncoding - definition level encoding

valuesEncoding - values encoding

Throws:

IOException

writePage

@Deprecated
default void writePage(BytesInput bytesInput,
                       int valueCount,
                       int rowCount,
                       Statistics<?> statistics,
                       SizeStatistics sizeStatistics,
                       Encoding rlEncoding,
                       Encoding dlEncoding,
                       Encoding valuesEncoding)
                throws IOException

Deprecated.

writes a single page

Parameters:

bytesInput - the bytes for the page

valueCount - the number of values in that page

rowCount - the number of rows in that page

statistics - the statistics for that page

sizeStatistics - the size statistics for that page

rlEncoding - repetition level encoding

dlEncoding - definition level encoding

valuesEncoding - values encoding

Throws:

IOException

writePage

default void writePage(BytesInput bytesInput,
                       int valueCount,
                       int rowCount,
                       Statistics<?> statistics,
                       SizeStatistics sizeStatistics,
                       GeospatialStatistics geospatialStatistics,
                       Encoding rlEncoding,
                       Encoding dlEncoding,
                       Encoding valuesEncoding)
                throws IOException

writes a single page

Parameters:

bytesInput - the bytes for the page

valueCount - the number of values in that page

rowCount - the number of rows in that page

statistics - the statistics for that page

sizeStatistics - the size statistics for that page

geospatialStatistics - the geospatial statistics for that page

rlEncoding - repetition level encoding

dlEncoding - definition level encoding

valuesEncoding - values encoding

Throws:

IOException

writePageV2

void writePageV2(int rowCount,
                 int nullCount,
                 int valueCount,
                 BytesInput repetitionLevels,
                 BytesInput definitionLevels,
                 Encoding dataEncoding,
                 BytesInput data,
                 Statistics<?> statistics)
          throws IOException

writes a single page in the new format

Parameters:

rowCount - the number of rows in this page

nullCount - the number of null values (out of valueCount)

valueCount - the number of values in that page (there could be multiple values per row for repeated fields)

repetitionLevels - the repetition levels encoded in RLE without any size header

definitionLevels - the definition levels encoded in RLE without any size header

dataEncoding - the encoding for the data

data - the data encoded with dataEncoding

statistics - optional stats for this page

Throws:

IOException - if there is an exception while writing page data

writePageV2

@Deprecated
default void writePageV2(int rowCount,
                         int nullCount,
                         int valueCount,
                         BytesInput repetitionLevels,
                         BytesInput definitionLevels,
                         Encoding dataEncoding,
                         BytesInput data,
                         Statistics<?> statistics,
                         SizeStatistics sizeStatistics)
                  throws IOException

Deprecated.

writes a single page in the new format

Parameters:

rowCount - the number of rows in this page

nullCount - the number of null values (out of valueCount)

valueCount - the number of values in that page (there could be multiple values per row for repeated fields)

repetitionLevels - the repetition levels encoded in RLE without any size header

definitionLevels - the definition levels encoded in RLE without any size header

dataEncoding - the encoding for the data

data - the data encoded with dataEncoding

statistics - optional stats for this page

sizeStatistics - optional size stats for this page

Throws:

IOException - if there is an exception while writing page data

writePageV2

default void writePageV2(int rowCount,
                         int nullCount,
                         int valueCount,
                         BytesInput repetitionLevels,
                         BytesInput definitionLevels,
                         Encoding dataEncoding,
                         BytesInput data,
                         Statistics<?> statistics,
                         SizeStatistics sizeStatistics,
                         GeospatialStatistics geospatialStatistics)
                  throws IOException

writes a single page in the new format

Parameters:

rowCount - the number of rows in this page

nullCount - the number of null values (out of valueCount)

valueCount - the number of values in that page (there could be multiple values per row for repeated fields)

repetitionLevels - the repetition levels encoded in RLE without any size header

definitionLevels - the definition levels encoded in RLE without any size header

dataEncoding - the encoding for the data

data - the data encoded with dataEncoding

statistics - optional stats for this page

sizeStatistics - optional size stats for this page

Throws:

IOException - if there is an exception while writing page data

getMemSize

long getMemSize()

Returns:

the current size used in the memory buffer for that column chunk

allocatedSize

long allocatedSize()

Returns:

the allocated size for the buffer ( > getMemSize() )

writeDictionaryPage

void writeDictionaryPage(DictionaryPage dictionaryPage)
                  throws IOException

writes a dictionary page

Parameters:

dictionaryPage - the dictionary page containing the dictionary data

Throws:

IOException - if there was an exception while writing

memUsageString

String memUsageString(String prefix)

Parameters:

prefix - a prefix header to add at every line

Returns:

a string presenting a summary of how memory is used

close

default void close()

Specified by:

close in interface AutoCloseable