org.apache.druid.storage.hdfs

Class HdfsDataSegmentPusher

java.lang.Object

org.apache.druid.storage.hdfs.HdfsDataSegmentPusher

All Implemented Interfaces:

DataSegmentPusher

public class HdfsDataSegmentPusher
extends Object
implements DataSegmentPusher

Field Summary

Fields inherited from interface org.apache.druid.segment.loading.DataSegmentPusher

JOINER

Constructor Summary

Constructors

Constructor and Description
HdfsDataSegmentPusher(HdfsDataSegmentPusherConfig config, org.apache.hadoop.conf.Configuration hadoopConfig, com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

Method Summary

Modifier and Type	Method and Description
String	getPathForHadoop()
String	getPathForHadoop(String dataSource) Deprecated.
String	getStorageDir(DataSegment segment, boolean useUniquePath) Due to https://issues.apache.org/jira/browse/HDFS-13 ":" are not allowed in path names.
String	makeIndexPathName(DataSegment dataSegment, String indexName)
Map<String,Object>	makeLoadSpec(URI finalIndexZipFilePath)
DataSegment	push(File inDir, DataSegment segment, boolean useUniquePath)
DataSegment	pushToPath(File inDir, DataSegment segment, String storageDirSuffix)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.druid.segment.loading.DataSegmentPusher

generateUniquePath, getAllowedPropertyPrefixesForHadoop, getDefaultStorageDir, getDefaultStorageDirWithExistingUniquePath, getStorageDir

Constructor Detail

HdfsDataSegmentPusher

@Inject
public HdfsDataSegmentPusher(HdfsDataSegmentPusherConfig config,
                                     org.apache.hadoop.conf.Configuration hadoopConfig,
                                     com.fasterxml.jackson.databind.ObjectMapper jsonMapper)

Method Detail

getPathForHadoop

@Deprecated
public String getPathForHadoop(String dataSource)

Deprecated.

Specified by:

getPathForHadoop in interface DataSegmentPusher

getPathForHadoop

public String getPathForHadoop()

Specified by:

getPathForHadoop in interface DataSegmentPusher

push

public DataSegment push(File inDir,
                        DataSegment segment,
                        boolean useUniquePath)
                 throws IOException

Specified by:

push in interface DataSegmentPusher

Throws:

IOException

pushToPath

public DataSegment pushToPath(File inDir,
                              DataSegment segment,
                              String storageDirSuffix)
                       throws IOException

Specified by:

pushToPath in interface DataSegmentPusher

Throws:

IOException

makeLoadSpec

public Map<String,Object> makeLoadSpec(URI finalIndexZipFilePath)

Specified by:

makeLoadSpec in interface DataSegmentPusher

getStorageDir

public String getStorageDir(DataSegment segment,
                            boolean useUniquePath)

Due to https://issues.apache.org/jira/browse/HDFS-13 ":" are not allowed in path names. So we format paths differently for HDFS.

Specified by:

getStorageDir in interface DataSegmentPusher

makeIndexPathName

public String makeIndexPathName(DataSegment dataSegment,
                                String indexName)

Specified by:

makeIndexPathName in interface DataSegmentPusher