public class PigInputFormatSpark extends PigInputFormat
PigInputFormat.RecordReaderFactorylog, PIG_INPUT_LIMITS, PIG_INPUT_SIGNATURES, PIG_INPUT_TARGETS, PIG_INPUTS| Constructor and Description |
|---|
PigInputFormatSpark() |
| Modifier and Type | Method and Description |
|---|---|
org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple> |
createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
org.apache.hadoop.mapreduce.TaskAttemptContext context) |
List<org.apache.hadoop.mapreduce.InputSplit> |
getSplits(org.apache.hadoop.mapreduce.JobContext jobcontext)
This is where we have to wrap PigSplits into SparkPigSplits
|
getPigSplitspublic org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,Tuple> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
createRecordReader in class PigInputFormatIOExceptionInterruptedExceptionpublic List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext jobcontext) throws IOException, InterruptedException
getSplits in class PigInputFormatjobcontext - IOExceptionInterruptedExceptionCopyright © 2007-2017 The Apache Software Foundation