public class DBStorage extends StoreFunc
| Modifier and Type | Field and Description |
|---|---|
protected ResourceSchema |
schema |
| Constructor and Description |
|---|
DBStorage(String driver,
String jdbcURL,
String insertQuery) |
DBStorage(String driver,
String jdbcURL,
String user,
String pass,
String insertQuery) |
DBStorage(String driver,
String jdbcURL,
String user,
String pass,
String insertQuery,
String batchSize) |
| Modifier and Type | Method and Description |
|---|---|
void |
checkSchema(ResourceSchema s)
Set the schema for data to be stored.
|
org.apache.hadoop.mapreduce.OutputFormat |
getOutputFormat()
Return the OutputFormat associated with StoreFunc.
|
void |
prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
Initialise the database connection and prepared statement here.
|
void |
putNext(Tuple tuple)
Write the tuple to Database directly here.
|
void |
setStoreFuncUDFContextSignature(String signature)
This method will be called by Pig both in the front end and back end to
pass a unique signature to the
StoreFunc which it can use to store
information in the UDFContext which it needs to store between
various method invocations in the front end and back end. |
void |
setStoreLocation(String location,
org.apache.hadoop.mapreduce.Job job)
Communicate to the storer the location where the data needs to be stored.
|
protected int |
sqlDataTypeFromPigDataType(byte pigDataType) |
Boolean |
supportsParallelWriteToStoreLocation()
DAG execution engines like Tez support optimizing union by writing to
output location in parallel from tasks of different vertices.
|
cleanupOnFailure, cleanupOnFailureImpl, cleanupOnSuccess, relToAbsPathForStoreLocation, warnprotected ResourceSchema schema
public DBStorage(String driver, String jdbcURL, String user, String pass, String insertQuery) throws SQLException
SQLExceptionpublic DBStorage(String driver, String jdbcURL, String user, String pass, String insertQuery, String batchSize) throws RuntimeException
RuntimeExceptionpublic void putNext(Tuple tuple) throws IOException
putNext in interface StoreFuncInterfaceputNext in class StoreFunctuple - the tuple to store.IOException - if an exception occurs during the writeprotected int sqlDataTypeFromPigDataType(byte pigDataType)
public org.apache.hadoop.mapreduce.OutputFormat getOutputFormat()
throws IOException
StoreFuncgetOutputFormat in interface StoreFuncInterfacegetOutputFormat in class StoreFuncOutputFormat associated with StoreFuncIOException - if an exception occurs while constructing the
OutputFormatpublic void prepareToWrite(org.apache.hadoop.mapreduce.RecordWriter writer)
throws IOException
prepareToWrite in interface StoreFuncInterfaceprepareToWrite in class StoreFuncwriter - RecordWriter to use.IOException - if an exception occurs during initializationpublic void setStoreLocation(String location, org.apache.hadoop.mapreduce.Job job) throws IOException
StoreFuncStoreFunc here is the
return value of StoreFunc.relToAbsPathForStoreLocation(String, Path)
This method will be called in the frontend and backend multiple times. Implementations
should bear in mind that this method is called multiple times and should
ensure there are no inconsistent side effects due to the multiple calls.
StoreFunc.checkSchema(ResourceSchema) will be called before any call to
StoreFunc.setStoreLocation(String, Job).setStoreLocation in interface StoreFuncInterfacesetStoreLocation in class StoreFunclocation - Location returned by
StoreFunc.relToAbsPathForStoreLocation(String, Path)job - The Job objectIOException - if the location is not valid.public void setStoreFuncUDFContextSignature(String signature)
StoreFuncStoreFunc which it can use to store
information in the UDFContext which it needs to store between
various method invocations in the front end and back end. This method
will be called before other methods in StoreFunc. This is necessary
because in a Pig Latin script with multiple stores, the different
instances of store functions need to be able to find their (and only their)
data in the UDFContext object. The default implementation is a no-op.setStoreFuncUDFContextSignature in interface StoreFuncInterfacesetStoreFuncUDFContextSignature in class StoreFuncsignature - a unique signature to identify this StoreFuncpublic void checkSchema(ResourceSchema s) throws IOException
StoreFunccheckSchema in interface StoreFuncInterfacecheckSchema in class StoreFuncs - to be checkedIOException - if this schema is not acceptable. It should include
a detailed error message indicating what is wrong with the schema.public Boolean supportsParallelWriteToStoreLocation()
StoreFuncPigConfiguration.PIG_TEZ_OPT_UNION_SUPPORTED_STOREFUNCS and
PigConfiguration.PIG_TEZ_OPT_UNION_UNSUPPORTED_STOREFUNCS
settings to determine if the StoreFunc supports it.supportsParallelWriteToStoreLocation in class StoreFuncCopyright © 2007-2017 The Apache Software Foundation