com.continuent.tungsten.replicator.applier.batch
Class SimpleBatchApplier

java.lang.Object
  com.continuent.tungsten.replicator.applier.batch.SimpleBatchApplier

All Implemented Interfaces:

RawApplier, ReplicatorPlugin

Direct Known Subclasses:

InfiniDBBatchApplier, VerticaStreamBatchApplier

public class SimpleBatchApplier
extends java.lang.Object
implements RawApplier

Implements an applier that bulk loads data into a SQL database via CSV files.

Version:

1.0

Author:

Robert Hodges

Field Summary

protected java.lang.String	charset
protected boolean	cleanUpFiles
protected CommitSeqnoTable	commitSeqnoTable
protected Database	conn
protected java.lang.String	consistencySelect
protected java.lang.String	consistencyTable
protected java.text.SimpleDateFormat	dateFormatter
static java.lang.String	DELETE Denotes a delete operation.
protected java.lang.String	driver
protected HeartbeatTable	heartbeatTable
protected java.util.regex.Pattern	ignoreSessionPattern
static java.lang.String	INSERT Denotes an insert operation.
protected java.lang.String	metadataSchema
protected LoadMismatch	onLoadMismatch
protected java.lang.String	password
protected boolean	showCommands
protected java.lang.String	stageColumnPrefix
protected java.lang.String	stageDirectory
protected java.lang.String	stageMergeScript
protected java.lang.String	stagePkeyColumn
protected java.lang.String	stageSchemaPrefix
protected java.lang.String	stageTablePrefix
protected java.lang.String	startupScript
protected java.sql.Statement	statement
protected java.lang.String	timezone
protected java.lang.String	url
protected java.lang.String	user

Constructor Summary

SimpleBatchApplier()

Method Summary

void	apply(DBMSEvent event, ReplDBMSHeader header, boolean doCommit, boolean doRollback) Applies row updates using a batch loading scheme.
void	assertNotNull(java.lang.String property, java.lang.String name)
protected void	clearStageTable(CsvInfo info)
void	commit() Commits current open transaction to ensure data applied up to current point are durable.
void	configure(PluginContext context) Complete plug-in configuration.
protected java.lang.String	getCsvString(OneRowChange.ColumnVal columnVal, OneRowChange.ColumnSpec columnSpec) Converts a column value to a suitable String for CSV loading.
ReplDBMSHeader	getLastEvent() Return header information corresponding to last committed event.
static SqlScriptGenerator	initializeGenerator(java.lang.String script)
void	prepare(PluginContext context) Prepare plug-in for use.
void	release(PluginContext context) Release all resources used by plug-in.
void	rollback() Rolls back any current work.
void	setCharset(java.lang.String charset) Sets the platform charset name.
void	setCleanUpFiles(boolean cleanUpFiles) If true, clean up files automatically.
void	setDriver(java.lang.String driver)
void	setOnLoadMismatch(java.lang.String onLoadMismatchString) Sets the proper handling of a load mismatch.
void	setPassword(java.lang.String password)
void	setShowCommands(boolean showCommands) If true, show commands in the log when loading batches.
void	setStageColumnPrefix(java.lang.String stageColumnPrefix) Set the prefix for staging table columns.
void	setStageDirectory(java.lang.String stageDirectory) Set the name of the staging directory.
void	setStageMergeScript(java.lang.String stageMergeScript) Set the name of the merge script.
void	setStagePkeyColumn(java.lang.String stagePkeyColumn) Set the default name of the staging table primary key.
void	setStageSchemaPrefix(java.lang.String stageSchemaPrefix) Set the schema prefix for staging tables.
void	setStageTablePrefix(java.lang.String stageTablePrefix)
void	setStartupScript(java.lang.String startupScript) Set the name of the connect script.
void	setTaskId(int id) Sets the ID of the task using this raw applier.
void	setTimezone(java.lang.String timezone) Sets the timezone.
void	setUrl(java.lang.String url)
void	setUser(java.lang.String user)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INSERT

public static java.lang.String INSERT

Denotes an insert operation.

DELETE

public static java.lang.String DELETE

Denotes a delete operation.

driver

protected java.lang.String driver

url

protected java.lang.String url

user

protected java.lang.String user

password

protected java.lang.String password

stageDirectory

protected java.lang.String stageDirectory

startupScript

protected java.lang.String startupScript

stageMergeScript

protected java.lang.String stageMergeScript

stageSchemaPrefix

protected java.lang.String stageSchemaPrefix

stageTablePrefix

protected java.lang.String stageTablePrefix

stageColumnPrefix

protected java.lang.String stageColumnPrefix

stagePkeyColumn

protected java.lang.String stagePkeyColumn

cleanUpFiles

protected boolean cleanUpFiles

charset

protected java.lang.String charset

timezone

protected java.lang.String timezone

onLoadMismatch

protected LoadMismatch onLoadMismatch

showCommands

protected boolean showCommands

metadataSchema

protected java.lang.String metadataSchema

consistencyTable

protected java.lang.String consistencyTable

consistencySelect

protected java.lang.String consistencySelect

conn

protected Database conn

statement

protected java.sql.Statement statement

ignoreSessionPattern

protected java.util.regex.Pattern ignoreSessionPattern

commitSeqnoTable

protected CommitSeqnoTable commitSeqnoTable

heartbeatTable

protected HeartbeatTable heartbeatTable

dateFormatter

protected volatile java.text.SimpleDateFormat dateFormatter

Constructor Detail

SimpleBatchApplier

public SimpleBatchApplier()

Method Detail

setDriver

public void setDriver(java.lang.String driver)

setUrl

public void setUrl(java.lang.String url)

setUser

public void setUser(java.lang.String user)

setPassword

public void setPassword(java.lang.String password)

setStartupScript

public void setStartupScript(java.lang.String startupScript)

Set the name of the connect script.

setStageMergeScript

public void setStageMergeScript(java.lang.String stageMergeScript)

Set the name of the merge script.

setStageSchemaPrefix

public void setStageSchemaPrefix(java.lang.String stageSchemaPrefix)

Set the schema prefix for staging tables.

setStageTablePrefix

public void setStageTablePrefix(java.lang.String stageTablePrefix)

setStagePkeyColumn

public void setStagePkeyColumn(java.lang.String stagePkeyColumn)

Set the default name of the staging table primary key.

setStageColumnPrefix

public void setStageColumnPrefix(java.lang.String stageColumnPrefix)

Set the prefix for staging table columns.

setStageDirectory

public void setStageDirectory(java.lang.String stageDirectory)

Set the name of the staging directory.

setCleanUpFiles

public void setCleanUpFiles(boolean cleanUpFiles)

If true, clean up files automatically.

setCharset

public void setCharset(java.lang.String charset)

Sets the platform charset name.

setTimezone

public void setTimezone(java.lang.String timezone)

Sets the timezone.

setOnLoadMismatch

public void setOnLoadMismatch(java.lang.String onLoadMismatchString)

Sets the proper handling of a load mismatch.

setShowCommands

public void setShowCommands(boolean showCommands)

If true, show commands in the log when loading batches.

apply

public void apply(DBMSEvent event,
                  ReplDBMSHeader header,
                  boolean doCommit,
                  boolean doRollback)
           throws ReplicatorException,
                  ConsistencyException,
                  java.lang.InterruptedException

Applies row updates using a batch loading scheme. Statements are discarded. Apply the proffered event to the replication target.

Specified by:

apply in interface RawApplier

Parameters:

event - Event to be applied. If a DBMSEmptyEvent, just mark the apply position.

header - Header data corresponding to event

doCommit - Boolean flag indicating whether this is the last part of multi-part event

doRollback - Boolean flag indicating whether this transaction should rollback

Throws:

ReplicatorException - Thrown if applier processing fails

ConsistencyException - Thrown if the applier detects that a consistency check has failed

java.lang.InterruptedException - Thrown if the applier is interrupted

See Also:

RawApplier.apply(com.continuent.tungsten.replicator.event.DBMSEvent, com.continuent.tungsten.replicator.event.ReplDBMSHeader, boolean, boolean)

commit

public void commit()
            throws ReplicatorException,
                   java.lang.InterruptedException

Commits current open transaction to ensure data applied up to current point are durable.

Specified by:

commit in interface RawApplier

Throws:

ReplicatorException - Thrown if applier processing fails

java.lang.InterruptedException - Thrown if the applier is interrupted

See Also:

RawApplier.commit()

getLastEvent

public ReplDBMSHeader getLastEvent()
                            throws ReplicatorException,
                                   java.lang.InterruptedException

Return header information corresponding to last committed event.

Specified by:

getLastEvent in interface RawApplier

Returns:

Header data for last committed event.

Throws:

ReplicatorException - Thrown if getting sequence number fails

java.lang.InterruptedException - Thrown if the applier is interrupted

See Also:

RawApplier.getLastEvent()

rollback

public void rollback()
              throws java.lang.InterruptedException

Rolls back any current work.

Specified by:

rollback in interface RawApplier

Throws:

java.lang.InterruptedException - Thrown if the applier is interrupted.

See Also:

RawApplier.rollback()

setTaskId

public void setTaskId(int id)

Sets the ID of the task using this raw applier.

Specified by:

setTaskId in interface RawApplier

Parameters:

id - Task ID

See Also:

RawApplier.setTaskId(int)

configure

public void configure(PluginContext context)
               throws ReplicatorException,
                      java.lang.InterruptedException

Complete plug-in configuration. This is called after setters are invoked at the time that the replicator goes through configuration.

Specified by:

configure in interface ReplicatorPlugin

Throws:

ReplicatorException - Thrown if configuration is incomplete or fails

java.lang.InterruptedException

See Also:

ReplicatorPlugin.configure(com.continuent.tungsten.replicator.plugin.PluginContext)

assertNotNull

public void assertNotNull(java.lang.String property,
                          java.lang.String name)
                   throws ReplicatorException

Throws:

ReplicatorException

prepare

public void prepare(PluginContext context)
             throws ReplicatorException,
                    java.lang.InterruptedException

Prepare plug-in for use. This method is assumed to allocate all required resources. It is called before the plug-in performs any operations.

Specified by:

prepare in interface ReplicatorPlugin

Throws:

ReplicatorException - Thrown if resource allocation fails

java.lang.InterruptedException

See Also:

ReplicatorPlugin.prepare(com.continuent.tungsten.replicator.plugin.PluginContext)

initializeGenerator

public static SqlScriptGenerator initializeGenerator(java.lang.String script)
                                              throws ReplicatorException

Throws:

ReplicatorException

release

public void release(PluginContext context)
             throws ReplicatorException,
                    java.lang.InterruptedException

Release all resources used by plug-in. This is called before the plug-in is deallocated.

Specified by:

release in interface ReplicatorPlugin

Throws:

ReplicatorException - Thrown if resource deallocation fails

java.lang.InterruptedException

See Also:

ReplicatorPlugin.release(com.continuent.tungsten.replicator.plugin.PluginContext)

clearStageTable

protected void clearStageTable(CsvInfo info)
                        throws ReplicatorException

Throws:

ReplicatorException

getCsvString

protected java.lang.String getCsvString(OneRowChange.ColumnVal columnVal,
                                        OneRowChange.ColumnSpec columnSpec)
                                 throws CsvException

Converts a column value to a suitable String for CSV loading. This can be overloaded for particular DBMS types.

Parameters:

columnVal - Column value

columnSpec - Column metadata

Returns:

String for loading

Throws:

CsvException