com.continuent.tungsten.replicator.applier
Class JdbcApplier

java.lang.Object
  com.continuent.tungsten.replicator.applier.JdbcApplier

All Implemented Interfaces:

RawApplier, ReplicatorPlugin

Direct Known Subclasses:

DrizzleApplier, MySQLApplier, OracleApplier, PostgreSQLApplier

public class JdbcApplier
extends java.lang.Object
implements RawApplier

Implements a DBMS implementation-independent applier. DBMS-specific features must be subclassed. This applier can be used directly by specifying the DBMS driver and full JDBC URL.

Version:

1.0

Author:

Teemu Ollakka

Field Summary

protected long	commitCount
protected CommitSeqnoTable	commitSeqnoTable
protected Database	conn
protected java.lang.String	consistencySelect
protected java.lang.String	consistencyTable
protected java.util.HashMap<java.lang.String,java.lang.String>	currentOptions
protected java.lang.String	currentSchema
protected long	currentTimestamp
protected java.lang.String	driver
protected long	eventCount
protected HeartbeatTable	heartbeatTable
protected java.util.regex.Pattern	ignoreSessionPattern
protected java.lang.String	ignoreSessionVars
protected java.lang.String	lastSessionId
protected int	maxSQLLogLength Maximum length of SQL string to log in case of an error.
protected java.lang.String	metadataSchema
protected java.lang.String	password
protected ReplicatorRuntime	runtime
protected java.sql.Statement	statement
protected int	taskId
protected java.lang.String	url
protected java.lang.String	user

Constructor Summary

JdbcApplier()

Method Summary

protected Column	addColumn(java.sql.ResultSet rs, java.lang.String columnName) Returns a new column definition.
void	apply(DBMSEvent event, ReplDBMSHeader header, boolean doCommit, boolean doRollback) Apply the proffered event to the replication target.
protected void	applyLoadDataLocal(LoadDataFileQuery data, java.io.File temporaryFile) applyLoadDataLocal executes the LoadDataFileQuery.
protected void	applyOneRowChangePrepared(OneRowChange oneRowChange)
protected void	applyRowChangeData(RowChangeData data, java.util.List<ReplOption> options)
protected void	applyRowIdData(RowIdData data)
protected boolean	applySessionVariables(java.util.List<ReplOption> options) applyOptionsToStatement adds to the batch queries used to change the connection options, if needed and if possible (if the database support such a feature)
protected void	applySetTimestamp(java.lang.Long timestamp) applySetTimestamp adds to the batch the query used to change the server timestamp, if needed and if possible (if the database support such a feature)
protected void	applyStatementData(StatementData data)
protected void	applyUseSchema(java.lang.String schema) applySetUseSchema adds to the batch the query used to change the current schema where queries should be executed, if needed and if possible (if the database support such a feature)
protected int	bindColumnValues(java.sql.PreparedStatement prepStatement, java.util.ArrayList<OneRowChange.ColumnVal> values, int startBindLoc, java.util.ArrayList<OneRowChange.ColumnSpec> specs, boolean skipNulls)
void	commit() Commits current open transaction to ensure data applied up to current point are durable.
void	configure(PluginContext context) Complete plug-in configuration.
protected int	fillColumnNames(OneRowChange data) Queries database for column names of a table that OneRowChange is affecting.
Database	getDatabase()
ReplDBMSHeader	getLastEvent() Return header information corresponding to last committed event.
protected void	logFailedStatementSQL(java.lang.String sql)
protected void	logFailedStatementSQL(java.lang.String sql, java.sql.SQLException ex) Logs SQL into error log stream.
protected boolean	needNewSQLStatement(int row, java.util.ArrayList<java.util.ArrayList<OneRowChange.ColumnVal>> keyValues, java.util.ArrayList<OneRowChange.ColumnSpec> keySpecs, java.util.ArrayList<java.util.ArrayList<OneRowChange.ColumnVal>> colValues, java.util.ArrayList<OneRowChange.ColumnSpec> colSpecs)
void	prepare(PluginContext context) Prepare plug-in for use.
protected void	printColumnSpec(java.lang.StringBuffer stmt, java.util.ArrayList<OneRowChange.ColumnSpec> cols, java.util.ArrayList<OneRowChange.ColumnVal> keyValues, java.util.ArrayList<OneRowChange.ColumnVal> colValues, com.continuent.tungsten.replicator.applier.JdbcApplier.PrintMode mode, java.lang.String separator)
void	release(PluginContext context) Release all resources used by plug-in.
void	rollback() Rolls back any current work.
void	setDriver(java.lang.String driver)
void	setGetColumnMetadataFromDB(boolean getColumnInformationFromDB)
void	setIgnoreSessionVars(java.lang.String ignoreSessionVars)
protected void	setObject(java.sql.PreparedStatement prepStatement, int bindLoc, OneRowChange.ColumnVal value, OneRowChange.ColumnSpec columnSpec)
void	setPassword(java.lang.String password)
void	setTaskId(int id) Sets the ID of the task using this raw applier.
void	setUrl(java.lang.String url)
void	setUser(java.lang.String user)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

taskId

protected int taskId

runtime

protected ReplicatorRuntime runtime

driver

protected java.lang.String driver

url

protected java.lang.String url

user

protected java.lang.String user

password

protected java.lang.String password

ignoreSessionVars

protected java.lang.String ignoreSessionVars

metadataSchema

protected java.lang.String metadataSchema

consistencyTable

protected java.lang.String consistencyTable

consistencySelect

protected java.lang.String consistencySelect

conn

protected Database conn

statement

protected java.sql.Statement statement

ignoreSessionPattern

protected java.util.regex.Pattern ignoreSessionPattern

commitSeqnoTable

protected CommitSeqnoTable commitSeqnoTable

heartbeatTable

protected HeartbeatTable heartbeatTable

lastSessionId

protected java.lang.String lastSessionId

currentSchema

protected java.lang.String currentSchema

currentTimestamp

protected long currentTimestamp

eventCount

protected long eventCount

commitCount

protected long commitCount

maxSQLLogLength

protected int maxSQLLogLength

Maximum length of SQL string to log in case of an error. This is needed because some statements may be very large. TODO: make this configurable via replicator.properties

currentOptions

protected java.util.HashMap<java.lang.String,java.lang.String> currentOptions

Constructor Detail

JdbcApplier

public JdbcApplier()

Method Detail

setTaskId

public void setTaskId(int id)

Sets the ID of the task using this raw applier.

Specified by:

setTaskId in interface RawApplier

Parameters:

id - Task ID

See Also:

RawApplier.setTaskId(int)

setDriver

public void setDriver(java.lang.String driver)

getDatabase

public Database getDatabase()

setUrl

public void setUrl(java.lang.String url)

setUser

public void setUser(java.lang.String user)

setPassword

public void setPassword(java.lang.String password)

setIgnoreSessionVars

public void setIgnoreSessionVars(java.lang.String ignoreSessionVars)

setGetColumnMetadataFromDB

public void setGetColumnMetadataFromDB(boolean getColumnInformationFromDB)

printColumnSpec

protected void printColumnSpec(java.lang.StringBuffer stmt,
                               java.util.ArrayList<OneRowChange.ColumnSpec> cols,
                               java.util.ArrayList<OneRowChange.ColumnVal> keyValues,
                               java.util.ArrayList<OneRowChange.ColumnVal> colValues,
                               com.continuent.tungsten.replicator.applier.JdbcApplier.PrintMode mode,
                               java.lang.String separator)

Parameters:

keyValues - Is used to identify NULL values, in which case, if the mode is ASSIGNMENT, "x IS ?" is constructed instead of "x = ?".

fillColumnNames

protected int fillColumnNames(OneRowChange data)
                       throws java.sql.SQLException,
                              ApplierException

Queries database for column names of a table that OneRowChange is affecting. Fills in column names and key names for the given OneRowChange.

Parameters:

data -

Returns:

Number of columns that a table has. Zero, if no columns were retrieved (table does not exist or has no columns).

Throws:

java.sql.SQLException

ApplierException

addColumn

protected Column addColumn(java.sql.ResultSet rs,
                           java.lang.String columnName)
                    throws java.sql.SQLException

Returns a new column definition.

Parameters:

rs - Metadata resultset

columnName - Name of the column to be added

Returns:

the column definition

Throws:

java.sql.SQLException - if an error occurs

bindColumnValues

protected int bindColumnValues(java.sql.PreparedStatement prepStatement,
                               java.util.ArrayList<OneRowChange.ColumnVal> values,
                               int startBindLoc,
                               java.util.ArrayList<OneRowChange.ColumnSpec> specs,
                               boolean skipNulls)
                        throws java.sql.SQLException

Throws:

java.sql.SQLException

setObject

protected void setObject(java.sql.PreparedStatement prepStatement,
                         int bindLoc,
                         OneRowChange.ColumnVal value,
                         OneRowChange.ColumnSpec columnSpec)
                  throws java.sql.SQLException

Throws:

java.sql.SQLException

applyRowIdData

protected void applyRowIdData(RowIdData data)
                       throws ReplicatorException

Throws:

ReplicatorException

applyStatementData

protected void applyStatementData(StatementData data)
                           throws ReplicatorException

Throws:

ReplicatorException

applySetTimestamp

protected void applySetTimestamp(java.lang.Long timestamp)
                          throws java.sql.SQLException

applySetTimestamp adds to the batch the query used to change the server timestamp, if needed and if possible (if the database support such a feature)

Parameters:

timestamp - the timestamp to be used

Throws:

java.sql.SQLException - if an error occurs

applyUseSchema

protected void applyUseSchema(java.lang.String schema)
                       throws java.sql.SQLException

applySetUseSchema adds to the batch the query used to change the current schema where queries should be executed, if needed and if possible (if the database support such a feature)

Parameters:

schema - the schema to be used

Throws:

java.sql.SQLException - if an error occurs

applySessionVariables

protected boolean applySessionVariables(java.util.List<ReplOption> options)
                                 throws java.sql.SQLException

applyOptionsToStatement adds to the batch queries used to change the connection options, if needed and if possible (if the database support such a feature)

Parameters:

options -

Returns:

true if any option changed

Throws:

java.sql.SQLException

logFailedStatementSQL

protected void logFailedStatementSQL(java.lang.String sql)

logFailedStatementSQL

protected void logFailedStatementSQL(java.lang.String sql,
                                     java.sql.SQLException ex)

Logs SQL into error log stream. Trims the message if it exceeds maxSQLLogLength.
In addition, extracts and logs next exception of the SQLException, if available. This extends logging detail that is provided by general exception logging mechanism.

Parameters:

sql - the sql statement to be logged

See Also:

maxSQLLogLength

needNewSQLStatement

protected boolean needNewSQLStatement(int row,
                                      java.util.ArrayList<java.util.ArrayList<OneRowChange.ColumnVal>> keyValues,
                                      java.util.ArrayList<OneRowChange.ColumnSpec> keySpecs,
                                      java.util.ArrayList<java.util.ArrayList<OneRowChange.ColumnVal>> colValues,
                                      java.util.ArrayList<OneRowChange.ColumnSpec> colSpecs)

applyOneRowChangePrepared

protected void applyOneRowChangePrepared(OneRowChange oneRowChange)
                                  throws ReplicatorException

Throws:

ReplicatorException

applyRowChangeData

protected void applyRowChangeData(RowChangeData data,
                                  java.util.List<ReplOption> options)
                           throws ReplicatorException

Throws:

ReplicatorException

apply

public void apply(DBMSEvent event,
                  ReplDBMSHeader header,
                  boolean doCommit,
                  boolean doRollback)
           throws ReplicatorException,
                  ConsistencyException

Apply the proffered event to the replication target.

Specified by:

apply in interface RawApplier

Parameters:

event - Event to be applied. If a DBMSEmptyEvent, just mark the apply position.

header - Header data corresponding to event

doCommit - Boolean flag indicating whether this is the last part of multi-part event

doRollback - Boolean flag indicating whether this transaction should rollback

Throws:

ReplicatorException - Thrown if applier processing fails

ConsistencyException - Thrown if the applier detects that a consistency check has failed

See Also:

RawApplier.apply(com.continuent.tungsten.replicator.event.DBMSEvent, com.continuent.tungsten.replicator.event.ReplDBMSHeader, boolean, boolean)

commit

public void commit()
            throws ReplicatorException,
                   java.lang.InterruptedException

Commits current open transaction to ensure data applied up to current point are durable.

Specified by:

commit in interface RawApplier

Throws:

ReplicatorException - Thrown if applier processing fails

java.lang.InterruptedException - Thrown if the applier is interrupted

See Also:

RawApplier.commit()

rollback

public void rollback()
              throws java.lang.InterruptedException

Rolls back any current work.

Specified by:

rollback in interface RawApplier

Throws:

java.lang.InterruptedException - Thrown if the applier is interrupted.

See Also:

RawApplier.rollback()

applyLoadDataLocal

protected void applyLoadDataLocal(LoadDataFileQuery data,
                                  java.io.File temporaryFile)
                           throws ReplicatorException

applyLoadDataLocal executes the LoadDataFileQuery. This method is most likely to be database specific. See for example :

Parameters:

data - The LoadDataFileQuery containing the query that has to be executed

temporaryFile - The file containing data to be loaded

Throws:

ReplicatorException - if an error occurs

See Also:

MySQLApplier.applyLoadDataLocal(LoadDataFileQuery, File)

getLastEvent

public ReplDBMSHeader getLastEvent()
                            throws ReplicatorException

Return header information corresponding to last committed event.

Specified by:

getLastEvent in interface RawApplier

Returns:

Header data for last committed event.

Throws:

ReplicatorException - Thrown if getting sequence number fails

See Also:

RawApplier.getLastEvent()

prepare

public void prepare(PluginContext context)
             throws ReplicatorException

Prepare plug-in for use. This method is assumed to allocate all required resources. It is called before the plug-in performs any operations.

Specified by:

prepare in interface ReplicatorPlugin

Throws:

ReplicatorException - Thrown if resource allocation fails

See Also:

ReplicatorPlugin.prepare(com.continuent.tungsten.replicator.plugin.PluginContext)

configure

public void configure(PluginContext context)
               throws ReplicatorException

Complete plug-in configuration. This is called after setters are invoked at the time that the replicator goes through configuration.

Specified by:

configure in interface ReplicatorPlugin

Throws:

ReplicatorException - Thrown if configuration is incomplete or fails

See Also:

ReplicatorPlugin.configure(com.continuent.tungsten.replicator.plugin.PluginContext)

release

public void release(PluginContext context)
             throws ReplicatorException

Release all resources used by plug-in. This is called before the plug-in is deallocated.

Specified by:

release in interface ReplicatorPlugin

Throws:

ReplicatorException - Thrown if resource deallocation fails

See Also:

ReplicatorPlugin.release(com.continuent.tungsten.replicator.plugin.PluginContext)