com.continuent.tungsten.replicator.thl.log
Class DiskLog

java.lang.Object
  com.continuent.tungsten.replicator.thl.log.DiskLog

public class DiskLog
extends java.lang.Object

This class implements a multi-thread disk log store.

Version:

1.0

Author:

Stephane Giron

Field Summary

protected int	bufferSize I/O buffer size for log file access.
protected java.lang.String	eventSerializerClass Name of the class used to serialize events.
protected int	logConnectionTimeoutMillis Number of milliseconds before timing out idle log connections.
protected java.lang.String	logDirName Name of the log directory.
protected long	logFileRetainMillis Number of milliseconds to retain old logs.
protected int	logFileSize Log file maximum size in bytes.
protected int	logRotateMillis Special timeout when waiting for a new log file after a rotate log.
protected boolean	readOnly Indicates whether access should be read only or not
protected int	timeoutMillis Wait timeout.
protected WriteLock	writeLock Write lock to prevent log file corruption by concurrent access.

Constructor Summary

DiskLog()
Creates a new log instance.

Method Summary

LogConnection	connect(boolean readonly) Creates a new log connection.
void	delete(LogConnection client, java.lang.Long low, java.lang.Long high) Deletes a portion of the log.
int	fileCount() Returns the count of files in the log.
long	getActiveSeqno() Returns the active sequence number.
Serializer	getEventSerializer() Returns the event serializer instance.
java.lang.String	getFilePrefix() Returns THL log filename prefix.
java.lang.String	getFirstFile() Returns the first index file or null if no such file exists.
long	getFlushIntervalMillis() Return flush interval in milliseconds.
java.lang.String	getIndex() getIndex returns a String representation of the index, built from the configured log directory.
java.lang.String	getLastFile() Returns the last index file or null if no such file exists.
java.lang.String	getLogDir() Returns the log directory.
java.lang.String[]	getLogFileNames() Returns an array of log files.
int	getLogFileSize() Returns the log file size.
int	getLogRotateMillis() Returns the current timeout value for log rotation.
long	getMaxSeqno() Return the maximum sequence number stored in the log.
long	getMinSeqno() Return the minimum sequence number stored in the log.
int	getTimeoutMillis() Returns the current timeout value for blocking reads.
boolean	isDoChecksum() Return true if checksums are enabled.
boolean	isWritable() Returns true if this log is writable.
static java.io.File[]	listLogFiles(java.io.File logDir, java.lang.String logFilePrefix) Returns a sorted list of log files.
void	prepare() Prepare the log for use, which includes ensuring that the log is created automatically on first use and building an index of log file contents.
void	release() Releases the log resources.
void	release(LogConnection connection) Releases a log connection.
void	setActiveSeqno(long activeSeqno) Updates the active sequence number.
void	setBufferSize(int bufferSize) Sets the log buffer size.
void	setDoChecksum(boolean doChecksum) Determines whether to checksum log records.
void	setEventSerializerClass(java.lang.String eventSerializerClass) Sets the event serializer class name.
void	setFlushIntervalMillis(long flushIntervalMillis) Set write flush interval in milliseconds.
void	setFsyncOnFlush(boolean fsyncOnFlush) If set to true, perform an fsync with every flush.
void	setLogConnectionTimeoutMillis(int logConnectionTimeoutMillis) Set the number of milliseconds before timing out idle log connections.
void	setLogDir(java.lang.String path) Sets the directory that will be used to store the log files
void	setLogFileRetainMillis(long logFileRetainMillis) Set the number of milliseconds to retain old log files.
void	setLogFileSize(int size) Sets the log file size.
void	setLogRotateMillis(int logRotateMillis) Sets the timeout value for reading a new file after a log rotation.
void	setMaxSeqno(long seqno) Sets the maximum sequence number stored in the log.
void	setReadOnly(boolean readOnly)
void	setTimeoutMillis(int timeoutMillis) Sets the timeout value for blocking reads.
void	validate() Validates the log to ensure there are no inconsistencies.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logDirName

protected java.lang.String logDirName

Name of the log directory.

eventSerializerClass

protected java.lang.String eventSerializerClass

Name of the class used to serialize events.

logFileSize

protected int logFileSize

Log file maximum size in bytes.

timeoutMillis

protected int timeoutMillis

Wait timeout. This is used for testing to prevent infinite timeouts.

logRotateMillis

protected int logRotateMillis

Special timeout when waiting for a new log file after a rotate log. This timeout will normally only expire when there is a corrupt log.

logFileRetainMillis

protected long logFileRetainMillis

Number of milliseconds to retain old logs.

logConnectionTimeoutMillis

protected int logConnectionTimeoutMillis

Number of milliseconds before timing out idle log connections. Defaults to 8 hours.

bufferSize

protected int bufferSize

I/O buffer size for log file access. Larger is better.

writeLock

protected WriteLock writeLock

Write lock to prevent log file corruption by concurrent access.

readOnly

protected boolean readOnly

Indicates whether access should be read only or not

Constructor Detail

DiskLog

public DiskLog()

Creates a new log instance.

Method Detail

setLogDir

public void setLogDir(java.lang.String path)

Sets the directory that will be used to store the log files

Parameters:

path - directory to be used. Last / is optional.

getLogDir

public java.lang.String getLogDir()

Returns the log directory.

getFilePrefix

public java.lang.String getFilePrefix()

Returns THL log filename prefix.

setLogFileSize

public void setLogFileSize(int size)

Sets the log file size. This is approximate as rotation will occur after storing an event that made the file grow above the given limit.

Parameters:

size - file size

getLogFileSize

public int getLogFileSize()

Returns the log file size.

setDoChecksum

public void setDoChecksum(boolean doChecksum)

Determines whether to checksum log records.

Parameters:

doChecksum - If true use checksums

isDoChecksum

public boolean isDoChecksum()

Return true if checksums are enabled.

setLogFileRetainMillis

public void setLogFileRetainMillis(long logFileRetainMillis)

Set the number of milliseconds to retain old log files.

Parameters:

logFileRetainMillis - If other than 0, logs are retained for this amount of time

setLogConnectionTimeoutMillis

public void setLogConnectionTimeoutMillis(int logConnectionTimeoutMillis)

Set the number of milliseconds before timing out idle log connections.

Parameters:

logConnectionTimeoutMillis - Time in milliseconds

setEventSerializerClass

public void setEventSerializerClass(java.lang.String eventSerializerClass)

Sets the event serializer class name.

getEventSerializer

public Serializer getEventSerializer()

Returns the event serializer instance.

setTimeoutMillis

public void setTimeoutMillis(int timeoutMillis)

Sets the timeout value for blocking reads. This value is only changed when testing.

getTimeoutMillis

public int getTimeoutMillis()

Returns the current timeout value for blocking reads.

setLogRotateMillis

public void setLogRotateMillis(int logRotateMillis)

Sets the timeout value for reading a new file after a log rotation.

getLogRotateMillis

public int getLogRotateMillis()

Returns the current timeout value for log rotation.

setBufferSize

public void setBufferSize(int bufferSize)

Sets the log buffer size.

setFlushIntervalMillis

public void setFlushIntervalMillis(long flushIntervalMillis)

Set write flush interval in milliseconds. 0 means flush on every write. This lowers latency.

getFlushIntervalMillis

public long getFlushIntervalMillis()

Return flush interval in milliseconds.

setFsyncOnFlush

public void setFsyncOnFlush(boolean fsyncOnFlush)

If set to true, perform an fsync with every flush. Warning: fsync is very slow, so you want a long flush interval in this case.

setReadOnly

public void setReadOnly(boolean readOnly)

prepare

public void prepare()
             throws ReplicatorException,
                    java.lang.InterruptedException

Prepare the log for use, which includes ensuring that the log is created automatically on first use and building an index of log file contents.

Throws:

ReplicatorException

java.lang.InterruptedException

release

public void release()
             throws ReplicatorException,
                    java.lang.InterruptedException

Releases the log resources. This should be called after use to ensure log sync task termination.

Throws:

ReplicatorException

java.lang.InterruptedException

setActiveSeqno

public void setActiveSeqno(long activeSeqno)

Updates the active sequence number. Log files will be retained if they contain this number or above.

getActiveSeqno

public long getActiveSeqno()

Returns the active sequence number.

getMaxSeqno

public long getMaxSeqno()

Return the maximum sequence number stored in the log.

setMaxSeqno

public void setMaxSeqno(long seqno)

Sets the maximum sequence number stored in the log.

getMinSeqno

public long getMinSeqno()

Return the minimum sequence number stored in the log.

fileCount

public int fileCount()

Returns the count of files in the log.

getLogFileNames

public java.lang.String[] getLogFileNames()

Returns an array of log files.

isWritable

public boolean isWritable()

Returns true if this log is writable.

connect

public LogConnection connect(boolean readonly)
                      throws ReplicatorException

Creates a new log connection.

Parameters:

readonly - If true, for read only. Only one active connection may write at any given time.

Returns:

A new log client.

Throws:

ReplicatorException

release

public void release(LogConnection connection)

Releases a log connection.

Parameters:

connection - Connection to release

validate

public void validate()
              throws LogConsistencyException

Validates the log to ensure there are no inconsistencies.

Throws:

LogConsistencyException - Thrown if log is not consistent

delete

public void delete(LogConnection client,
                   java.lang.Long low,
                   java.lang.Long high)
            throws ReplicatorException,
                   java.lang.InterruptedException

Deletes a portion of the log. This operation requires a file lock to accomplish.

Parameters:

client - Disk log client used for deletion

low - Sequence number specifying the beginning of the range. Leave null to start from the current beginning of the log.

high - Sequence number specifying the end of the range. Leave null to delete to the end of the log.

Throws:

ReplicatorException - Thrown if delete fails

java.lang.InterruptedException

getIndex

public java.lang.String getIndex()

getIndex returns a String representation of the index, built from the configured log directory.

Returns:

a string representation of the index

getFirstFile

public java.lang.String getFirstFile()

Returns the first index file or null if no such file exists.

getLastFile

public java.lang.String getLastFile()

Returns the last index file or null if no such file exists.

listLogFiles

public static java.io.File[] listLogFiles(java.io.File logDir,
                                          java.lang.String logFilePrefix)

Returns a sorted list of log files.

Parameters:

logDir - Directory containing logs

logFilePrefix - Prefix for log file names

Returns:

Array of logfiles (zero-length if log is not initialized)