Skip to main content

%SYS.Journal.File

Class %SYS.Journal.File Extends %Persistent [ StorageStrategy = JournalFileStorage, System = 4 ]

Journal file related API

Parameters​

READONLY​

Parameter READONLY = 1;

DOMAIN​

Parameter DOMAIN = "%Utility";

Default Localization Domain

Properties​

Name​

Property Name As %SysPath [ ReadOnly ];

Path of the journal file

Chan​

Property Chan As %Integer [ InitialExpression = -1, Internal, ReadOnly ];

End​

Property End As %Integer [ Calculated ];

The end of the last valid record in the journal file (applicable to open file only)

FirstRecord​

Property FirstRecord As %SYS.Journal.Record [ Calculated ];

Object reference to the first record in the journal file (applicable to open file only)

LastRecord​

Property LastRecord As %SYS.Journal.Record [ Calculated ];

Object reference to the last record in the journal file (applicable to open file only)

LastMarker​

Property LastMarker As %SYS.Journal.Marker [ Calculated ];

Object reference to the last marker record in the journal file (applicable to open file only)

ClusterStartTime0​

Property ClusterStartTime0 As %Integer [ Calculated ];

Cluster start time in time(0) format (i.e., # of seconds since the Epoch) or 0 if not a cluster journal file

ClusterStartTime​

Property ClusterStartTime As %String [ Calculated ];

Cluster start time in %TimeStamp format or "" if not a cluster journal file

Methods​

%LoadData​

Method %LoadData(FileName As %String, flag As %Boolean) As %Status [ Private ]

Open​

Method Open(FileName As %String) As %Status

To open a journal file for record retrieval

%OnClose​

Method %OnClose() As %Status [ Private ]

%OnNew​

Method %OnNew() As %Status [ Private ]

Use​

Method Use() As %Integer

To use an open journal file for record retrieval (necessary when you have multiple journal file instances open at the same time and need to switch from one file to another).

EndGet​

Method EndGet() As %Integer

IsValidJournal​

ClassMethod IsValidJournal(FileName As %String) As %Status

TRUE if the given FileName is a journal file

FirstRecordGet​

Method FirstRecordGet() As %SYS.Journal.Record

LastRecordGet​

Method LastRecordGet() As %SYS.Journal.Record

ClusterStartTime0Get​

Method ClusterStartTime0Get() As %Integer

ClusterStartTimeGet​

Method ClusterStartTimeGet() As %String

GetRecordAt​

Method GetRecordAt(Address As %Integer) As %SYS.Journal.Record

Return the object reference to the record at the given journal location (Address)

GetRecordAfter​

Method GetRecordAfter(Address As %Integer) As %SYS.Journal.Record

Return the object reference to the record after the given journal location (Address)

GetRecordBefore​

Method GetRecordBefore(Address As %Integer) As %SYS.Journal.Record

Return the object reference to the record prior to the given journal location (Address)

LastMarkerGet​

Method LastMarkerGet() As %SYS.Journal.Record

GetPrev​

ClassMethod GetPrev(FileName As %String = "", ByRef PrevFileName As %String, InSession As %Boolean, MirOnly As %Boolean = 0, MirrorSetName As %String = "") As %Status

Purpose: Get journal file prior to the given journal file

InSession: whether to limit the search within a journal session (default=NO)

GetNext​

ClassMethod GetNext(FileName As %String, ByRef NextFileName As %String, MirOnly As %Boolean = 0, MirrorSetName As %String = "") As %Status

Purpose: Get journal file following the given journal file

GetMaxSize​

ClassMethod GetMaxSize(FileName As %String) As %Integer

Purpose: Get max size of a journal file

GetGUID​

ClassMethod GetGUID(FileName As %String) As %String [ CodeMode = expression ]

Get the GUID assigned to the given journal file.

GetSystemGUID​

ClassMethod GetSystemGUID(FileName As %String) As %String [ CodeMode = expression ]

Get the GUID assigned to this system in the mirror configuration file.
This is used to help people identify which system created the journal file.
Since this comes from the config file, it's possible that this can change over time but its not supposed to.

GetMirrorGUID​

ClassMethod GetMirrorGUID(FileName As %String) As %String [ CodeMode = expression ]

CheckIntegrity​

ClassMethod CheckIntegrity(ByRef jrnlist As %String, Detail As %Integer = 0, ByRef SrcDirs As %String, quietly As %Integer = 1) As %Status

Purpose: Check integrity of the journal file(s)
jrnlist can be either a file path or, if passed by reference, a number-indexed array of file paths, e.g.:
jrnlist(1)=path1, jrnlist(2)=path2, etc.
The file paths in the array should be in chronicle order, that is, path1 and path2 are assumed to have been created consecutively in the example above. If it is determined that the files in the given array do not meet that criterion, a single value 0 is returned (for "Invalid Parameter") and the top node of the array is set to the number index of the offending path.
Detail: If 1, scan each given journal file forward to locate the last record and verify it is the last record to be expected.
If 2, verify values stored in each record are retrievable.
If SrcDirs is set to 1 and passed by reference, the database directories used in the specified journal file(s) will be stored in SrcDirs array upon return.

CheckIntegrityOne​

ClassMethod CheckIntegrityOne(path As %String, scan As %Integer = 0, jrnend As %Integer, ByRef jrnsrcdir As %String, quietly As %Integer) As %Status [ Internal ]

Purpose: Check integrity of a journal file

Locate​

ClassMethod Locate(ByRef LocatedFiles As %String, ByRef JrnFileList As %String, LocDirList As %String) As %Status [ Internal ]

Locate journal files given in JrnFileList in the directories given in LocDirList

SelectRange​

ClassMethod SelectRange(ByRef SelectedFiles As %String, FirstFile As %String = "", LastFile As %String = "", JrnLogPath As %String = "") As %Status [ Internal ]

Returns OK and a list of local journal files (as SelectedFiles) between FirstFile and LastFile (inclusive). Error if either FirstFile or LastFile is not found.
FirstFile (and LastFile) can be a full path, a file name (yyyymmdd.nnn) with or without a prefix or a GUID representing the file.
If FirstFile="" or not specified, it defaults to the first file in journal log.
If LastFile="" or not specified, it defaults to the last file in journal log.

Match​

ClassMethod Match(File As %String, Path As %String, GUID As %String) As %Status [ Internal ]

File can be a full path, a file name (yyyymmdd.nnn) or a GUID representing the file.
Path is a full path in canonical form on the original platform.
Returns

  • 0: no match;
  • 1: File matches the full path;
  • 2: File matches the file name (including prefix);
  • 3: File matches the file name without prefix;
  • 4: File matches GUID.

ExtractFileName​

ClassMethod ExtractFileName(Path As %SysPath) As %String [ Internal ]

Given the full path of a journal file, extract its file name portion

Restore​

ClassMethod Restore(FileName As %String, Detail As %Integer = 0) As %Status

Purpose: (virtually) restore the journal

RestorePIDList​

ClassMethod RestorePIDList(FileName As %String) As %Status

Purpose: Regenerate the jobid->PID translation table based on records stored in the specified journal file

RequiredForClusterRecovery​

ClassMethod RequiredForClusterRecovery(ByRef FileName As %String, ByRef FileCount As %Integer, SkipSelf As %Boolean = 0) As %Status

Return the path of the oldest journal file on each node that is required for cluster recovery, which includes rolling forward and back transactions.
FileName = # of nodes where journal recovery is required
FileName(csn) = the path of the oldest journal file on node <#csn>
If SkipSelf = 1, skip the node where the method is called (local node is covered by RequiredForRecovery()

RequiredForRecovery​

ClassMethod RequiredForRecovery(ByRef FileName As %String, ByRef FileCount As %Integer) As %Status

Return the name of the oldest journal file required for crash recovery, which includes rolling forward and backward transactions.
FileName="" if journaling is disabled, meaning that none is required, as transactions cannot be rolled forward or backward from a pre-disabling file

RequiredForRollback​

ClassMethod RequiredForRollback(ByRef FileName As %String, ByRef FileCount As %Integer) As %Status

Return the path of the oldest journal file containing open transactions.
FileName: the oldest file containing open transactions or current journal file
FileCount: the file counter value assigned to the file

LastEncrypted​

ClassMethod LastEncrypted(OldestFile As %String = "", csn As %Integer = -1) As %String

Return the path of the last encrypted file no older than OldestFile if given

Purge​

ClassMethod Purge(NDaysOld As %Integer, NBackupsOld As %Integer) As %Status

Purpose: Purge old journal files based on criteria given in number of days or backups, taking care not to purge files required for transaction rollbacks or crash recovery.

Parameters:

  • NDaysOld: journal files must be at least this # of days old to be purged
  • NBackupsOld: journal files must be older than this # of successive successful backups to be purged If both parameters are specified, only one criterion has to be met to qualify a journal file for purging (subject to the restriction about rollback and crash recovery).
    Note: It is the completion date, rather than the creation date, of a file that must meet the date criterion if specified. Thus, a file may not be OK to purge even though the date in its name appears to meet the date criterion.

PurgeAll​

ClassMethod PurgeAll() As %Status

Purge all journal files except those required for transaction rollbacks or crash recovery.
Warning: Post-backup journal files are not necessarily preserved.

PurgeOne​

ClassMethod PurgeOne(FilePath As %String, PurgeTimeH As %String, LogOnly As %Integer = 0) As %Status

Purpose: Purge the journal file with the given name

Warning: This method does NOT check for open transactions before deleting the journal file.

FilePath: the full path of the journal file to purge
PurgeTimeH: Time of purging in $H format
LogOnly: if 1, skip purging and log the purged file only. Default = 0

Archive​

ClassMethod Archive(NSecondsOld As %Integer, NBackupsOld As %Integer) As %Status

Purpose: Archive old journal files based on criteria given in number of seconds or backups, taking care not to archive files required for transaction rollbacks or crash recovery.

Parameters:

  • NSecondsOld: journal files must be at least this # of seconds old to be archived
  • NBackupsOld: journal files must be older than this # of successive successful backups to be archived If both parameters are specified, only one criterion has to be met to qualify a journal file for archiving (subject to the restriction about rollback and crash recovery).
    Note: It is the completion time, rather than the creation time, of a file that must meet the time criterion if specified. Thus, a file may not be OK to archive even though its timestamp appears to meet the time criterion.

ArchiveAll​

ClassMethod ArchiveAll() As %Status

Archive all journal files except those required for transaction rollbacks or crash recovery.
Warning: Post-backup journal files are not necessarily preserved.

ArchiveOne​

ClassMethod ArchiveOne(FilePath As %String) As %Status

Purpose: Archive the journal file with the given name

Warning: This method does NOT check for open transactions before deleting the journal file.

FilePath: the full path of the journal file to archive
ArchiveTimeH: Time of archiving in $H format
LogOnly: if 1, skip archiving and log the archived file only. Default = 0

TranslateReason​

ClassMethod TranslateReason(reason As %Integer = 0) As %String

Translate the numeric reason for the creation of a new journal file to string form

GetReason​

ClassMethod GetReason(File As %SysPath) As %String [ CodeMode = expression ]

Return the reason for the creation of the given journal file (File) in string form

Summary​

ClassMethod Summary(FilePath As %String, ByRef DataFields As %String) As %Status

FileCount: the file counter value assigned to the file

DatabaseListExecute​

ClassMethod DatabaseListExecute(ByRef qHandle As %Binary, FilePath As %String) As %Status [ Internal ]

Returns sfn's and paths/names of databases in a journal file

DatabaseListFetch​

ClassMethod DatabaseListFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ Internal ]

DatabaseListClose​

ClassMethod DatabaseListClose(ByRef qHandle As %Binary) As %Status [ Internal ]

ByTimeReverseOrderExecute​

ClassMethod ByTimeReverseOrderExecute(ByRef qHandle As %Binary, FileName As %String = "") As %Status [ Internal ]

Returns journal file names and file sizes in the reverse order of time (i.e., latest file first)
The file size is the actual size on disk which for compressed files should be smaller than the amount of journal data in the file.

ByTimeReverseOrderFetch​

ClassMethod ByTimeReverseOrderFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ Internal ]

ByTimeReverseOrderClose​

ClassMethod ByTimeReverseOrderClose(ByRef qHandle As %Binary) As %Status [ Internal ]

MirrorByTimeReverseOrderExecute​

ClassMethod MirrorByTimeReverseOrderExecute(ByRef qHandle As %Binary, FileName As %String = "", MirrorSetName As %String = "") As %Status [ Internal ]

Returns journal file names and file sizes in the reverse order of time (i.e., latest file first) for the specified mirror
The file size is the actual size on disk which for compressed files should be smaller than the amount of journal data in the file.

MirrorByTimeReverseOrderFetch​

ClassMethod MirrorByTimeReverseOrderFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ Internal ]

MirrorByTimeReverseOrderClose​

ClassMethod MirrorByTimeReverseOrderClose(ByRef qHandle As %Binary) As %Status [ Internal ]

PurgeableExecute​

ClassMethod PurgeableExecute(ByRef qHandle As %Binary, DaysToKeep As %Integer = "") As %Status [ Internal ]

Returns paths of journal files (along with their mirror names if any) that are eligible for purging based on the default or given criteria.
DaysToKeep: How many days of journal files to keep. Use default (current system setting) if left unspecified. If 0, none is purgeable; if <0, all files are purgeable except those required for transaction rollback and crash recovery.

PurgeableFetch​

ClassMethod PurgeableFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ Internal ]

PurgeableClose​

ClassMethod PurgeableClose(ByRef qHandle As %Binary) As %Status [ Internal ]

ClusterListExecute​

ClassMethod ClusterListExecute(ByRef qHandle As %Binary, FileName As %String = "", TimeOrder As %Boolean = 0, CSN As %Integer = -1) As %Status [ Internal, PublicList = (jrninfo, jrninfor) ]

Information on cluster journal files
By default, files are listed in reverse order of time, unless TimeOrder = 1
If FileName is specified, files are listed until FileName is reached (depend on TimeOrder)
If CSN is given, only files belong to the node are listed.

ClusterListFetch​

ClassMethod ClusterListFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ Internal ]

ClusterListClose​

ClassMethod ClusterListClose(ByRef qHandle As %Binary) As %Status [ Internal ]

SearchExecute​

ClassMethod SearchExecute(ByRef qHandle As %Binary, String As %String, FileName As %String, Offset As %Integer, Backward As %Boolean = 0) As %Status [ Internal ]

Returns addresses of records in a given journal file that match a given string

SearchFetch​

ClassMethod SearchFetch(ByRef qHandle As %Binary, ByRef Row As %List, ByRef AtEnd As %Integer = 0) As %Status [ Internal ]

SearchClose​

ClassMethod SearchClose(ByRef qHandle As %Binary) As %Status [ Internal ]

Compress​

ClassMethod Compress(infile As %String, outfile As %String, enginename As %String = "", chunksz As %Integer = 0) As %Status

Compress a journal file

  • engine is the name of one of the supported compression engines (eg. lz4, zstd or zlib)
  • chunksz is in MB (range=1-16) and specifies how much uncompressed journal data to pack into one compressed chunk. Larger chunks increase compression efficiency but requires larger buffers and reading more data from disk to access a journal file record. The engine and chunksz parameters use the default system values if omitted.

If compression fails a partially created output file is deleted.
The output file must not exist to start with or an error is returned.
In order to compress an encrypted file the encryption key must be active and the output file will be encrypted.

Caution: Be careful with file names. If the intention is to create a compressed version of a journal file with the system's naming scheme by appending a 'z' to the name, specify a temporary output file and then rename/move the resulting file if the operation is succesful to the target name. It is best to avoid having incomplete files with a file name the system might look for. Also take care to ensure the output file has the proper filesystem level protection.

Decompress​

ClassMethod Decompress(infile As %String, outfile As %String) As %Status

Decompress a journal file

  • The output file must not exist or an error is returned
  • If decompression fails, the output file will be deleted
  • To decompress an encrypted file the encryption key must be active and the output file will be encrypted. Caution: Be careful with file names and locations. If the intention is to create a uncompressed version of a journal file with the system's naming scheme by removing the 'z' from the name it is best to specify a temporary file name and then rename/move the resulting file if the operation is succesful to the desired target name. When opening a journal file the system first tries to open the uncompressed version. If it opens a file which is being generated it may think that the file is smaller than it actually is (the size will be determined when it is opened) and may return incorrect results. Also take care to ensure that the output file has the proper filesystem level protection.