iSharkFly-Open
/
fis-gtm
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
fis-gtm/sr_port/mupip.hlp

2181 lines
86 KiB
Plaintext
Raw Normal View History

ENH: Initial import from sourceforge. These source tree was directly imported from http://sourceforge.net/projects/fis-gtm/files/GT.M-x86-Linux-src/V5.4-002B/ by extracting the file: gtm_V54002B_linux_i686_src.tar.gz
2012-02-05 11:35:58 -05:00
1 BACKUP
B[ACKUP]
BACKUP copies blocks from one or more Greystone Technology Database
Structure (GDS) files to a new file or files. BACKUP suspends updates
to all regions specified by the BACKUP command from the time it starts
the first region until it finishes the last region. This ensures that
BACKUP captures a consistent application state. BACKUP does not
suspend processes that only perform retrievals.
The format of the BACKUP command is:
B[ACKUP][-qualifier[...]] region-list[,...] file-spec
By default, BACKUP is -COMPREHENSIVE.
The first argument may specify more than one region of the current
Global Directory in a list separated with commas.
To BACKUP only one region, the file-specification must resolve to a
UNIX file or directory name. To BACKUP several regions, the
file-specification must be a directory specification. If the
file-specification is a directory, MUPIP assigns the backup files the
same name as the file associated with the dynamic segment of each
region. Therefore, the target directory must not contain any of the
regions included in the BACKUP.
2 Qualifiers
-COMPREHENSIVE
-C[OMPREHENSIVE]
Specifies that BACKUP copy the entire file from disk to disk. On
completion, the result is ready for use as a GT.M database. This
option does not support operation to magnetic tape.
BACKUP -COMPREHENSIVE has the following advantages:
o It does not require exclusive access to the file
o It can interlock multiple files simultaneously
The -COMPREHENSIVE qualifier is not compatible with any other
qualifier.
By default, BACKUP operates -COMPREHENSIVE.
-INCREMENTAL
-I[NCREMENTAL]
Specifies that BACKUP include only blocks from the database that
have changed since a prior point specified by the -SINCE or
-TRANSACTION qualifier. MUPIP RESTORE integrates the results of
a BACKUP -INCREMENTAL into a database.
The -INCREMENTAL qualifier is not compatible with the
-COMPREHENSIVE qualifier.
-RECORD
-R[ECORD]
Specifies that the BACKUP utility record this backup as a
reference point for subsequent backups. Each time a BACKUP
specifies -RECORD, that backup replaces the previous recorded
backup as the RECORD reference point for the file.
-SINCE
-S[INCE]=keyword
Specifies that a BACKUP -INCREMENTAL includes blocks changed
since the last specified BACKUP.
-SINCE accepts the keywords:
o C[OMPREHENSIVE] - Backup all changes since the last BACKUP
-COMPREHENSIVE
o I[NCREMENTAL] - Backup all changes since the last BACKUP
-INCREMENTAL
o R[ECORD] - Backup all changes since the last BACKUP -RECORD
The -SINCE qualifier is incompatible with the -COMPREHENSIVE and
-TRANSACTION qualifiers.
By default, BACKUP -INCREMENTAL operates -SINCE=COMPREHENSIVE.
-TRANSACTION
-T[RANSACTION]=transaction-number
Specifies a hexadecimal starting transaction which causes BACKUP
-INCREMENTAL to copy all blocks that have been changed by the
specified and all subsequent transactions. Transaction numbers
appear in a DSE DUMP -FILEHEADER, with a "Current TN" label. If
the transaction number is invalid, BACKUP reports an error and
rejects the command.
BACKUP -INCREMENTAL -TRANSACTION=1 copies all in-use blocks and
has the following advantages:
o It does not require exclusive access to the file
o It can interlock multiple files simultaneously
Different regions do not normally have a single transaction
number that marks a meaningful point. Therefore, a BACKUP
command specifying multiple regions and using the -TRANSACTION
qualifier with arguments other than one (1) is unlikely to
produce desirable results.
The -TRANSACTION qualifier is incompatible with the
-COMPREHENSIVE and -SINCE qualifiers.
1 CREATE
CR[EATE]
CREATE generates database files using the characteristics stored in a
Global Directory by the Global Directory Editor (GDE). CREATE uses the
Global Directory to map a region to a segment and a segment to a file.
Use the MUPIP CREATE command to create a new database or a new copy of
a previously existing file during a database reorganization. If a
database file already exists for the segment, CREATE takes no action.
If a file does not exist, CREATE sets up the file. CREATE also
initializes the database structure (GDS).
The format of the CREATE command is:
CR[EATE] [-R[EGION]=region-name]
The optional -REGION qualifier specifies a single region for which to
create a database file.
By default, CREATE sets up database files for all regions in the
current Global Directory.
2 Qualifiers
-REGION
-R[EGION]=region-name
Specifies a single region for creation of a database file. By
default, CREATE sets up (creates) database files for all regions
in the current Global Directory.
1 EXIT
EXI[T]
EXIT terminates MUPIP and returns control to the point where MUPIP was
invoked. This command is useful when you invoke MUPIP without an
action and wish to leave without performing one, or after using MUPIP
HELP.
The format of the EXIT command is:
EXI[T]
The Exit command does not accept any qualifiers.
1 EXTEND
EXTE[ND]
EXTEND expands a GDS database file. Databases generally extend
automatically, but this command allows you to control the time and
amount of extension.
The format of the EXTEND command is:
EXTE[ND] region-name [-B[LOCKS]=blocks]
The required region-name parameter specifies the name of the region to
expand. EXTEND uses the Global Directory to map the region to the
dynamic segment and the segment to the file.
2 Qualifiers
-BLOCKS
-B[LOCKS]=blocks
Specifies the number of GDS database blocks by which GT.M should
extend the file. GDS files use some blocks for bit maps. EXTEND
adds the specified number of blocks and the bit map blocks
required as overhead. For more information about bit maps, refer
to the "GDS" chapter of the GT.M Administration and Operations
Guide.
EXTEND uses the value in the fileheader as the number of GDS
blocks by which to extend the database file.
1 EXTRACT
EXTR[ACT]
EXTRACT copies specified globals from the current database to a
sequential output file in one of two formats (i.e., GO, or BINARY).
Use EXTRACT to back up specific globals or when extracting data from
the database for use by another system. EXTRACT uses the Global
Directory to determine which database files to use. EXTRACT may
operate concurrently with normal GT.M database access. To ensure that
an EXTRACT reflects a consistent application state, suspend database
updates to all regions involved in the extract with the -FREEZE
qualifier.
The format of the EXTRACT command is:
EXTR[ACT][-qualifier[...]] file-specification
EXTRACT places its output in the file defined by the
file-specification. EXTRACT may output to a UNIX file on any device
that supports such files, including magnetic tapes. Note that magnetic
tapes may have a smaller file maximum size than disks. <CTRL C>
produces a status message from EXTRACT. Entering <CTRL C> twice in
quick succession aborts EXTRACT. An EXTRACT terminated abnormally by
operator action or error produces incomplete output.
2 Qualifiers
-SELECT
-S[ELECT]=global-name-list
Specifies the globals to extract. The "^" in the specification
of the global name is optional. Enclose lowercase global names
in quotes ("").
The global-specification can be:
o A global name, such as MEF
o A range of global names, such as A7:B6
o A list, such as A,B,C
o Global names with the same prefix, such as TMP*
In the first case, EXTRACT selects only global ^MEF. In the
second case, EXTRACT selects all global names between ^A7 and
^B6, inclusive. In the third case, EXTRACT selects globals ^A,
^B, and ^C. In the fourth case, EXTRACT selects all global names
from ^TMP through ^TMPzzzzz.
By default, EXTRACT selects all globals, as if it had the
qualifier -SELECT=*.
-FORMAT
-FO[RMAT]=GO|B[INARY]
Specifies the format of the output file.
The format codes are:
o GO - Global Output format, used for files you want to
transport or archive
o B[INARY] - Binary format, used for database reorganization
or short term backups
-FORMAT=GO stores the data in record pairs. Each global node
produces one record for the key and one for the data. FORMAT=GO
has two header records.
-FORMAT=BINARY only applies for Greystone Technology Database
Structure (GDS) files. EXTRACT -FORMAT=BINARY works much faster
than EXTRACT -FORMAT=GO.
By default, EXTRACT uses -FORMAT=GO.
-FREEZE
-FR[EEZE]
Prevents database updates to all regions of the Global Directory
used by the EXTRACT for the duration of the EXTRACT.
By default, EXTRACT does not freeze regions during operation.
-LABEL
-LA[BEL]=text
Specifies a text string which becomes the first record in the
output file. Enclose labels containing punctuation or lowercase
labels in quotes (""). EXTRACT -FORMAT=BINARY truncates the
label text to 32 characters.
By default, EXTRACT uses the label "GT.M MUPIP EXTRACT".
For a description of the -FORMAT=BINARY header label, refer to
the subsequent section on EXTRACT -FORMAT=BINARY.
-LOG
-[NO]LO[G]
Specifies whether or not to display a message on SYS$OUTPUT for
each global extracted. The message shows the number of global
nodes, the maximum subscript length and maximum data length for
each global.
By default, EXTRACT operates -LOG.
1 Global_dir
MUPIP and the Global Directory
Some of the MUPIP commands require information contained in the Global
Directory. Therefore, a process must have access to a valid Global
Directory before using any MUPIP database service commands other than
JOURNAL, RESTORE, and the -FILE options of INTEG and SET.
The environment variable gtmgbldir specifies the Global Directory.
Define gtmgbldir at the shell level. Individual users define gtmgbldir
in their login or other shell scripts.
Example
$ gtmgbldir=prod.gld
$ export gtmgbldir
1 HELP
H[ELP]
HELP provides online information about MUPIP commands and qualifiers.
The format of the HELP command is:
H[ELP] [options...]
The HELP command does not accept any qualifiers. Enter the MUPIP
command for which you want information at the Topic prompt. Use
<RETURN> or <CTRL Z> to leave the help facility.
1 INTEG
I[NTEG]
The INTEG command performs an integrity check on a GDS database file.
INTEG operates on one or more regions in the current global directory
by suspending concurrent updates to those regions. INTEG of a single
file database without a Global Directory requires exclusive
(stand-alone) access to that file.
Use INTEG at the following times:
o Periodically - to insure ongoing integrity of database(s); frequent
INTEGs help catch integrity problems before they spread through the
database file
o After a crash - to insure the database was not corrupted
o When database errors are reported - to troubleshoot the problem
The format of the INTEG command is:
I[NTEG][-qualifier[...]] file-spec | region-list
The filename directly identifies the GDS file to INTEG. The
region-list identifies one or more regions that in turn identify GDS
files through the current Global Directory.
Always analyze errors reported by INTEG immediately to prevent further
corruption. Greystone strongly recommends fixing the following errors
as soon as they are discovered:
o Blocks incorrectly marked free - these may cause accelerating
damage when processes make updates to any part of the database
region.
o Integrity errors in an index block - these may cause accelerating
damage when processes make updates to that area of the database
region that uses the faulty index.
INTEG -FAST and the "regular" INTEG both report these errors. Other
database errors do not pose the threat of rapidly spreading problems
in GDS files, but if operations continue the errors may cause the
following:
o Invalid application operation due to "missing" data
o Process errors when a database access encounters an error
o Degrading application level integrity as a result of incomplete
update sequences caused by the prior symptoms
You must assess the type of damage, the risk of continued operations,
and the disruption of stopping normal operation for database repair.
For information on analyzing and correcting database errors, refer to
the "Database Integrity" chapter in the GT.M Administration and
Operations Guide.
<CTRL C> aborts INTEG. Because INTEG does most of its reporting at the
end, aborting the process before it completes may not give you all the
information you need.
2 -FAST
-FA[ST]
Specifies that INTEG checks only index blocks. INTEG -FAST does not
check data blocks. INTEG -FAST produces results dramatically faster
than a full INTEG. While INTEG -FAST is not a replacement for a
full INTEG, it very quickly detects the most dangerous structural
errors in a database.
The -FAST qualifier is incompatible with the -TN_RESET qualifier.
By default, INTEG checks all active index and data blocks in the
database.
2 -REGION
-R[EGION]
Specifies that the INTEG parameter identifies one or more regions
rather than a database file.
INTEG -REGION does not require sole access to databases. Instead,
it freezes updates to the database during the check. The
region-list argument may specify more than one region of the
current Global Directory in a list separated with commas. INTEG
-REGION requires the environment variable gtmgbldir to specify a
valid Global Directory. For more information on defining gtmgbldir,
refer to the "Global Directory Editor" chapter of the GT.M
Administration and Operations Guide.
Note: Because a KILL may briefly defer marking the blocks it
releases "free" in the bit maps, INTEG -REGION may report spurious
block incorrectly marked busy errors. Because block incorrectly
marked busy errors are benign, ignore these errors unless INTEG
consistently reports a block as incorrectly marked busy.
The -REGION qualifier is incompatible with the -FILE and -TN_RESET
qualifiers.
By default, INTEG operates -FILE.
2 -FILE
-FI[LE]
Specifies that the parameter to the INTEG command is a filename.
INTEG -FILE requires exclusive (stand-alone) access to a database
file and does not require a Global Directory. Because it has
stand-alone access to the file, INTEG -FILE is able to check
reference counts.
The -FILE qualifier is incompatible with the -REGION qualifier.
By default, INTEG operates -FILE.
2 -TN_RESET
-TN[_RESET]
Instructs an INTEG -FILE to reset the transaction number to one in
every database block currently holding valid data.
Transaction number overflow back to 0 disrupts the integrity of the
database.
The -TN_RESET qualifier is incompatible with the -BLOCK, -FAST,
-REGION and -SUBSCRIPT qualifiers.
By default, INTEG does not modify the block transaction numbers.
2 -SUBSCRIPT
-S[UBSCRIPT]=subscript
Specifies a global or a range of keys to INTEG. Enclose the global
key in quotes ("") and identify a range by separating two
subscripts with a colon (:). -SUBSCRIPT limits map checking to
incorrectly marked free errors.
Use -SUBSCRIPT only if you know the path to the keys in the
subscript and have reason to believe the path is not damaged. If
the path is questionable or known to be damaged, use DSE to find
the block(s) and INTEG -BLOCK.
The -SUBSCRIPT qualifier is incompatible with the -BLOCK and
-TN_RESET qualifiers.
Use -FULL to have INTEG report all global-names covered by a range.
2 Examples
INTEG -SUBSCRIPT= Examples
Example
MUPIP INTEG -SUBSCRIPT="^a" MUMPS.DAT
This INTEGs the global variable ^a in the database file MUMPS.DAT.
Example
MUPIP INTEG-SUBSCRIPT="^a(100)":"^b(""c"")"-reg DEFAULT
This INTEGs all global variables greater than or equal to ^a(100)
and less than ^b("c") in the default region.
Note: To specify a literal in the command string, use double quotes
e.g., ^b(""c"").
2 -BLOCK
-BL[OCK]=block-number
Specifies the block at which to start checking a sub-tree of the
database. -BLOCK limits map checking to incorrectly marked free
errors.
The -BLOCK qualifier is incompatible with the -SUBSCRIPT and
-TN_RESET qualifiers.
2 -KEYRANGES
-[NO]K[EYRANGES]
Specifies whether or not the INTEG report includes key ranges that
identify the data suspected of problems detected by INTEG.
By default, INTEG displays -KEYRANGES.
2 -MAP
-[NO]MAP[=integer]
Specifies the maximum number of incorrectly marked busy errors that
INTEG reports.
-NOMAP removes limits on incorrectly marked busy reporting, i.e.,
INTEG reports all map errors. -NOMAP does not accept assignment of
an argument.
Because incorrectly marked free errors are very dangerous, INTEG
always reports them, and -MAP does not affect them.
An error in an index block prevents INTEG from processing
potentially large areas of the database. A single "primary" error
may cause large numbers of "secondary" incorrectly marked busy
errors. Because "real" or primary incorrectly marked busy errors
only make "empty" blocks unavailable to the system, they are
relatively benign.
By default, INTEG reports a maximum of 10 map errors (-MAP=10).
2 -MAXKEYSIZE
-[NO]MAX[KEYSIZE][=integer]
Specifies the maximum number of key size too large errors that
INTEG reports.
-NOMAXKEYSIZE removes limits on key size reporting, i.e., INTEG
reports all key size too large errors. -NOMAXKEYSIZE does not
accept assignment of an argument.
Key size too large error should only occur after someone uses DSE
CHANGE -FILEHEADER -KEY_MAX_SIZE to reduce the maximum key-size.
By default, INTEG reports a maximum of 10 key size errors
(-NOMAXKEYSIZE=10).
2 -TRANSACTION
-[NO]TR[ANSACTION][=integer]
Specifies the maximum number of block transaction number too large
errors that INTEG reports.
-NOTRANSACTION removes limits on transaction reporting, i.e.,
INTEG reports all transaction number errors. -NOTRANSACTION does
not accept assignment of an argument.
A system crash may generate many block transaction number too large
errors. These errors can cause problems for BACKUP -INCREMENTAL,
but have no effect on the run-time environment. The DSE CHANGE
-FILEHEADER -BLOCKS_FREE= command quickly fixes block transaction
number too large number errors.
By default, INTEG reports a maximum of 10 block transaction errors
(-TRANSACTION=10).
2 -BRIEF
-BR[IEF]
Specifies an INTEG summary report which displays the total number
of directory, index and data blocks. The -BRIEF qualifier is
incompatible with the -FULL qualifier.
By default, INTEG reports are -BRIEF.
2 -FULL
-FU[LL]
Specifies an expanded INTEG report which displays the number of
index and data blocks in the directory tree and in each global tree
as well as the total number of directory, index and data blocks.
The -FULL qualifier is incompatible with the -BRIEF qualifier.
By default, INTEG reports are -BRIEF.
2 -ADJACENCY
-A[DJACENCY]=integer
Specifies the range of blocks within which INTEG considers a block
adjacent to another database block on the same level. The adjacency
report from INTEG gives an approximation of physical data density
which directly affects efficient database access. Use the
-ADJACENCY qualifier to adjust the reporting to reflect
characteristics of your disks, e.g., pick a factor that matches the
number of sectors on a drive.
By default, INTEG uses -ADJACENCY=10.
1 JOURNAL
J[OURNAL]
The MUPIP JOURNAL command analyzes, extracts from, reports on, and
recovers journal files.
Another MUPIP command, SET, turns journaling on and off, identifies
the type of journaling, and sets some database journaling
characteristics.
The format for the JOURNAL command is:
MUPIP J[OURNAL] -qualifier[...] file-specification[,...]
2 Action_qualifiers
Action Qualifiers
-RECOVER
-REC[OVER]
Instructs the JOURNAL command to replay database updates in the
specified journal file into the appropriate database. -RECOVER
initiates the central JOURNAL operation. JOURNAL commands may
specify -RECOVER alone or with other action qualifiers.
-VERIFY
-[NO]V[ERIFY]
Checks a journal file for proper form. JOURNAL commands may
specify -VERIFY alone or with other action qualifiers. JOURNAL
-RECOVER commands implicitly -VERIFY the file(s) on which they
operate. JOURNAL -RECOVER ignores -NOVERIFY for all qualifier
combinations that do not include -FORWARD and -FENCES=NONE.
-EXTRACT
-EX[TRACT][=file-specification]
Specifies that JOURNAL transfer the contents of one or more
journal files to a single output file in a format intended for
processing by a MUMPS program. For a description of -EXTRACT
output record formats, refer to the section on JOURNAL -EXTRACT
output records. JOURNAL commands may specify -EXTRACT alone or
with other action qualifiers.
-EXTRACT takes an optional argument, which provides an output
file-specification.
By default, MUPIP JOURNAL derives the output file specification
using the name of the original database file associated with the
journal and a file type of .MJF. If the command specifies more
than one journal file, JOURNAL -EXTRACT derives the default
output file specification from the name of the first database.
-SHOW
-SH[OW]=show-option-list
Specifies what information the JOURNAL command displays about a
journal file. JOURNAL commands may specify -SHOW alone or with
other action qualifiers.
For information on the options refer to the show-option-list
topic.
2 show-option-list
show-option-list
The following topics detail the show-option-list elements.
3 ALL
AL[L]
ALL displays every available type of information about the
journal file. For additional information, refer to the
descriptions of each of the other SHOW keywords.
3 HEADER
H[EADER]
HEADER displays the journal file header information.
This includes:
o Database file name
o Journal file name
o Journal file version label (.e.g. GDSJNLnn)
o Whether before-images were captured
o Journal creation time/date
o Journal creator
o The last user to open the journal
o The last time the journal file was opened
The information for the creator and last user includes:
o Process Name
o Process Identification Number
o Node Name
o Terminal Number
3 PROCESSES
P[ROCESSES]
PROCESSES displays all processes active during the period
specified implicitly or explicitly by JOURNAL command time
qualifiers.
3 ACTIVE_PROCESSES
AC[TIVE_PROCESSES]
ACTIVE_PROCESSES displays all processes active at the end of the
period specified implicitly or explicitly by JOURNAL command
time qualifiers.
3 BROKEN_TRANSACTIONS
B[ROKEN_TRANSACTIONS]
BROKEN_TRANSACTIONS displays all processes that had incomplete
fenced transactions at the end of the period covered by the
JOURNAL command.
3 STATISTICS
S[TATISTICS]
STATISTICS displays a count of all journal record types
processed during the period specified implicitly or explicitly
by JOURNAL command time qualifiers.
2 Direction_qualifiers
Direction Qualifiers
-FORWARD
-FO[RWARD]
Specifies that JOURNAL processing should proceed from the
beginning of the given journal files. If the actions include
-RECOVER, the target database file should contain a copy of that
database made at the time when MUPIP SET -JOURNAL= created the
journal files.
-FORWARD is incompatible with -BACKWARD.
-BACKWARD
-BA[CKWARD]
Specifies that JOURNAL processing should proceed from the end of
the journal files. If the actions include -RECOVER, JOURNAL
-BACKWARD starts restoring before-images starting at the end of
the file, back to an explicitly or implicitly specified point
before it "reverses" and processes database updates in the
forward direction. The target database file should "match" the
end of the journal file, i.e., be the same as when GT.M wrote
the last record of the journal.
-BACKWARD is incompatible with -FORWARD.
2 Time_qualifiers
Journal time specifications
Journal qualifiers specifying time take arguments in absolute or
delta time format. Enclose time arguments in quotes ("") and
include all leading delimiters. Absolute format is "day-mm-yyyy
hh:mm:ss:cc" , where cc represents hundredths of a second. Absolute
time may indicate today's date with "--" before the hours. Delta
format is "day hh:mm:ss:cc" , where cc represents hundredths of a
second. If delta time is less than a day, it must start with zero
(0) followed by a space, or just a space, before the hours. Delta
time is always relative to the time of the last record in all
journal file arguments to the MUPIP JOURNAL command. A normal
database closure, caused by the last accessing process leaving
GT.M, also properly closes the associated journal file.
Alternatively, a system failure causes the journal to end in an
abnormal or "disorganized" fashion. JOURNAL processing deals with
both cases.
The time qualifiers perform as follows:
o -AFTER= only applies to JOURNAL -EXTRACT -FORWARD and specifies
a starting time; processing for all other -FORWARD actions must
start at the beginning of the journal files
o -BEFORE= specifies an ending time for any action -FORWARD or
-BACKWARD
o -SINCE= specifies a starting time for any action -BACKWARD
o -LOOKBACK_LIMIT= specifies a "safety zone" for resolving open
fenced transactions when processing any action -BACKWARD; the
-LOOKBACK_LIMIT= argument may be a list of limits: "TIME=time"
and/or "OPERATIONS=integer"
Because GT.M rounds time-stamps within the journal to hundredths of
a second and the JOURNAL processing can only determine time as
exactly as the journal records permit, the JOURNAL command
processes specified times in a "fuzzy" fashion. Because they deal
with processing completed logical transactions, -SINCE= and
-LOOKBACK= times have more "blur" than -AFTER= and -BEFORE= times.
-AFTER
-A[FTER]=time
Specifies the starting time for JOURNAL -EXTRACT -FORWARD to
commence output. The time specified references time stamps in
the journal and identifies the point after which JOURNAL
processing starts extracting information out of the journal
file. -AFTER= specifies time in absolute or delta time formats.
Delta format specifies an offset from the time of the last
record of the journal file. If -AFTER= provides a time following
the last time recorded in the journal file or following any
-BEFORE= time, JOURNAL processing produces no result. Using
-BEFORE= with -AFTER= restricts -EXTRACT to a particular period
of time.
-AFTER= is incompatible with -BACKWARD and with all action
qualifiers except -EXTRACT.
By default, -EXTRACT starts at the beginning of the journal
file.
-BEFORE
-BE[FORE]=time
Specifies the ending time at which JOURNAL processing stops
extracting or recovering data. The time specified references
time stamps in the journal files. -BEFORE= specifies time in
absolute or delta time formats. Delta format specifies an offset
from the time stamp in the last record of each the journal file.
If -BEFORE= provides a time preceding the first time recorded in
the journal file or preceding any -AFTER= or -SINCE= time,
JOURNAL processing produces no result.
-BEFORE= is compatible with all other JOURNAL qualifiers.
By default, JOURNAL processing terminates at the end of the
journal file.
-SINCE
-SI[NCE]=time
Specifies how far back in time JOURNAL -BACKWARD should process
from the end of the journal file, before starting its forward
processing. The time specified references time stamps in the
journal files. -SINCE= specifies time in absolute or delta time
formats. Delta format specifies an offset from the time of the
last record of the journal file. When JOURNAL -BACKWARD locates
the -SINCE= time, if it has open fenced transactions, it
continues processing backward to resolve them unless the command
also specifies -FENCES=NONE. The -LOOKBACK= qualifier controls
the length of processing backward past the -SINCE= time.
If -SINCE= time exceeds the last time recorded in the journal
files, JOURNAL processing effectively ignores the qualifier.
-SINCE= is incompatible with -FORWARD. If -SINCE= provides a
time preceding any -BEFORE= time, JOURNAL processing produces no
result.
By default, JOURNAL -BACKWARD processes the last five (5)
minutes of the journal file(s).
-LOOKBACK_LIMIT
-[NO]LOO[KBACK_LIMIT][=lookback-option-list]
Specifies how far JOURNAL -BACKWARD processes back past the
explicit (-SINCE=) or implicit turn around time while attempting
to resolve open transaction fences. -LOOKBACK= options include
time and transaction counts.
-NOLOOKBACK_LIMIT specifies that JOURNAL -BACKWARD can process
all the way to the beginning of the journal file, if necessary,
to resolve open transaction fences.
-LOOKBACK_LIMIT= is incompatible with -FORWARD. When
-FENCES=NONE, JOURNAL processing ignores -LOOKBACK_LIMIT=.
The lookback-options are:
TIME=time - limits lookback by a specified amount of delta or
absolute journal time.
OPERATIONS=integer - limits lookback to some number of database
updates.
The lookback-option names and values must be enclosed in quotes
(""), e.g., "Time=0 00:05" or "Oper=10." When -LOOKBACK=
specifies both options, they must be enclosed in parentheses ()
and separated by a comma (,). When -LOOKBACK= specifies both
options, the first limit reached terminates the lookback.
By default, MUPIP JOURNAL uses -LOOKBACK_LIMIT="TIME=0 00:05"
providing five minutes of journal time prior to -SINCE= in which
to resolve open fences.
2 Control_qualifiers
Control Qualifiers
-REDIRECT
-RED[IRECT]=file-pair-list
Specifies that JOURNAL -RECOVER replay the journal file to a
database different than the one for which it was created. Use
this qualifier to create or maintain databases for training or
testing.
JOURNAL rejects -REDIRECT unless it appears with -RECOVER.
The file-pair-list consists of one or more pairs of
file-specifications enclosed in parentheses () and separated by
commas (,). The pairs are separated by an equal sign in the
form:
old-file-specification=new-file-specification
where the first file-specification names the original database
file and the second file-specification names the target of the
-RECOVER. When -REDIRECT specifies only one file pair, the
parentheses are optional.
By default, JOURNAL directs -RECOVER to the database file from
which the journal was made.
-FENCES
-FE[NCES]=fence-option
Specifies how JOURNAL processes fenced transactions. Fenced
transactions are logical transactions made up of database
updates preceded in MUMPS by a ZTSTART command and followed by a
ZTCOMMIT command. All updates between a ZTSTART and a ZTCOMMIT
are designed such that they should all occur together, i.e., no
one of them should reach the database unless they all do.
The fence options are:
o NONE, which causes JOURNAL to apply all individual updates as
if transaction fences did not exist
o ALWAYS, which causes JOURNAL to treat any unfenced or
improperly fenced updates as errors
o PROCESS, which causes JOURNAL to accept unfenced database
updates, and also to observe fences when they appear,
generating an error in the case of a ZTSTART with no
corresponding ZTCOMMIT
By default, MUPIP JOURNAL uses -FENCES=PROCESS.
-INTERACTIVE
-[NO]IN[TERACTIVE]
Specifies whether, for each error over the -ERROR_LIMIT, JOURNAL
processing prompts the invoking operator for an answer
controlling continuation of processing. If the operator responds
that processing should not continue, the JOURNAL command
terminates.
-NOINTERACTIVE terminates the journal processing as soon as the
process generates the number of errors specified in
-ERROR_LIMIT.
When processing in INTERACTIVE mode, the default is
-INTERACTIVE, otherwise, e.g., BATCH mode, the default is
-NOINTERACTIVE.
-ERROR_LIMIT
-[NO]ER[ROR_LIMIT][=integer]
Specifies the number of errors that JOURNAL processing treats as
acceptable. When the number of errors exceeds the -ERROR_LIMIT,
the -INTERACTIVE qualifier determines whether JOURNAL processing
halts or defers to the operator.
-NOERROR_LIMIT prevents JOURNAL from stopping because of errors.
Journal processing will continue until it reaches the end of the
journal file, regardless of the number of errors. Note that
-NOERROR_LIMIT is not the same as -ERROR_LIMIT=0.
By default, MUPIP JOURNAL uses -ERROR_LIMIT=0, causing the first
error to initiate the appropriate error action.
-CHECKTN
-[NO]C[HECKTN]
-CHECKTN specifies that JOURNAL -FORWARD must ensure that the
first database update in the journal file has the next
transaction number after the current transaction in the database
file.
-NOCHECKTN suppresses checking of the starting journal
transaction against the ending database transaction number.
-CHECKTN is incompatible with the -BACKWARD qualifier.
By default, JOURNAL -FORWARD uses -CHECKTN.
2 Selection_qualifiers
Selection Qualifiers
-GLOBAL
-G[LOBAL]=global-list
Specifies globals for JOURNAL to include or exclude from
processing. You may find this qualifier useful for extracting
and analyzing specific data.
The global-list contains names of one or more global-names (not
including subscripts) preceded by "^" enclosed in parentheses ()
and separated by commas (,). If -GLOBAL specifies only one item,
the parentheses are optional. The names may include the
wildcards ? and *. The entire list or each name may optionally
be preceded by a minus (-), requiring JOURNAL to exclude
database updates that update the specified global(s). When the
global-list with a JOURNAL -GLOBAL does not start with a minus,
JOURNAL processes only the explicitly named globals.
By default, JOURNAL processes all globals.
-USER
-U[SER]=user-list
Specifies that JOURNAL processing include or exclude database
updates generated by one or more users. You may use this
qualifier to "back-out" database updates erroneously entered by
a specific user.
The user-list contains names of one or more users enclosed in
parentheses () and separated by commas (,). If -USER specifies
only one item, the parentheses are optional. The names may
include the wildcards ? and *. The entire list or each name may
optionally be preceded by a minus (-), requiring JOURNAL to
exclude database updates initiated by the specified user(s).
When the user-list with a JOURNAL -USER does not start with a
minus, JOURNAL processes only database updates generated by the
explicitly named users.
By default, JOURNAL processes database updates regardless of the
user by which they were initiated.
-ID
-ID=pid-list
Specifies that JOURNAL processing include or exclude database
updates generated by one or more processes, identified by
process identification numbers (PIDs) in hexadecimal. You may
use this qualifier for troubleshooting or analysis of data.
The pid-list contains one or more PIDs enclosed in parentheses
() and separated by commas (,). If -ID specifies only one item,
the parentheses are optional. The entire list or each PID may
optionally be preceded by a minus (-), requiring JOURNAL to
exclude database updates associated with the specified PID(s).
When the pid-list with a JOURNAL -ID does not start with a
minus, JOURNAL processes only database updates associated with
the explicitly listed PIDs.
By default, JOURNAL processes database updates regardless of the
process by which they were initiated.
-PROCESS
-P[ROCESS]=process-name-list
Specifies that JOURNAL processing include or exclude database
updates generated by one or more processes, identified by names.
You may use this qualifier to extract specific process-related
data for testing or troubleshooting.
The process-name-list contains names of one or more processes
enclosed in parentheses () and separated by commas (,). If
-PROCESS specifies only one item, the parentheses are optional.
The names may include the wildcards ? and *. The entire list or
each process-name may optionally be preceded by a minus (-)
requiring JOURNAL to exclude database updates associated with
the specified process name(s). When the process-name-list with a
JOURNAL -PROCESS does not start with a minus, JOURNAL processes
only database updates associated with the explicitly named
processes.
By default, JOURNAL processes database updates regardless of the
process by which they were initiated.
-TRANSACTION
-T[RANSACTION]=transaction-type
Specifies transaction-types for JOURNAL to include or exclude
from processing. For example, you may use this qualifier to
exclude KILL transactions and prevent an accidental KILL from
reoccurring during replay.
The transaction-types are:
SET
KILL
These types correspond to the MUMPS commands of the same names.
The transaction-type may optionally be preceded by a minus (-)
requiring JOURNAL to exclude transactions of the specified type.
When the transaction-type with a JOURNAL -TRANSACTION does not
start with a minus, JOURNAL processes only transactions of the
explicitly named type.
By default, JOURNAL processes transactions regardless of type.
2 Examples
Journal Examples
Example
$ cp /dat/cus.cat
$ mupip journal -recover -forward -fences=none cus.mjl
The cp command copies a backup copy of the database for use in
recovery. The MUPIP JOURNAL command recovers the database using the
cus.mjl journal file. The JOURNAL command processes the journal
file in a forward direction (from the beginning of the journal).
The -FENCES=NONE directs JOURNAL to ignore any fences and recover
all individual updates.
Example
$ mupip set -file -journal=(before,buff=128) cus.dat
$ mupip set -file -journal=(before,buff=128) acc.dat
...
$ mupip journal -recover -verify -back -error=2 cus.mjl,acc.mjl
The first two command lines initiate journaling for the two
specified database files. MUPIP JOURNAL does not accept wildcards
in database file names. Because the MUPIP SET command specifies
JOURNAL=BEFORE, subsequent JOURNAL -RECOVER may have either
-FORWARD or -BACKWARD direction. The last line contains the command
to recover the two database files using the two specified journal
files. The journal recover processes in a BACKWARD direction using
before-image processing. If JOURNAL processing encounters two or
more errors (-ERROR=2), the recovery process terminates.
Example
$ mupip journal -forw -befo="-- 10:30" -glob="^bv%r*" -extr=bv
cus.mjl
This command line extracts database updates that occurred from the
beginning of the journal until 10:30 to global variables with the
prefix ^bv , followed by some character, followed by r, optionally
followed by more characters. The JOURNAL -EXTRACT places the
updates in a file called bv.mjf
Because the command does not specify an extension, JOURNAL assigns
the default extension .mjf to the output file.
Example
$ mupip jour -rec -back -befo="-- 10:30" -since="-- 9:30"
-lookback="time=0 00:05" cus.mjl
This command line performs a -RECOVERY -BACKWARD of the database
file that corresponds to the journal file cus.mjl. JOURNAL
-RECOVER processes from the journal time 9:30 to journal time
10:30. If the JOURNAL finds open fenced transactions, it "looks
back" an additional five minutes to resolve them. The -BEFORE= and
-SINCE= qualifiers in this example use absolute time. Because the
time specification omits the date, JOURNAL assumes today's date.
2 extract_formats
JOURNAL -EXTRACT formats
EXTRACT output records are constructed of fields or "pieces"
delimited by back-slashes (\).
The first piece of an EXTRACT output record always contains the
two-digit decimal transaction record type, e.g., 01 for a process
initialization record.
The second piece always contains the full date and time of
operation, represented in $HOROLOG-format, with decimal seconds,
e.g., 54271,44580.55.
The third piece always contains the process id (PID) of the process
that performed the operation, represented as a decimal number. The
remainder of the record depends on the record type.
The fields described as "database transaction number" contain a
GT.M assigned number that is unique within the journal file.
3 proc_initialization
Process Initialization Record
A type 1 record indicates an image-initiated contact with the
GT.M database region for the first time. The format for a
process initialization record is:
01\time\pid\nnam\unam\term
where
time full time/date
pid process id
nnam node name
unam user name
term terminal name
3 proc_termination
Type 2 - Process Termination Record
A type 2 record indicates an image terminated and dropped
interest in the GT.M database region. The format for a process
termination record is:
02\time\pid\nnam\unam\term\tnum
where
time full time/date
pid process id
nnam node name
unam user name
term terminal name
tnum database transaction number
3 end_of_file
Type 3 - End of File Record
A type 3 record indicates all GT.M images dropped interest in
the region and the journal file was closed normally. The format
for an end-of-file record is:
03\time\pid\nnam\unam\term\tnum
where
time full time/date
pid process id
nnam node name
unam user name
term terminal name
tnum database transaction number
3 Kill
Type 4 - Kill Record
A type 4 record indicates a database update caused by a KILL
command. The format for a KILL record is:
04\time\pid\tnum\node
where
time full time/date
pid process id
tnum database transaction number
node a MUMPS node reference in external format
3 Set
Type 5 - Set Record
A type 5 record indicates a database update caused by a SET
command. The format for a SET record is:
05\time\pid\tnum\sarg
where
time full time/date
pid process id
tnum database transaction number
sarg a MUMPS set argument
Note a MUMPS SET argument has a node reference followed by an
equal-sign (=) and MUMPS data string expression.
3 tr_start
Type 6 - Transaction Start Record
A type 6 record indicates a ZTSTART command. The format for a
transaction start record is:
06\time\pid
where
time full time/date
pid process id
3 tr_commit
Type 7 - Transaction Commit Record
A type 7 record indicates a ZTCOMMIT command. The format for a
transaction commit record is:
07\time\pid\tnum\part
where
time full time/date
pid process id
tnum database transaction number
part number of journal entries in the transaction
1 Jrnl_examples
Journaling Examples
The following examples present a typical use of database journaling to
prevent loss of data. In our examples the database consists of three
regions, ACC, MAIN, and TMP, mapped respectively to files
/usr/prod/acc.dat, /usr/prod/mumps.dat and /usr/prod/tmp.dat.
2 Setting_Database_Regions_for_Journaling
We assume that region TMP holds only process-local data, and,
therefore, does not require backups or journaling. We assume, on the
other hand, that regions ACC and MAIN hold production application data
that should be protected by journaling. Moreover, our application
requires a high degree of availability. Therefore, we set up
journaling with BEFORE_IMAGES for regions ACC and MAIN. The
BEFORE_IMAGES allow for JOURNAL RECOVER -BACKWARD, which generally
works faster than JOURNAL RECOVER -FORWARD. Because both our journaled
regions map to the database files on the usr device, we choose a
different disk with a different controller to accommodate the journal
files. This choice improves resiliency against hardware failures.
Example
$ mupip set -region -journal=(off,buff=200,file=/jnl/prod/acc) ACC
$ mupip set -region -journal=(off,buff=200,file=/jnl/prod/main) MAIN
These commands must be issued when the database files are available
for exclusive (stand-alone) access. They establish several journal
characteristics. The example increases the journal buffer size from
the default of 128 pages to 200 pages.
2 Journal_Maintenance
First backup the database files. Copy and store your existing journal
files, thencreate and initialize new journal files.
Example
$ mupip backup ACC,MAIN /dev/rst8/051590
$ mupip set -region -journal=(on,before) ACC,MAIN
This sequence of commands backs up the database files to disk and
initializes new journal files for each. MUPIP BACKUP can operate
without exclusive access to the database files by freezing all
updates. However, in order to ensure that the BACKUP captures an
application state matching the beginning of the journal files, it
should run stand-alone.
Some applications with high rates of updates may create considerable
amount of journaling data. To save the disk space, you may archive
journal files to magnetic tapes until the next database backup. Before
archiving a journal file, first create a new one.
Example
$ mupip set -file -journal=(on,before) ACC.DAT
$ cp /dev/rst8/jnl.bck acc.mjl ~{
$ purge JNL:[PROD]ACC.MJL\
This sequence creates a new journal file and backs up the old journal
file to the tape. Notice that the MUPIP SET requires exclusive access
to the database file to ensure that the old journal stops and the new
journal starts with consistent transaction states.
2 Recovery_from_Journal_Files
Because we set up our databases with BEFORE_IMAGE journaling, when
both the database and journal files are available after a failure
event, we can use JOURNAL -RECOVER -BACKWARD.
Example
$ delete /usr/prod/tmp.dat
$ mupip create -region=tmp
$ mupip journal -reco -show -back -nointer
/jnl/prod/acc./jnl/prod/main
$ mupip integ -region *
This first deletes and recreates tmp.dat. The MUPIP JOURNAL command
includes the -SHOW qualifier to generate a report on the status of
activity in the journal files. It also includes the -NOINTERACTIVE
qualifier to prevent operator interaction when JOURNAL processing
encounters an error. By default, this JOURNAL -RECOVER has an implicit
-ERROR_LIMIT=0, which causes the first broken transaction to terminate
processing. In addition to checking database integrity, the MUPIP
INTEG -REGION acts as the first database access after the recovery
and, if the old journal file terminates improperly, creates a new
journal file. Unlike INTEG -REGION, the command MUPIP INTEG -FILE does
not initialize a new journal file. However, if the old journal file
has damage, any other access to the data base creates a new version.
If MUPIP JOURNAL -RECOVER reports broken transactions during recovery,
reenter the transactions.
WARNING: The new journal file created by commands such as mupip integ
-region will overwrite the existing journal file. To preserve the
existing file, save it under a different name before executing any
commands that will cause a new journal file to be created.
If the databases were lost, for instance due to a disk drive failure,
we would have to use JOURNAL -RECOVER -FORWARD after replacing the
drive and retrieving backups of the databases. This recovery may take
much longer than backward recovery, depending on the amount of data in
the journal.
Example
$ lprm /usr/prod/*.dat
$ cp /usr/prod /usr/051590/acc.dat
$ cp /usr/prod /usr/051190/mumps.dat
$ mupip journal -reco -forw -err=5 /jnl/jnl/acc,/jnl/jnl/main
$ mupip create -region=TMP
$ mupip integ -region -fast *
This sequence of commands recreates the databases to the point of the
last backup. Then it recovers all updates from the journal files. The
-ERROR_LIMIT= qualifier causes JOURNAL -RECOVER to attempt processing
through up to five (5) errors. By default, when the JOURNAL -RECOVER
executes in batch, processing terminates after five errors. However if
the command executes interactively, after five errors, MUPIP JOURNAL
prompts the the operator at the invoking terminal to choose between
continuing or terminating. Also by default, the JOURNAL -RECOVER
command implies a -VERIFY, causing a check of the journal prior to
-RECOVER processing. The commands up to this point require exclusive
(stand-alone) access to the database files. MUPIP INTEG -REGION
verifies the integrity of the recovered database. If the journal was
not properly closed, the INTEG -REGION also creates a new version of
the journal file, overwriting the existing version. Because the
databases are large, we use the -FAST qualifier on the INTEG. Because
INTEG freezes all updates and we wish to ensure database integrity
before going back into production, we continue to restrict access to
the database until the INTEG completes. Once the database has been
verified, we can resume work. One of the first items of business
should be to reenter any broken transactions.
Should the system crash again and something such as a hard disk
failure prevent backward recovery, we must recover the database
forward, twice. First, restore the database from backup, then recover
to the point of the first crash, then finally, recover from that point
to the point of the second crash.
Example
$ delete *.DAT;*
$ mupip restore acc.dat usr/051590/acc.bck
$ cp /usr/prod /usr/051590/acc.dat
$ cp /usr/prod /usr/051590/mumps.dat
$ mupip journal -reco -veri -forw -err=5 /jnl/acc.mjl
$ mupip journal -reco -veri -forw -err=5 /jnl/acc.mjl
$ mupip create -region=TMP
$ mupip integ -region -fast *
This is similar to the previous example, however, this sequence
recovers the database regions from two consecutive journal files.
1 Jrnl_overview
Journaling overview
Journaling records an extra copy of information during database
updates in order to provide resiliency against hardware and software
failures. Journaling can reduce the "window of exposure" from some of
the most common types of failure: power loss and media loss due to
head-to-disk interference. Journals also provide a valuable tool in
cases of software errors and operational miscues. A journal file has
questionable value only in the case where the database and the journal
share a common point of failure that affects the information in both,
over a significant period of time. Therefore, using different disks
and, when possible, different disk controllers for the journal and the
database files improves the likelihood of the journal serving its
intended purpose.
The database management portion of a MUMPS implementation ensures that
multiple concurrent updates and retrievals of the same information (or
information "close together" in ordered sequence) are handled in a
predictable and logical fashion. The database manager may have to
change multiple records, usually indices, as a result of a single
update. Therefore, interrupting a process performing such a
"multi-point" update violates a design assumption of the MUMPS
implementation and results in a malformed database. Access to a
damaged area of the database does not produce the desired result.
Instead, such an "integrity" problem causes symptoms including system
hangs, misplaced updates, failure to find information that exists,
finding information out of sequence, and run-time errors. If the bad
records contain no valid information or redundant information, the
simplest cures for integrity errors entail deleting incorrectly
formatted records. However, sometimes crashes damage information of
value and, in any case, database repair requires time and skill. GT.M
journaling provides a means to recover or replace databases that have
integrity problems. Use of journaling at this "global" level requires
no MUMPS programming.
MUPIP and its documentation uses the term transaction to mean database
update. In journaling, the term transaction may refer to multiple
related database updates.
2 Forward_Recovery
Forward Recovery
Forward Recovery consists of restoring a backup copy of the
database and applying the journal file to that database file. The
journal file contains copies of each database update. Forward
Recovery reads the entire journal file from beginning to end (in a
"forward" direction) and updates the backup copy of the database.
The optional MUPIP JOURNAL -BEFORE= qualifier specifies a journal
ending time that stops journal processing before the physical end
of the journal file. In general, Forward Recovery takes longer than
Backward Recovery. However, if the current database is somehow
destroyed, you must use Forward Recovery. Also, if a journal file
was created NOBEFORE_IMAGE with a MUPIP SET, that journal only
permits Forward Recovery.
Example of Forward Recovery:
MUPIP JOURNAL -RECOVER -FORWARD
-BEFORE
|
--------------------------------V-----------X--------->time
10:30 10:32
>>+++++++++++++++++++++++++++++>>++++++++++>>
This shows a recovery after a system crash at 10:32, which
processes the entire journal file forward. If we add -BEFORE="--
10:30" to the command, the recovery stops when processing
encounters updates that originally occurred after 10:30.
2 Backward_Recovery
Backward Recovery
Backward Recovery works by processing from the end of the journal
file that contains information for the period just prior to the
failure event, thereby minimizing recovery time. Backward Recovery
uses "before-image" journaling. With "before-image" journaling,
GT.M captures the database updates, as well as "snap-shots" of
portions of the database immediately prior to the change caused by
the update. In effect, MUPIP JOURNAL=BEFORE_IMAGE creates
"mini-backups" preceding each database update. Backward Recovery
uses the mini-backups to restore the database as far back in time
as specified, then it goes forward in time replaying the database
updates. Using Backward Recovery with the MUPIP JOURNAL qualifiers
-BEFORE=, -SINCE=, and -LOOKBACK=, you can specify a block of time
to recover. JOURNAL -RECOVER -BACKWARD only works if the production
database is useable, and if the MUPIP SET command that created the
journal file specified the BEFORE_IMAGE characteristic.
Note: Before-images require more disk I/O and storage space.
Example of Backward Recovery:
MUPIP JOURNAL -RECOVER -BACKWARD -SINCE="-- 9:30"
-LOOKBACK_LIMIT
| -SINCE
| | -BEFORE
| | |
----------------V-------V-------V------X--------->time
9:30 10:30 10:32
<*******<++++++++++++++<<
********++++++++>++++++>>
This shows a recovery after a system crash at 10:32. The recovery
"undoes" the database updates backward to 9:30 and then forward
until the crash. If we add -BEFORE="-- 10:30" to the command, the
recovery stops when forward processing encounters updates that
originally occurred after 10:30. If the application includes
ZTSTART and ZTCOMMIT commands to fence a group of transactions,
backwards processing may continue back prior to 9:30 searching to
resolve fenced transactions that were incomplete at 9:30. The
-LOOKBACK_LIMIT= qualifier controls the maximum amount of
additional backward processing.
2 Fencing_Transactions
Fencing Transactions
Journaling without fences in MUMPS addresses the fact that a system
crash can damage the database integrity. However, sound design
frequently dictates modelling a single "real-world" event in
updates to more than one global variable. Such real-world events
are usually captured in a single data entry session and are
referred to as logical transactions. Therefore, interrupting a
process performing a "multi-node" logical transaction violates a
design assumption of the application and results in logical
inconsistencies in the database. Such logical inconsistencies
produce symptoms including run-time errors, inappropriate branching
and incorrect reports. Sometimes logical inconsistencies are
referred to as application-level database integrity problems.
Standard MUMPS does not yet include a method to identify the fact
that a single logical transaction may be made up of multiple global
updates. Therefore, a journal recovery that corrects database
integrity problems may perform an update that is part of an
incomplete sequence of updates intended as a single logical unit.
GT.M provides the MUMPS commands ZTSTART to mark the beginning of a
logical transaction and ZTCOMMIT to mark the end of a logical
transaction. When ZTSTART and ZTCOMMIT fence a logical transaction,
the journal recovery can refrain from starting an incomplete
update. To take advantage of this additional level of journaling
functionality, the application must use ZTSTART and ZTCOMMIT
commands.
Journaling does not require modification of application programs.
However, using ZTSTART and ZTCOMMIT to add transaction fences
around updates that comprise a logical unit significantly improves
the benefit of journaling. For instance, the logical transaction
"transfer funds between accounts" consists of a debit update to one
account and a credit update to another account. One of the updates
made without the other is not valid. When recovering from journal
files, JOURNAL processing recovers either all updates within the
transaction fences or none of them. MUPIP JOURNAL -RECOVER reports
the latter case during recovery.
1 LOAD
L[OAD]
LOAD enters global variable names and their corresponding data values
into a GT.M database from a sequential file. LOAD uses the Global
Directory to determine which database files to use. LOAD may operate
concurrently with normal GT.M database access. However, a LOAD does
not use MUMPS LOCKs and therefore may produce application-level
integrity problems if run concurrently with many applications.
The format of the LOAD command is:
L[OAD] [qualifier...] file-specification
LOAD takes its input from the file defined by the file-specification.
<CTRL C> produces a status message from LOAD. Entering <CTRL C> twice
in quick succession aborts LOAD. A LOAD terminated abnormally by
operator action or error is incomplete but does not adversely affect
the database structure.
2 -FORMAT
-FO[RMAT]=keyword
Specifies the format of the input file. The format must match the
actual format of the input file for LOAD to operate.
At present, the only available format code is:
GO - Global Output format
3 GO
-FORMAT=GO
-FORMAT=GO stores the data in record pairs. Each global node produces
one record for the key
and one for the data. -FORMAT=GO has two header records, therefore
LOAD -FORMAT=GO
starts active work with record number three (3).
2 -BEGIN
-BE[GIN]=integer
Specifies the record number of the input file with which LOAD should
begin. Directing LOAD to begin
at a point other than the beginning of a valid key causes an error.
It is important to consider the number of header records when choosing a
-BEGIN point. For more
information, refer to the section on -FORMAT.
For -FORMAT=GO input, normally the value should be an odd number.
By default, LOAD starts at the beginning of the input file.
2 -END
-E[ND]=integer
Specifies the record number of the input file at which LOAD should stop.
The -END=integer must be
greater than the -BEGIN=integer for LOAD to operate. LOAD terminates
after processing the record of
the number specified by -END or reaching the end of the input file.
For -FORMAT=GO input, normally the value should be an even number.
By default, LOAD continues to the end of the input file.
2 -FILL_FACTOR
-FI[LL_FACTOR]=integer
Specifies the target fill density for the data blocks updated in the
database file. -FILL_FACTOR must be
an integer between 5 and 100 specifying the percentage fill rate for the
block.
In general, maximum data densities provide the best performance.
However, if you have an
understanding of the update patterns of your application, you may
achieve a performance benefit over
time from lowering the initial fill-factor.
By default, LOAD uses -FILL_FACTOR=100 for maximum data density.
1 Overview
MUPIP overview
The GT.M MUMPS Peripheral Interchange Program, MUPIP, is a utility
that provides an assortment of tools for GT.M database management and
database journaling.
MUPIP provides the following commands:
For stand-alone database services:
o CREATE database files
o EXTEND Mapped Memory database files
o JOURNAL, recover database files and extract journal records
o INTEG, check the integrity of GDS database files
o RESTORE incremental backups to GDS database files
o SET database file characteristics
For concurrent database services:
o BACKUP GDS database files
o BACKUP-INCREMENTAL changes to GDS database files
o EXTEND Buffered Global database files
o EXTRACT data from GDS databases
o INTEG, check the integrity of GDS databases
o LOAD databases from sequential files
o REORGanize database files to optimize performance
o RUNDOWN database files that are not currently accessed
For non-database services:
o HELP for MUPIP commands
o STOP GT.M processes
1 REORG
REO[RG]
MUPIP REORG offers a tool for optimizing your database files for peak
database performance. REORG
requires minimal operator resources, and runs concurrently with other
database activity, including updates.
Competing activity generally increases the time to perform a REORG, as
well as that of the competing
operations. If you use REORG concurrently with normal database access, you
may wish to lower the priority
of the process performing the REORG to minimize its impact on normal
operations.
Note that while REORG optimizes the GDS structure of database files, it
does not deal with native file system
file fragmentation. Because native file fragmentation may significantly
impair database performance, its
prevention and control is still important. Always create files with
appropriate allocations and extensions, on
disks with large contiguous free-space. Use native utilities and,
depending on your procedures, MUPIP
utilities to eliminate file fragmentation when database files have been
extended more than a dozen times.
The format of the REORG command is:
REO[RG] [qualifier...] file-specification
<CTRL C> produces a status message from REORG. Entering <CTRL C> twice in
quick succession
aborts REORG. A REORG terminated abnormally by operator action or error is
incomplete but does not
adversely affect the database structure.
2 Qualifiers
-SELECT
-SELECT=global-name-list
Restricts REORG operation to a subset of specified globals. By
default, REORG operates on all
globals in all database files identified by the current global
directory for the process executing the
MUPIP command.
Arguments to this qualifier may be an individual global name, a prefix
followed by an asterisk (*)
wild-card symbol, or a list of names and/or prefixes followed by the
wild-card symbol. The "^" in
the specification of the global name is optional. Enclose lowercase
global names in quotes ("").
The global-specification can be:
o A global name, such as MEF
o A range of global names, such as A7:B6
o A list, such as A,B,C
o Global names with the same prefix, such as TMP*
In the first case, EXTRACT selects only global ^MEF. In the second
case, EXTRACT selects all
global names between ^A7 and ^B6, inclusive. In the third case,
EXTRACT selects globals ^A, ^B,
and ^C. In the fourth case, EXTRACT selects all global names from ^TMP
through ^TMPzzzzz.
-FILL_FACTOR
-FILL_FACTOR=percent-qualifier
Directs REORG to leave free space within blocks for future updates.
Arguments to this qualifier
must be integers from 5 to 100. REORG uses this figure in deciding
whether to place more
information in a block; currently REORG does not move information out
of a block to make more
room. By default, REORG attempts to fill blocks to their maximum
capacity.
1 RESTORE
RE[STORE]
RESTORE integrates one or more BACKUP -INCREMENTAL files into a
corresponding database. For a
RESTORE to work, the transaction numbers in the incremental file(s) must
sequentially follow those in the
database. Gaps or overlaps in the transaction numbers at RESTORE input
file boundaries cause RESTORE to
issue errors.
The format of the RESTORE command is:
RE[STORE] [qualifier] file-spec file-list
The file-specification identifies the name of the database file that
RESTORE uses as a starting point. The
transaction number in the database must match the starting transaction
number of each successive input to the
RESTORE. If the BACKUP -INCREMENTAL was created using -TRANSACTION=1, set
the database up
with MUPIP CREATE and do not access it with anything except the MUPIP
commands INTEG, EXTEND,
and SET before initiating the RESTORE.
The file list specifies one or more files produced by BACKUP -INCREMENTAL
to RESTORE into the
database. The file-specifications are separated by commas (,) and must be
in sequential order, from the oldest
transaction number to the most recent. RESTORE may take its input from a
UNIX file on any device that
supports such files.
2 Qualifiers
-EXTEND
-[NO]EXTEND
Specifies whether RESTORE should extend automatically if its target
database file is smaller than
the file size identified by the input BACKUP -INCREMENTAL file. MUMPS
activity between
backups may automatically extend a database file. Therefore, the
database file specified as the
starting point for a RESTORE may require an extension before the
RESTORE. If the database needs
an extension, MUPIP displays a message. The message gives the sizes of
the input and output
database files and the number of blocks by which to extend the
database. If the RESTORE specifies
more than one incremental backup with a file list, the database file
may require more than one
extension.
By default, RESTORE automatically extends the database file.
1 RUNDOWN
RU[NDOWN]
When database files have not been properly closed, RUNDOWN closes those
currently inactive databases and
releases the central memory they claim. In normal operation, the last
process to close a database file performs
the RUNDOWN actions.
The format of the RUNDOWN command is:
RU[NDOWN] [-qualifier file-spec or region-list]
The filename or region-list identifies the target of the RUNDOWN. If
RUNDOWN has no parameter, it
ignores any qualifier and flushes the memory associated with all database
files that currently have associated
memory and no active users.
Use RUNDOWN after a system crash or after the last process accessing a
database terminates abnormally.
RUNDOWN ensures that an inactive database is properly closed and ready for
subsequent use. RUNDOWN
has no effect on any database that is actively being accessed at the time
the RUNDOWN is issued.
A RUNDOWN that specifies a target file or region can correct file state
problems which a RUNDOWN with
no parameters does not find.
2 Qualifiers
-FILE
-F[ILE]
Specifies that the argument is a filename for a single database file.
The -FILE qualifier is
incompatible with the -REGION qualifier. If the rundown parameter
consists of a list of files, the
command only operates on the first item in the list.
-REGION
-R[EGION]
Specifies that the argument contains one or more region-names which
identify database files mapped
by the current Global Directory. The -REGION qualifier is incompatible
with the -FILE qualifier.
2 SEt
SET modifies certain database characteristics. SET requires exclusive
(stand-alone) access to the
database file. SET operates on either regions or files.
The format of the SET command is:
SE[T] -qualifier... filename or region-list
The filename or region-list identifies the target of the SET.
The SET command must include one of the following qualifiers:
o -FILE
o -REGION
The optional qualifiers are:
o -G[LOBAL_BUFFERS]=integer
o -[NO]JOURNAL[=journal-option-list]
2 Qualifiers
-FILE
-F[ILE]
Specifies that the argument is a filename for a single database file.
The -FILE qualifier is
incompatible with the -REGION qualifier.
-REGION
-R[EGION]
Specifies that the argument is a region-list which identifies database
file(s) mapped by the current
Global Directory. SET -REGION changes multiple files when the
parameter contains a list and-or
wildcards. The -REGION qualifier is incompatible with the -FILE
qualifier.
The following qualifiers determine the action(s) for the SET.
-GLOBAL_BUFFERS
-G[LOBAL_BUFFERS]=integer
Specifies the number of cache buffers for a BG database. For
information on determining good
working sizes for GLOBAL_BUFFERS, refer to the "Global Directory
Editor" chapter of the GT.M
Administration and Operations Guide.
The minimum is 64 buffers and the maximum is 4096 buffers. By default,
MUPIP CREATE
establishes GLOBAL_BUFFERS using information entered in the Global
Directory with GDE.
-JOURNAL
-[NO]J[OURNAL][=journal-option-list]
Specifies whether the database allows journaling and, if it does,
characteristics for the journal file.
-NOJOURNAL specifies the database does not allow journaling.
-NOJOURNAL does not accept an
argument assignment.
-JOURNAL specifies journaling is allowed. -JOURNAL takes one or more
arguments in a
journal-option-list.
For a full description of the -JOURNAL qualifier and its keywords,
refer the "GT.M Journaling"
chapter in the GT.M Administration and Operations Guide.
1 SET
SE[T]
MUPIP SET -JOURNAL determines whether a specified file or region(s)
have journaling activated. SET requires sole access to the database.
SET operates on either regions or files.
The format for the SET command is:
SE[T] -qualifier... file-spec or region-list
The file-specification or region-list identifies the target of the
SET. Region-names separated by commas (,) make up a region-list. For
a summary table of MUPIP commands and qualifiers including MUPIP SET,
refer to the MUPIP chapter in the GT.M Administration and Operations
Guide.
2 Object_qualifiers
Object Qualifiers
-FILE
-F[ILE]
Specifies that the argument contains a file-specification for a
single database file. The -FILE qualifier is incompatible with
the -REGION qualifier.
-REGION
-R[EGION]
Specifies that the argument contains a region-name which,
through the mapping of the current Global Directory, identifies
a database file. SET -REGION modifies multiple files when the
parameter contains more than one name. The -REGION qualifier is
incompatible with the -FILE qualifier.
2 Action_qualifiers
Action Qualifiers
/GLOBAL_BUFFERS
/G[LOBAL_BUFFERS]=integer
Specifies the number of cache buffers for a BG database. For
information on determining good working sizes of GLOBAL_BUFFERS,
refer to the "Global Directory Editor" chapter of the GT.M
Administration and Operations Guide.
The minimum is 64 buffers and the maximum is 4096 buffers. By
default, MUPIP CREATE establishes GLOBAL_BUFFERS from
information entered in the Global Directory with GDE.
-JOURNAL
-[NO]J[OURNAL][=journal-option-list]
Specifies whether the database allows journaling and, if it
does, characteristics for the journal file.
-NOJOURNAL specifies that the database does not allow
journaling. -NOJOURNAL does not accept an argument assignment.
-NOJOURNAL does not create new journal files. When a database
has been SET -NOJOURNAL, it appears to have no journaling file
name or other characteristics.
-JOURNAL= enables journaling for a database file. -JOURNAL=
takes one or more arguments in a journal-option-list. Except
when used with the OFF option, SET -JOURNAL= always overwrites
any existing versions of the specified journal file(s). The
journal-option-list contains keywords separated with commas (,)
enclosed in parentheses (). When the list contains only one
keyword, the parentheses are optional.
For details on the list refer to the journal-option-list topic.
2 journal-option-list
journal-option-list elements
The following topics detail the journal-option-list elements.
3 ON
ON
ON specifies that MUPIP create a new journal file and that GT.M
record subsequent updates to the database in that journal file.
A SET -JOURNAL=ON must include either BEFORE_IMAGE or
NOBEFORE_IMAGE in the accompanying journal-option-list. When a
database has been SET -JOURNAL=ON, GT.M journals updates to that
file.
By default, SET -JOURNAL= turns journaling on.
3 OFF
OFF
OFF specifies that GT.M not record subsequent updates to the
database in the journal file. OFF may also be used to set up
journaling characteristics without creating a journal file or
starting journaling. When a database has been SET -JOURNAL=OFF,
it has established journal characteristics ready to turn ON, but
GT.M does not journal updates to that file.
By default, SET -JOURNAL= turns journaling on.
3 BEFORE_IMAGE
[NO]BE[FORE_IMAGE]
[NO]BEFORE_IMAGE controls whether the journal should capture
before-images of information that an update is about to modify.
MM databases must use NOBEFORE_IMAGE journaling. A SET
-JOURNAL=ON must include either BEFORE_IMAGE or NOBEFORE_IMAGE
in the accompanying journal-option-list.
A BEFORE_IMAGE journal permits the possibility of performing
"roll-back" recovery (i.e., Backward Recovery) of the associated
database. BEFORE_IMAGE increases the load on I/O and CPU
resources and therefore may affect performance.
3 FILE_NAME
F[ILE_NAME]=file-specification
FILE_NAME=file-specification specifies the name of the journal
file. FILE_NAME is incompatible with SET -REGION.
Journal file-specifications are limited to 55 characters.
By default, MUPIP CREATE establishes the journal
file-specification from the Global Directory. If the Global
Directory does not contain a journal file-specification SET
-JOURNAL derives the journal file-specification from the
database file-specification using a file type of .MJL. Note that
because the default usually places the journal file on the same
disk drive as the database file, it does not protect well
against disk hardware failures.
3 ALLOCATION
A[LLOCATION]=blocks
ALLOCATION=blocks specifies the initial size of the journal file
in blocks. Because frequent journal file extensions degrade
run-time performance, make journal file allocation ample for a
production database.
The minimum ALLOCATION is 10 blocks and the maximum is
16,777,216 blocks.
If journaling characteristics have not been previously
established by GDE or a prior SET -FILE -JOURNAL and a SET
-JOURNAL= specifies ALLOCATION but does not specify EXTENSION,
the command automatically changes EXTENSION to equal 10% of the
new ALLOCATION.
By default, MUPIP CREATE establishes the ALLOCATION from the
Global Directory, where the Greystone supplied default is 100
blocks. If the Global Directory does not contain ALLOCATION
information, SET -JOURNAL uses a default of 100 blocks.
3 EXTENSION=blocks
E[XTENSION]=blocks
EXTENSION=blocks specifies the size by which a journal file
extends when it becomes full. EXTENSION=0 disables automatic
journal file extension. While this technique exerts firm control
over disk space consumption by a journal file, running out of
journal file space terminates journaling for the region. Because
frequent journal file extensions degrade run-time performance,
make the journal file extension ample for a production database.
The minimum EXTENSION is 0 blocks and the maximum is 65,536
blocks.
If journaling characteristics have not been previously
established by GDE or a prior SET -FILE -JOURNAL and a SET
-JOURNAL= specifies ALLOCATION but does not specify EXTENSION,
the command automatically changes EXTENSION to equal 10% of the
new ALLOCATION.
By default, MUPIP CREATE establishes the EXTENSION from the
Global Directory, where the Greystone-supplied default is 100
blocks. If the Global Directory does not contain EXTENSION
information and the SET -JOURNAL does not specify either
ALLOCATION or EXTENSION, MUPIP uses a default of 100 blocks.
3 BUFFER_SIZE=pages
BU[FFER_SIZE]=pages
BUFFER_SIZE=pages specifies the amount of memory used to buffer
journal file output.
A larger BUFFER_SIZE usually smooths and improves run-time
performance by allowing larger, less frequent writes. On the
other hand, a larger BUFFER_SIZE requires more memory resources,
which may be scarce. A larger BUFFER_SIZE provides more room for
journal records in buffered memory and therefore increases the
number of update records that may be lost in a system failure.
The minimum BUFFER_SIZE is enough 512-byte pages to hold two GDS
database blocks and the maximum is 2000 pages.
By default, MUPIP CREATE establishes the BUFFER_SIZE from the
Global Directory, where the Greystone-supplied default is 128
pages. If the Global Directory does not contain BUFFER_SIZE
information, SET -JOURNAL uses a default of 128 pages.
2 Examples
SET -JOURNAL Examples
Example
$ mupip set -file -journal=(nobefore,buff=128) cus.dat
This initiates journaling for the database file cus.dat. Because
the parameters include NOBEFORE, subsequent JOURNAL commands to
-RECOVER the database updates in the journal must specify -FORWARD.
The journal file created has the name cus.mjl.
Example
mupip set -region -journal=(before,alloc=50000,ext=5000)
This enables journaling with BEFORE_IMAGES on all regions of the
current Global Directory and gives each journal an ALLOCATION of
50000 blocks and an EXTENSION of 5000 blocks. If the regions have
significantly different levels of update, either set the ALLOCATION
and EXTENSION in the Global Directory before the MUPIP CREATE(s)
or use several MUPIP SET -FILE commands.
Example
mupip set -region -journal=before joel.dat
This declares journaling active with before-images for all regions
of the current Global Directory when they are next opened.
Example
mupip set -file -nojournal MUMPS.DAT
This disables journaling on the database file MUMPS.DAT in the
current default directory.
1 STOP
ST[OP]
STOP terminates a GT.M image. The image executes an orderly rundown of all
databases in which it
currently has an interest and then exits. A MUPIP STOP may also be used to
stop non-GT.M images.
The format of the STOP command is:
ST[OP] process-id
Use the shell command ps to display a list of active process names and
process identifiers (PIDs).