Backup: Long Answer

An EMEN2 database environment contains three types of files: database files, log files, and region files.

Database files contain key/value pairs that comprise all the records in the database, as well as a number of database files used for indexes. Database files are contained in $DB_HOME/data and subdirectories. Log files contain data from all committed transactions, and are stored in $DB_HOME/log as log.XX, where XX are consecutive integers starting from 1.

To provide guarantees about transaction atomicity and durability, changes are first written to log files on stable storage before a transaction is marked as committed. The database files are not updated until this has been completed. In the event of a crash or hardware failure, the database files can be checked against the log files to correct any errors or missing data.

Because a cold backup copies the database files, the database must be stopped so they are not changed while the backup is in progress. Once a cold backup is made, it can be updated with a hot backup. A hot backup only copies new log files, which are append-only, and does not require the database files to be stable during the backup.

backup.py

This page is currently being rewritten to avoid displaying incorrect or out-of-date information. EMEN2 provides several mechanisms for backing up metadata and raw data -- please contact email me if you would like specifics while I rewrite this page.