CCC offers several different ways to verify data on the source and destination. The procedure that you use will depend on when you want to verify data and why you want to verify the data.
- Backup Health Check: Verify before copying, automatically replace corrupted destination files
- Postflight verification: Verify files that were copied during the current task event [New in CCC 6!]
- Ad hoc verification: Verify the source or the destination against the "last known state" [New in CCC 6!]
Backup Health Check: Verify before copying, automatically replace corrupted destination files
CCC normally uses file size and modification date to determine whether a file should be copied. When you use the Find and replace corrupted files setting (Advanced Settings > Performance & Analysis), CCC will calculate a checksum of every file on the source and every corresponding file on the destination. If the checksums differ:
- If the source file is 100% readable, CCC will recopy the file to the destination.
- If the source file is not completely readable, the existing destination file will be left in place. CCC will record an error for the file in Task History and raise it to your attention when the task completes.
This option will increase your backup time (because CCC is tasked with re-reading every file on the source and destination), but it will expose any corrupted files within your backup set on the source and destination.
When and why would I use this feature?
Media failures occur on nearly every hard drive at some point in the hard drive's life. These errors affect your data randomly, and go undetected until an attempt is made to read data from the failed sector of media. If a file has not been modified since a previous (successful) backup, CCC will not ordinarily attempt to read every byte of that file's content. As a result, it is possible for a corrupted file to go unnoticed on your source or destination volume. Obviously this is a concern if the file is important, and one day you actually need to recover the contents of that file. Use the "Find and replace corrupted files" feature to avoid and to proactively deter bit rot.
Frequent use of the checksum calculation option is unnecessary and may be a burden upon your productivity, so CCC offers additional options to limit how frequently the checksumming occurs (e.g. weekly, monthly, quarterly, on specific days of the week, etc.).
Note: CCC will never replace a valid file on your destination with an unreadable, corrupt file from the source. If CCC cannot read a file on your source volume, any existing backup of that file will remain intact on your backup volume and CCC will report an error, advising you to replace the source file with the intact backup version. The Find and replace corrupted files setting will only automatically replace corrupted files on the destination, and only when the source file is completely readable.
What is a "corrupted" or "unreadable" file?
With regard to files on the source, CCC's Find and replace corrupted files option specifically refers to files that cannot be physically read from the disk. It does not refer to files that have been mistakenly or maliciously altered such that they cannot be opened by the application that created them.
Postflight verification: Verify files that were copied during the current task event
As CCC copies files to the destination, it calculates a checksum of the data it's writing. If your task is configured to use the Re-verify files that were copied setting (Advanced Settings > Postflight), at the end of the task, CCC will read the destination files that were copied and verify that the data matches the data that was initially read from the source.
When and why would I use this feature?
Generally this kind of verification is unnecessary — if no errors were reported by the destination filesystem while copying a file nor when closing the file on the destination, you should expect the destination device to have permanently retained the data of that file. However, media failures are only discovered when data is read from the destination device, so it is possible for a device to accept writes without failure, but then fail to deliver the data on a subsequent read due to media failure. Especially if you are migrating data to a new device, or if you are planning to delete items from the source after completing the backup, this additional verification confirms that the freshly-written files are intact on the destination.
Ad hoc verification: Verify the source or the destination against the "last known state"
When CCC copies files to your destination, it maintains a record of the files that were copied (by default; that setting is available in Advanced Settings > Performance & Analysis > "Maintain a record of transactions"). That record includes the size, modification date, and a checksum of the latest version of each file. On demand, CCC can evaluate either the source or the destination against those records to determine if any files differ since the files were copied. Click on the Source or Destination selector, then choose Verify files copied by this task to start that verification.
When and why would I use this feature?
Unlike the previous two features that offer automated verification of files based on a comparison of the source against the destination, this feature is something you would use in an ad hoc manner. Suppose, for example, that you just installed some software, and now you're a bit concerned that something untoward has happened to your source volume. You can open CCC, click on the Source selector, then choose Verify files copied by this task. CCC will then read every file on the source and compare its checksum against the checksum of the file when it was last copied by the selected task. If any files have been modified since then, CCC will present those to you, along with context about the change (e.g. modification date, size, and/or checksum differences).
Another example: Suppose you want to restore some files from your backup, but prior to doing so, you want to verify that the files haven't been modified since the last CCC backup task. Open CCC, click on the Destination selector, then choose Verify files copied by this task. This time CCC will read the files on the destination and compare them against the same task records that hold the "last known state" information about those files.
The verification report shows some differences. What do these mean?
The verification report shows the status of items found on the selected volume based on the attributes of the file at the last backup event:
This item matches the transaction record
This item was added since the task last ran
This item's content changed without affecting the size or modification date (flagged false positive, see below)
The modification date of this item differs
The size of this item differs
This item's checksum differs
This file is no longer present
No transaction record (see below)
Click on the status icon of the selected item to reveal the actual and expected size, modification date, and checksum of the selected item.
False positives
There are a handful of file types whose content can change without affecting the size or modification date. Database memory files are a good example. Based on our past experience, CCC will flag some items as "false positives", meaning that while the content changed without affecting the size or modification date, the modification is unlikely to be malevolent, nor an indication of something wrong with the file or the backup procedure.
No transaction record
Transaction records are created when CCC 6 copies a file from the source to the destination. If you've recently upgraded to CCC 6, your destination may have an existing backup, but CCC won't have any transaction records for those files that were copied with an older version of CCC. If you perform a verification on an existing source or backup volume, only the files that were copied since upgrading to CCC v6 will have transaction records. Likewise, any items that are excluded from the backup task or protected on the destination by a filter or the SafetyNet feature will not have transaction records.
Rather than erasing your destination and re-establishing the backup to create those transactions, you can enable the Find and replace corrupted files setting in Advanced Settings (Performance & Analysis) and run your task once to establish the transaction records.
What do I do about differences noted in the verification report?
When the verification report shows differences, that means that the files on the selected volume are different now than they were when the selected task last copied those items. Before you draw any conclusions about differences identified by CCC's verification report, it's essential to keep in mind:
- CCC can only verify files that were copied by the selected task. Files that were (legitimately) modified by another backup task or another application will appear as "different". Likewise, files that are excluded from the backup task cannot be verified, and will appear as differences.
- It's normal for files to be modified on the source; differences identified on the source do not necessarily indicate an error condition, you may simply need to run your backup task again to get those files updated on the destination, and updated in CCC's transaction records.
If you're seeing differences on a destination volume, run the backup task again using CCC's Find and replace corrupted files setting:
- For reference, you can save a copy of the verification report before closing the window. Click the "Save Verification Report" icon in the top-right corner to save the report.
- Close the Verification window
- Click the Advanced Settings button at the bottom of CCC's window
- Click the Performance & Analysis tab
- Check the box next to Find and replace corrupted files
- Choose Only on the next run from the popup menu to the right of the "Find and replace corrupted files" setting
- Click the Done button
- Click the Start button (or Save, then Start)
- When the task has completed, click on the Destination selector and choose Verify files copied by this task to repeat the verification.
Differences found on a source volume indicate changes that were made to the source since the backup task last ran, or otherwise outside of the purview of the selected CCC task. If you are seeing differences on the source, then you should consider each difference that is noted and decide if the transaction records are simply out of date (i.e. if a file has been modified since the last backup, then you may simply need to re-run the backup to update the backup file and the transaction record), or if the files should be restored from a verified backup instead.
Verification cannot be effective when "Strict volume identification" is disabled and multiple destination volumes are used
If you're using a single task with multiple destinations, CCC isn't going to track transactions for each destination volume separately. As a result, attempts to verify a volume will only be effective for the last volume that was updated by your task. If you use the verification feature frequently, then we recommend that you use separate tasks for each of your destination volumes.
Transaction privacy and disabling transaction collection
Transaction records are maintained on a per-task basis in an encrypted database. These databases are only accessible to administrator users, and can only be accessed via CCC, and only on the Mac upon which they are created.
Transaction records for any given task are deleted when:
- The CCC task is deleted
- All task events associated with the task are removed in the Task History window
- After changing the source or destination of the task, if you choose the Remove Audit option
- When you specifically delete the Audit records for a task in CCC's Settings > DB Diagnostic > Audit Records
- When transaction collection is disabled for the task (see below)
To disable the collection of transactions in any particular task:
- Click the Advanced Settings button at the bottom of the window
- Click the Performance & Analysis tab
- Uncheck the box next to Maintain a record of transactions
Can I delete, or otherwise reduce the size of the task audit records?
Task audit records are stored in a database on your startup disk, at Macintosh HD > Library > Application Support > com.bombich.ccc > TaskDBsV3. Tasks that record a lot of transactions will eventually create a large database file. CCC does take measures to limit the size and growth of these files, but that activity is balanced against a desire to retain transactions for as many task events as possible. Once transaction records are removed, you can no longer see the changes that were associated with a particular task event. If all transactions are removed for a task, the verification functionality noted above will no longer be available for that task.
You can see a list of these databases, as well as their size and health status, in CCC's Settings > DB Diagnostic > Audit Records.
If you would like to delete all of the records associated with a task:
- Open CCC Settings
- Click DB Diagnostic in the toolbar
- Select the Audit Records tab
- Select a task
- Click the Delete Records button
Alternatively, if you would simply prefer to reduce the size of a task's database, you can delete some of the task history events associated with that task:
- Choose Task History from CCC's Window menu
- Select a task from the filter popup menu at the top of the window
- Select the some of the oldest events (i.e. click and drag to select multiple items)
- Right-click on the selection and choose Remove