Mike's picture by Mike | January 18, 2023

We field a lot of support requests, and similar to a doctor's office, we see some extreme cases. One of those interesting extreme cases are folders with high file counts. Any time a folder has more than a few thousand items in it, the filesystem is going to be a lot slower when working with that folder. Adding a new file, for example, requires that the filesystem compare the new item name to the name of every other file in the folder to check for conflicts, so trivial tasks like that will take progressively longer as the file count increases. Gathering the enormous file list will also take progressively longer as the list gets larger. The performance hit is even more noticeable on rotational disks and network volumes, so we often see these sticking out in backup tasks.

Sometimes high folder counts can bring a backup task to a halt

Task encountering a folder with a high file count

Last week, one of our users found the task as shown above. Upon closer analysis, we determined that the "media" folder had 181,274 files in it. In other words, more than 10% of the files on the whole startup disk were in that "media" folder. In extreme cases like this, the delay to retrieve a file list can be so long (i.e. longer than 10 minutes) that the task aborts with an error, e.g.:

The task was aborted because a subtask did not complete in a reasonable amount of time. Or,

The task was aborted because the destination filesystem is not responding.

These are typically pretty wild cases that benefit from some human intervention. For example, I recall seeing an AddressBook application support folder that had more than 2.5 million image thumbnails in it. Nobody has that many contacts! That wasn't an isolated incident, either, we've seen that same AddressBook folder implicated at least a half dozen times. Cases like that usually point out a software failure in the application that is creating the files – a failure to sanity-check its data store, or to put a sane limit on how many log files it creates. Sometimes, like in the Telegram example above, it's just short-sighted design.

For a contrasting example, consider how Mail organizes a potentially astronomic list of files. If you navigate to the hidden Library folder in your home folder, then to Mail > V10 > {any UUID} > {any mailbox} > {another UUID} > Data, you'll see folders named by number, four layers deep, until you finally get to a Messages folder with actual files in it. While this nested hierarchy makes it annoying for a human to find a specific email file, that organization limits any individual folder's file count and optimizes Mail's ability to quickly collect those resources from the filesystem.

Some data stores with potentially-high file counts

Here's a list of some common culprits we'... Read More

ccc

Mike's picture by Mike | October 18, 2022
CCC is ready for macOS Ventura

It's Fall here on the top of the globe, which means that temps are getting cooler, pumpkin seems to be in everything, and apples are in season. And of course, Apple is about to drop another new upgrade to macOS: Ventura. We've been testing the new OS over the summer, and I'm pleased to report that CCC is ready to protect your data before and after you apply this upgrade – we added official Ventura support to CCC 6.1.3, which we posted back in September.

Every year we make a handful of changes to CCC to support changes that Apple makes in the new OS. We have a mixed bag this year, and I wanted to point out just a couple things that work a little differently. Overall, the changes are pretty bland, which means this article will not be very exciting. So to spice things up, I added a recipe for Pumpkin Spice Muffins at the bottom.

System Preferences → System Settings

The name change seems innocuous, but the changes that Apple made to this application are really significant. Initially I was really flustered with the new layout. I finally gave up and accepted that I'll just have to use the search bar to find things. Apple was always meddling with the order of items in System Preferences, though, and I recall feeling disoriented every year in this application. Now that I've accepted using the Search feature, I think I'm actually a bit faster at using it.

Privacy Settings

Apple introduced new privacy settings in macOS Mojave, and they've been tweaking them in every OS release. These settings have been pretty annoying for developers and users alike. Apple doesn't offer a programmatic interface for the "Full Disk Access" controls, so the best we can do is coach users through the procedure. Normally that works fine, but it was still multiple steps, and users could take a wrong turn at any of those steps.

I'm delighted to report that Apple made a modicum of improvement to this process in Ventura. They finally dropped the padlock paradigm, so pre-authentication is no longer required to unlock the settings in the Privacy panel. As a result, the procedure for granting Full Disk access to CCC and its helper tool on Ventura is now one step! When you start dragging CCC's "privacy fish", CCC will open the System Settings application and reveal the Full Disk Access controls, ready for the drop. At that point you're prompted to authenticate, and then you're done. Progress!

Login Items

Yeah, "login items" – you can imagine how exciting my summer was 😉. In Ventura, Apple deprecated an older mechanism for adding login items, and replaced it with a new mechanism. Normally I don't immediately adopt brand-new features like this (because it makes it more complicated to support multiple OSes with one... Read More

ccc

Mike's picture by Mike | May 19, 2021

We've had so many new features in the oven for a while, and now we're finally ready to share it with the world! CCC 6 offers unprecedented accountability for your backups and insight into what's changing on your Mac, plus a brand new file copier that's faster, smarter, and designed to adapt to Apple's fast pace of OS and filesystem innovation.


Take a look at what's new in CCC 6:

The core CCC backup features you know and love are now better than ever!

Faster backups with our next-generation file copier

We've completely rebuilt our file copier to take advantage of the performance characteristics of modern storage. Built on a multi-threaded design, our new file copier provides a foundation for many of the new features noted below, and paves the way for many new features in the future.

Quick Update: Update your backups up to 20X faster

CCC can now tap into the macOS FSEvents service for a list of folders modified on the source since the last backup rather than scanning every folder for changes. Especially for tasks involving a destination network volume, the performance benefit of this feature cannot be overstated!

Redesigned Interface with Dark Mode

It’s been a while since we’ve given CCC’s interface an update and think you’ll enjoy its sleek new look. CCC’s main window is reorganized to make it smaller while making many of the controls and font sizes larger. We completely redesigned every element in CCC to offer a high quality Dark Mode experience. CCC now offers more detailed progress indication while a task is running, including a time remaining estimate. File processing and transfer rates are now... Read More

ccc

Mike's picture by Mike | May 19, 2021

CCC 5.1.27 and CCC 6 can make bootable copies of the system on Intel and Apple Silicon Macs (11.3+) right now, and we'll continue to support that functionality as long as macOS supports it.

But as Apple's platform continues to evolve, we have to design our recovery strategies around the current hardware capabilities. A bootable external device may not be a part of that strategy. CCC can do so much more than just make copies of the system, and now is the right time to revisit your backup strategy and make it even better with some of the new features in CCC 6.


For decades, Mac users have taken for granted the Mac's "External Boot" feature. Prior to Mac OS X, people could simply drag and drop the System folder from one volume to another; presto, external boot volume. When Apple made it more complicated with Mac OS X, we pioneered the "bootable backup" solution (nearly 20 years ago!), and this has been a feature we've reliably supported on every new Mac and every new OS since then.

But Apple has never been afraid of shaking things up to blaze new trails. Big Sur and the new Apple Silicon Macs have shaken up the way Mac users will recover from hardware failure.

Big Sur's Big Change

All on its own, Big Sur introduced a significant new change to the creation of an external boot device. The operating system now resides on a cryptographically sealed "Signed System Volume" that can only be copied by an Apple-proprietary utility, "Apple Software Restore" (ASR). We were already familiar with ASR, so fairly quickly we were making bootable copies of Big Sur back in November. It hasn't been perfect though. We've performed tens of thousands of ASR clones at this point, and we've discovered that ASR is just not as robust as our own file copier. There are many scenarios where ASR simply fails with no explanation. ASR is also very one-dimensional; choosing to copy the system requires that we sacrifice other backup features, e.g. we cannot copy the system and retain versioned backups of your data, we can't evaluate what was copied, we can't exclude items from the initial backup, we can't save checksum data for later verification. So, while we're certainly able to make a bootable copy of the system with ASR, it starts to feel like using it causes us to lose sight of what's actually important to back up – your irreplaceable data.

Apple Software Restore isn't quite ready for the new Apple Silicon Mac storage

When Apple introduced Apple Silicon Macs, we discovered another snag. The "Apple Fabric" storage in these Macs offers per-file encryption keys (like the storage in iOS devices), and for months, ASR didn't work with it. Apple partially resolved that in macOS 11.3, but even now using ASR to clone the system back to the internal storage of... Read More

Mike's picture by Mike | November 3, 2020

CCC 5.1.23+ can make bootable backups of Big Sur on Intel and Apple Silicon Macs.

Update May 13, 2021: CCC 5.1.27 can now make bootable backups of a Big Sur startup disk on all Macs.

Update Nov 24, 2020: CCC 5.1.23 can now make bootable backups of a Big Sur startup disk on Intel-based Macs. Support for System volume cloning on Apple Silicon Macs is disabled for now because Apple's APFS replication utility does not currently work on that platform. When Apple fixes that, we'll post an update to CCC that restores support for making bootable backups on Apple Silicon Macs.

CCC is a native application on Apple Silicon and is 100% compatible with Apple Silicon Macs
CCC will automatically proceed with a Data Volume backup when backing up an APFS Volume Group on Apple Silicon Macs — that's a complete backup of your data, applications, and system settings. If you would like to make your Apple Silicon Mac backup bootable, you can install Big Sur onto the CCC Data Volume backup. Please keep in mind, however, that your CCC backup does not have to be bootable for you to be able to restore data from it.


With the announcement of macOS Big Sur, Apple has retired Mac OS X (10) and replaced it with macOS 11. As with every upgrade since the original release of Mac OS X, we have to make changes to CCC to accommodate the changes in this new OS. As the numeric change would suggest, though, this is the biggest change to macOS since Apple introduced Mac OS X roughly 20 years ago. The system now resides on a "Signed System Volume". This volume is cryptographically sealed, and that seal can only be applied by Apple; ordinary copies of the System volume are non-bootable without Apple's seal. To create a functional copy of the macOS 11 System volume, we have to use an Apple tool to copy the system, or install macOS onto the backup.

Does this mean that we can no longer have bootable backups?

I can certainly understand why people are concerned about the future of this solution. Thanks to these massive system changes and some bugs in the version of Big Sur that Apple intends to ship, nobody can make a proper copy of the System volume right now, not even with Apple's proprietary utilities. Based on that statement alone, and a suggestion from one of my competitors to just give up and use Time Machine instead (which does not make bootable backups, nor back up the System), someone could falsely conclude that it's impossible to ... Read More