If you’re using a Mac to access files on a NAS using AFP, and it’s taking a long time to list folder contents and open files — or you’re seeing timeout errors — your CNID DB may be unhappy.
Say what?
Macs talk to networked storage using any of four different protocols: Apple File Protocol (AFP), Server Message Block (SMB, a.k.a. CIFS), Network File System (NFS), or the snappily-named Web Distributed Authoring and Versioning (WebDAV). Indeed, many NAS boxes offer three or more of these simultaneously.
Macs historically used AFP, and while Apple has deprecated AFP in favor of SMB/CIFS, AFP still has some advantages. On my NetGear ReadyNAS boxes, for example, AFP properly preserves file-creation times, while SMB does not. AFP connections have historically been faster than SMB connections, too, at least before SMB v3 became widespread. Even using MacOS 11, I’m finding that AFP connections behave more reliably and consistently than SMB connections, especially on NAS volumes with restricted access credentials.
AFP originated with the classic Mac OS, and it handles a rich selection of Mac-specific metadata: resource forks, type and creator codes, and the like. These metadata don’t map neatly into the native filesystems of NASes — or, really, anything other than Macs.
The critical difference at the heart of this story is that AFP, like the Mac’s file systems dating back to HFS, refers to files not by name, but by CNID: a Content Node ID. The open-source Netatalk fileserver that provides AFP services in most common NAS boxes stores the mapping between CNIDs and filenames in a database.
CNID Conniptions
As long as the CNID database is in sync with reality, AFP file services run smoothly. It’s when the database no longer matches the files on disk that slowdowns occur, as Netatalk has to resolve discrepancies on the fly. The database can fall out of sync when you access the same volume with multiple services: files created or deleted via SMB or NFS aren’t tracked in the CNID database. Drastic database de-optimization may also happen during software updates, though that is rare.
As differences accumulate, AFP services bog down, sometimes to the point where the Mac Finder times out trying to fetch directory listings. It takes longer to open files; TimeMachine, SuperDuper!, and Carbon Copy Cloner (CCC ) backups take more time to execute; you might not even be able to browse a network volume.
The Fix for AFP
Netatalk provides the dbd tool to update and resync the CNID database.
dbd [-cfFstuvV] volumepath
As far as I know, no NAS vendor makes dbd available in their management GUI, so you’ll have to open a terminal session to invoke it (how to do so is left as an exercise for the reader, as it varies by vendor and by firmware version). dbd can be run in an incremental-update, non-destructive mode, and I recommend trying that first. In my experience, you can still perform normal file ops while dbd is running, with little impact. Here’s a run with the -t “show statistics” and -v “verbose” flags on a test volume holding exactly two files:
(Note the “file without meta EA” warning; the “white shading.jpg” file was copied to the NAS using SMB, so Mac-specific metadata, a.k.a. Extended Attributes, did not transfer.)
If the database is so far gone that you see timeout and disconnect messages on the console as dbd itself runs, try re-running with the -f “nuke it from orbit and start fresh” flag.
In this case, the -f option took longer to run, but with a badly out-of-sync database it runs quite a bit faster than dbd without -f.
With my Ultra 6 boxes and their weedy little Atom CPUs, AFP access is at a standstill while nuke-it-from-orbit is running, but at least Netatalk won’t be tripping over its own damaged database and dbd will have the chance to create a new, clean version.
In my case, I had one data volume with 10+ TB of data that had been accessed from Macs via AFP, SMB, and NFS (the latter two for edit-in-place sessions with FCPX); I also had Windows 7 and Windows 10 machines writing to it using SMB. After several months of this abuse AFP access became noticeably slower. Despite running the server’s defrag and balance processes, AFP remained poky, and finally it started throwing timeouts both in Finder and during nightly CCC backups. dbd run incrementally ran very slowly, and spewed multiple timeout errors; after several hours I killed it and ran dbd -f, which ran much faster… though it still took about a day to make everything neat and clean again. But once it was done, AFP worked perfectly again. And when I run dbd in update mode on this rebuilt database, it processes 1.15 million files in a mere 638 seconds.
I now run dbd every three months as part of regular maintenance, just like defrag and balance.
Why not avoid the problem in the first place?
A practical rule is, “if it hurts, don’t do it”, and that certainly holds in this case: pick one file-sharing protocol, and stick to it. Indeed, most NAS vendors warn you away from using multiple protocols on the same volume, though sometimes that warning is buried in a knowledge-base article instead of being presented in the user manual or management UI.*
But, warnings or not, you may still have a need or a desire to use AFP with some Macs on a filesystem that is (or was) also accessed using other protocols. Running dbd can clear up any lingering problems; adding it to your maintenance schedule can keep those problems from recurring in the future.
* “It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’.” —Douglas Adams, The Hitchhiker’s Guide to the Galaxy