The DiskArbitration framework, and diskarbitrationd specifically, is the primary arbiter of volumes and disks under Mac OS X. Like Filesystem Bundles, there is no existing documentation of the commands DiskArbitration expects you to implement, and what information it expects you to return.

I’m here to explain it in gruesome detail.

NOTICE: it is assumed you have read the documents on Filesystem Bundles and are familiar with the terminology, at least as far as FSMediaTypes and FSPersonalities are concerned. The terms “media type” and “personality” will be used extensively, so please know them well.

In every media type, you specify a probe executable and arguments. In every personality, you specify formatting and repair executables. As of Mac OS X 10.4, diskarbitrationd chooses the media type and personality using calls to the internal function ___CFDictionaryGetAnyValue(). This function returns the first value as obtained using CFDictionaryGetKeysAndValues(), which is nondeterministic; it could be any media type, or any personality.

Thus, it is advisable that all of your media types and personalities use the same executables. Arguments may differ on a per-type or per-personality basis, as those are only respected by DiskManagement-based applications. The flags used by diskarbitrationd are hard-coded.

Without further ado, here are the major groups of commands in this document. Make certain you read through the terminology section first.

• Probe commands
• Repair commands
• Mounting
• Formatting

Terminology

Before diving into the actual specifications, it is crucial that you be familiar with the terminology I’ll be using. Many of the terms were discussed in my post about Filesystem Bundles. Here are the new terms I’ll use, with which you should make yourself familiar before continuing:

Executables

• [PROBE]

  This refers to the probe executable referenced by the FSProbeExecutable inside the “randomly”-selected FSMediaTypes record chosen by diskarbitrationd.

• [FORMAT]

  This refers to the formatting executable referenced by the FSFormatExecutable inside the “randomly”-selected FSPersonalities record chosen by diskarbitrationd.

• [REPAIR]

  This refers to the repair executable referenced by the FSRepairExecutable inside the same FSPersonalities record as used to find the [FORMAT] command.

Arguments

• [bsd_name]

  This refers to the basename of a disk device; this is my notation of the value that Mac OS X also calls the “BSD Name” of a device. Given /dev/disk0s3, the [bsd_name] is just disk0s3.

• [device]

  This refers to the block device for a given disk. This is the argument that people familiar with Linux and/or UNIX would expect. Given /dev/disk0s3, the [device] is precisely /dev/disk0s3.

• [raw_device]

  This refers to the “raw” counterpart to a disk’s block device as referenced by [device]. The effects of using one versus the other can be quite pronounced, and diskarbitrationd uses different variations in different places. Given /dev/disk0s3, the [raw_device] is /dev/rdisk0s3.

Probe commands

Since the concept of probing a disk is specific to diskarbitration and BSD’s idea of a loadable filesystem, the probe commands return statuses are expected to use the FSUR_* constants found in <sys/lodable_fs.h>. These commands should never be accessed by a shell, as they don’t use the “zero-as-success” exit code expected of command-line tools.

These probing commands will be first contact between your filesystem and diskarbitrationd. When a disk first appears, either at boot or when a device is attached to the system, diskarbitrationd goes through the following procedures to check if your filesystem is actually present. However, these checks are only performed if your filesystem has a matching record in FSMediaTypes. The checks are as follows:

1. Probe device
2. Get UUID
3. Set UUID

Probe device

When diskarbitrationd finds a matching media type, it will decide to probe the disk using your filesystem. Since the checks to match a media type are only rudimentary, this is basically diskarbitrationd doing finer-grained checks to determine if the new device actually contains a filesystem you can mount. You will see the following invocation from diskarbitrationd:

[PROBE] -p [bsd_name] removable readonly

You’ll notice that the last two arguments, removable and readonly are hard-coded. They will always be passed by diskarbitrationd; they are two of the string constants defined in <sys/loadable_fs.h>. Their presence seems irrelevant and arcane; in most (if not all) cases, they can be safely ignored.

RETURNS: FSUR_RECOGNIZED.

Any other value is taken to indicate failure, even if it’s another FSUR_ constant such as FSUR_IO_SUCCESS. Also, avoid returning zero, as other tools may interpret that as success when an operation actually failed.

Get UUID

When the initial probe succeeds, diskarbitrationd will immediately invoke your probe executable again. This time, since you claimed the filesystem, it’s looking for the volume’s UUID. This may seem like a menial task you can overlook, but if at all possible, it is of great benefit to diskarbitrationd and other Mac OS X facilities if you can provide a UUID. Your tool will be called as such:

[PROBE] -k [bsd_name]

The UUID is returned to diskarbitrationd by printing it to stdout. You are expected to print the uppercase hexadecimal equivalent of your 64- or 128-bit identifier as text. A newline is not required, and your output should not be hyphenated.

If you only print a 64-bit value (16 characters), diskarbitrationd will automatically conver it into a full 128-bit UUID in a deterministic fashion. For example, the HFS+ utility only returns a 64-bit value. NTFS identifiers are also only 64 bits in length.

RETURNS: FSUR_IO_SUCCESS.

This value indicates that you successfully fetched and output the UUID for diskarbitrationd to parse. Any other value indicates failure. Be very careful with this. If you return success, but your output on stdout fails to parse into a UUID, diskarbitrationd will then ask you to set one via the Set UUID operation.

Set UUID

If your volume returns success for the Get UUID operation, yet fails to provide text that can be converted into a UUID, diskarbitrationd will ask you to set a valid UUID on the volume. The assumption is that the conversion failed because the on-disk data was not a valid ID—for example, if your on-disk value was all zeroes. The invocation is as follows:

[PROBE] -s [bsd_name]

You’ll note that diskarbitrationd doesn’t give you a UUID. It just politely asks that you generate a new identifier for the volume and store it on disk, wherever and however that is appropriately done. For something like NTFS, mkntfs just picks a new random 64-bit value and writes it to disk.

RETURNS: FSUR_IO_SUCCESS.

This indicates that you successfully wrote a new identifier to the disk. If you return success, the new UUID should be immediately readable via the Get UUID operation, as diskarbitrationd will try to fetch it immediately.

Repair commands

These repair commands use the same executable as given by its FSPersonalities, but the arguments are ignored, and diskarbitrationd uses its own set. In the context of diskarbitrationd, these are used for basic verficiation (and/or repair) prior to mounting a volume. Essentially, diskarbitrationd wants to make sure it doesn’t ask you to mount an unclean volume.

If a volume fails verification (and repair, if attempted), then diskarbitrationd WILL NOT fall through to the next media type in the probe order. Thus, if you try to override an internal filesystem, and choke somewhere in verification or repair, then diskarbitrationd will not revert to the old implementation—the user’s volumes will simply fail to mount.

From experience, I can say this confuses a lot of people. When overriding Apple-provided implementations, it is imperative that you only return success during the probe operation if you can actually mount the system; you should not postpone verification logic to this stage.

Here are the two ways (in order) in which diskarbitrationd will call your tools:

• Verify volume
• Repair volume

Verify volume

At this point, diskarbitrationd just wants to do basic verification on the volume. For reasons I cannot entirely explain, diskarbitrationd will actually pass you the raw device associated with the disk, as such:

[REPAIR] -q [raw_device]

This should do checks necessary to make sure the disk is in working order prior to mounting it. If these checks fail, diskarbitrationd WILL NOT fall through to the next filesystem in the probing order; it will just attempt repair.

RETURNS: 0 on success, non-zero on failure.

Repair volume

If your verification fails, then diskarbitrationd will request that you do a lightweight repair on the volume. If you cannot safely repair the volume (in an automatic, without-user-input way), then this should fail. Like the verification, the arguments are simple:

[REPAIR] -y [raw_device]

Again, it uses the raw device, for reasons I do not understand. If you can successfully repair and clean the volume without intervention, that is fantastic; otherwise, this should fail, lest the user will be using an inconsistent filesystem.

RETURNS: 0 on success, non-zero on failure.

Mounting

True to the BSD nature of Mac OS X, diskarbitrationd actually goes through the standard BSD disk-mounting channels. Though you can set a FSMountExecutable in your FSPersonalities records, those are completely ignored by diskarbitrationd. Once diskarbitrationd has successfully located the first filesystem claiming it can mount a volume, it will mount it using the following command:

/sbin/mount -t [type] (-o [options]) [device] [mountpoint]

Of course, since your filesystem is not one known to /sbin/mount, it will in turn launch the following command to actually mount your filesystem:

/sbin/mount_[type] (-o [options]) [device] [mountpoint]

The -o [options] argumenst will only be present if diskarbitrationd decides there are additional arguments that must be passed. If they are present, [options] will be precisely one argument, as a comma-separated list of flags. The [device] argument is described above, and the [mountpoint] flag is self-explanatory. This is how a UNIX mount command would normally look, so there’s nothing extraordinary here.

Here’s what you do need to know: the value of [type] as passed to /sbin/mount and translated into /sbin/mount_[type] is just the value of your filesystem bundle’s CFBundleName string.

RETURNS: 0 on success, non-zero on failure.

Formatting

This is almost entirely undocumented, and not actually handled by anything related to DiskArbitration. Only tools based on the private DiskManagement framework will actually be able to format volumes. Thus, they respect the argument strings you set in your FSPersonalities records. The format executable is invoked as follows:

[FORMAT] [options] (-v) [vol_name] [raw_device]

The [options] are all of the space-separated options you specify in the selected personality’s FSFormatArguments string. Make sure your utility knows how to parse any flags you use in your bundle’s personalities. Optionally, DiskManagement will pass the -v flag if it wants verbose output, but it’s not guaranteed to be there.

The [vol_name] argument is the volume name the user has requested, encoded as UTF-8 bytes. DiskManagement seems to restrict the length of this to 16 characters in some cases, but I cannot explain why—at least, when I used diskutil to create NTFS volumes using mkntfs, the operation would fail on names longer than 12 or 16 characters, even though mkntfs says it can take up to 128 UTF-8 bytes.

The [raw_device] is the raw disk device as described above.

RETURNS: 0 on success, non-zero on failure.

The inner workings and underpinnings of pluggable filesystems on Mac OS X have never really been documented before. At least, Google couldn’t find any meaningful results if they have, so this is the first in a long series of articles that will (hopefully) demystify the creation of third-party filesystems for OS X.

There are many pieces of software involved in managing filesystems, but the central part that links them all is the filesystem bundle. Apple-supplied filesystems are installed in /System/Library/Filesystems. Unfortunately, /Library/Filesystems is currently not checked by any of the subsystems involved. So, third-party filesystems must also install their bundles into /System/Library/Filesystems, blatantly violating the Apple-only nature of /System.

This article aims to fully document the construction of a filesystem bundle (in the CFBundle and NSBundle sense of the term). Find the information you need in one of the following sections:

• Info.plist: Basics
• Info.plist: Media Types
• Info.plist: Personalities

Info.plist: basics

First things first: any filesystem bundle is still a bundle, so it must contain all of the standard bundle keys. The following are the most important keys to specify with your bundle.

CFBundleIdentifier (String)

This key, like elsewhere, is just a unique identifier for your filesystem bundle following the reverse-DNS convention. For example, Apple uses com.apple.filesystems.util.msdos for MS-DOS (i.e. FAT) volumes.

CFBundleName (String)

Likwise, this standard key should match the basename of your filesystem. For example, if you install ntfs-3g.fs, then your CFBundleName is just ntfs-3g.

CFBundlePackageType (String)

For all filesystem bundles found in /System/Library/Filesystems, they their package type code (CFBundlePackageType) is set to the value “fs  ”.

Other keys…

Other basic keys have the same relevance as they do in other bundles, but these are the three which every filesystem developer will need to consider when creating the Contents/Info.plist that goes into their bundle. There are two other property list keys specific to filesystem bundles, and their details are much more involved, and much less documented.

Info.plist: Media Types

In every filesystem bundle’s Info.plist, you can find the key FSMediaTypes which is a dictionary. It is used by DiskArbitration when scanning filesystems. Essentially, the grossly simplified process looks like this:

1. A new disk appears
2. diskarbitrationd probes each new partition
3. Partition is mounted using first successful probe

In another post, I will document the exact process of a single filesystem probe, and information diskarbitrationd expects you to provide. For now, don’t worry about that just yet. The FSMediaTypes key helps diskarbitrationd more efficiently identify which filesystems should be checked.

The top-level FSMediaTypes key maps to a dictionary. For all intents and purposes, it appears as though an array would have sufficed—that is, the keys inside the FSMediaTypes dictionary appear to be largely irrelevant. Each value inside the FSMediaTypes dictionary is also, itself, a dictionary. To prevent ambiguity, I will refer to these values as the FSMediaTypeDict.

For the purposes of this document, the keys inside FSMediaTypes exist solely as a means for you to give each mapping a unique name. Typically, the key used mirrors any Content Hint given, or is otherwise descriptive if a Content Hint is lacking. Some examples include Whole or Partitionless when there is no Content Hint.

As mentioned above, the individual keys are:

• FSMediaProperties
• FSProbeExecutable
• FSProbeArguments
• FSProbeOrder

FSMediaProperties (Dictionary)

The FSMediaProperties dictionary contains the most important information about your filesystem. diskarbitrationd uses these data to determine which filesystem bundles should be used to probe a partition. So if a new partition doesn’t match the criteria your FSMediaProperties sets forth, diskarbitration will not attempt to probe the partition using your filesystem bundle.

Criteria inside of each FSMediaProperties are checked using IOServiceMatchPropertyTable() from I/O Kit. Useful keys for matching are described in the IOMedia reference.

Though I cannot find documentation on ADC that details the precise matching algorithm, I can surmise this from the use of keys in filesystems:

“For every key in the match dictionary, a device matches if either (a) it lacks the key or (b) the values are equivalent.”

In this case, the “match dictionary” is the FSMediaProperties dictionary. If the algorithm were anything else, the FSMediaTypeDict entries that specify both Leaf = 1 and Whole = 1 could never match any device, since the two concepts seem to be mutually exclusive.

• Leaf (Boolean)

  Match a “leaf” device, which will typically have an identifier of the form diskXsY. See also the Whole flag.

• Whole (Boolean)

  Match a “whole” device, which will typically have an identifier of the form diskX. See also the Leaf flag.

• IOProviderClass (String)

  This string tells diskarbitrationd that only devices whose IOProviderClass (from I/O Kit) should be probed using this FSMediaTypeDict. The only instance of this is in cddafs.fs, which only wants to operate on physical CD media, and therefore specifies IOProviderClass = IOCDMedia.

• Content Hint (String)

  The Content Hint provided by any FSMediaTypeDict is singlehandedly the most useful piece of information you can give diskarbitrationd. When this key is present, the given FSMediaTypeDict will only be used to probe partitions or devices whose “Content Hint” string matches the one you specify.

  For disks partitioned using the GUID Partition Table, the “Content Hint” provided is the GUID of a given partition’s type. The linked article has a list of known partition type GUIDs.

  For disks partitioned using an Apple Partition Map, the “Content Hint” is the pmParType field in each partition record. Known strings include Apple_HFS, Apple_HFSX, Windows_NTFS, and a slew of others for MS-DOS and CD filesystems. As a developer, you’ll know the GUID or pmParType string to expect for filesystems you create.

  It is unknown if any Content Hint data is provided for disks partitiond using the Master Boot Record scheme. Current indications are that no hints are provided or checked; likely, all MBR can only match as whole or leaf devices.

FSProbeExecutable (String)

The FSProbeExecutable specifies the path to the utility that diskarbitrationd should launch when when it wants your filesystem bundle to probe a new partition—that is, when it wants you to verify that you actually can mount a volume.

Relative paths are considered relative to your bundle’s Resources folder. Utilities should be placed with your Resources, but in Apple’s bundles they are typically at the root of the bundle, not even underneath Contents.

FSProbeArguments (String)

This string specifies what command-line arguments should be passed to a probe executable specified with FSProbeExecutable. The arguments are passed first—that is, before any disk identifiers.

FSProbeOrder (Integer)

This integer ensures that each of your FSMediaTypeDict values can be executed in a deterministic order. This sort order is global across all known filesystems. For example, if FSMediaTypeDict in ntfs.fs has an FSProbeOrder of 1000, and a matching equivalent in ntfs-3g.fs has an FSProbeOrder of 750, then ntfs-3g.fs will always be asked to probe matching volumes prior to Apple’s ntfs.fs.

A lower value is a higher priority.

Typical values are around 1000. To make your bundle take precedence over other bundles, set your FSProbeOrder values lower, and to make your bundle appear as a last-chance filesystem, set your FSProbeOrder values higher.

Info.plist: Personalities

To avoid any confusion, take note that the FSPersonalities list plays absolutely no role in filesystem mounting. That is, if you want to create a filesystem bundle that only mounts volumes, then you do not need to have the FSPersonalities key.

This set of keys is utilized by Apple’s private DiskManagement framework, so there is no publicly-available source code or documentation on its expectations. Thus, all facts and assertions found herein are, more or less, educated guesses as to how DiskManagement actually works. If someone has more time than I, it may be useful to dig through DiskManagement.framework using otx.

Akin to the FSMediaTypes above, FSPersonalities is simply a dictionary of dictionaries. The nested dictionaries each reprsent an invididual “personality,” or method of accessing the filesystem.

The key-value pairs that can be used in each record are described below:

• Identification
  • FSName
  • FSFormatContentMask
  • FSSubType
• Formatting and Mounting
  • FSFormatExecutable
  • FSFormatArguments
  • FSFormatMinimumSize
  • FSFormatMaximumSize
  • FSMountExecutable
  • FSMountArguments
• Verifying and Reparing
  • FSVerificationExecutable
  • FSVerificationArguments
  • FSRepairExecutable
  • FSRepairArguments

Mind your <key/>

Unlike FSMediaTypes, the top-level keys are important. There are two known uses for the top-level keys you choose in your FSPersonalities. They are as follows:

1. You need the top-level keys to properly localize each personality’s FSName strings. The localized strings are what will appearin Disk Utility.app, if your filesystem manages to appear in its lists.

2. When formatting volumes with diskutil eraseVolume, the top-level key of each personality is the textual string that is used to choose a specific format. For example, if you have a personality with the key NTFS-3G, then you can create an NTFS-3G volume using:

   diskutil eraseVolume NTFS-3G VolumeName /Volumes/Somedisk
   

   Or whatever localized name appears in Disk Utility.app, if it actually parses all filesystem bundles and makes their formats readily available to the user.

So make sure you carefully choose your top-level personality keys, because their text will be exposed to the user by the output of diskutil eraseVolume.

Personality identification

FSName (Localizable String)

This is the localized string that will appear in Disk Utility.app or any other localized application that wishes to verbosely describe a particular personality that your filesystem bundle provides. For an example of a filesystem with many personalities, look at hfs.fs.

Be mindful of the text you put here, because users will see it when anything higher-level than Terminal.app references your filesystem. While this includes Disk Utility.app, it also includes Finder’s “Get Info” windows, among other references that will be found.

These strings are localized inside language projects in your Contents/Resources folder like any other bundle. They end up in the InfoPlist.strings file for each language project.

FSFormatContentMask (String)

This string is the formatting analog of the Content Hint in the FSMediaTypes dictionary. However, where a filesystem may have multiple hints depending on a disk’s partitioning scheme, ther does not appear to be a viable method to indicate multiple FSFormatContentMask fields in a personality.

Ideally, each personality would include both the GUID associated with the format on a GPT disk as well as the pmParType hint given for APM disks. Unfortunately, you can only specify one. For most Apple-provided filesystems, the FSFormatContentMask is the string associated with APM-partitioned disks.

There may be cases where specifying the GUID associated with GPT-style disks may be appropriate, but no current filesystem (as of Mac OS X 10.4) does.

FSSubType (Integer)

Where FSFormatContentMask is used to store information the filesystem type during creation, FSSubType is used to determine information about a filesystem during analysis. The FSSubType field referenced here can be found in the f_reserved1 field in struct statfs as returned by statfs() and friends. For those of you curious, I looked through the BSD VFS layer in xnu to determine that; feel free to verify by downloading the xnu source from Apple.

Presumably, the DiskManagement framework uses f_fstypename from statfs() to locate the correct bundle for a filesystem, and then uses the FSSubType to locate the proper personality. This is purely speculation, but it explains why fusefs.fs includes localized strings for several common FUSE filesystems, and why those strings wer used despite ntfs-3g.fs including its own. Currently, all MacFUSE volumes belong to “fusefs”.

For VFS developers:

If you’re developing a VFS plugin, you can set the FSSubType value directly by indicating you support the f_fssubtype attribute, and then supplying a real value in your vfs_getattr() implementation. The default value for f_fssubtype is 0, if you don’t supply one yourself.

For MacFUSE developers:

Since your code does not reach down into the kernel, MacFUSE provides a convenient argument through which you can set the f_fssubtype for your mount. You set it using -o subtype=<value>. Unfortunately, this will get you nowhere right now. The f_fstypename field in struct statfs is set by mount_fusefs to always be “fusefs.” Thus, all lookups are done in fusefs.fs, not your own bundle (if any).

I will petition for MacFUSE to allow an fstype= or fstypename= parameter so that MacFUSE developers can get the intened behavior. The only side effect would be breaking my own fuse_daemon program which checked for “fusefs” as the f_fstypename. Other programs that use getmntinfo() to find MacFUSE mounts would have to alter their code too.

With creative hackery, there is a deterministic way to hide a “FUSE” token in the f_fstypename. The kernel, in vfs_fsadd() zeroes out the entire vfstable entry, which includes the default f_fstypename. This entire buffer is then copied into each mount point. So the unused portions of f_fstypename will always be zero. Thus by sticking a non-zero token at the end, we can deterministically say only an actual FUSE volume created that mointpoint, while still allowing developers to use their own f_fstypename (i.e. ntfs-3g).

Update: I have mentioned the idea to the MacFUSE development team on the macfuse-devel mailing list, and I also filed an enhancement request. Follow those for future updates.

Formatting and Mounting

FSFormatExecutable (String)

This string specifies the executable used to create a new filesystem with this personality. Also see the executables section.

FSFormatArguments (String)

This string specifies the arguments passed to the FSFormatExecutable when a volume is being created.

FSFormatMinimumSize (Integer)

This integer specifies the minimum size, in bytes, that the personality can support. Essentially, if a partition is not this size, DiskManagement will disallow this personality (in theory). Though many filesystem limits are documented on Wikipedia, most values are for maximum storage, not minimum requirements.

FSFormatMaximumSize (Integer)

This integer specifies the maximum size, in bytes, that the personality can support. If a partition is larger than this, DiskManagement will—in theory—disallow the use of this personality. For a non-normative list of limits, see the filesystem limits page on Wikipedia.

FSMountExecutable (String)

This string specifies the executable used to mount a volume with this personality. Also see the executables section.

This key is not used by diskarbitrationd when automatically mounting filesystems. So, in most cases, it is prudent to guarantee that this executable and that accessed by diskarbitrationd when mounting filesystems.

In short, diskarbitration will call /sbin/mount_<CFBundleName>. Since this file lies outside the installation path of your bundle, you may be inclined to use absolute paths; don’t.

FSMountArguments (String)

This string specifies the arguments passed to the FSMountExecutable when a volume is being mounted.

Verifying and Reparing

FSVerificationExecutable (String)

This string specifies the executable used to verify the contents of a volume with this personality. Also see the executables section.

FSVerificationArguments (String)

This string specifies the arguments passed to the FSVerificationExecutable when a volume is being verified.

FSRepairExecutable (String)

This string specifies the executable used to repair the contents of a volume with this personality. Also see the executables section.

FSRepairArguments (String)

This string specifies the arguments passed to the FSRepairExecutable when a volume is being repaired.

Executables and Arguments

Each record in FSMediaTypes and FSPersonalities contain references to executables on disk. Each reference comes with two values: one to specify the executable path, and another to specify arguments that should be passed. The gory details of the executable and argument strings are discussed below:

Executable strings

All executable strings are evaluated as if they are relative to a bundle’s Contents/Resources folder. It may be possible to specify an executable using an absolute path, but there none of Apple’s filesystems do, so it may not work. In general, it is better to reference all of your executables using relative paths, even if they lie outside your bundle.

The advantage to this method is that no matter how your bundle is located, you will always reference the correct executables. For example, if someone were to symlink directly to your filesystem bundle on another Mac OS X volume, the executables from that volume would still be referenced correctly. Absolute pathnames would fail in that case.

I’m well aware that this example is contrived, generally unsupported, and susceptible to each volume containing the full install of your filesystem’s utilities—not to mention requiring compatible versions of OS X (i.e. both are 10.4, etc…).

Argument strings

Argument strings are passed before any programmatically-determined values. For example, if diskarbitrationd will probe a volume using a command of the form [binary] [...], arguments will be inserted after [executable] but before whatever’s left ([...]). If the list contains spaces, they will be passed as separate arguments to the executable. Thus, the effective result is [executable] [arg_list] [...], where [arg_list] is the space-separated list of arguments you provide.

However, the arguments do not pass through any shell. They are split on whitespace and passed to the executable as such.

Conclusions

There is a ton of information to be learned about how filesystems work under Mac OS X. This document only covers the first of several pieces. Everything you need to know to build a structurally-correct filesystem bundle should be contained here. If there are any ambiguities or missing information, please let me know and I will update this post to reflect any new findings.

I hope this helps someone in their quest to build filesystems for Mac OS X.

After far too long, I have finally received an answer from my employer about releasing my packages for NTFS-3G under Mac OS X. Unfortunately, it is not the answer I nor Mac users at large wanted to hear.

My version of NTFS-3G for Mac OS X is dead.

Upper management (i.e. the top) would “rather not” have me working on this and releasing it on my own. The reasons have not been disclosed to me, all I know is that it’s probably in my best interest not to directly voilate this creed.

Since I pulled the package, I’ve received countless emails and messages voicing support, asking for help, looking for packages, and more. So, despite my silence until now, I’m not going to abandon this idea and everyone that liked it. I’m acutely aware of the fact that people used this package heavily, and would really like to have it available.

I will do everything I can to help similar projects succeed.

And before I continue, I owe a huge thanks to the NTFS-3G project and Google’s MacFUSE for making this possible in the first place. If you used and enjoyed my package, maybe you should drop by one of those places and let them know their work is appreciated. Just as a nice gesture, nothing more.

“Absolutely not.”

In general, this will be my response to inquiries related to my NTFS-3G package. So please don’t get your hopes up.

This much needs to be crystal clear: I cannot release source or binary packages beyond what circulated the Internet in the first half of 2007. I have been in contact with the NTFS-3G maintainers to return my one-line compilation fixes as required by the GPL.

Yes, I was working on a new set of binaries that would integrate better with Mac OS X. No, I cannot release them. Don’t ask me to send either set of packages to you “under the table,” because I won’t do it. I’m just not interested in making any legal missteps.

Maintaining progress

As stated above, I cannot release software or source code.

However:

I can publish all of the information I gathered and documented while I was working on NTFS-3G integration for Mac OS X. Apple provides literally no documentation on how to do this, so I will do my best to make this information easily accessible.

There are some important tools I forged while working on my package. Most of these were only for my convenience, and I think other people will find them useful. They are briefly described as follows:

• Xcode scripts
  • Build Universal binaries for autoconf-based sources
  • Automatically generate installer packages and disk images
• MacFUSE Tools
  • Simplify integration of MacFUSE and Mac OS X

I’m sure the scripts are fine; they just make it possible to use Xcode as your one-stop-shop for everything from development to distribution. As for the MacFUSE Tools… I’m still undecided. They’re useful and fairly trivial, so I may still go for the “clean room” approach, and just document what they did so others can easily recreate them.

For future developments in this area, keep an eye on my blog. I’ve also created keywords for Mac Filesystems, MacFUSE, and NTFS.

The “clean room” strategy

So, truth be told, I really want NTFS (via NTFS-3G) to become a first-class citizen on Mac OS X. So I’m going to document as much of my amassed knowledge as possible.

For those of you unfimiliar with the term “clean room” in this context, I’m referring to clean room design. The idea is that one person or team creates a specification, and another recreates an implementation, so no “infringing” knowledge of previous implementations is transfered to the new code.

This is particularly important when creating open-source implementations based on reverse engineering. Since my predicament is sufficiently akin, I’m choosing to adopt the “clean room” model. Given my experience, I will provide as much information and informal specifications as possible.

That said, these are the topics I wish to discuss:

• Anatomy of a filesystem bundle in /System/Library/Filesystems
• Working with diskarbitrationd (DiskArbitration framework)
• Working with diskutil and Disk Utility.app (DiskManagement framework)
• Useful tricks for FUSE filesystems under Mac OS X
• Making use of status information provided by MacFUSE
• A few “gotchas” specific to NTFS

There is a wealth of information to cover. Not everything will be available from day one, so be patient—I have real-life responsibilities just like the rest of us. Rest assured, though, that I will do my absolute best to provide the information and tools necessary for someone else to recreate my work. Most importantly, without providing any source code or releasing any binaries.

Everyone wins. Except the users caught in the gap while the wheel is reinvented.

Update

The reason I stopped releasing this package was at the request of my employer. I was still employed part-time during the spring when I released the packages, and didn’t realize my error. Before I continued development on this when I became a full-time intern for the summer, I asked for permission to avoid any problems later. They said “no,” so I’ve stopped.

I’ve heard that some people are incorrectly blaming the fine folks working on the NTFS-3G project, and they had nothing to do with my packages being discontinued. I returned my work to them, because that’s just how open-source works. They very much wanted me to continue, but I can’t. They were great to work with while I was developing my package.

I hope that clarifies everything. Cheers!

Back in high school, I grew bored with the simple pseudonym I used while on the Internet. The old handle I used was pmarks00, which I’ve tried desperately to shed. It still lingers, for example, as my primary username on AIM, but that’s about it. For most other online activities—forums, games, and otherwise—I started to use the name ShadowOfGed.

It’s original. It’s unique. Somehow, it’s strangely… me.

For those of you who don’t get the reference, Ged is a character out of Ursula K. Leguin’s series, The Wizard of Earthsea. In the first book, which I read in ninth grade, Ged is a wizard-in-training whose darker half is physically separated from him. It appears throughout the book as his shadow, hence my name. ShadowOfGed—my online personality is merely a shadow of the real me.

As surely as moss on a stone, the name grew on me.

Ruled by shadows

The Internet is a fantastic tool. It’s also a frightening reality that we have never faced before. It has dark corners that most try to avoid. These corners exist as a direct result of the Internet’s most distinctive feature. A feature that is, singlehandedly, its greatest asset and its greatest danger: anonymity.

Anonymity, or the perception thereof, brings with it the power to transform people. And when I think hard enough, I’m certain that it has affected me as much as any other.

Who I wanted to seem

One of my better-kept secrets is that I occasionally have an extremely volatile temper. Many years ago, it got me into lots of trouble with my parents, and even a few times at school—I always got my ass kicked. It was one of these volatile fits that ended my first and last relationship. Somewhere, I picked up the notion that I should learn to control it.

So I tried. And depending on your definition of “control,” I succeeded. Really, I’ve just gotten better at subduing my temper when it flares up. For some reason, I’ve always wanted to seem a nice person. Note that I say “seem” instead of “be.” Though I want to be nice, I think I’ve been more driven by how I want others to perceive me.

The Internet has helped that case, which is why I brought it up. I’ve got a couple places I visit frequently. I only enter discussion where I have something humorous, informative, or friendly to add. I’ve spent so much time trying to help others with their problems that I’ve neglected my own.

To be honest, I’m not sure I know how to tackle my own problems. I just look around, trying to help others as a means to ignore them. I’m a victim of the expectations I set for myself.

And occasionally, this fact strikes me like a deftly-wielded sword—with sharp and painful accuracy, precisely where it hurts. It gets under my skin, and for no apparent reason whatsoever, I’m irritated by everything. Tonight has been one of those nights.

Dangerously close

Men and women wiser than I have stated that we treat our closest friends the worst. I’m guilty as charged, and I think it’s coming back to haunt me. For the second summer in a row, I’ve traveled out to sunny California by myself. Living three thousand miles from the only friends I’ve ever known isn’t exactly easy. There are no casual hangouts, no shared meals, and no inside jokes. The simple things are missing, and it hurts.

The only thing that hurts more is knowing I’ll be here again.

And when everything is gone, I find myself wondering who I am. Trying to find some meaning in what I’ve been doing for the past 4 years. Hell, for the past 21 years. And I’m at a loss. People call this success; they tell me I’m lucky.

It’s nonsense. I’ve worked hard. So hard, in fact, that despite four years at a university, I’ve never been to college; despite seven years as such, I never was a teenager; despite my youth, I never was young.

At the end of a day, all that’s left is the shadow I cast of myself. That outline of a person who is smart, funny, and caring. An empty shell. A small tremor and I break. The rock-solid foundation is an illusion. I’m not perfect; nobody is. To believe otherwise is fallacious.

The silence of three thousand miles shook me today, and I’m busy picking up the pieces. Damn.

Taking responsibility

I’m not here to blame anyone. I built a house out of cards and the wind came blustering through today. I should work harder to keep in touch. Despite facing the same lessons for a third summer in a row, I forget how important friends are.

You all rock. I need to play as hard as I work. For me, that’s easier said than done. I need to be less Shadow and more Paul.