Saturday, March 10, 2007

NTFS-3G for Mac OS X (Original)

Through many posts in the MacFUSE thread over at AppleNova, I’ve informally published binary packages to make NTFS-3G accessible to non-technical folk. It lacks a central website, and following that Very Long Thread™ is quite tedious. So here’s a makeshift homepage.

Use the permalink to this post to find the latest versions of these packages. If you have space available, please mirror these downloads, because my iDisk can’t take a hit from the entire Internet. I will try to keep these packages current as the upstream packages make changes that necessitate new builds.

Latest packages

News

Wednesday May 23, 2007

It’s late and I’m tired, so I won’t put much for now. I’ve updated both packages. For those who wanted simpler distributions, you got it. NTFS-3G is now a meta-package that installs everything you need. MacFUSE Tools got some code cleanups, so a potential race condition is now fixed. There’s one outstanding change I’d like to make to fuse_wait, but I haven’t the time tonight. Enjoy the new versions!

UPDATE: Initial versions were broken because of a silly syntax mistake in my script. The newly-uploaded package works for me. New in this release is a fail-over to read-only in the case of an uncleanly mounted drive, so no volumes will “vanish” like they did before. Otherwise, there’s nothing particularly interesting.

Sunday May 13, 2007

I am going to be traveling across the country until sometime later this week. Consequently, I will be unable to answer emails and comments until then. I hope this latest release has fixed the couple issues people mentioned, and that I’ve helped resolve the other problems people have seen.

I’ll be back around Saturday at the latest. Good luck until then!

Saturday May 12, 2007

After the performance-enhancing patches came through, people noticed some strange bugs with the Finder. I’ve fixed them. They stemmed from DiskArbitration asking NTFS-3G to mount, say, /dev/disk0s3, but instead having /dev/rdisk0s3 get mounted instead. Using a handy FUSE option called fsname, I make it look like /dev/disk0s3 got mounted, as expected. NTFS-3G typically forbids the use of the fsname parameter, so I had to comment out that code. It’s OK in this case, because I Know What I Am Doing™.

Anyhow, the new release (1.417-r4) seems to behave as expected. If you see reproducible problems from these changes, let me know, either here or via email. Cheers!

Tuesday May 8, 2007

The biggest change everyone has been waiting for: [performance gains][speed].

Other than that, I’ve included a few rudimentary changes. For anyone encountering permission problems, I’ve set NTFS-3G to mount volumes for with uid and gid set to 99:99, which translates to unknown:unknown. This has a special meaning for Mac OS X. Files owned by unknown:unknown always appear to be owned by the user/group viewing them. Thus, if you’re accessing the drive from a non-administrator account, the files will still look like they belong to you.

Likewise, I corrected the default modes. Consequently, you can no longer execute OS X binaries from NTFS volumes. If that presents a problem, and enough people would like the +x mode set again, I can make the fix quickly and have a new version out soon. Otherwise, the NTFS volume now appear more sane if you look at the details as seen from the Terminal.

That’s it for this update!

Monday April 23, 2007

I’ve uploaded new versions of everything. Now the packages are properly versioned from the perspective of OS X. Scripts to uninstall each package are included in all the disk images; no separate downloads are necessary. The NTFS-3G driver got an overhaul to match MacFUSE 0.2.5, which added the noping_diskarb option. It only uses this if a new-enough version of MacFUSE is present.

I cleaned up the fuse_daemon code so that it behaves as this: SIGTERM causes forceful unmounts of all FUSE volumes; SIGQUIT causes a friendly unmount of all volumes through Disk Arbitration; and SIGINT causes a clean exit with all FUSE volumes still mounted.

Please download these, they should work better all around. I have done no performance benchmarks, though, so I make no guarantees in that department.

Saturday April 07, 2007

I have uploaded a new version of the MacFUSE Tools (0.2.2-r2) package that alters its behavior. The old algorithm tried to do “polite” unmounting when receiving SIGTERM on shutdown. This appears to stall the shutdown process for some users. It now responds to two signals: SIGINT and SIGTERM.

Upon receiving SIGTERM, it forcibly unmounts all FUSE volumes, not through DiskArbitration. Thus, do not send SIGTERM to fuse_daemon while still running your system. As a regular user, only send it SIGINT, which will perform the old, “polite” unmounting procedure.

Added uninstaller package and instructions in the FAQ.

Summary

What’s special about this distribution of NTFS-3G? What are the MacFUSE Tools?

This NTFS-3G package provides Universal builds of NTFS-3G, ntfsprogs, and the added Mac-friendly ability to automatically mount NTFS volumes as read/write. This eliminates the command-line knowledge previously needed to get read/write access to your NTFS volumes.

The MacFUSE tools provide a set of utilities that any MacFUSE volume can use to ensure proper interaction with Mac OS X, including the Finder, DiskArbitration, and the system shutdown. These tools are invisible to end-users, but are useful for developers wishing to create wrappers similar to the ntfs-3g.fs bundle distributed here.

Background

When MacFUSE was first released, the only way to access NTFS volumes as read/write was to compile your own copy of NTFS-3G. Moreover, you always had to re-mount your NTFS volumes from the command line, as NTFS-3G was not the default filesystem for NTFS volumes; Mac OS X’s own read-only implementation was the default choice.

This method left many of the non-technical Mac users in the dark; there was no way they could compile, install, and mount volumes using NTFS-3G with any reliability. Thanks to the architecture of Mac OS X, automatic disk mounting is handled through a modular service called Disk Arbitration, which scans known filesystem bundles in /System/Library/Filesystems. Thus, to make read/write NTFS access simple, I set out to create an NTFS-3G filesystem bundle that would override the default read-only implementation.

With much effort from Sören Nils Kuklau (chucker) and myself (ShadowOfGed), a set of scripts were developed that formed an ntfs-3g.fs bundle that automatically mounts NTFS volumes as read/write. This bundle was then packaged with Universal binaries of the NTFS-3G project, which is the NTFS driver, and also the ntfsprogs suite for NTFS administration (formatting, renaming, and more). The result is a very Mac-friendly, integrated version of NTFS-3G that requires zero user interaction to work properly.

After the initial release, and after MacFUSE added distributed notifications, it became apparent that a few tools would be needed to relieve a few common problems. First was the problem of ensuring FUSE filesystems initialize prior to exposing them to Finder. Second was the problem of ensuring that FUSE filesystems properly unmount on shutdown to prevent apparent hangs/slowdowns. Thus, the MacFUSE Tools package was developed. Since these problems are not exclusive to NTFS-3G, it is distributed separately in case anyone else can make use of them.

Support for these tools was integrated into NTFS-3G, so as of NTFS-3G 1.0, NTFS volumes will mount read/write seamlessly, quickly, and without any Finder or shutdown glitches. However, the problems of interacting with Boot Camp and NTFS locales also caused people grief, so there are now two features to address those, and they are discussed below.

Features

The following constitute the major features and programs included in the MacFUSE Tools and NTFS-3G distributions:

Boot Camp

Since some tools look for the use of Mac OS X’s default NTFS implementation, it is occasionally desirable to have NTFS-3G politely decline to mount an NTFS volume as read/write, allowing the Mac OS X default mounter to take responsibility. This allows things like Boot Camp to work properly. For volumes that you want NTFS-3G to bypass, simply create a file called .ntfs-readonly on the root of the NTFS volume. When this file is present, NTFS-3G will decline to mount the volume as read/write.

NTFS locales

To support NTFS locales, the NTFS-3G mounter must know the desired locale when it mounts a volume. To make this happen automatically, the NTFS-3G bundle looks for a file called .ntfs-locale in the root directory of every NTFS volume. If found, it uses the contents and passes them as the locale= argument to ntfs-3g. Thus, for English, one might specify a locale of en_US. Other examples include de_DE for German, and ru_RU for Russian. For more information on locales, see the NTFS-3G website and Google for further details.

Frequently asked questions

After releasing the software, I’ve been asked many questions about the software, how to do things, what’s wrong, and any other inquiry you might be able to imagine. Here are answers to some of the most frequent questions:

How do I uninstall NTFS-3G and MacFUSE Tools?

As of MacFUSE Tools 0.2.5 and NTFS-3G 1.417, just use the Uninstall scripts included in the downloadable disk images. They will prompt for your password to get the administrative access they need, and then they will quietly remove the package and all its files.

How do I find the BSD device for my NTFS volume?

Determining the BSD device for your volume is a simple and straightforward task. You’ll need to open Terminal.app to do this. Run the following code from your terminal:

mount | grep '/Volumes/My Volume' | awk '{ print $1 }'

The string /Volumes/My Volume represents the mounted location of your NTFS volume. If you don’t know what this is, instead of typing it in quotes as shown above, you can simply drag the volume from your Desktop into the Terminal window. This will automatically type the right folder for you.

You should get output that resembles this:

/dev/disk1s3

That string, /dev/disk1s3, is the BSD device associated with your NTFS volume. You’ll need it in some places below, so become familiar with this procedure.

How do I (un)mount my NTFS volume from a terminal?

Depending on whether you want to mount or unmount your volume, use one of the following commands after you have found the BSD device for your volume.

diskutil mount /dev/disk1s3
diskutil unmount /dev/disk1s3

How do I rename an NTFS volume?

There are two possible solutions to this problem. One is simple, but requires Windows, and the other is more complicated, but does not require Windows.

Option 1: Connect your NTFS volume to a Windows machine, through Parallels, Boot Camp, or plugging an external drive into a Windows machine. Then, within Windows, right-click on the desired drive in My Computer, and edit the volume name right there. These changes will take effect the next time Mac OS X mounts your volume. This is probably the most reliable method, although it requires Windows and more effort.

Option 2: Find the ntfslabel utility included with the NTFS-3G distribution here. This is a command-line solution, so you’ll need Terminal.app open. If you’re not comfortable with the command line, DO NOT DO THIS. Anyhow… the binaries are in /usr/local/bin, so if that is not in your PATH, you must either (a) use the full path to ntfslabel, or add /usr/local/bin to your PATH. The procedure looks roughly like this:

  1. Find the BSD Device associated with the volume.
  2. Unmount the volume.
  3. Rename using ntfslabel. sudo /usr/local/bin/ntfslabel /dev/disk1s3 “New Label”
  4. Re-mount the volume.

Your NTFS volume should be renamed. There is a limit of 128 Unicode characters for NTFS volume names—if you need more than that, please consider making your volume name more reasonable.

Why are NTFS copy speeds so slow?

This is a known issue with NTFS-3G run atop MacFUSE, but there’s little to be done about it. On internal drives, transmission speeds are typically around 10 MB/s, whereas external USB2.0 drives typically see around 1.0 MB/s. Yes, there is a performance bottleneck somewhere, but it’s not a problem this distribution can solve.

Be happy that we have read/write access to NTFS volumes through MacFUSE. Performance optimizations will come later (hopefully).

Why won’t my NTFS volume mount?

There are only two reasons that an NTFS volume, internal or external, will fail to mount automatically using this NTFS-3G distribution. It’s easy to determine which of the following reasons are causing your NTFS volume to mount using the default read-only implementation.

  1. Your volume contains a .ntfs-readonly file, explicitly telling NTFS-3G to bypass the volume.

    In order to remove the .ntfs-readonly file, some extra work is going to be involved. Follow the outlined procedure to remove that file and allow your NTFS volume to once again mount as a read/write volume:

    1. Find the BSD device associated with the volume.

    2. Unmount the volume.

    3. Create a temporary mount point.

      mkdir ~/my-ntfs3g
      
    4. Mount through ntfs-3g.

      sudo /usr/local/bin/ntfs-3g /dev/disk1s3 ~/my-ntfs3g
      
    5. Remove the .ntfs-readonly file.

      rm ~/my-ntfs-3g/.ntfs-readonly
      
    6. Unmount from ntfs-3g

      sudo umount /dev/disk1s3
      
    7. Remove the temporary mount point.

      rmdir ~/my-ntfs3g
      
    8. Mount the volume again.

    If you followed the directions properly, the NTFS volume should be mounted as read/write. If it still mounts as read-only, then you likely have the same problem described next.

  2. Your NTFS volume was not unmounted or ejected cleanly by Windows.

    The only way to safely fix this problem is to attach your NTFS volume to a Windows machine, and have Windows run CHKDSK on it. For internal drives accessed by Parallels or through Boot Camp, Windows should run CHKDSK automatically while booting up for inconsistent NTFS volumes. For external hard drives, you will likely need to run CHKDSK manually.

    After running CHKDSK and cleanly unmounting the volume, it should work under NTFS-3G as a read/write volume. To ensure a clean unmount make sure that (a) for internal volumes, you shut down Windows properly, and (b) for external volumes, you “eject” the device before unplugging it.

If your problems persist, then it is likely there are bigger problems with your NTFS volume. I’d recommend trying to manually mount your volume using ntfs-3g and seeing what output it gives—likely, it will tell you why it won’t mount the volume.

How do I change the volume icon for my NTFS volume?

Once your NTFS volume is mounted as read/write, you can change the volume icon in precisely the same manner that you can change it for any other volume seen by Mac OS X. If, for some reason, the icon does not persist across boots, check for the presence of a file named .VolumeIcon.icns at the root of the volume. If this file doesn’t exist, Mac OS X was unable to create it. There are only a few possible reasons:

  • The volume was mounted read-only when you first changed the icon.
  • Your NTFS volume is out of disk space, and couldn’t create the icon file.
  • Something deleted .VolumeIcon.icns when you rebooted.

In any case, setting custom icons is known to work with NTFS-3G, and to persist across reboots. Thus, any problems encountered with this are external to NTFS-3G. Please make sure none of these conditions caused your problem, and if you continue to have trouble, please try thoroughly to determine the underlying cause.

Warranty

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Nothing new. Basically, I’m not responsible if these packages do anything bad to you, your computer, or our data. Such is the price we pay for free software.