Sunday, July 08, 2007

Bad news on the doorstep

After far too long, I have finally received an answer from my employer about releasing my packages for NTFS-3G under Mac OS X. Unfortunately, it is not the answer I nor Mac users at large wanted to hear.

My version of NTFS-3G for Mac OS X is dead.

Upper management (i.e. the top) would “rather not” have me working on this and releasing it on my own. The reasons have not been disclosed to me, all I know is that it’s probably in my best interest not to directly voilate this creed.

Since I pulled the package, I’ve received countless emails and messages voicing support, asking for help, looking for packages, and more. So, despite my silence until now, I’m not going to abandon this idea and everyone that liked it. I’m acutely aware of the fact that people used this package heavily, and would really like to have it available.

I will do everything I can to help similar projects succeed.

And before I continue, I owe a huge thanks to the NTFS-3G project and Google’s MacFUSE for making this possible in the first place. If you used and enjoyed my package, maybe you should drop by one of those places and let them know their work is appreciated. Just as a nice gesture, nothing more.

“Absolutely not.”

In general, this will be my response to inquiries related to my NTFS-3G package. So please don’t get your hopes up.

This much needs to be crystal clear: I cannot release source or binary packages beyond what circulated the Internet in the first half of 2007. I have been in contact with the NTFS-3G maintainers to return my one-line compilation fixes as required by the GPL.

Yes, I was working on a new set of binaries that would integrate better with Mac OS X. No, I cannot release them. Don’t ask me to send either set of packages to you “under the table,” because I won’t do it. I’m just not interested in making any legal missteps.

Maintaining progress

As stated above, I cannot release software or source code.

However:

I can publish all of the information I gathered and documented while I was working on NTFS-3G integration for Mac OS X. Apple provides literally no documentation on how to do this, so I will do my best to make this information easily accessible.

There are some important tools I forged while working on my package. Most of these were only for my convenience, and I think other people will find them useful. They are briefly described as follows:

  • Xcode scripts
    • Build Universal binaries for autoconf-based sources
    • Automatically generate installer packages and disk images
  • MacFUSE Tools
    • Simplify integration of MacFUSE and Mac OS X

I’m sure the scripts are fine; they just make it possible to use Xcode as your one-stop-shop for everything from development to distribution. As for the MacFUSE Tools… I’m still undecided. They’re useful and fairly trivial, so I may still go for the “clean room” approach, and just document what they did so others can easily recreate them.

For future developments in this area, keep an eye on my blog. I’ve also created keywords for Mac Filesystems, MacFUSE, and NTFS.

The “clean room” strategy

So, truth be told, I really want NTFS (via NTFS-3G) to become a first-class citizen on Mac OS X. So I’m going to document as much of my amassed knowledge as possible.

For those of you unfimiliar with the term “clean room” in this context, I’m referring to clean room design. The idea is that one person or team creates a specification, and another recreates an implementation, so no “infringing” knowledge of previous implementations is transfered to the new code.

This is particularly important when creating open-source implementations based on reverse engineering. Since my predicament is sufficiently akin, I’m choosing to adopt the “clean room” model. Given my experience, I will provide as much information and informal specifications as possible.

That said, these are the topics I wish to discuss:

  • Anatomy of a filesystem bundle in /System/Library/Filesystems
  • Working with diskarbitrationd (DiskArbitration framework)
  • Working with diskutil and Disk Utility.app (DiskManagement framework)
  • Useful tricks for FUSE filesystems under Mac OS X
  • Making use of status information provided by MacFUSE
  • A few “gotchas” specific to NTFS

There is a wealth of information to cover. Not everything will be available from day one, so be patient—I have real-life responsibilities just like the rest of us. Rest assured, though, that I will do my absolute best to provide the information and tools necessary for someone else to recreate my work. Most importantly, without providing any source code or releasing any binaries.

Everyone wins. Except the users caught in the gap while the wheel is reinvented.

Update

The reason I stopped releasing this package was at the request of my employer. I was still employed part-time during the spring when I released the packages, and didn’t realize my error. Before I continued development on this when I became a full-time intern for the summer, I asked for permission to avoid any problems later. They said “no,” so I’ve stopped.

I’ve heard that some people are incorrectly blaming the fine folks working on the NTFS-3G project, and they had nothing to do with my packages being discontinued. I returned my work to them, because that’s just how open-source works. They very much wanted me to continue, but I can’t. They were great to work with while I was developing my package.

I hope that clarifies everything. Cheers!

8 comments:

GSB said...

thanks for the entertainment you provided. Please accept my sincere appreciation of your honesty, integrity and also dedication to both your work and employer.

Rob said...

I'm sorry to hear this, and please take what I'm about to say as constructive criticism. Allowing someone to erode your rights like that, employer or not, is disgraceful. Stand up for yourself, if not for your sake, for technology and intellectual freedom's sake. If someone imposed anything like this on me I'd walk out the door in extreme disgust. You have a brain, a good one judging from your contributions to technology. Don't let these bullies screw you around. I know you probably don't see it that way, and I think that's also a great shame, that it's almost become 'normal' for people to get away with enforcing this kind of totalitarianism.

Paul said...

It's not eroding my rights. I was not forced into this, and in fact I initiated it. There is just cause for my employer to ask that I not do this, as there could easily be a conflict of interest.

I took this down voluntarily after they responded to my inquiry, because I didn't want any conflicts down the road. And my employer is far from totalitarian.

Rob said...

Thanks for your response. I've thought about it, and I must admit I'm confused.

"Unfortunately, it is not the answer I nor Mac users at large wanted to hear."

"Upper management (i.e. the top) would “rather not” have me working on this and releasing it on my own. The reasons have not been disclosed to me, all I know is that it’s probably in my best interest"

Why are you saying one thing in your comments but another in your post? Just because you volunteer to give up your rights, doesn't mean they aren't given up. Unless an NDA or copyright is being broken, your employer is NOT justified in telling you what you can do with your own, unpaid time, in your private life. If everybody puts up with it, voluntarily or not, rights *are* being eroded.

I can see you're not too disappointed so I guess it's not so bad for you. But what about the next person who can't create a company from his bedroom because he unwittingly signed his private, intellectual thoughts away to his day job? Steve Wozniak of Apple Computer almost had to give away the rights of his first computer prototype to Packard Bell, luckily they didn't want it. Imagine if they had...?

Also how can you say your employer has a 'just cause' when you admit yourself that your employer didn't actually give you any reason? But then you assume it's in your best interests and follow blindly?

That, friend, is the very definition of totalitarianism.

Paul said...

I don't know where you're from or what laws and contracts you've lived with, but it's common practice for U.S. companies to have their employees to sign intellectual property agreements upon hire.

In some cases they claim rights to anything you create while in their employ. Those companies are ridiculous. In my case, they only claim rights to (a) work done on the clock, (b) on company resources, or (c) that "relate to the business and products" of the company.

All three are reasonable claims, I believe. This project, though not susceptible to claims under (a) and (b), does unambiguously fall under (c). I had forgotten about that clause, but that's why I voluntarily asked for an exception, and stopped when it was declined.

Alexx said...

Paul,
I am sorry but I defintely agree with Rob, though I partially understand and share your decision.
I am from Italy, and luckily here in very few cases employees are subject to sign such half-legal (at least in Italy) agreements...
If I were in you, I would have gone much deeper, by investigating what really was fallin' "unambigously" under point (c). And last, I wouldn't have accepted such a dumb answer: "Don't do it and don't ask why", which sounds really like a totalitarian behaviour.

That said (and sad, too), keep going, man.

Paul said...

I'm going to leave this buried in the comments, because I don't really want to announce it publicly. However, here's the missing piece of the puzzle:

I work for Apple.

So, it's completely understandable that they wouldn't want me distributing a package that (a) intentionally stomps on their read-only NTFS driver, and (b) could theoretically cause instability/damage.

That's why I asked, and why the response was not surprising; I could infer their reasons without having need to argue.

However, I took several posts to detail what I learned from publicly-available sources and information, since the ADC documents don't really cover it. I should write a few more, but haven't yet.

Reid said...

Ah, okay, it's pretty clear now why you acted the way you did. However, I would point out to you that other Apple employees have released software that enhances Mac OS without a problem. Mike Bombich does exactly this with Carbon Copy Cloner.

I encourage you to seek out other Apple software devs who have relelased tools and utilities for Mac OS X. But hey, it's your choice, and I respect that.