One of my “most desired features” for my Windows box is something that I’ve referred to as a merged mountpoint. Last night I made a significant amount of progress towards implementing this feature on my own. Here’s some background information:
Most users have one hard drive in your system, usually C:\. This drive will probably have your Windows installation, Program Files, Documents and Settings, and a few temporary files to start off; depending on how much stuff is installed, you’re looking at about 10GB taken up with a typical “workstation” installation. With a 120GB drive, that leaves you (after formatting and space conversion) about 100GB to work with.
On my system, I have random utilities, other applications like Visual Studio and Eclipse, and a few games. All in all, my 120GB C:\ is down to about 60GB for random media files, like music and video. I could probably fill this space up in about a week if I was really going at it, too.
If I add another hard drive to the system (D:\), there’s another 120GB of space, but it’s on a different physical drive. I have a choice to make – do I start installing programs and utilities to the D:\ drive, or do I start locating any new media files on D:\?
Hit the jump for more details…
What this usually means is that I end up having multiple copies of folders on different drives – for example, C:\share\TV Shows and D:\share\TV Shows. This gets fairly confusing for organization, especially when you have six internal hard drives and one external one occupying drive letters. I absolutely love the P5B Deluxe board in my Windows workstation; there are seven internal SATA ports and one external eSATA plug.
There are more or less “official” solutions for this sort of problem – one would be striping two drives using RAID-0. This, however, is only effective for two drives and still poses a problem for three or more physical hard disks. What’s more, RAID-0 increases the likelihood of data loss; if one drive goes down, the entire filesystem across both drives is ruined.
Another solution involves using RAID-5, but it requires sacrificing a disk for parity checking. There are different methods of implementing RAID-5, as well; hardware implementations require expensive cards and are vendor-locked, while software implementations can have performance issues and be difficult to set up.
The most promising way of achieving what I want would be a filesystem called ZFS, but it’s primarily for Solaris and Linux. I already have too much data invested in NTFS-formatted partitions to really make this change successful, and I do tend to use my Windows workstation for PC applications.
With all this in mind, I’ve set out to create an application that can do what I want – namely, merge multiple folders into a single large mountpoint. A relatively unknown feature of NTFS is called “junctions” or symlinks – they’re much like the symbolic links in BSD or *nix operating systems. As a result, I’ve created a program that:
- scans all specified folders on different drives
- merges contents of folders with the same name
- creates a new virtual folder structure on a drive
- creates symbolic links to files on different drives
This process works transparently across the network – so if I mapped my main system, \\monolith\c\vroot and tried to access an MP3 file, the file would transparently be delivered from \\monolith\e\MP3\.
I’m still working out the details, but will release the app as an open-source Java project once it’s in a reasonably stable state. One of the primary goals is cross-platform support; right now, I have the test version working on my OS X system; it should work with any *nix system supporting the “ln” command. My next goal will be to get it working with Vista’s built-in “symlink.exe”, as well as SysInternals’ “junction.exe” for 2000 and XP boxes.
I want this system for two major reasons – one being my FTP server, which could use this system instead of multiple /c, /d, /e, etc… shares; the other one is for my Xbox Media Center installation, to keep all the TV shows and music in one place.
I’ll keep you posted once the application is working on Windows platforms and I’ve successfully used it for the Xbox.