LUCHS.AT - System Administration - Of File Servers and Compulsive Hoarders

Digital assets have a major drawback compared to their solid counterparts in the analogue world - you cannot see them. While most people hail this attribute as one of the major building blocks of the digital age, most system administrators tend to disagree. This is especially true if you ever had to switch the hardware of file servers. Usually keeping the data is the goal of upgrades of any kind. When changing storage devices this involves a lot of copying and inspecting the logs of copied file and directory names reveals - your file server is dealing with compulsive hoarders!

We are fully aware that keeping track of every single file and directory on your typical computer, mobile phone or digital gadget is next to impossible. The same is true for networked environments where user data is stored centrally. The typical work environment features thousands of files. Settings, temporary files, icons, e-mails, caches, credentials, multimedia files, documents and lots of stuff needs to be stored for every user. Most people think that storing data is cheap and thus they don't care. Storage media keeps getting larger and cheaper, right?

Wrong! Let's state it clearly: Proper storage is not cheap!

But why? It depends how valuable the data is for you. Storage is only cheap if you can afford to lose data. If you don't care if your data disappears from one second to the next, then storage is really cheap. However if you want to keep your data for a longer period of time, then you need copies. Lots of copies! A typical storage environment features mirrored disks, a primary backup server, possibly a secondary backup server (or storage media) and an archive. Assuming you go with a minimal amount of mirroring (double) per stage, then you end up with a factor of 8! This means that multiplying your cheap disk from the store with a factor of 8 gets you much nearer to the real price of storing data. And we haven't even talked about data transport yet. Have you ever tried to copy your live data via USB 2.0 or 100 Mbit/s Ethernet? Both methods are cheap and widely used. Once you need to copy 500+ GB you start to think about faster data transports, thus increasing the effort for proper storage and raising the costs.

So, do you know what the cheapest part of proper storage is? Deleting data you do not need any more. You can do this any time. Please do it! Your system administrator and all devices in the backup chain will be glad. Deleting data is a small step for you, but it is a big step for your storage systems.

Note: Yes, we know about data deduplication and other features of storage subsystems, but this isn't the point. There is no silver bullet and we cannot use endless resources forever.