Cheap And Fast Storage And Backup Methods For Large Work Files And Data

In today's 'Information Age' climate, having enough storage space to use for storing data is never enough it seems. Photos, video, movies, database and music files are getting ever bigger due to the demand of higher resolution equipment such as high-end digital cameras, 4K video players and recorders etc.

Due to the variety of choices available, developing an effective backup plan for your valuable data can be a daunting process. It isn't just about choosing a fast SSD or external USB drive for your system, but looking at the overall setup including the connection types, drive storage size, system reliability, equipment and software used. So what is considered an effective storage or backup solution? Here are some pointers on what you should choose:

1) Speed Of Backup And Retrieval

This is the first and foremost question of any storage setup. Whilst many backup solutions of recent times have offered the solution using sparse-based image files such as with Apple's Time Machine, such setups are considered too slow for work or business use, since they take a lot of time to retrieve especially with slow connections from a cloud link like Dropbox.

Therefore the method that is best recommended for storage for businesses, personal or work projects today is direct connection types - USB 3.0, 3.1, USB-C, SATA, 5G wifi, 10G networking and Thunderbolt. These methods currently offer the best and fastest storage and backup methods available.

Alongside these, are the options of using a fast drive or set of drives such as an SSD or 2.5"/3.5" standard disk hard drive for both internal or external use. For internal, SATA is the main connector type whilst for external, USB 3.1 or Thunderbolt docks are now the norm (such as Highpoint's RocketRaid TB3 Dock). Drive sizes up to 16TB are available from brands such as WD, HGST, Toshiba and Seagate, whilst SSDs up to 4TB are available from Samsung and Sandisk.

As for drive boxes, there is a choice between choosing a DAS (direct attached storage) or NAS (network attached storage). DAS using Thunderbolt or 10Gbe are currently the fastest methods of connecting. NAS drives tend to run only to 1GbE speeds unless they are upgraded to 10GbE speeds like with some from QNAP's or Synology's range. For DAS and NAS, only four brands offer the most useful and cost effective equipment for backup at present - QNAP, Drobo, Synology, and Areca. Whilst each company has their own approach to managing data, their core attractiveness is their speed and reliability - they are some of the fastest available drives on the market (versus using cloud storage, which can be slow if you have a slow internet connection). Whilst cloud storage is a known method of backup, it is considered the last and slowest option if speed of recovery is a concern in the workflow.

QNAP and Synology's RAID drives are preferred by both home, office, and Linux users whilst Drobo is preferred mainly by Mac users and digital photographers and creatives because of their ease-of-use though they also work with Windows. Some of QNAP's and Synology's drives are able to be converted into a mini-PC and run virtualisation software (running different OSes on the drive itself), whilst Drobo and Areca drives are purely storage drives. And while Drobo's main range like 5D, 8D and 5Dt are direct-attached and require installation of a desktop app for Mac OSX or Windows, their 5N and 5N2 drives are for network use only. Areca drives are purely RAID drives.


2) Ease-Of-Use and Equipment Reliability

The next important questions after the speed of storage and retrieval, is the ease of use and reliability of the equipment. Whilst equipment is never guaranteed to not fail, there are ways to mitigate the likelihood of getting faulty gear by using good equipment.

In my experience, the most easy to use and reliable drives and brands I've used have been Toshiba, Sandisk and HGST branded cards and drives and drives from Drobo and Areca. Drives and boxes from these brands have typically lasted me more than 5 years at a time.

It is a general rule to replace hard drives every 3-4 years including SSDs due to the amount of wear they have. For SSDs, even though they may not have moving parts, they are have a limited number of write-cycles before they completely degrade, so they still need replacing at some point in their life.

For ease-of-use, Drobo drives are highly recommended in my experience. The reason why
it is favoured by seasoned photographers and videographers to store their data is because apart from the raw speed it offers, it is very easy to upgrade the existing drive. Unlike a traditional RAID where the setup and maintenance is dependent on the exact make, size and model of drives installed, Drobo's BeyondRAID system doesn't' require matching drives to increase or repair the hard drive space. Different hard drive brands and sizes can be mixed and matched on a Drobo. So long as the new drive is equal or larger than the drive being replaced, the drive will quickly rebuild itself after replacement.

3) Developing A Strong Backup Plan

Apart from using good equipment, a good backup plan using a combination of good workflow methods, a set schedule and apps/software are essential. Fortunately there are a lot of options available,  because of the maturity of the backup market. Using a combination of both hardware and software solutions, here are some examples:

Apps For Windows:
EaseToDo Backup, Macrium Reflect & Acronis True Image - drive cloning
DriveImage XML & Acronis True Image - disk copy scheduling
Macrium Reflect - Remote backup

Apps For Mac:
Superduper & Carbon Copy Cloner- drive cloning
Chronosync - disk copy scheduling
Carbon Copy Cloner - Remote backup

Drive based apps:
Synology - Hyper Backup
QNAP - Hybrid Backup

4) Upgradeability

DAS and NAS drives today offer users a great number of choices and configurations.
Typical choices include: numbers of drive bays, the storage size the drive can hold; the CPU used, the number of Ethernet or USB ports on the drive for connectivity and the amount of RAM. Depending on the make and model some of these drives allow for upgrading of the original drive spec. While most will allow the user to increase their drive capacity through the addition of a drive or larger sized drive, some allow the use of extra cache memory (such as the Drobo 5Dt/5D3/5N2 series) while others allow extra RAM or even PCIe 10GbE cards (such as those from QNAP and Synology).


5) Setup Options

Below are a number of example setup options for a backup system depending on the workload.

Typical Windows based backup:
DAS (WD Mybook/QNAP TR004/external drive in USB dock/Drobo 5C connected to PC running EaseToDo Backup or equivalent software)
NAS (QNAP, Synology or other NAS drive connected for network access, Hyper or Hybrid backup set to a schedule). 2nd USB external drive attached to NAS as backup copy. Can be used with 1 or more NAS to offsite location for extra redundancy using RTRR.

Typical Mac DAS backup:
DAS (WD Mybook/QNAP TR004/external drive in USB dock/Drobo 5D connected to Mac, running Time Machine and/or Chronosync across devices)
NAS (QNAP, Synology or other NAS drive connected for network access, Hyper or Hybrid backup set to a schedule). 2nd USB external drive attached to NAS as backup copy. Can be used with 1 or more NAS for redundancy. Can be used with 1 or more NAS to offsite location for extra redundancy using RTRR.

Low Volume Backup
For PC:
DAS (Drobo 5Dt connected to PC X2/Areca Thunderbolt RAID X2 running Carbon Copy Cloner/EaseToDo Backup across devices)
NAS: NAS (QNAP, Synology or other NAS drive connected for network access, Hyper or Hybrid backup set to a schedule). 2nd USB external drive attached to NAS as backup copy.

For Mac:
DAS (Drobo 5D/5Dt connected to Mac X2/Areca Thunderbolt RAID X2 running Carbon Copy Cloner/Time Machine and/or Chronosync across devices)
NAS (QNAP, Synology or other NAS drive connected for network access, Hyper or Hybrid backup set to a schedule). 2nd USB external drive attached to NAS as backup copy.

High Volume Backup

For PC:
DAS (Drobo 5Dt connected to PC X2/Areca Thunderbolt RAID X2/QNAP Thunderbolt running Carbon Copy Cloner/EaseToDo Backup across devices)
NAS (QNAP, Synology or other NAS drive connected for network access, Hyper or Hybrid backup set to a schedule). Thunderbolt or 10Gbe based. 2nd USB external drive attached to NAS as backup copy.

For Mac:
DAS (Drobo 8D connected to Mac X2/Areca Thunderbolt RAID X2/QNAP Thunderbolt running Carbon Copy Cloner/Time Machine and/or Chronosync across devices)
NAS (QNAP, Synology or other NAS drive connected for network access, Hyper or Hybrid backup set to a schedule). Thunderbolt or 10Gbe based. 2nd USB external drive attached to NAS as backup copy.