What is the Best Way to Backup Your Data Every Day?
I know it is not going to be shocking news when I say that Data is really, really important. That should not come as a staggering fact. If you found this article thanks to a rather perceptive google search, then clearly you think data is very important too. It is all too easy to rely on your data living on multiple machines in your office or home. Centralized backup is a very unappealing idea. For a start, it is expensive. You will spend hundreds, if not thousands of pounds on storage, then you find out that you need to have it in ANOTHER location too (as otherwise, it isn’t truly a backup) and then worse still it isn’t even being spent on making more space, but actually to duplicate your old data and not be used. You are spending all this money on what can be described as a remarkably large insurance policy. Worse still if you have all the machines in your home or office backing up to a device in that same location, almost all data safety and storage experts will scream to high heaven that this is still not enough. Sadly this is true, because not only do you put yourself at risk of complete critical loss in the event of fire or flooding, but also in the case of theft you end up having all your eggs in one basket and making the thieves lives much easier. So ultimate you have to stop thinking about this only in terms of how much this storage is going to cost. If you think like that, you will never move past stage one. No, you have to think about how much will this data cost you to LOSE. Your personal media (family, events, memories), your client’s data or perhaps your surveillance data. Take a moment and think how much it would cost your business right now if you lost your data – hundreds? Thousands? Close the business? The ideal backup solution should always be based on the cost of loss, not gain! Although a little trite, it is still a reliable rule of thumb to have a 3-2-1 backup solution operation that looks like this:
- Primary Data (Where data is initially created/collected, the PCs, the Macs, the Phones, etc)
- On-site Backup solution (where multiple devices are backup’ed up to internally, also more commonly referred to as the bare-metal backup)
- Off-site/different location Backup (where the copy of the on-site backup lives that is separate network and/or physical location. This can range from another NAS, to a cloud provider and even a USB drive)
Stages 2 and 3 should be encrypted in case they are stolen/entered, as well as feature login credentials and an admin system in the event of them being occupied/accessed illegally. Today I want to focus on the 2nd and 3rd stages of your backup routine and help you decide the best ways to spend your budget on the most effective safety net and recovery system for your needs.
Backup and Redundancy – DO NOT GET THEM MIXED UP!
Way, WAY too many NAS (network-attached storage) or DAS (direct-attached storage) owners think their data actually have a backup layer in place when they refer to their RAID configuration, their versioning or their snapshots. These are NOT backup methods and are actually designed primarily as a means of recovering your data in the event of an HDD/SSD dying, an accidentally deleted file or rolling a file back to a previous revision. Think of RAID/Snapshots/Versioning as ‘safety nets’, The only work from INSIDE the infrastructure NOT outside of it. If the NAS server or RAID enabled DAS box does, then there is a very high chance that your data cannot be recovered via these methods (not impossible, just touch and certainly not foolproof). Below is a video that breaks down a number of different Backup and Redundancy methods inside a single NAS system:
How Expensive is a 2-Stage Backup and Do I need it?
The cost of a 2-stage backup (i.e having 2 backups of your data) operation can be measured by the amount of data you/your company produces on a daily basis. Additionally, depending on your commitment to holding onto your data (so, for business, you might have a 6 year or 12-month retention policy) there are ways to make economies in between the stages. However, whether you are a home user of a business user, there are few instances where a double-layered backup is not ideal. Let’s look at this from a business and home user point of view:
Why Business and Enterprise users need an Extensive Data Backup Plan
Example. Your company has 10 employees. Each has their own workstation and they contact clients on a daily basis to drum up new sales, fulfil existing quotes and maintain a customer relationship manager (CRM). You have both #1 AND #2 stage backups (so local PCs and A NAS in-house maybe) covered. Plus you have business insurance in case of a fire or flood. One morning you arrive to find your office has been flooded/burnt/burgled/struck by lightning and everything is fried. At first, you think, lucky we have insurance. They will pay up for whole new office equipment, PCs and your server. However, what about all that customer data? Not only can the insurance company not replace it but they will not pay its consequential value (with VERY few insurance policies covering data recovery services). So now you have to start from square 1. Plus now you also have a bundle of rather angry customers from the previous days and weeks whose requirements go unfulfilled. This coupled with starting your business network from scratch, employees salaries continuing as normal and several IT guys (or 1 guy working for days) setting everything up from scratch again (this WILL be the case from fire, flood or theft) could easily KILL a company. Now, in that context, isn’t a few thousand put towards an off-site #3 Backup so bad? Thought not.
Why Home and Private users need an Extensive Data Backup Plan Too
Ok, so a home user has a smaller ecosystem to maintain and now the mission-critical data and the life or death nature of your information is less so. Or is it? Example #2. What about all those important house documents you’ve scanned? Those TV shows and movies you bought on a one time download? What about your wedding picture or those of your children growing up? Those videos of your friends and relatives that are no longer with us? Not to be bleak, but it is often the case that although much of a person’s data is not of huge financial value, it is still utterly and completely irreplaceable in the literal sense.
Likewise, if your many devices (phones, computers, hard drives) get corrupted, hacked with malware/ransomware or simply broken, don’t you want the peace of mind of knowing that there is always a backup of EVERYTHING? If you are copying the data of all your devices to a large hard drive enclosure in your home, this is NOT a perfect backup. It just protects you from the loss of one or more of your mobile/individual devices. Additionally, the minute you start deleting files on your phone/laptop to ‘make some space because it is already backed up on the NAS‘, your NAS is not a backup, it is the single repository for that data! So, as you can see, the need for a reliable true backup is paramount regardless of whether you are a home user or business user. However accepting that you need a backup is not enough, you need to know what to consider when choosing the right backup. Home users and their data volume/frequency can always consider USB backups of course, which can be good but only upto the time that the last backup took place AND on the assumption that the USB drive is stored offsite periodically.
What are the factors I need to take into account when considering my Backup Solution?
Choosing the best full backup for your data can be a little difficult. With so many variables ranging from cost to size to speed and more, it can be easy to go around in circles and still end up choosing nothing. In almost all cases, the deciding factor is cost. However, this is closely followed by speed. Having a backup is all well and good, but if it takes too long to finish, it can often be slower than the speed at which you create data and create a bottleneck. Likewise, if you choose an unsuitable connection of choice between your primary backup storage and/or off-site 2nd backup storage, then they may communicate inefficiently. Below are the main overheads to consider when choosing your backup.
Distance & Speed – How far is the 2nd backup going to be from the primary backup and/or primary data source? Unless you are considering fibre cabling between different physical sites, you are going to be limited by your upload/download speed and if your network is busy with multiple users, then you will need to provision priority of service to these backup connections, or else risk it affecting day to day operation (particularly VoIP connections – a growing necessity in 2022 with shifting patterns in the way we are working during/ost pandemic). The local connections between your client devices in-house and the primary backup may well only be using an internet network/LAN to communicate, but that can still become oversaturated. So be sure to provision switches where appropriate too (we will touch on 10GbE later)
Power (Watts etc) – The power of the hardware inside the primary (bare-metal) data storage device, secondary backup target and (depending on your setup) even the power of your client hardware in house is also an important factor to take into consideration when scaling and provisioning an appropriate backup strategy. Critical power failure (i.e a power cut) in one or more of these locations can not only result in some data not being backed up, but more importantly can compromise the backups themselves, with some systems suffering critical hardware failure and SSD/HDD damage in the process. It is HIGHLY recommended for users who are considering a private server for their data have it connected to the mains power via a USB device, in order to allow the system to still operate in the event of power failure AND shut itself down safely.
Physical Media – The server of choice for your primary backup is only half of the battle. The media that you are storing the data too can make a huge impact too. Many have their own maximum speed or capacity, so even with a super-fast enclosure and/or connection, you will be bottlenecked by the drives themselves. All media types e.g. HDD, SSD or Tape have their own maximum performance and some are lower than the connection maximum and some are faster and therefore end up creating a limited speed inadvertently. This can be negated in a number of ways. These range from the use of RAID (redundant array of independent disks) that combines media into single storage pools that benefit from redundancy (i.e a safety net if one drive dies) and/or increases in performance. Additionally, you can look at SSD caching, a service that is particularly popular in private NAS servers released after around 2018 and continues to be included in most systems. SSD caching allows the system to have one of more SSDs installed and then utilize the increased performance they offer to benefit write actions (i.e you write data onto the SSDs first and then the NAS moves it over to the slower HDDs afterwards) and/or read actions (where the most frequently accessed files are cloned onto the SSDs and it speeds up their access to client devices) though this is less useful for backups, it is still a useful option.
The Media connection internally – As mentioned, the SATA port on most commercial HDD/SSD has a maximum of 6 gigabits (Gbs) in SATA III – SAS at 12 Gigabits. All of these are internal and are maximum bandwidths available to each media drive installed in each bay. However, realistically, most typical Hard Drives max out at 272MB/s at the very top end (and even then, this is using enterprise-grade HDDs) and most conventional 2.5″ SSDs (with the exception of U.2 and U.3) max out at about 550MB/s). So you are going to need several of these drives inside your primary backup system in a decent RAID configuration to facilitate performance AND redundancy (like a RAID 5 or RAID 5). For your external/2nd backup target, a lot of enterprise users take advance of cold storage such as tape drive media. Designed to be used for data that needs to be held for insurance/archive for years and years, the current highest generation of tape architecture is is LTO-9 = 400MB/s in uncompressed/RAW form and 1000MB/s in compressed, with storage capacities per tape hitting 45 Terabytes! However, a full backup would take (at best, so with everything at full!) over 12 hours to transfer. Additionally, tape backup is fantastically expensive and therefore largely enterprise only as an option.
The external connection – Lastly and possibly the most overlooked part, is the communication between your primary client devices to send data, the primary backup data and the secondary backup. Not just the speed, but the resilience and future-proofing. You need to consider what connection you are going to use today, tomorrow and years from now. The last thing you want is to saddle yourself with a connection now and later when you upgrade your primary hardware on individual clients or locations, end up with a device you cannot access or use with an unchangeable bottleneck. Then your data just becomes a chore to access at best and potentially days or weeks or time to restore. Popular connections and their speed between host and client devices are:
Network Backups (backups that share a LAN or vLANs in a greater network)
- 1GBe LAN/Ethernet = 1Gbit/s, or 100-109MB/s
- 2.5GbE LAN/Ethernet = 2.5Gbit/s, or 250-270MB/s
- 5GbE LAN/Ethernet = 5Gbit/s, or 500-545MB/s
- 10GbE LAN/Ethernet = 10Gbit/s, or 1000-1024MB/s
USB and local Backups directly connected and stored offsite
- USB 3.0 (3.2 Gen 1) = 5 Gbit/s
- USB 3.2 Gen 2= 10 Gbit/s
- USB 3.2×2 = 20 Gbit/s
- Thunderbolt 3 & Thunderbolt 4 = 40 Gbit/s
Fast Local Backups or Direct Connections over large distances
- Fibrechannel / FC – 1, 2, 4, 8, 16, 32, and 128 gigabit per second rates depending to start
- Tape Based backups, such as LTO 7,8,9 – 300/360/400MB/s uncompressed and 750/900/1000MB/s compressed
- Cloud storage – dependant on connection. Additionally, factors such as upload and download speeds, fair usage policies and more affect the performance. If you are fortunate to have a 1 Gigabit internet connection, that means around 100MB/s for download speeds and a fraction of that for upload speeds
So, as you can see, there are many choices out there for a means of sending backup between the 1st, 2nd and 3rd parts of your backup environment.
What is the Difference in Speed and Cost between different Backup Solutions?
And so to the meat of the subject. Different solutions cost money and in the interests of SPEED, below i have detailed numerous solutions that will provide a backup solution of upto 10TB of storage. All Costs and speeds are based on a solution that is an acceptable distance away for maximum efficiency. Perfect speed results were provided with http://www.calctool.org/ , however, it is worth noting that these are ‘perfect situation’ based and it would be tough to see this maximum threshold. you will comfortably see around 10-20% below this, but that is fine. Finally, it is always worth remembering that the speeds mentioned below are based on an initial 10 Terabyte backup. After that, there is the likely possibility that future backups will be much more granular (this can also be referred to as differential backups, where only the changed files are backed up each time afterwards). But some users may wish to take advantage of time managed backups, where a full backup per day is kept, for X number of days (when eventually the newest backup will overwrite the oldest on rotation).
ALSO IMPORTANT – In all examples where a 4TB SSD (which average around £350-400 over SATA) is mentioned, you can use a 4TB HDDs at around £75-100 (brand depending) to save a considerable sum – but you will effective quadruple or more the time the initial backups will take. Likewise, future incremental backups will be significantly reduced also. In examples where the SSD would have been substantially bottlenecked by a connection, I have used HDD as you will not need to spend the extra.
The Best Value 1/2.5GbE based Backup Solution for 10TB of data
For a solid LAN based backup (with optional internet access as needed for off-site work) I would recommend the Synology DS220+ 2 Bay Pentium NAS at around £250. Alongside this, you will need a smart Switch (to take advantage of LAG and 2GbE, 2x 10TB HDD at £230 each as you will not see any speed difference on a network connection with SSD (RAID 1). This will cost around just under £800. Alternatively, there have been other improvements in the base level ethernet connectivity in 2-4 Bay NAS solutions in the last few years, with 2.5GbE or even 5GbE available on the likes of the QNAP TS-x53D, TS-x73A or Asustor Lockerstor 2/4 series. These solutions arrive at only a fraction more and although they can possibly add another £100-200 to the price tag, this can be offset by time saved in the backup operations AND by purchasing a 4-Bay NAS over a 2-Bay and leveraging the price vs the redundancy vs the capacity (eg instead of 2x 10TB in RAID 1, purchasing 4x 3TB and getting better performance and an extra 2TB in RAID 5).
What do CalcTool.org have to say about 1GBe
Over 1 Gigabit per second, in a perfect scenario – just over 20 hours. Realistically closer to 25 or 30 hours. SO the first few backups should be conducted over the weekend but all future ‘difference only’ backups should be fin at 12-hour intervals without harming the bandwidth too much, Likewise, you can scale this down as you utilize link aggregation (also known as port trunking, when ports are combined) and/or 2.5GbE/5GbE to reduce this figure down to 8 hours or just 4 hours respectively.
The Best Value 10GBe Network-Based Solution for 10TB of Data
In order to create the perfect cost-effective yet powerful 10GBe Network-based backup solution (so 10x faster than normal LAN) I would recommend the QNAP TS-431X3 with 10GBe (at around £450) with SFP+ Connection and SFP+ Cables with transceivers attached (Cable price dependant on length and transceivers). Additionally, you will need a 10GBe switch (at least £150 realistically), and for MAXIMUM speed 4x 4TB Seagate Ironwolf 125 SSD (at £450 per drive) in a RAID 5 which slows things a pinch but gives you the safety of 1 drive worth of redundancy. Of course, you can downgrade to Seagate Ironwolf 4TB Hard drives (at £85 per drive) and save over £1500, but you will see a noticeable dip in performance of around 200-400MB/s (NAS dependant). So the choice is yours. Lastly, you will need a 10GBe interface on the machine(s) you are backing up from in order to maintain the 10GBe throughput (or alternatively just have the NAS on a 10GbE connection and the clients all on 1GbE with bandwidth being shared as appropriate). In total this will cost around £2400+ for the SSD based solution and just £900-1000 for the HDD solution. None of this is set in stone, of course, you can scale things in a number of ways, using bigger NAS solutions (i.e. more bays) or using multiple 10GbE connections in LAG/Trunked connections.
What do CalcTool.org have to say about 10GBe
The performance will largely be dictated by the distance of the backups, choice of HDD or SSD and types of files. However, over 10 Gigabit per second, in a perfect scenario – just over 2 hours. However this is a little optimistic and in practice, it will realistically weigh in closer to 3+ hours, as the complexity of the files (thousand so smaller emails and documents or fewer BIG media files, everyone is different). This of course is for the first few backups of a FULL 10 Terabytes of data. Later with incremental and ‘difference only’ backups, you will see times slashed heavily for the better.
The Cost of Thunderbolt 1, 2 or 3 Backup Solutions for 10TB of data
Fast becoming a connection of choice for photo and video editors in both the Mac and Windows community, Thunderbolt is the no-fuss connection that promises speed, without the technical nonsense. Much like before you can choose to go with SSD drives for supreme speed (at a hefty price tag) or HDD if you want to make economies. Below are the options best suited for a Thunderbolt 1 (although still legacy, still is in use), Thunderbolt 2 and Thunderbolt 3 Backups:
- TB 1 DAS, 4-Bay, Cable, 4x 4TB SSD, RAID 5 enabled = £3000+ —- 2Hours + Backup time with SSD population / 3.5+ Hour for HDD Population
- TB 2 DAS, 4-Bay, Cable, 4x 4TB SSD, RAID 5 enabled = £3500+ —- 1 Hour + Backup time with SSD population / 2-3+ Hour for HDD Population
- TB 3 DAS, 4-Bay, Cable, 4x 4TB SSD, RAID 5 enabled = £4500+ —- 30-45min + Backup time with SSD population / 1.5 Hour for HDD Population
What do CalcTool.org have to say about Thunderbolt 1, 2 and 3
Thunderbolt does not lose speed over distance, however, most conventional cables you can buy max out around 5 metres and the ones included with the above enclosures arrive at 1-1.8m. In a real-world scenario you can realistically double this length of time listed above in the initial backups. However, it will MASSIVELY improve with subsequent backups. With the exception of a few, most Thunderbolt backups arrive with only Thunderbolt ports, so in order to maintain the speed levels of this backup, you need to either ensure that it is connected to your centralised depositary via Thunderbolt, or if it’s backing up multiple devices, that they are using a good networking device, as Thunderbolt Direct attached storage only allows a single connected device at any one time.
The Price, Speed & Suitability LTO-7, LTO-8 and LTO-9 Tape Backup 10TB Solutions
In the case of LTO / tapes, this kind of storage for 10TB can be incredibly inefficient for an extra layer of storage. You can purchase much smaller 1 and 2 tape frames/storage devices, but for what you are paying and the overall accessibility for all machines involved, it isn’t great. If you were regularly backing up 5x or 10x this amount of storage, it would be a different story. Internal operations can be upto 1000MB/s with compressed data and 400MB for raw uncompressed data. SO unless you are synchronizing between two LTO tape loading machines, you will almost certainly use uncompressed. However these are internal operations and as we are discussing backing up from existing systems to a storage device, we have to focus on the external connection. Most likely 10GBe network or 12GB/s SAS will be the means of backing up to your tape device. But Cost is hard to pin – easily £2000-3000 and upwards, over at least two tapes (capacity differed at each LTO generation) etc. Most likely around over 1-3 hours transfer time, but hugely impractical at this scale and most likely much higher in practice. Definite a good option for those at the enterprise tier, but anything less will be squandering their budgets significant with a weak ROI.
The Best Value USB 3.2 Gen 2 Backup Solution for 10TB of data
The latest available version of USB, also known as USB 3.1 Gen 2, is easily the cheapest way to store a 10TB backup at a very respectable 10 Gb/s (comparable to Thunderbolt 1) speed. You will need to ensure that the connected device(s) that you are backing up too/from use the newer USB 3.1 Gen 2 port to ensure you do not get bottlenecked at 5 Gb/s, but this Highpoint 6124V RAID 5 enabled USB 2 Gen 2 DAS enclosure arrives at £350+ and if populated with either 4x 4TB SSD or 4x HDD (same price difference as before), RAID 5 enabled = £2100+ for an SSD Based solution and just £700+ for a HDD based version
What do CalcTool.org have to say about USB 3.1 Gen 2?
With the best drives available you will have this 10TB localized backup over USB 3.1 Gen 2 finished in just over 2 hours. However taking system overheads into consideration, as well as the RAID 5 into account (something you could counter with a RAID 10 and 4x 6TB HDD perhaps) you are looking at between 2.5 and 3.5 hours completion. Additionally, a number of NAS Drive vendors have started implementing USB 3.2 Gen 2 in their systems, which means that a 10Gb/s USB DAS solution can be used as a means of creating a secondary backup by connecting it with your primary backup and running one of the many free backup/sync tools that NAS systems include with your purchase. Just remember that in order for this to be a truly safe 2nd tier backup, it cannot constantly reside in the same location as the primary backup.
Is Cloud Suitable as a 10TB Backup solution?
You may wonder why I have not suggested the cloud as a regular backup yet. It is certainly appealing. No parts are needed, just a healthy internet connection. You already have all the hardware you will need to establish this kind of synchronised backup – this should be by far the cheapest and easiest backup, right? Well yes and no. It IS cheap – in the short-term. Even if you take into account that your Business internet connection costs, from as little as £10 to £50 a month, reaching much higher once you consider fiber channelling, it is still pretty attractive. However you have to consider the time this backup will take and how it will affect the bandwidth throughout your business – otherwise, you will need to be conducted them overnight due to limit consumption. Some brands, such as Synology with their C2 platform, have rolled in an additional cloud service alongside their bare-metal solution. This is intended to be used as an alternative means of accessing an existing storage area that is synchronized with your NAS/local solution remotely (as well as benefiting from significantly easier remote access to data). NAS+Cloud can serve as a great 2-3/double backup option, as well as ensuring that your backups are in very different physical locations. Lastly, with intelligent caching and background synchronization between the NAS-THE CLOUD-CLIENT HARDWARE happening, a lot of the actual backing up can be hugely incremental and lessen the impact on the end-user, whether they are local to the NAS or remotely communicating with the C2 cloud (or 3rd party clouds such as Google Drive and Dropbox that are synced with the NAS). But what about the cloud on its own, not used in tandem with a NAS/Private server?
The first thing to factor in is the data being sent TO the cloud. Remember we are talking about Uploads, not downloads (downloads and a backup should only be considered during recovery, not day to day operations). Most internet services advertise incredible download speeds, but backups are almost exclusively upload based and upload speeds are normally a 10th or less than advertised download speeds. lastly, we can talk about costs. As although the initial costs are much less, let’s go for £50 a month for a dedicated high upload speed connection privately (closer to £500 for a business line) for your off-site backup. That is £600 a year. In 5 years, that is £3,000 (a cost that is the same or higher than most of the solutions discussed previously). The real kicker is that after those 5 years, you either have to continue paying to maintain this backup OR buy a suitable local storage drive to download it too – something you could have had ALREADY by going for the other solutions and thereby saving you thousands of pounds more.
I took the trouble of using the awesome tool at http://www.thecloudcalculator.com/ and if you have a 30Mbps upload speed (fairly respectable and the entry point outside of central city hubs). backing up 10TB initially would take 33 Days, 22 Hours, and 27 Minutes, 11 Seconds
That is horrendously long and you cannot just assume this is a one-off and negotiable with incremental backups and difference-only changes. You need a reliable and adaptive backup solution – not one that will do the job as long as you work within its limits. If you want to entertain the idea of a cloud-based backup of 10TB on a regular basis, we have to look into fiber and at least 2Gbps (so 2000 Megabits) to get to 12 hours for an overnight full backup (non-incremental). This is going to cost a small fortune and unless you intend to take advantage of this speed during the day-time, is a huge outlay for something that is not hugely accessible or reliable.
So, those are your options. Remember you are not limited to just ONE option and each of these solutions can be used in combinations, depending on your budget, physical environment, internet speeds or volume of client devices. In summary:
Thanks for reading! Need any more help? Why not take advantage of the free advice section below. It is genuinely free, supported by voluntary donations and ad revenue. Nothing is done with your email address and all questions are answered by two humans – Me (Robbie) and Eddie the Web Guy!
LET ME KNOW ABOUT NEW POSTS
Enter your email address to subscribe to this blog and receive notifications of new posts by email.
Get an alert every time something gets added to this specific article!
This description contains links to Amazon. These links will take you to some of the products mentioned in today's content. As an Amazon Associate, I earn from qualifying purchases. Visit the NASCompares Deal Finder to find the best place to buy this device in your region, based on Service, Support and Reputation - Just Search for your NAS Drive in the Box Below