Can replication replace backup?

Just my 2c worth

Without arguing replication and backup, as already stated they're two different things, I really need to give my 2c worth regarding using the cloud as an alternative.

I wouldn't, and you're going to have a hard time convincing me it's a good idea.

Firstly, if the cloud target isn't your own equipment, you can't trust it. I don't care who says what, since we have no control over their data storage, we have no guarantee it's going to be secure.

If it *IS* your own, then you need to pay for and manage that too.

Also, unless you have an encrypted tunnel to the target, again, as far as security goes, it's free for all.

And tell me why, that once you mention "cloud", people somehow manage to forget their ISP data costs are going rise dramatically.

Put those three points together, and dollar for dollar, tape looks good.

Sheeze. And people wonder why.

Hear, hear!

Replication != Backup. CDP is a different story, but you need both replicated copies (in case of primary storage systems failure) and backups (in case of "oops, I deleted it!"

Easy for geeks to understand. Not so easy for business types.

Backup vs Replication

Chris Evans is wrong:

"Traditional replication goes against the premise for which backups are taken - to recover to a point in time at which data was lost or corrupted....The second point (corruption) is probably the most important reason why traditional replication can’t replace backup - corrupted data would be replicated...In summary, traditional replication will not replace backup"

If "replication" equals "raid", then he is correct that logical corruption affects all copies, but then this solution is a bit unimaginative.

One tool which should be on everyone's list is rsync, which can do versioning on disk. In fact it even prevents the need for multiple copies of the same file via hardlinks, making it efficient enough for many scenarios.

Evan Unrue's claims are all valid:

"CDP replication mitigates the risk of replicating corruption because you have very granular restore capability being able to recover to a specific point in time"

I'm not sure that a point in time is necessary over an hourly or daily snapshot. Maybe for database records "point in time" is advantageous, but it's not obviously clear that it's always necessary to restore a file to the minute or second. For some, higher performance and efficiency may be more critical.

Claus Egge makes sense too:

Claus does a good job explaining how they can be unified.

His description of backup/replication contradicts Chris's.

Interestingly, Claus calls Continuous Data Protection a "backup" techology, while Evan called it a "replication" technology. Personally, I don't care either way since I acknowledge there is functional overlap.

"Yet, there is no technical reason for not eliminating tape entirely."

Ultimately, when it comes to capabilities, disks are supersets of tape. They can do anything a tape can do, and then some.

Phil Jones:

"you need both replication and backup because the two are not the same, and nor are they alternatives to each other...all need careful thought when building your backup strategy, and cannot simply be achieved just by implementing some form of replication."

While I have no specific disagreement, Phil seems to imply that they cannot be unified into a single solution that does both. CDP really blurs the lines as well. There's nothing wrong with being traditional, but Phil should consider updating his wording.

Replication V Backup.

They used to say that the best word processor was the one you knew how to use. This seems to be the argument that is generally put forward for back up tapes etc.

Replication is not easy, and I do not feel that there is a decent piece of software out there that does the job properley, a bit like voice recognition software, we all want it, we all know it is Holy Grail and we all know it is not here yet!

If you are ever faced with a real life disaster recovery, you will wish you had replication to be able to very quickly rebuild your whole systems.

If you have back ups, this is the time that you find you have not got what you thought you had, and you realise that you have to painstakingly rebuild everything from scratch, this where you find that what you thought was being backed up was infact considered by your predecessor/boss/assistant to be not to be necessary and would save media space/time.

This is the real trouble - too many IT managers believe the sales blurb, and may have passed all the exams but have never got their hands dirty! So real life is not so real until it bites them in the ass!

Replication v Backup

I am going to go out on a limb and suggest that most backups are a complete waste of time. Very few people actually check to make sure that the backup process completed or that it worked. Even fewer then check to see if the restore function will actually restore data correctly.

The big problem of course though, is that most backup procedures work on the basis that when something goes wrong, you will have access to the relevant type of tape drive, and a suitable server with the required OS and backup software installed with which to carry out the restore function. If your server is a smoking pile of metal and plastic, you are in a bit of a jam.

Backup therefore needs to be part of a complete BC/DR strategy - and replication can be a more sensible first step for many companies. Virtualisation may also be appropriate method of protection, as well as off site facilties or BC/DR partners. Although people may complain about the costs, the impact of not having protection in the event of a disaster may be far greater.

Data usage and structure not discussed, but very important to choosing a backup solution.

A key thing that appears to have been missed, is about the type of data being stored and how that data is used, for example, if all you ever do is append data, i.e. nothing is ever deleted, then replication is a perfect form of backup, because you never have to roll the system back, so provided that you can ensure consistency between the two copies, all you ever need to do, is add a new entry that is correct to supersede the old information. If you need to see what the data looked like at a particular point in time, then you just ignore all data after that time.

Where using replication is more difficult to justify as a form of backup, is when data is changed or deleted, because the changed data is no longer kept in the data structure and the blocks on which the data was stored could be re-used, this makes it possible to look at the data as it was at a previous point in time.

You can get around this in some cases by partitioning data, an example of this is in compliance environments where you must retain data for X and destroy it by Y. Between X and Y is an amount of time, you halve it and that is your partition period. A partition (once created) is active for the duration of one partition period and data may only be appended to it during the time that it is active, at the end of the period the partition becomes read only and a new active partition is created for new data. Once the newest piece of data in any partition reaches the end of the retention period, the partition and all data within it is destroyed.

This is fine, when you work with systems, where the data structure is designed to have old entries retired, a classic example of this is a financial system, which records a transaction and a running total with each entry. We can see this with an example for a bank statement, when we look at statements, your bank statement is designed to be completely stand alone, for example:

Balance in: £4135.83

Credit - Pay £1982.32

Debit - Electricity £250.12

Debit - Gas £35.00

Debit - Apple Store £599.99

Balance Out: £5233.04

Your bank statement contains the incoming balance, the transactions that took place within the statement period and the outgoing balance. Next month, you'll get a new statement with the new information, if all you care about is how much is in your account, you can shred the old one and it has no repercussions on the new statement, however, if you care about how much you spent on Gas last January, you will keep the statement for as long as that information is relevant.

Financial systems are ideally suited to this kind of process, because bank accounts are not generally related to each other and retention periods are always longer than a year, every account that has not been closed will be updated at least once a year when interest is calculated, this means that there is always a current record in a partition that is within it's retention period and the closure of one account does not affect the status of another.

This is fine... but what happens if you have a relational database and different elements within the database having different retention periods for compliance reasons, a typical example might be a case management system for use in law enforcement, where there will be People (or organisations), Objects (e.g. Vehicles, Weapons), Locations and Events (Phone call, robbery, stabbing) stored within the system and related in any number of ways. The answer is that you cant easily partition this data, so the application has to manage the data's life cycle, retention and disposal and the applications repositories have to have snapshots copies taken at regular points in time in order to ensure that it's possible to go back to a specific piece of information as it looked at that time. Another method is that the application can store a snapshot of elements related to a case within the case record, this helps to insulate against differing retention policies at the expense of storage, it also can make predicting the rate of change difficult, especially if cases become inter-related.

For data that like this that is not linear, you can back the data up to tape as a snapshot, or use a disk based technology and replicate the disk and the snapshots to a second system, but every snapshot technology in the market works on the principle of space reservation, i.e. a certain amount of space is reserved for changed data that pertains to a specific snapshot. If the live volume runs out of space or the rate of change is too great, snapshots will be retired in order to preserve the live data or the most recent snapshot(s), if your business needs to keep a copy from a year ago for any reason and you do not manage the storage system, ensure sufficient space is always maintained and that data change is kept fairly constant or at a predictable rate, it's likely that you will run low on storage and the oldest snapshots will begin to vanish prior to their retention period ending.

What this means is that if the rate of change of your data is massive or unpredictable, then you need to have incredibly large amounts of storage available for the space reservation on both your primary storage and your replica, otherwise your likely to fall out of compliance with your backup retention policy. There are other things to think about if your rate of change spikes. What about the links between the replica's? Will the changes be written to the replica fast enough? If the system is synchronous, will performance of the application be impacted while the system waits for the replica system to confirm the data has been committed?

Replication can do the job, if the volumes and rate of change for data are well understood, or if the data stored has certain characteristics, however, for most cases tape is easier to manage, easier to dispose of and could be more economical to store.

Crib notes: don't use rsync because it's free

Sure, it does everything you need (fault tolerant, versioned, preserving deleted versions), but come on, it was written by hippies who don't even charge you anything - how much can it be worth?

You must, must, must pay fat £££ to some greasy consultant first to produce 50 pages of powerpoint which nobody will ever read, then he'll recommend whatever Oracle or, I dunno, Sysbase charge the most for.

Use RAID

Sent to me:-

I've worked in many Small Office/Home Office (SOHO) environments where data backup is a massive expense that many are simply not prepared to pay for.

Tape backup is limited by capacity against cost and does not allow a small business to get back up and running as quickly as possible (the smaller the business, the greater the loss for downtime).

One common approach that I now use for data backup of servers is the use of external drives to create a snapshot of a full working system that acts as a drop-in replacement. This is achievable through RAID.

OK, RAID is not a backup solution per se, but imagine plugging a large drive into an existing array using, adding it to the array and letting it synchronise before faulting it and removing it. Given the high bandwidth of modern SCSI and SATA controllers, you can easily copy the entire array within normal business hours and remove it for offsite storage to cover this environment against a disaster.

Fair enough, even given the low cost of drives, you could not (realistically) implement the Grandfather/Father/Son backup system but you can at least, with a small number of drives, maintain a consistent copy of data that is not only a drop-in replacement for a buggered hard drive, but also have that computer up and running instantly with little administrative overhead.

Now I've missed out a load of the finer points in my comments, but I think the general point is made.

Anon.

-----------

Chris.

On SSDs and Sun/Oracle/Fowler

Sent to me and posted anonymously:-

If we think at the right abstraction level, both do the same: making a copy of our data. The key issues with these techniques is: what are the points where I can recover from and how long will a recovery take. Both techniques have their own merits, but in general, if well designed and implemented, a replicated, exact copy on disk is preferable above a backup approach where we have to restore from when a disaster strikes.

Both techniques also have their own payload on the storage environments, where tape in backup setups has a strong sequential requirement, compared with disk.

That's why John Fowler from Oracle attracted me with the combination of flash disk (SSD) and tape. Both are very fast: tape fast sequential, SSD fast random access, but also fast for sequential access, limited mostly by filesystem (management) overhead. So this combination can be an alternative for a 'green' backup of our data.

Disk storage those days is also used with snapshots, scheduled reguarly or at writes all the time, to build a recoverable history in our storage system. This approach, in combination with a second copy (backup or (a)synchroneous replication) has to be adjusted to business needs.

This becomes more complicated nowadays because Service Oriented Architectures with Service Busses and messages flying all around between services. This means that databases underneat, services that store their own data and the transaction bus environments has to have a consistent recovery for the whole chain. Scheduled maintenance or backup windows will stay around for a while, I guess...

-------------

Chris.

use both but test data integrity

As someone who has installed and setup a few replication systems, its great when it works, but for the dba at the sharp end when replication stops working its a nightmare. Support in europe is very limited and troubleshooting a replication system can be either described as rocket science or a black art. It needs to be constantly monitored and the licenses are expensive. When there is a failure of the primary db, you still always lose some data depending on what the failure was on the primary server.

The backup software comes as part of the original package so its free. Its easy to understand. However daily monitoring of the backups of logs and transactions needs to take place.

In both cases, the big issue is data integrity. You can lock away your backups into a safe inside a mountain but if they are corrupt then they are worthless. Checking of data integrify of the backup and the replicated data must be undertaken daily. Unfortunately maintenance of these systems is the achilles heal where must organisations fail.

Backup also a green initiative

I've been working for/with several large vendors for the last 15 years, and all that time I've been hearing that tape should die. Surprisingly it has not yet died, development is kind of slowing down, but is still out there.

Besides of the point of local/government regulation, there is also a green issue about storing data on tapes.

Think if you need to have one copy/backup of your company data stored for 5-10 years. If it's only replicated you'll need to have disks spinning at all times. I do know vendors offer spindown etc. but still you need to have electricity to the arrays. Tape is by far the cheapest solution, also considering the new technologies emerging like LTO-5, with it's huge capacity.

I'm daily in the business advocating replication both sync and a-sync, but I do find that most customers still need tape to secure the data, at least some of the versions older than let's say 6 months or 1 year. Within this timeframe they are happy to "only" have replication, but they still wants the tape, and do not want to have triple amount of disks spinning.

The issue is data integrity

The comment below was sent to me by Ian Smith who says:

"I have spent over 12 years working in Backup/Recovery and Storage for vendors, consultancies and the clients. I am now the Chief Technology Officer in a company called Backup Migrator. We produce Butterfly software, which are innovative software products to address some of the major issues in the world of Backup and Recovery. Backup Analysis, Backup Migration, Media Migration etc .You can find our website: www.butterflysoftware.net ... [and] I do have some thoughts on the Replication v Backup discussion. I feel some of the views discussed are quite traditional, and would like to give my perspective on it."

Terrific. Here is what Ian mailed:-

The question here has led to a number of over simplified answers, focusing too much on explaining the technology than actually looking forward into technology development and progress.

The ultimate fact is that traditional backup operations, as we currently know them- are finished. There are a number of factors driving this, the primary one being the intelligence that is now being built into the primary storage, host and application environment. This area has advanced significantly. Virtualisation, thin provisioning, source de-duplication, thin replication, application replication are all technology points that are driving down cost and improving efficiencies in the primary environment. There has become a large overlap between what the primary host environment is doing and what traditional backup products are doing.

Content aware primary storage platforms such as NetApp's OnTap are gaining more market share and this can be bound to their cost effective capability for managing and storing data. This data management includes snapshot and replication capabilities that are integrated into the platform. This brings massive benefits to the integration into the primary storage- reducing the volumes of data under management, improving RPO periods and giving near instant recovery capability. This 'replication' style backup methods are significantly more capable than dropping backup images to physical tape. Replication of de-duplicated primary data brings space savings and time savings at getting a consistent image/recovery point off-site.

However, I have always been an advocate of keeping the backup data on different technology: vendor, type, location than the primary environment. If not there is always a risk of total failure through the primary data and recovery data. So whilst the above snapshot and replication capabilities bring huge benefits they are not the whole solution.

This is where I feel traditional 'backup' operations are becoming pincer-ed by the primary intelligence snapshots/replication and a more robust compliance/archive function. This is where a more clear distinction between business continuance and actual compliance recovery becomes apparent. Currently, some environments have tried to create a disaster recovery solution with a backup product- and this is where full system/environment recovery can never be achieved.

A unified solution, of a primary snapshot/replica style environment, with a more robust/off technology long term retention compliance solution sits better. It avoids the overlap that is currently there and simplifies the recovery procedure in the event of a short term, or a long term recovery requirement.

To support this we can already see backup technology and replication/snapshot technology converging. A good example of this is the IBM Tivoli FastBack product. Whilst having a traditional backup capability, it has the drive snapshot, instant mount recovery from the backup server. We can see here that the lines between replication and backup are blurred and the benefits of both are realised to the client.

It's an exciting time in backup at the moment, retention requirements are getting longer, recovery requirements are getting more stringent and technology is moving at a fast pace. Backup Migrator's Butterfly products allow risk free, fully automated migration of backup data between different backup products and technologies- allowing a client to break free of any lock-in to a specific platform. This allows customers to realise the cost and functional benefits of the developments in these technology areas.

Ian Smith