Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Data Preservation Presentation

Leveraging high performance networks to enhance data protection, improve survivability, and simplify disaster recovery.
by

Timothy Pearson

on 22 May 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Data Preservation Presentation

Tim Pearson, Pitt State
tpearson@pittstate.edu

Data Integrity Assurance and Disaster Recovery – Heightened Awareness... a Fresh Look.
Overview
The Status Quo Bias
"We've always done it this way."
"Nothing bad is going to happen."
Traditional approach
Nightly backups (D 2 D 2 T)
Tapes sat in the tape drive, right there in the same datacenter... all week
Same tool for Disaster Recovery & Data Integrity Assurance
Never really believed anything too "bad" would happen
What Motivated us to Change?
Two distinct sides of the "backup" coin:
Disaster Recovery
Data Integrity Assurance
Concepts:
Disaster Recovery
: We want the most recent data possible, in a reliable & usable form, positioned to be placed into production use with short notice and ultra-rapid execution. Usually deals with entire data or file sets or whole virtual machines, collections of VMs, or physical server “images”.
Integrity Assurance
: Making data as “oops”-proof as possible. Maintain multiple point-in-time copies of data and provide a mechanism to restore or “parallel” that data quickly.
Some tools are well suited for Disaster Recovery, some are well suited to data Integrity Assurance use cases. Few tools are well suited for both.
Developing an overall strategy
Status Quo Bias
Motivation
Broad concepts for Disaster Recovery and Data Integrity Assurance
Specific Example (and in use) Solutions
External links
Your comments, successes, and recommendations.
A real Wake-up Call...
How did we react?
Disaster Recovery Planning
Much more emphasis on "off site" backup
Considered DR and DIA as separate issues
Researched tools for our toolkit
Available
Applicable
Affordable
Make it work.
"Backup"
Classify the importance of each data "object" (Volume/Datastore/Database).
How frequently do the contents change:
Highly dynamic to static "archival" data.
What's "do-able" (possible, practical, affordable)
Techniques
Traditional Approach
- DIA / DR use the same tool and simple schedule
:
Runs nightly (Full / Differential / Incremental)
Systems sometimes taken offline to assure logically consistent backups.
Techniques Providing Continuous Protection
Near-realtime
:
Detects changes and copies changed data as quickly as possible (best effort). Write operations are asynchronous.
Usually includes a scheduled component for maintenance or to make certain no changes were missed.
Carbonite, CrashPlan, Google Drive, Sky Drive, etc.
Some are marketed as backup solutions, some are marketed as easy access tools. All of them use very similar approaches and technology.
Some vendors offer "enterprise" solutions in both spaces.
Global De-Duplication
New Possibilities
Unique data is written only once. All recurrences of the same data are stored as "pointers" to the initial occurrence. Space savings are phenomenal!
Pitt's Attempt at a Comprehensive Disaster Recovery & Data Integrity Assurance Strategy
Storage:
Multi-tier design...
Tier 1 - "Big and fast"
: EqualLogic iSCSI arrays - some in the primary datacenter, some in the backup datacenter across campus.
Big and "not so fast" - but immediately offsite
: Dell PowerVault in the Wichita State datacenter.
Big, slow, inexpensive
:
iSCSI "White Box" Linux hosts
Western Digital 12TB "MyCloud" Appliances
All of the storage subsystems can grow in a fairly linear-cost fashion. Performance scales more or less proportionally with growth in capacity.
Backup Strategies:
Traditional daily (CommVault) -
Backs up "everything".
File level. Globally de-duplicated.
One disk library lives in DR Datacenter across campus.
Duplicate library lives on Dell array at Wichita State (1Gb "iSCSI friendly" circuit from KanRen)
Primarily a Data Integrity Assurance tool.
Not primary for DR but can be used for that purpose (slow restoration process).
45 Day retention.
Backup Strategies:
Disaster Recovery Imaging -
vmWare Data Protection Appliances
Image level, globally de-duplicated
Deduplicated VM Images live in DR Datacenter across campus
A DR tool, but could be used to recover individual files.
Very fast. Backs up four 1TB Zimbra "mailbox" servers in under 2 hours.
Variable retention (up to 3 months)
Blindingly fast restoration. "Zero to server" in 15 or 20 minutes for average sized VM.
With addition of another host at WSU, we will be replicating these images off site too.
We use the "free" version. The commercial version supports file level restoration and database awareness (among other enhanced features)
Completely independent of CommVault solution.
Backup Strategies:
Snapshots (EqualLogic/vmWare Integration) -
Scheduled (from every 15 minutes to every couple of hours)
No interruption in operations
Primarily a Data Integrity Assurance tool but could be used for DR as well.
Every virtual machine and every MS-SQL database (quiesced)
Snapshots are retained for a day to a few days
Snapshots can come on-line in place of or along side the volume they protect (very quickly).
Backup Strategies:
Synchronous Replication (SyncRep) - EqualLogic
EqualLogic SAN split into two storage pools consisting of "leggo block" disk array modules. Two high performance arrays in the datacenter make up the primary storage pool and 3 slightly lower performance arrays in the Disaster Recovery datacenter make up the secondary pool.
Data from the primary pool is replicated in real-time to identically provisioned volumes in the secondary one. Only write operations have to be transmitted to the sync partner.
Automatic and manual "failover" from one to the other - transparent to all clients.
User Data Protection and Ubiquitous Access
Open Source - no installation and no subscription costs for software.
Uses off-the-shelf, RAID-capable 12TB appliances with access via iSCSI.
Host is a CentOS virtual machine running LAMP
Clients for Windows, MAC, Linux, Android, IOS
Data remains in Kansas/USA (Regulatory compliance)
Authentication via Active Directory - A/D group membership controls access and quota.
See owncloud.org for more information
Development and Testing
Currently Testing -
Microsoft 2012 R2 "Work Folders"
Currently requires Windows 8 or better
Other clients under development
Not yet ready for "prime time" but shows promise.

Server 2012 R2 Volume Deduplication
Nearly 50% reduction in space consumed by faculty / staff "home" folders and department shares (i.e. wide variety of data).

Owncloud / Work Folders for Students
Script to map to student's folder on virtual (lab) desktops and physical lab machines.
Even with a modest quota, backup will be challenging - no benchmark on deduplication.
Resource List:
Image Backups of Virtual Machines:
vmWare Data Protection: http://www.vmware.com/products/datacenter-virtualization/vsphere/data-protection.html

EqualLogic SANs and vmWare Integration for SnapShots and Smart Copies:
http://www.equallogic.com

Dell Storage:
http://www.dell.com/storage

OwnCloud
http://www.owncloud.org/

Western Digital "MyCloud"
http://www.wd.com/mycloudex4
Questions?
Information to share?
Recommendations?
May 22, 2011 - EF5 - 161 Fatalities
This is a view of the aftermath from an orbiting satellite taken a few days after the incident.
Techniques using schedules
Intra-Day (one to many times during "production" hours):


For
Data Integrity Assurance
:
Snapshotting -
SAN or vmWare are typical examples.
Run on a timed schedule. Quiesce (suspend write operations) the disk device and create a point-in-time version of the storage object. Uses "deltas" that only consume space to accommodate changes.
File level copies -
Can use “volume shadow copy” and other techniques to copy open files.

Techniques using schedules
Intra-Day (one to many times during "production" hours):

For
Disaster Recovery
:
Storage Object
Scheduled Replication
- Copies a “snapshot” to another, remote device.

Techniques Providing Continuous Protection
Real-Time (Synchronous) Replication:

SAN-Synchronous Replication from one storage device to another. Write operations are committed to both the primary and secondary storage before the write is declared complete to the client machine’s operating system.
vmWare vSphere Replication (a storage agnostic replication engine). Runs as a virtual appliance.
SAN Vendor's Intrinsic Feature Set (ex. EqualLogic calls this "SyncRep."
Can negatively impact performance
Requires lots of bandwidth with low latency to work well

Can propagate “oops”’s to your “backup” storage very efficiently
. Not a data integrity assurance solution!
Techniques Providing Continuous Protection
Near-Realtime:
OwnCloud is a very full-featured free, open source alternative to the commercial offerings. (
http://owncloud.org
)
Clients for IOS, Android, Windows, Linux and MAC
Web Access (html and WebDAV)
Folders and/or files can be made sharable, with or without a password
Viewer for popular formats built into the web interface (PDF, DOC, etc)
Versioning (Retains a configurable number of previous file versions)
Recycle-bin / Undelete
A simple LAMP Server (PSU uses CentOS 6 VM with repository for RPM-like installation and maintenance via "yum.")
Supports lots of users on very inexpensive storage (PSU uses two 12TB Western Digital "MyCloud" RAID-capable appliances connected to the OwnCloud server via iSCSI. 12TB MyCloud costs around $1,000.)
Hundreds of clients on a single processor VM with 2GB of memory
tpearson@pittstate.edu
(620) 235-6562
"Compute" Redundancy
Two IBM BladeCenter chassis'
One in primary datacenter and one in DR Datacenter across campus.
50% of the physical blade servers live in each BladeCenter chassis.
Cluster is loaded at about 50% - meaning either location could assume the entire workload.
So long as one of the two SyncRep pools and one of the two BladeCenters are functioning, we're up and running.
In all cases, failover is automatic (SyncRep for storage and HA for hosts).
Full transcript