Disaster Recovery - Symantec

2010 Symantec Disaster Recovery Study Global Results

Methodology • Applied Research performed survey • 1,700 enterprises worldwide • 5,000 employees or more • Cross-industry

2

Key Findings • Virtualization and Cloud Make DR Complex • The Downtime Recovery Gap • Impact of Disaster Recovery Testing • Recommendations

3

Virtualization and Cloud Make DR Complex

4

Virtual Environments Protected Properly? • 56% of data on virtual systems is regularly backed up • Only 20% of virtual environments protected by replication or failover technologies

5

Lack of Tools, Decrease of Virtual Protection • 58% report different tools for virtual and physical environments is a challenge

• Virtualization led 84% to reevaluate DR plans in 2010 • 60% of virtualized environments not covered in DR plans

6

Storage and Resource Constraints an Issue • 59% identified resource constraints (people, budget, and space) as the top challenge when backing up virtual machines

• 57% state that the lack of primary and 60% state that lack of backup storage hampers protecting mission critical data

7

Cloud Causes Security and Control Issues • Organizations put 50% of applications in the cloud • 66% say security is main concern of cloud • 55% say control is biggest challenge of cloud

8

The Downtime Recovery Gap

9

Downtime Recovery Gap • Expectation of downtime for outage = 2 hours • Actual downtime in last 12 months = 5 hours • Median of 4 incidents in past 12 months

10

Major Causes of Downtime • 72% experience downtime from system upgrades (50.9 hours)

• 70% experience downtime from power outages and failures (11.3 hours) • 26% conducted a power outage and failure impact assessment • 63% experience cyber attacks (52.7 hours)

11

Impact of Disaster Recovery Testing

12

Improvement In Testing Frequency and Success • 82% test more frequently than once a year • Significant increase from 66% who reported same in 2009 • 40% of tests fail to meet RTO/RPOs

13

Reasons for not testing • Budget (60%) • Disruption to employees (59%) • Disruption to customers, sales & revenue stream (24%) • Lack of people’s time (26%) • Cost of testing: $606,948

14

Symantec Recommendations

15

Recommendations • Ensure that mission-critical data and applications are treated the same across environments (virtual, cloud, physical) in terms of DR assessments and planning • Use integrated tool sets for managing physical, virtual and cloud environments to save time, training costs and help better automate processes. • Embrace low-impact backup methods and deduplication to ensure that mission-critical data in virtual environments is backed up, efficiently replicated off campus • Prioritize planning activities and tools that automate and perform processes which minimize downtime during system upgrades • Implement solutions that detect issues, reduce downtime and recover faster to be more in line with expectations • Don’t cut corners on basic technologies and processes that protect in case of an outage

Appendix All questions included

17

Demographics

Company titles D: What is your title? 0%

10%

20%

Chief Information Officer (CIO) / Chief Technology Officer (CTO)

40%

50%

24%

VP / SVP

43%

Data Center Maanger or Data Center Director

7%

IT Manager

17%

IT Staff

Other (Please specify)

30%

7%

2%

Industries E: What is your market? 0%

5%

10%

Financial

10%

Manufacturing

10%

Technology

10%

Telecommunications

9%

Healthcare

8%

Automotive

7%

Consumer

7%

Insurance

7%

Retail

7%

Education

4%

Energy

4%

Media

3%

Online

3%

Public sector

3%

Transportation

3%

Real estate

2%


2%

Hospitality

1%

15%

20%

25%

Data Center Questions

Downtime Q1: How many of each of the following has caused your organization to experience downtime in the past five years? (Mark all that apply.) 0%

20%

40%

60%

80%

System upgrades

72%

Power outage / failure / issues

70%

Fire

69%

Configuration change management issues

64%

Cyber attacks

63%

Malicious employee behavior

63%

Data leakage or loss

63%

Flood

48%

Hurricane

47%

Earthquake

46%

Tornado

46%

Terrorism

45%

Tsunami

44%

Volcano

42%

War

42%


1%

100%

Downtime Q2: How many hours of downtime has your organization experienced in the past 12 months for each of the following? (Means shown) 0.0

10.0

20.0

Cyber attacks

40.0

50.0

60.0 52.7

System upgrades

50.9


15.1

Fire

15.0


11.3


10.4

Terrorism

9.6

Earthquake

9.3


9.1

Flood

8.3

Hurricane

7.8

Tornado

7.4

War

7.2

Volcano

6.9

Tsunami

6.9


30.0

1.6

Downtime Q3: As measured by hours of downtime, what is your number one cause of downtime? 0%

10%

20%

System upgrades

30%

40%

50% 48%

Cyber attacks

13%


8%

Fire

6%

Flood

4%


4%


4%

Earthquake

2%


2%

Tsunami

2%

Volcano

2%

Terrorism

2%

Hurricane

1%

Tornado

1%

War

1%


1%

Threat assessments Q4: Which of the following threats has your organization conducted an impact assessment? 0%

20%

40%

60%

Cyber attacks

69%

System upgrades

67%

Earthquake

48%

Terrorism

48%

Hurricane

44%


26%


26%


25%

Fire

24%


23%

Flood

16%

Tsunami

6%

Tornado

6%

Volcano

5%

War

4%


80%

1%

100%

DR responsibility Q5: Which person in your organization has the ultimate responsibility for managing the disaster recovery plan? 0%

20%

Chief Information Officer (CIO) / Chief Technology Officer (CTO)

40%

60%

80% 61%

IT Manager

12%

Disaster Recovery Manager (DRM)

9%

Data Center Manager or Data Center Director

6%

VP / SVP

4%

Business Continuity Manager (BCM)

3%

IT Staff

2%

External consultant / outsourcer

1%

None - we do not have a disaster recovery committee

1%


0%

Don't know

0%

100%

DR committees Q6: Which of the following people are on your organization's disaster recovery committee? (Mark all that apply.) 0%

20%

40%

60%

Disaster Recovery Manager (DRM)

80% 65%

Systems / infrastructure manager

56%

Chief Information Officer (CIO) / Chief Technology Officer (CTO) / IT Director

32%

Chief Executive Officer (CEO)

25%

Chief Security Officer (CSO)

25%

Divisional / Departmental IT manager

21%

Chief Financial Officer (CFO)

18%

Business Continuity Manager (BCM)

15%

Line of business executives / managers

11%

Other directors

8%

External consultant

8%

Non-IT senior managers

7%

None - we do not have a disaster recovery committee

1%


1%

Don't know

1%

100%

DR plans Q9: What of the following are covered by your DR plan? (Mark all that apply.) 0%

20%

40%

60%

HP-UX

55%

AIX

50%

Windows

40%

Solaris

23%

RedHat

18%

VMware

SUSE Linux

16%

11%

80%

100%

Replication Q10a: Do you replicate critical applications between data centers?

No 8%

Yes 92%

Replication Q10b: What replication technologies are used? (Only asked of those who replicate critical applications between data centers) (Mark all that apply.) 0%

20%

40%

60%

80%

Database-based replication

69%

Application-based replication

68%

Array-based replication

65%

Host-based replication

Other (please specify)

34%

0%

100%

Replication challenges Q11: What is your primary challenge with storage array-based replication? 0%

20%

40%

Complexity of replication solutions

55%

Cost

25%

Limited WAN bandwidth (too much data)

Hardware lock-in

60%

17%

3%

80%

100%

Disaster impact Q13: How would you rate the potential impact that could results from a disaster your organization is concerned about? 1 - Absolutely no impact

2 - Low impact

3 - Neutral

4 - Somewhat high impact

5 - Extremely high impact

100% 90%

19%

14%

11%

13%

14%

11%

15%

14%

10%

10%

39%

37%

36%

40%

42%

41%

42%

37%

44%

32%

34%

80% 70% 60%

41%

50% 40% 30% 29%

32%

32%

33%

32%

34%

32%

31%

20%

10% 0%

10% 6% 4%

7%

7%

8%

9%

7%

10%

11%

5%

5%

5%

5%

6%

6%

7%

6%

Data loss

Cost of downtime

Reduction in profits

Reduction in revenue

Damage to brand reputation

Damage to customer loyalty

Damage to supplier relationships

Damage to Configuration competitive drift issues standing in the marketplace

10% 12%

Decreased employee productivity

Downtime costs Q14: What would you estimate is the cost of an hour of downtime for each of the following in your organization? (Means shown) $0

$10,000

$20,000

$30,000

$40,000

$50,000

$60,000

Web servers

$62,063

Custom line of business applications

$55,324

Databases

$47,769

ERPs / CRMs

$42,265

Web commerce applications

$41,117

Application servers

$39,590

Messaging applications

$24,571

Collaboration software

$21,748

Email Other (Please specify)

$70,000

$18,409 $10,523

Outages Q15: How many outages did you have in the past 12 months? Mean

13.8

Downtime Q16: In your estimation, how long was the average time of downtime per incident in hours? Mean

20.4

Disaster recovery budget Q17: What is your annual disaster recovery budget? Mean

$964,599

Disaster recovery budget Q18: In your opinion, which of the following best describes your disaster recovery budget? 1 - Increasing 100%

2 - Staying the same

3 - Decreasing

3%

90% 80%

43%

70% 67% 60% 50% 26% 40%

30% 20% 31%

31%

Over the past 12 months

In the next 12 months

10% 0%

Recession impact Q19: How has the global recession impacted the resources available for your disaster recovery planning? 0%

10%

Extremely negative impact

20%

40%

50%

12%

Some negative impact

23%

No impact whatsoever

17%

Some positive impact

Extremely positive impact

30%

46%

2%

Annual IT budget Q20: What is your total annual IT budget? Mean

$13,573,258

IT budget allocation Q21: What percentage of your IT budget is allocated towards disaster recovery initiatives including backup, recovery, clustering, archiving, spare servers, replication, tape, services, DR plan development and offsite costs, etc.? Median

26%

DR site status Q23: What is the status of your disaster recovery site? (Mark all that apply.) 0%

20%

40%

60%

80%

It is hot standby

72%

It is managed by an outside vendor

63%

It is cold standby

We don't have a disaster recovery site

17%

3%

100%

Failover / recoveries Q24: What percentage of your failover / recoveries you perform is each of the following types? (Means shown) 0%

10%

20%

30%

Same-site failover / recovery

31%

Cloud failover / recovery

29%

Campus failover / recovery

Global failover / recovery

40%

22%

18%

50%

Recovery time Q25: If a significant disaster were to occur at your organization that destroyed the main data center, how soon would the organization be able to do each of the following? (In hours) (Means shown) 2.5 2.4

2.4

2.3 2.2

2.2

2.2

2.1

2.1

2.0

1.9

1.8 Skeleton operations

Mostly back up and running

100 percent up and running

Operations would be able to continue as normal despite the disaster

Recovery objectives Q26: for the Tier 1 applications in your disaster recovery plan, what are your recovery time objectives? What are your recovery point objectives? (Medians shown) Recovery Time Objectives

4

Recovery Point Objectives

5

Recovery objectives Q27: For virtualized applications in your disaster recovery plan, what are your recovery time objectives? What are your recovery point objectives? (Medians shown) Recovery Time Objectives

4.0

Recovery Point Objectives

5.0

Reevaluation Q28: How often do you reevaluate your TO / RPO requirements or change them for new applications? 0%

20%

Monthly

16%

Every 6 months

52%

Once a year

10%

Every 1 - 2 years

Less frequently than every 3 years

60%

14%

Quarterly

Every 2 - 3 years

40%

4%

1%

1%

On an ad-hoc basis

1%

Never

1%

80%

100%

Full scenario testing Q29: How frequently does your organization carry out full scenario testing of its disaster recovery plan, involving relevant people, processes, and technologies? 0%

20%

Monthly

40%

60%

16%

Quarterly

15%

Every 6 months

51%

Once a year

11%

Every 1 - 2 years

3%

Every 2 - 3 years

1%

Less frequently than every 3 years

1%

On an ad-hoc basis

1%

Never

1%

80%

100%

DR testing cost Q30: How much did you spend in the past year on DR testing? Mean

$606,948

DR testing cost Q31: What was the cost of testing your disaster recovery plans in the past year? Mean

$769,686

Successful tests Q32: What percentage of disaster recovery tests successfully recovered critical data and applications within RTOs / RPOs? Median

70%

Recovery barriers Q33: How many times did each of the following challenges prevent you from recovery within the RPOs / RTOs? (Medians shown) 0

1

2

3

4

Insufficient IT infrastructure at the DR site

3

Configuration issues

3

Discovery that the plan has become out of date

3

People do not do as they are supposed to

3

Processes turn out to be inappropriate

3

Technology does not do what it is supposed to


2

0

Testing barriers Q34: Which of the following do you consider to be barriers to running a full scenario test on your disaster recovery plan? (Mark all that apply.) 0%

20%

40%

60%

Resources, in terms of budget

60%

Disruption to employees

59%

Resources, in terms of people's time

26%

Disruption to customers

16%

Lack the technology to run the test

15%

Disruption to sales and the revenue stream

14%

Other IT projects taking a higher priority

13%

Not seen as a priority by top management

4%

None

3%


0%

80%

100%

Deduplication Q35: How far along are you in implementing deduplication? 0%

10%

20%

Considering / planning, but have not yet purchased capabilities

30%

40%

50%

20%

Purchased capabilities, but have not yet implemented

19%

Implemented, but have not been able to see ROI

10%

Implemented, able to demonstrate ROI

48%

Implemented, fell short of ROI

1%

Implemented, but too soon to demonstrate ROI

1%

Deduplication Q36: How much budget would you estimate you save / would save by implementing deduplication? Mean

$893,405

Deduplication Q37: How much storage space, in terms of gigabytes, would you estimate you save / would save by implementing deduplication? Mean

45,735 GB

Appliance form vs. Software model Q38: Do you prefer an appliance form factor with software for deduplication or a software delivery model built into existing backup software that lets you use commodity hardware?

Appliance with software 44% Software delivery model 56%

Reevaluating Q39: Has implementing server virtualization caused you to reevaluate your disaster recovery plan?

No 16%

Yes 85%

Virtual servers Q40: What percentage of virtual servers is covered in your disaster recovery plan? Median

40%

Virtual applications Q41: What percentage of the following applications are being put into virtual environments at present? (Medians shown) 0%

10%

20%

30%

Databases

26%

Application servers

25%

Web servers

25%


23%

ERPs / CRMs

23%



22%

0%

40%

50%

Virtual applications Q42: What percentage of each of the following applications will be put into virtual environments 12 months from now? (Medians shown) 0%

10%

20%

30%

Databases

26%

Application servers

25%

Web servers

25%

ERPs / CRMs

24%


22%


22%


0%

40%

50%

Virtual servers Q43: What percentage of the servers in your data centers are being virtualized in each of the following? (Medians shown) 0%

10%

20%

30%

Application test environment

30%

Patch testing environment

30%

Application development environment

30%

Production environment

30%

40%

50%

Backing up virtual environments Q44: How do you back up virtual environments? (Medians shown) 0%

20%

40%

We utilize off-host technology (e.g., VMware VCB / v-Storage API) for "clientless" backups of VMs

50%

Like a physical machine - standard Client (non deduplication) inside each virtual machine

30%

Like a physical machine - except with deduplication client inside each virtual machine

30%

Not backing up virtual machines

60%

24%

80%

100%

Virtualization Q45: What are the main reasons you have not virtualized more applications? (Mark all that apply.) 0%

20%

40%

60%

Performance

60%

Manpower / human resources

60%

Application vendor support issues

53%

Cost

29%

Skills

25%

Storage inefficiencies / storage costs too high

13%

Inability to meet service levels / availability requirements of the business

10%

Ability to recover and manage virtual environments Haven't though much about it

8% 2%

80%

100%

Virtual server testing Q46: How often do you test virtual servers as part of your disaster recovery plan? 0%

20%

Daily

40%

60%

9%

Weekly

50%

Monthly

14%

Quarterly

13%

Semi-annually

7%

Yearly

5%

Less than once a year

2%

Never

2%

80%

100%

Challenges Q47: What challenges have you faced in protecting mission critical data and applications in virtual environments? (Mark all that apply.) 0%

20%

40%

60%

Lack of available backup storage capacity

60%

Lack of primary storage capacity

57%

Lack of automated recovery

55%

Insufficient backup tools

39%

Lack of enterprise high availability

37%

Lack of enterprise storage management

19%

Different tools for physical and virtual environments

15%

Lack of scalability Other (Please specify)

7% 1%

80%

100%

Challenges Q48: How much of a challenge do each of the following present in protecting mission critical data and applications in virtual environments? 1 - Small Challenge

2 - Neutral

3 - Large Challenge

100% 19%

90%

80%

34%

36%

40%

36%

30% 40% 58%

70%

60% 50%

54%

21% 28%

44%

29%

30%

30%

30% 40%

23%

30%

30%

49% 20%

38%

35%

30%

35%

38%

30%

10%

20%

16%

Different tools for physical and virtual environments

Lack of scalability

0%

Lack of available Lack of primary backup storage storage capacity capacity

Lack of automated recovery

Insufficient backup tools

Lack of enterprise high availability

Lack of enterprise storage management


Virtual applications Q49: What percentage of your organization's data and mission critical applications in virtual environments are protected by each of the following? (Medians shown) 0%

10%

20%

30%

Disk backup

25%

Continuous data protection

23%

Tape backup Online / cloud storage (ie online)

22% 21%

Optical removable media (CDs, DVDs, Blu-ray, etc.)

20%

Data replication

20%

High availability failover

20%

Global or wide area failover

20%

40%

50%

Data backup Q50: What percentage of the data on your virtual systems is regularly backed up? Median

56%

Virtual backup Q51: How often do you back up the data on your virtual systems? 0%

20%

Daily

54%

Monthly

12%

Quarterly

9%

Semi-annually

Less than once a year

Never

60%

18%

Weekly

Yearly

40%

4%

2%

0%

1%

80%

100%

Virtual backup challenges Q52: What is the top challenge with backing up virtual machines as opposed to physical ones? 0%

20%

Resource constraints (people, budgets, and space)

40%

60%

59%

Application-consistent backups

16%

Lack of efficient technology / hardware / software

16%

Lack of efficient restore options

5%

Too much time required

4%

80%

100%

Email recovery Q53: In terms of email or Exchange, which of the following is your primary disaster recovery strategy? 0%

10%

20%

30%

Continuous data protection

34%

Email as a service

26%

Global failover

16%

Local failover

14%

Regular backup

5%

Cloud-based hosting

4%

Protecting data with snapshots

40%

1%

50%

Multi-tiered services Q54: What challenges does your organization have with managing high availability and disaster recovery for multi-tiered IT services? (Mark all that apply.) 0%

20%

40%

Failure to protect all components of the IT service

80%

62%

Lack of coordination between application and data recovery solutions

57%

Having inconsisten levels of protection for different components of the IT service

25%

Lack of understanding application dependencies

18%

Using manual recovery of the application, which is slow and increases the risk of error

14%

Cross-functional teamwork and communication is lacking


60%

9%

2%

100%

Multi-tiered services Q55: How many hours does it take to recover your multitiered services? Mean

22.8

Cloud storage Q56: How far along are you in implementing cloud storage? 0%

20%

40%

Considering / planning, but have not yet purchased capabilities

Already implemented

80%

61%

Purchased capabilities, but have not yet implemented

Not considering

60%

23%

7%

8%

100%

Cloud storage Q57: Have you been able to measure an ROI for cloud storage? 0%

20%

Have not been able to see ROI

60%

80%

14%

Are able to demonstrate ROI

65%

Fell short of ROI

Too soon to demonstrate

40%

11%

9%

100%

Cloud computing Q58: How are you using cloud computing initiatives to help with your data center's disaster recovery plan? 0%

20%

Software as a service

17%

Failover to the cloud

11%

Not using cloud computing

Deploying cloud applications

60%

57%

Backup to the cloud

Recovery from the cloud

40%

6%

6%

4%

80%

100%

Cloud computing impact Q59: What has been the impact of cloud computing to your disaster recovery plan? 0%

20%

Extremely easier

60%

80%

16%

Easier

67%

No change

13%

More difficult

Extremely difficult

40%

4%

0%

100%

Cloud computing challenges Q60: What are the biggest disaster recovery challenges you face when considering implementing cloud computing / cloud storage? 0%

20%

Control failovers / make resources highly available

60%

55%

Control of management of resources

14%

Ability to backup

14%

Security

12%

Expertise


40%

4%

1%

80%

100%

Cloud computing policies Q61: Do you have written guidelines or policies in place for approving cloud applications that use business sensitive or confidential information?

No 15%

Yes 85%

Cloud computing Q62: Who drives cloud computing initiatives? 0%

20%

40%

CEO

55%

CIO / CTO

25%

IT managers

14%

Employee end users / business managers

Employees who implement their own

60%

5%

1%

80%

100%

Cloud computing Q63: What percentages of the following types of applications are you putting into the cloud? (Medians shown) 100% 90% 80% 70% 60% 50%

50%

Mission-critical applications

Non-mission critical applications

50% 40% 30% 20% 10% 0%

Cloud computing concerns Q64: What is the biggest concern with putting mission-critical applications in the cloud? 0%

20%

Security

60%

80%

66%

Accessibility

14%

Control

12%

Management

Backup

40%

6%

3%

100%