Testing BC and DR planning is an essential component of any “healthy” continuity management program and as such, should be undertaken on a regular basis. While this is generally “good practice,” organisations are often under internal and external compliance and governance pressures to complete additional and more complex or mature testing regimes.
There is a broad range of testing options across the spectrum depending upon the maturity of the organisation’s planning. If this is the first time that a test has been undertaken – (“green fields”) planning can start with a plan walk through (Table Top or White Board) test. These are paper based scenario workshops with business and/or technical personnel attending. The test is generally a few hours long and should question the information and logical sequence of priorities contained in the planning documentation.
At OpsCentre, we often are engaged by a client to assist with an upcoming test and use our experience to add complexity and interest to the activity. The client organisation has successfully completed tested (often the same test) on a number of occasions and would like us to provide more in-depth rigor around the process in general.
Other than experiencing a full blown disaster (which by the way is the best form of test – although not recommended on an annual basis) we have orchestrated testing workshops to assist our clients as detailed below:
1. Applications Functional Testing
- Technical failover of applications or services from the primary production facility to the alternate recovery site
- Insure that the test is isolated from production and that no “cheating” occurs whereby test attendees liaise with production resources or documentation that would not be available in a disaster
2. BC and DR End to end process flow testing
- Complete testing of the recovery facilities by business and technical units including up and downstream application restoration in the disaster recovery environment
- This can be an expensive and resource intensive exercise. The results are extensive and recommended to establish detailed baselines for all aspects of BC/DR planning
3. Denial of Access Testing
- Business site-wide tests for recovery personnel to perform a normal day’s work from their alternate recovery sites with applications/systems pointed to normal production services
- Try this test at 3:00AM – convening disaster personnel and timing their response. Disasters can occur at any time and if not possible to physically attempt this type of test, logically the process flow should include “out of hours” scenarios.
4. Facility Power Downs
- During essential mechanical and electrical maintenance activities at key facilities. Contingency plans are executed/tested concurrently
- If the production infrastructure is going to be off-line due to maintenance that is predetermined, use this opportunity to test your planning and response mechanisms to their fullest.
Completing any of the scenarios illustrated will take a fair amount of project planning and management buy-in. Considerations should be thought out well in advance of the test/audit/governance/compliance schedule so that the test exercises run as smooth as possible and the best results are achieved.